LLMs are changing how we do science

AI SCIENCE

LLMs are changing how we do science

LLMs are now more accurate than human experts at predicting neuroscience study results, according to a study by University College London (UCL).

Published in Nature Human Behaviour, the research shows that LLMs, trained on large amounts of text, can spot patterns in scientific studies and forecast outcomes with exceptional accuracy.

The study used BrainBench, a tool designed to test how well LLMs could identify the correct results from pairs of neuroscience study abstracts.

Each pair included one real abstract and one with plausible but incorrect results.

Both LLMs and human experts were tasked with choosing the correct one.

Here are the key findings:

• LLMs had an average accuracy of 81%, while human neuroscience experts averaged 63%.

• A specialised model, BrainGPT, trained on neuroscience studies, boosted accuracy to 86%.

• LLMs were more accurate when confident in their predictions, suggesting they could work alongside researchers.

This shows the potential of AI to go beyond summarising past knowledge and start predicting future outcomes.

For example, BrainGPT, a version of the Mistral model retrained on neuroscience studies, performed better than general-purpose LLMs.

Professor Bradley Love (UCL Psychology & Language Sciences) commented:

“It won’t be long before scientists use AI tools to design better experiments. While this study focused on neuroscience, the same approach could work across other scientific fields.”

The findings also raise questions about how innovative science really is, as AI’s success suggests many research results follow predictable patterns.

The team is now developing tools to help scientists test their experimental ideas by predicting potential outcomes, which could save time and resources.

This international study involved top institutions, including the University of Cambridge, the University of Oxford, and others across Europe, North America, and Australia, highlighting AI’s growing role in advancing research.

81% accuracy? That’s AI saying, “Hold my coffee.”