LLMs outperforming humans in complex text analysis is good news for researchers

A study that tested the capacity of LLMs to analyze news articles has found a level of “unprecedented accuracy” that consistently outperforms human coders.

As social and behavioral research increasingly relies on gaining insights from large volumes of text data, from central bank statements to movie scripts, new findings indicate that LLMs offer a cost-effective method of analysis that will drastically cut time and costs for researchers.  

The authors of the research — Vicente J Bermejo (Assistant Professor at Esade), Andrés Gago and Ramiro H. Gálvez (Universidad Torcuato Di Tella), and Nicolás Harari (Boston University)— say the benefits of using LLMs to analyze complex and lengthy texts will continue to improve as the technology develops.