GPT-4 outperforms human analysts in financial statement analysis, claims research

New Delhi : A new research paper claims that GPT-4 can outperform human analysts when it comes to predicting the future based on financial statement analysis. The paper, which has been published in a preprint journal, found in its tests that GPT-4 delivered better results than its human counterparts in the short-term period (between one month and six months). It achieved 60.31 per cent accuracy in its predictions, compared to 56.7 per cent for human analysts. However, the paper did not suggest that the AI ​​model can replace humans.

Objective of the research paper

Published in the preprint journal Social Science Research Network (SSRN), the 54-page paper titled “Financial Statement Analysis with Large Language Models” attempts to explore what role traditional artificial intelligence (AI) models can play in analysing the financial statements of an organisation and predicting its performance in the stock market in the near future.

This type of analysis has always been considered very complex because a wide range of factors can affect the performance of companies. Even some financial companies use artificial neural networks (ANNs) to assist individuals in their analysis, but large language models (LLMs) have not been used for this. The researchers wanted to see if a state-of-the-art (SOTA) LLM like GPT-4 could be a valuable addition to it.

What was found in the GPT-4 research paper?

The researchers gave GPT-4 anonymized and standardized corporate financial statements (to prevent biases that might emerge from mentioning the company name). Next, the researchers used two methods to test LLM’s capabilities. The first was designing a simple prompt that instructed the chatbot to analyze the statements and predict future earnings. The second was to use a “chain-of-thought” (CoT) prompt that taught the AI ​​model to imitate financial analysts.

The COT method asked GPT-4 to identify notable trends, calculate key financial ratios and form expectations about future earnings. While simple signals did not yield remarkable results, COT signals achieved 60.31 percent which was higher than the performance of the average human analyst.

“We have found that LLMs excel at a quantitative task that requires intuition and human-like reasoning. The ability to function across different domains points to the emergence of Artificial General Intelligence,” the paper states.

However, the researchers were quick to point out that GPT and human analysts complement each other, not that the former replaces the latter. In particular, the paper claimed that GPTs have an advantage in areas where humans show bias and disagreement. Humans, similarly, add value when analysis requires additional contextual information that is unlikely to be available within the financial data.