For a while, it seemed like there was nothing that AI chatbots like ChatGPT couldn’t do. But now researchers say they've found at least one thing with which AI is decidedly unspectacular: buying winning stocks.
Economic researchers in Brazil set up an experiment to see if a large language model, in this case Google’s Gemini, could beat either a simple, equal-weighted "naive" portfolio or the S&P 500.
“The ability of LLMs to process and analyze vast amounts of unstructured data, coupled with their ability to generate human-like text, has raised the question of whether they can be leveraged to make efficient decisions in real life,” the researchers wrote, adding that stock trading, managing risk and portfolio management are three areas of particular interest in finance.
The LLM failed to outperform either benchmark, the researchers found.
"The findings reveal no significant empirical evidence that Gemini consistently outperforms these benchmarks," the researchers wrote. "Specifically, Gemini’s success rate in generating higher returns is approximately 50%, indicating no substantial advantage over naive investment strategies."
The study also found that as the investment horizon stretched out over more years, the performance of AI worsened.
"A notable trend observed is the progressive decline in performance as the trading horizon extends, suggesting that Gemini’s predictive capabilities may weaken over longer periods," researchers wrote.
Publishing their findings last week in “Can AI Beat A Naïve Portfolio?: An Experiment With Anonymized Data,” four researchers at the Federal University of Rio Grande do Sul created three stock data sets—one based just on financials, one based just on price data, and one based on both. They also took into consideration different investment time horizons.
In all, they ran 18,000 simulations for 1,522 companies, incorporating 20 years of data. They then asked Gemini to choose five different stocks that met some basic criteria—such as no penny stocks and at least five years of data—and allocate $10,000 as it saw fit.
The scenario the researchers gave Gemini was the following: "You are a financial analyst with expertise in analyzing the past performance of companies and picking winning stocks. In this task, you are analyzing past information about five publicly traded companies."
And the prompt was, "You have $10,000 to invest for the next [time horizon of one month, six months, 12 months or 36 months]. You have six choices—the five publicly traded companies or an investment at the risk-free rate."