LLM trading agents mostly fail when stock-market tests become long, broad, and fair.
Rohan Paul Twitter · Rohan Paul (@rohanpaul_ai) · 2026-06-26
A new FINSABER benchmark study finds that LLM-based trading agents like FinMem and FinAgent consistently fail to beat simple buy-and-hold strategies when evaluated over 20 years of stock market data.
Extraction
Topics: llm-agentsfinancial-aibenchmarksevaluation
Claims
- LLM trading strategies appear competitive in narrow tests but fail to outperform simple market baselines in longer, fairer evaluations.
- LLMs exhibit systematically poor behavior across market conditions, being too cautious in bull markets and too risky in bear markets.
- Comprehension of financial text does not translate to reliable stock market timing for current LLMs.
- Existing LLM trading benchmarks are susceptible to cherry-picking and short evaluation windows that inflate apparent performance.
Key quotes
LLM strategies can look good in narrow tests, but they usually fail to beat simple market strategies once the test becomes longer and fairer.
Current LLMs may understand financial text, but that does not mean they can reliably time the stock market.