AI Stock Analysis: Lessons from a Withdrawn Working Paper
Headlines saying AI outperforms stock analysts need two checks: what was measured, and what is the research’s current status? Financial Statement Analysis with Large Language Models illustrates why older coverage should be checked against the latest original record.

First check: the working paper is temporarily withdrawn
As checked on October 29, 2026, the Financial Statement Analysis with Large Language Models on arXiv states that the working paper is withdrawn, with v3 dated 20 February 2025. The authors say co-researchers found data and analysis inconsistencies while attempting replication, prompting a temporary withdrawal for review.
What can be confirmed is the status and reason stated in the announcement on that page. We should not expand this into allegations of author misconduct or claim that all AI systems cannot analyze financial statements. Retraction is a reason to stop using the original results as evidence of performance until corrected, verifiable results become available.
This point is different from reading general ability scores in AI Benchmark Reading Guide: Here we have to ask more about the extent to which financial test results are connected to decisions and actual returns.
What did the original abstract claim, and what can it still not establish?
The abstract on arXiv describes giving GPT-4 standardized, anonymized financial statements to predict earnings direction. It also makes claims about comparisons with analysts and investment strategies. These statements are the claims of the withdrawn working paper, not the results endorsed by KrobClick. Therefore, the original performance figures are not used to rank or recommend models.
Hiding company names is an experimental-design detail, not proof against future-information leakage or evidence of practical usability. Check model inputs, publication timing, and sample selection. “Working paper” indicates preliminary status; do not call this a journal-article retraction without further evidence.
Separate three questions: reading statements accurately, forecasting earnings accurately, and investing profitably
| Question | Required evidence | What cannot yet be inferred |
|---|---|---|
| Did it read the financial statements correctly? | Numbers, periods, units, and references to the original source. | Does not indicate next year's earnings |
| Was the profit prediction correct? | Out-of-development-set performance, compared with a baseline using data from the same time | Earnings growth does not guarantee a rising share price: it may already be priced in |
| Does the strategy give good returns? | Practical trading rules, appropriate costs, risks, and testing periods | Retrospective results do not guarantee the future or suitability for everyone. |
A plausible explanation still needs its answer checked and tested on out-of-sample data. If a report moves from saying a model can predict earnings to saying it can pick market-beating stocks, check the original paper for evidence supporting that additional claim.
Seven questions before trusting AI investment experiments
- What to predict: Predicting earnings direction, stock prices, and returns after costs are different objectives. Check what “winning” means.
- Compared with what: An analyst, a simple model, or a baseline assuming next year resembles this year? Do all receive comparable information and decision time?
- When is the data available: Financial statements must have been published before the strategy's assumed purchase time. Using subsequently revised databases without accounting for timing may reveal information investors did not yet know.
- How to separate development and test sets: Repeatedly adjusting instructions based on test-set answers means the results are no longer an independent fresh test.
- Are failed or delisted companies included?: Selecting only surviving stocks overlooks failed or delisted companies, making historical results look unrealistically good
- Which expenses are deducted?: Fees, bid-ask spreads, executable prices, short-selling costs where relevant, and liquidity limits
- Is it possible to repeat and check status: Are model versions, dates, datasets, rules, and calculation methods documented enough for review? Does the latest source show corrections or withdrawal?
This is KrobClick's evidence-review framework, not an allegation that the cited study failed every criterion. The only known withdrawal explanation is the authors' announcement; diagnosing further requires more information.
Hypothetical example: Accuracy does not determine net profit.
Suppose strategy A returns 8% per year before costs and baseline B returns 7%. If A's total costs are 2 percentage points and B's are 0.5 percentage points, simple net returns are 6% and 6.5%, respectively. This example subtracts costs from returns to illustrate the concept. It excludes taxes, the timing of costs, and compounding, and is not a result from actual research or a real model.
Even if A has a higher net result in another example, examine drawdowns, borrowing, and risk taken. A single average return can conceal the investor's experience. For averages versus compound growth, read CAGR Article
Check the status of the AI study before citing old coverage
An evidence log might read: “News claims AI beats people at stock analysis; source arXiv:2407.17866; page marked withdrawn; original results cannot currently support performance claims.” Add the actual review date. Do not assume v3 if unspecified, or treat withdrawal as a new experiment proving AI lost to humans.
Record the title and authors, open the original, and check the latest version date and notes. If it is withdrawn, mark the result as unconfirmed and retain that status when sharing older coverage. One study is insufficient to judge all AI systems.
Investor.gov warns to check AI data with other sources. Also, this applies to both chatbot answers and news stories that claim how good the chatbot is. Good evidence should reveal the conditions under which it works and where it remains unknown. This article is about research evaluation. It is not trading advice or a guarantee of investment results.
Sources
- Kim, Muhn and Nikolaev: Financial Statement Analysis with Large Language Models — withdrawn, v3 (20 February 2025)
- SEC Investor.gov: Limitations of AI Data and Warning of Investing Fraud
Primary sources checked October 29, 2026 (Thailand time). Examples and recommendations are KrobClick’s own, not AI test results or actual investment returns. Educational content, not personalized financial, investment, legal, or tax advice; no return is guaranteed.