Most writing about AI in finance describes what it might do. This describes what it does not do, and specifically which of those limits are structural rather than temporary. The distinction matters, because a limitation waiting on better models is a planning assumption, while a structural one is a permanent constraint.
Structural vs Temporary Limits
| Limitation | Type |
|---|---|
| Cannot reliably predict market prices | Structural |
| Cannot establish causation from correlation | Structural |
| Cannot learn from data that does not exist | Structural |
| Cannot resolve conflicting fairness definitions | Structural — mathematical |
| Hallucination in language models | Reducible, not eliminable |
| Explainability of complex models | Improving, with limits |
| Accuracy on structured prediction | Temporary — genuinely improving |
The top four do not yield to more compute or better architectures. They are properties of the problems, not of the tools.
It Cannot Predict What Is Not Predictable
Market prices already reflect known information. Any exploitable pattern attracts capital until it disappears — meaning the model’s own success destroys its edge.
This is not a modelling shortcoming. A model can only extract signal that exists, and in an actively contested market that signal is small and transient by construction. See AI in algorithmic trading.
Contrast with fraud detection, where AI works well: fraudulent behaviour genuinely differs from normal behaviour in observable ways, so there is real signal to find.
It Cannot Learn From Data That Does Not Exist
This limitation is subtle and affects lending permanently.
A credit model trains on applicants who were approved, because only they have repayment outcomes. Rejected applicants have no outcome — you cannot observe how someone would have repaid a loan they never received.
So the model learns from a population its predecessors selected. If earlier criteria wrongly excluded a group, the model never discovers they would have repaid, and retraining on its own filtered output reinforces the error.
Reject inference and random approvals below cut-off help at the margin. The problem cannot be fully solved because the missing data is unobtainable.
It Cannot Establish Cause
Models find association. They do not distinguish a causal relationship from a coincidental one.
This matters for two reasons. Decisions based on a correlation fail when the underlying circumstance shifts — nothing anchors the relationship. And in lending it produces proxy discrimination, where a variable predicts well because it stands in for something the model is not permitted to use.
Fairness Cannot Be Fully Satisfied
A mathematical result rather than an engineering gap: several reasonable definitions of fairness are mutually incompatible where base rates differ between groups.
You cannot simultaneously equalise approval rates and keep scores equally calibrated across groups except in special cases. No model resolves this, because it is arithmetic. Institutions must choose a definition and defend the choice.
It Assumes Tomorrow Resembles Yesterday
Every model encodes relationships from its training period. When conditions change, accuracy decays without any visible failure — model drift.
In credit this is compounded by feedback delay. Default outcomes take twelve to twenty-four months, so a model that began drifting today cannot be proven to have drifted for over a year.
Genuine regime changes — a pandemic, a regulatory shift, a rate cycle unlike anything in the training window — are exactly the situations where models are least reliable and decisions matter most.
Language Models Cannot Verify Themselves
LLMs optimise for plausible text, not true text. Hallucination is a consequence of the design rather than a defect.
RAG reduces it substantially by grounding answers in supplied documents. It does not eliminate it — retrieval can miss, documents can be stale, and the model can extrapolate past what the text says.
Critically, the model cannot tell you when it is wrong. Self-reported confidence is not calibrated, so asking “are you sure?” produces reassurance rather than information.
Where AI Is Genuinely Strong
Stating the limits fairly requires stating the strengths. AI works well when the problem has:
| Property | Example |
|---|---|
| Large volumes of labelled data | Millions of past transactions with fraud outcomes |
| Fast feedback | Fraud confirmed within days |
| Genuine signal | Fraudulent behaviour really does differ |
| Stable relationships | Document layouts, cheque formats |
| Repetitive high volume | Reconciliation, extraction, classification |
| Tolerable error cost | A misrouted query, not a misapproved loan |
Fraud detection, document processing, KYC verification and back-office automation satisfy most of these. Market prediction satisfies almost none — which is why one works and the other does not.
How to Assess a Claim
- Is there genuine signal? If outcomes are largely random, no model helps.
- Does labelled data exist? No outcomes means no supervised learning.
- How fast is feedback? Slow feedback means slow drift detection.
- Does the target adapt? Adversarial problems degrade faster.
- What does a wrong answer cost? High cost demands human review.
- Can the decision be explained? If not, it may be unusable regardless of accuracy.
Key Takeaways
- Some limits are structural — more compute does not resolve them
- Market prediction fails because signal is faint and self-destroying
- Credit models cannot learn from rejected applicants — permanently
- Models find correlation, never causation
- Fairness definitions are mathematically incompatible
- All models assume tomorrow resembles yesterday
- Hallucination is reducible, not eliminable, and models cannot self-verify
- AI works where there is signal, labelled data, fast feedback and tolerable error cost
Frequently Asked Questions (FAQ)
Q: What are the main limitations of AI in finance?
It cannot reliably predict market prices, cannot learn from data that does not exist such as rejected loan applicants, cannot establish causation, cannot satisfy conflicting fairness definitions simultaneously, and degrades as conditions change. The first four are structural rather than temporary.
Q: Will better AI models solve these problems?
Some yes, some no. Accuracy on structured prediction is genuinely improving. But market unpredictability, the absence of outcome data for rejected applicants, and the incompatibility of fairness definitions are properties of the problems rather than of current tools.
Q: Why does AI work for fraud detection but not stock prediction?
Fraudulent behaviour genuinely differs from normal behaviour in observable ways, feedback arrives within days, and there is abundant labelled data. Market prices already reflect known information, and any exploitable pattern disappears as capital pursues it.
Q: Can AI eliminate bias in lending?
No. Models trained on historical decisions learn historical patterns including discriminatory ones, and removing protected variables does not help because other variables act as proxies. Additionally, competing fairness definitions cannot all be satisfied at once.
Q: Why can’t AI models tell when they are wrong?
Because they have no internal mechanism for verifying truth. Language models produce plausible text and express confidence regardless of accuracy. Self-reported certainty is not calibrated, so asking whether the model is sure provides reassurance rather than information.
Q: What should I check before believing an AI claim in finance?
Whether genuine predictive signal exists, whether labelled outcome data is available, how quickly feedback arrives, whether the target adapts adversarially, what a wrong answer costs, and whether the decision can be explained. Claims that fail these tests warrant scepticism.
Q: Is AI overhyped in financial services?
The applications are frequently overstated while the deployed value is real but unglamorous. Document processing, fraud detection and back-office automation deliver measurable benefit. Market prediction and autonomous decision-making are where claims most often exceed what the technology supports.
Related Reading: