Suppose you had to write rules deciding who gets a loan. Income above a level, employment for a period, no recent defaults. You would write perhaps twenty rules and still miss most of what actually predicts repayment. Machine learning takes the opposite approach: show the system a hundred thousand past loans and let it work out the pattern itself.
What is Machine Learning?
Machine learning (ML) is a method of building systems that learn patterns from data rather than being explicitly programmed with rules.
The distinction from ordinary software is genuine:
| Traditional software | Machine learning | |
|---|---|---|
| How behaviour is defined | A person writes the rules | The system infers rules from examples |
| Input | Rules + data → output | Data + outcomes → rules |
| Handles unforeseen cases | Poorly — no rule exists | Better — generalises from patterns |
| Explainability | Complete — read the code | Varies from clear to opaque |
| Behaviour over time | Constant until changed | Degrades as the world shifts |
That final row matters more than people expect and is covered below.
ML, AI and Deep Learning
These nest inside one another:
- Artificial intelligence — the broad field of machines performing tasks associated with human intelligence
- Machine learning — a subset of AI where systems learn from data
- Deep learning — a subset of ML using What is Machine Learning? with many layers
Most of what is marketed as “AI” in Indian financial services is machine learning. Much of it is not deep learning — simpler models often perform better on structured data like loan applications, and they are far easier to explain to a regulator.
The Three Main Types
| Type | How it learns | Finance example |
|---|---|---|
| Supervised | From labelled examples where the answer is known | Credit scoring — past loans labelled repaid or defaulted |
| Unsupervised | Finds structure without labels | Customer segmentation, anomaly detection in transactions |
| Reinforcement | Learns by trial and feedback | Trade execution optimisation |
Supervised learning dominates finance, because financial data usually comes with outcomes attached. You know which loans defaulted. You know which transactions were fraudulent. That labelled history is exactly what supervised learning needs.
How a Credit Model Is Actually Built
The process is worth understanding because it explains most of what goes wrong.
| Step | What happens |
|---|---|
| 1. Assemble data | Historical applications with known outcomes — repaid or defaulted |
| 2. Engineer features | Turn raw data into predictive variables — income stability, credit utilisation, tenure |
| 3. Split the data | Training set to learn from, test set held back to evaluate honestly |
| 4. Train | The algorithm adjusts until predictions match known outcomes |
| 5. Validate | Test on data the model never saw |
| 6. Deploy and monitor | Track whether performance holds in production |
Step 3 is where inexperienced teams fail. Evaluating a model on data it trained on always looks excellent and means nothing — the model has memorised rather than learned. This is overfitting, and it is the most common reason a model that looked outstanding in development performs poorly in production.
Why Models Degrade
A trained model captures relationships as they existed in the training data. When those relationships change, the model quietly becomes less accurate — model drift.
Causes are ordinary: economic conditions shift, a new product changes the customer mix, a competitor’s exit alters who applies, consumer behaviour moves. Nothing breaks visibly. The model keeps producing confident scores that are progressively less correct.
This is why deployment is a beginning rather than an end, and why RBI’s framework emphasises ongoing monitoring rather than one-time validation. See model drift for the detail.
What ML Needs to Work
- Enough data. Predicting a rare event from a few hundred examples does not work.
- Representative data. A model trained only on approved applicants never sees how rejected applicants would have performed — a structural blind spot in lending.
- Genuine signal. If the outcome is largely random, no algorithm will predict it. This is why market prediction resists ML far more than fraud detection does.
- A stable relationship. ML assumes the pattern persists. Where it does not, accuracy decays.
Correlation, Not Causation
An ML model finds statistical association. It does not establish cause.
A model might learn that applications submitted at 3 am default more often. That could reflect something real about the applicants, or it could be an artefact of which channel operates overnight. The model cannot tell the difference — it simply reports the association.
This matters practically. Acting on a correlation as though it were causal produces decisions that fail when the underlying circumstance changes, and in lending it can produce Explainable AI in Lending where a variable stands in for a protected characteristic.
Simple Models Often Win
There is a persistent assumption that more complex means more accurate. In finance this frequently fails to hold.
| Simple models (logistic regression, decision trees) | Complex models (deep neural networks) | |
|---|---|---|
| Structured data like loan applications | Often comparable accuracy | Marginal gain at best |
| Explainability | Strong | Weak |
| Regulatory acceptance | Straightforward | Harder to defend |
| Data required | Less | Much more |
For unstructured data — images, text, voice — deep learning is genuinely superior. For a table of applicant attributes, a well-built simple model is often both accurate enough and far easier to justify.
Key Takeaways
- Machine learning infers rules from data instead of following rules a person wrote
- ML sits inside AI; deep learning sits inside ML
- Supervised learning dominates finance because outcomes are known
- Overfitting — memorising rather than learning — is the classic development failure
- Model drift means accuracy decays silently as the world changes
- ML finds correlation, not causation
- Simple models often match complex ones on structured financial data, and explain far better
- A model trained only on approved applicants carries a structural blind spot
Frequently Asked Questions (FAQ)
Q: What is machine learning in simple terms?
Machine learning is a way of building systems that work out patterns from examples rather than being given explicit rules. Show a system a hundred thousand past loans with their outcomes and it learns which characteristics predict repayment, without anyone writing those rules down.
Q: What is the difference between AI and machine learning?
Artificial intelligence is the broad field of machines performing tasks associated with intelligence. Machine learning is a subset where systems learn from data. Deep learning is a further subset using multi-layered neural networks. Most financial “AI” is machine learning.
Q: What are the types of machine learning?
Supervised learning uses labelled examples where the answer is known, and dominates finance. Unsupervised learning finds structure without labels, used in segmentation and anomaly detection. Reinforcement learning improves through trial and feedback, used in areas like trade execution.
Q: What is overfitting?
Overfitting occurs when a model memorises its training data rather than learning generalisable patterns. It performs superbly on data it has seen and poorly on new cases. Holding back a test set the model never trains on is how this is detected.
Q: Why do machine learning models get worse over time?
Because they capture relationships as they existed in the training data. When economic conditions, customer mix or behaviour change, those relationships shift and accuracy decays — a phenomenon called model drift. Nothing visibly breaks, which is what makes it dangerous.
Q: Does machine learning show cause and effect?
No. ML identifies statistical correlation. It cannot distinguish a causal relationship from a coincidental one, which is why acting on model output as though it explained why something happens can produce decisions that fail when circumstances change.
Q: Are complex models always better?
No. On structured data such as loan applications, well-built simple models often match complex ones while being far easier to explain and defend to a regulator. Deep learning genuinely outperforms on unstructured data like images, text and voice.
Related Reading: