Neural networks are named after brain neurons, which is a comparison that causes more confusion than it resolves. A neural network is not a simplified brain. It is a mathematical function with a very large number of adjustable dials, tuned until its outputs match known answers.
The Basic Structure
A neural network is arranged in layers. Each layer contains nodes, and each node connects to nodes in the next layer with an associated weight — a number expressing how strongly that connection matters.
| Layer | Role | Loan example |
|---|---|---|
| Input | Receives the raw variables | Income, tenure, existing debt, bureau score |
| Hidden | Combines inputs into intermediate representations | Learned combinations nobody named |
| Output | Produces the final result | Probability of default |
The hidden layers are where the interesting and the awkward both live. A network might learn that a particular ratio of income to obligation, combined with employment tenure, predicts something useful. Nobody instructed it to construct that combination and nobody named it. It exists as a pattern of weights.
How Training Works
Training is repeated correction. The process is conceptually simple even though the arithmetic is not.
- Weights start at random values, so the first predictions are meaningless
- The network makes a prediction on a training example
- The prediction is compared with the known answer, producing an error
- The error is propagated backwards, and each weight is nudged in the direction that would have reduced it
- This repeats across millions of examples
Step 4 is backpropagation, and it is the central idea. Each weight is adjusted slightly according to how much it contributed to being wrong. Repeat enough times and the network settles into a configuration that produces good predictions.
Nothing here resembles reasoning. It is optimisation — finding the dial settings that minimise error.
What “Deep” Means
Deep learning simply means a neural network with many hidden layers, as opposed to one or two.
Depth allows a network to build representations in stages. In image recognition, early layers detect edges, middle layers detect shapes, later layers detect objects. Each layer works with what the previous one produced.
This layered abstraction is why deep learning transformed image, speech and language processing. It is also why deep networks need enormous amounts of data — there are far more weights to determine.
Why Finance Often Uses Something Simpler
This is the practically important part, and it runs against the assumption that more sophisticated is better.
| Neural network | Logistic regression / gradient boosting | |
|---|---|---|
| Structured application data | Marginal advantage at best | Often comparable or better |
| Explainability | Poor — weights do not translate to reasons | Good to excellent |
| Data required | Very large | Moderate |
| Training cost | High | Low |
| Regulatory defensibility | Difficult | Straightforward |
| Images, voice, free text | Clearly superior | Poor |
For a table of loan applicant attributes, gradient-boosted trees frequently match or beat a neural network while remaining explainable. Given that explainability is a regulatory expectation in lending, the simpler model often wins on every dimension that matters.
Where neural networks are genuinely necessary in finance: reading documents, processing cheque images, voice analysis in call centres, and the language models that are themselves very large neural networks.
The Explainability Problem
A logistic regression has a coefficient per variable. You can read it: income contributes this much, tenure that much.
A neural network has weights distributed across thousands or millions of connections. No individual weight means anything interpretable. The knowledge is in the pattern, not in any component of it.
This is why post-hoc explanation techniques exist — they approximate why the network decided as it did, since the network cannot be read directly. It is an approximation of reasoning rather than the reasoning itself.
Common Misconceptions
- “It works like a brain.” The inspiration is loose and largely historical. A neural network is a mathematical function, not a simulation of cognition.
- “It understands the data.” It finds statistical associations. Understanding, causation and meaning are not involved.
- “More layers means better.” Beyond what the data supports, additional depth causes overfitting rather than improvement.
- “It keeps learning after deployment.” Almost always no. Weights are frozen at deployment; the model must be explicitly retrained.
- “It removes human bias.” It learns whatever is in the training data, including bias, and can entrench it while appearing objective.
Key Takeaways
- A neural network is layers of nodes connected by weights, tuned to minimise error
- Backpropagation adjusts each weight according to its contribution to being wrong
- Deep learning means many hidden layers, enabling staged abstraction
- Depth transformed image, speech and language processing
- On structured financial data, simpler models often match or beat neural networks
- Weights cannot be read as reasons — hence the explainability problem
- Networks do not understand, do not reason, and do not keep learning after deployment
Frequently Asked Questions (FAQ)
Q: What is a neural network in simple terms?
A neural network is a mathematical system arranged in layers, where connections between nodes carry adjustable weights. Training repeatedly adjusts those weights until the network’s outputs match known correct answers on the training data.
Q: How does a neural network learn?
Through backpropagation. The network makes a prediction, the error against the known answer is measured, and that error is propagated backwards so each weight is nudged in the direction that would have reduced it. Repeating this across millions of examples produces a working model.
Q: What is the difference between a neural network and deep learning?
Deep learning is a neural network with many hidden layers rather than one or two. The additional depth lets the network build representations in stages, which is what made it effective on images, speech and text.
Q: Are neural networks used for credit scoring?
Sometimes, but less often than assumed. On structured application data, gradient-boosted trees and logistic regression frequently perform comparably while remaining explainable — which matters when regulators expect decisions to be accounted for.
Q: Why are neural networks hard to explain?
Because knowledge is distributed across thousands or millions of weights, none of which means anything interpretable on its own. Unlike a regression coefficient, an individual weight cannot be read as a reason, so explanation requires approximation techniques applied afterwards.
Q: Do neural networks work like the human brain?
Only in loose historical inspiration. A neural network is a mathematical function optimised to minimise prediction error. It does not reason, understand or represent meaning in the way the comparison implies.
Q: Does a neural network keep learning after it is deployed?
Almost never. Weights are frozen when the model goes live, and improvement requires an explicit retraining cycle. This is why deployed models degrade over time as conditions change — a problem called model drift.
Related Reading: