How does a machine learn to recognize handwritten digits
— turning a blurry sketch into a confident answer?
Start with an image. A grid of pixels, each holding a
brightness value between zero and one. Seven hundred and eighty-four numbers.
Stretch them into a column. This is the input layer of a neural
network — every pixel becomes a neuron, lit up by its own activation.
Then come hidden layers, neurons that don't see the image directly,
only the layers before them. And finally an output layer of
ten neurons, one for each digit, zero through nine.
Every neuron in one layer connects to every neuron in
the next. Each connection carries a weight. Big weights mean
strong attention. Small weights mean almost nothing flows through.
A neuron's activation is the weighted sum of everything feeding in,
plus a bias. We squash that number through a non-linear function
— making the neuron either quiet, or confidently active.
Feed the pixels in. The signal cascades layer by layer, transformed by thousands
of weights and squashed at every step, until at the end one output
neuron lights up — the network's guess of what digit it saw.
But at first, the weights are random — chosen with no thought
behind them. The guess is noise. The network has no idea what
a three looks like, or a seven, or a four.
So we measure the damage. Compare the guess to the truth, square the
difference, and average across thousands of training examples. This single number — the
cost — is what we want to make as small as possible.
Imagine the cost as a landscape, one axis for
every weight in the network. We find the steepest
direction down, and take a tiny step that way.
Repeat this billions of times, across millions of training
examples, nudging weights in microscopic steps. Each example tugs
the network just a little. Slowly, the cost falls.
Slowly, the noise resolves into something else.
The hidden layers start to detect edges, then loops, then the
strokes that make up digits. Patterns no one programmed — only
learned, distilled out of the data, one example at a time.
This is a neural network. Layers of
weighted sums, sliding downhill toward truth.