Chapter 5: Neural Networks Demystified

In this chapter

• Krish from Pune builds a leaf-species classifier from scratch.
• A perceptron — the simplest neural net. The maths fits on one line.
• Why ‘deep’ learning means many layers.
• Build a 3-layer net in pure numpy. No magic.
• Then meet TensorFlow / PyTorch — what real engineers use.
• Train a digit classifier on the MNIST dataset.

⊕ MATH MINUTE The Perceptron Formula

A perceptron takes inputs (x1, x2, ...), multiplies each by a weight (w1, w2, ...), adds a bias (b), then passes the sum through an activation function f. The OUTPUT is one number. The ‘learning’ is just finding the right weights. That is ALL it does.

output = f(w₁x₁ + w₂x₂ + ... + wₙxₙ + b)

💻 CODE LAB A Perceptron in 6 Lines of numpy

import numpy as np

# Inputs: leaf length, leaf width
x = np.array([12.5, 4.2])
w = np.array([0.7, -0.3])   # weights
b = 0.5

z = np.dot(w, x) + b
output = 1 if z > 0 else 0   # activation (step function)
print('Prediction:', output)  # 1 = mango leaf, 0 = not mango

Why ‘deep’ learning?

One neuron can’t learn much. A million can. A single perceptron can only draw a STRAIGHT line between two classes, but real life is curved. So we stack them into a LAYER of neurons; each neuron in the next layer takes the outputs of the previous one. With 3 layers, a network can learn ANY pattern in theory (the Universal Approximation Theorem, proved in 1989). “Deep” learning means networks with many layers — ChatGPT has 96, Krutrim has 32, and Krish’s leaf network has 3.

💻 CODE LAB Forward Pass — a 3-layer Network in numpy

import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))     # activation function

x = np.array([12.5, 4.2])            # input: leaf [length, width]

# Layer 1: 2 inputs -> 4 hidden neurons
W1 = np.random.randn(4, 2)
b1 = np.random.randn(4)
h1 = sigmoid(W1 @ x + b1)            # hidden layer output

# Layer 2: 4 hidden -> 2 outputs (mango or not)
W2 = np.random.randn(2, 4)
b2 = np.random.randn(2)
output = sigmoid(W2 @ h1 + b2)

print('Class scores:', output)
print('Prediction:', np.argmax(output))

⚙ TRY IT TODAY

The code above does a FORWARD pass with RANDOM weights. To learn, the network needs a BACKWARD pass — adjusting weights based on errors. This is called BACKPROPAGATION. Search ‘backpropagation numpy’ and add training code to make this net actually learn. Once you can train this from scratch, you understand neural nets better than 99% of engineers.

💻 CODE LAB Same 3-layer Net — in PyTorch

import torch
import torch.nn as nn

class LeafNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(2, 4)     # input -> hidden
        self.fc2 = nn.Linear(4, 2)     # hidden -> output

    def forward(self, x):
        x = torch.sigmoid(self.fc1(x))
        x = torch.sigmoid(self.fc2(x))
        return x

model = LeafNet()
x = torch.tensor([12.5, 4.2])
print(model(x))   # PyTorch handles everything for us!

💻 CODE LAB Train an MNIST Digit Classifier in 25 Lines

import torch, torch.nn as nn, torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

train = datasets.MNIST('.', train=True, download=True,
                       transform=transforms.ToTensor())
loader = DataLoader(train, batch_size=64, shuffle=True)

model = nn.Sequential(
    nn.Flatten(),
    nn.Linear(28*28, 128), nn.ReLU(),
    nn.Linear(128, 10))

opt = optim.Adam(model.parameters(), lr=0.001)
loss_fn = nn.CrossEntropyLoss()

for epoch in range(3):
    for xb, yb in loader:
        opt.zero_grad()
        loss = loss_fn(model(xb), yb)
        loss.backward()
        opt.step()
    print('Epoch', epoch, 'loss', round(loss.item(), 3))
# After 3 epochs: ~97% accuracy on the test set

Questions — Chapter 5

1. Write the perceptron formula.
2. What activation function did our examples use? Name one other.
3. ChatGPT has how many transformer layers? (Hint: a number from this chapter.)
4. Convert the numpy 3-layer net to PyTorch — write the LeafNet class.
5. Why did Krish’s accuracy jump from 51% to 86%? What technique did he use?

Check your understanding — Chapter 5

Tap an answer to check it instantly. These are for practice — nothing is saved or sent.