The Perceptron
The perceptron is the atom of neural networks. Split it open and you find weights, a bias, and a decision.
Prerequisites: Phase 1 (Linear Algebra Intuition)
Video coming soon
An AI-narrated walkthrough for this module is on the way. Everything you need is in the lesson below.
#Learning Objectives
- Implement a perceptron from scratch in Python, including the weight update rule and step activation function
- Explain why a single perceptron can only solve linearly separable problems and demonstrate the XOR failure case
- Construct a multi-layer perceptron by composing OR, NAND, and AND gates to solve XOR
- Train a two-layer network with sigmoid activation and backpropagation to learn XOR automatically
#The Problem
You know vectors and dot products. You know that a matrix transforms inputs into outputs. But how does a machine learn which transformation to use?
The perceptron answers this. It's the simplest possible learning machine: take some inputs, multiply by weights, add a bias, and make a binary decision. Then adjust. That's it. Every neural network ever built is layers of this idea stacked together.
Understanding the perceptron means understanding what "learning" actually means in code: adjusting numbers until the output matches reality.
#The Concept
#One Neuron, One Decision
A perceptron takes n inputs, multiplies each by a weight, sums them up, adds a bias, and passes the result through an activation function.
graph LR
x1["x1"] -- "w1" --> sum["Σ(wi*xi) + b"]
x2["x2"] -- "w2" --> sum
x3["x3"] -- "w3" --> sum
bias["bias"] --> sum
sum --> step["step(z)"]
step --> out["output (0 or 1)"]
The step function is brutal: if the weighted sum plus bias is >= 0, output 1. Otherwise, output 0.
step(z) = 1 if z >= 0
0 if z < 0This is a linear classifier. The weights and bias define a line (or hyperplane in higher dimensions) that splits the input space into two regions.
#The Decision Boundary
For two inputs, the perceptron draws a line through 2D space:
x2
┤
│ Class 1 /
│ (0) /
│ /
│ / w1·x1 + w2·x2 + b = 0
│ /
│ / Class 2
│ / (1)
┼───────────/──────────── x1Everything on one side of the line outputs 0. Everything on the other side outputs 1. Training moves this line until it correctly separates the classes.
#The Learning Rule
The perceptron learning rule is simple:
For each training example (x, y_true):
y_pred = predict(x)
error = y_true - y_pred
For each weight:
w_i = w_i + learning_rate * error * x_i
bias = bias + learning_rate * errorIf the prediction is correct, error = 0, nothing changes. If it predicts 0 but should be 1, weights increase. If it predicts 1 but should be 0, weights decrease. The learning rate controls how big each adjustment is.
#The XOR Problem
Here's where it breaks. Look at these logic gates:
AND gate: OR gate: XOR gate:
x1 x2 out x1 x2 out x1 x2 out
0 0 0 0 0 0 0 0 0
0 1 0 0 1 1 0 1 1
1 0 0 1 0 1 1 0 1
1 1 1 1 1 1 1 1 0AND and OR are linearly separable: you can draw a single line to separate the 0s from the 1s. XOR is not. No single line can separate [0,1] and [1,0] from [0,0] and [1,1].
AND (separable): XOR (not separable):
x2 x2
1 ┤ 0 1 1 ┤ 1 0
│ / │
0 ┤ 0 / 0 0 ┤ 0 1
┼──/──────── x1 ┼──────────── x1
line works! no single line works!This is a fundamental limit. A single perceptron can only solve linearly separable problems. Minsky and Papert proved this in 1969 and it nearly killed neural network research for a decade.
The fix: stack perceptrons into layers. A multi-layer perceptron can solve XOR by combining two linear decisions into a nonlinear one.
perceptron-boundary#Build It
#Step 1: The Perceptron class
class Perceptron:
def __init__(self, n_inputs, learning_rate=0.1):
self.weights = [0.0] * n_inputs
self.bias = 0.0
self.lr = learning_rate
def predict(self, inputs):
total = sum(w * x for w, x in zip(self.weights, inputs))
total += self.bias
return 1 if total >= 0 else 0
def train(self, training_data, epochs=100):
for epoch in range(epochs):
errors = 0
for inputs, target in training_data:
prediction = self.predict(inputs)
error = target - prediction
if error != 0:
errors += 1
for i in range(len(self.weights)):
self.weights[i] += self.lr * error * inputs[i]
self.bias += self.lr * error
if errors == 0:
print(f"Converged at epoch {epoch + 1}")
return
print(f"Did not converge after {epochs} epochs")#Step 2: Train on logic gates
and_data = [
([0, 0], 0),
([0, 1], 0),
([1, 0], 0),
([1, 1], 1),
]
or_data = [
([0, 0], 0),
([0, 1], 1),
([1, 0], 1),
([1, 1], 1),
]
not_data = [
([0], 1),
([1], 0),
]
print("=== AND Gate ===")
p_and = Perceptron(2)
p_and.train(and_data)
for inputs, _ in and_data:
print(f" {inputs} -> {p_and.predict(inputs)}")
print("\n=== OR Gate ===")
p_or = Perceptron(2)
p_or.train(or_data)
for inputs, _ in or_data:
print(f" {inputs} -> {p_or.predict(inputs)}")
print("\n=== NOT Gate ===")
p_not = Perceptron(1)
p_not.train(not_data)
for inputs, _ in not_data:
print(f" {inputs} -> {p_not.predict(inputs)}")#Step 3: Watch XOR fail
xor_data = [
([0, 0], 0),
([0, 1], 1),
([1, 0], 1),
([1, 1], 0),
]
print("\n=== XOR Gate (single perceptron) ===")
p_xor = Perceptron(2)
p_xor.train(xor_data, epochs=1000)
for inputs, expected in xor_data:
result = p_xor.predict(inputs)
status = "OK" if result == expected else "WRONG"
print(f" {inputs} -> {result} (expected {expected}) {status}")It will never converge. This is the hard proof that a single perceptron cannot learn XOR.
#Step 4: Solve XOR with two layers
The trick: XOR = (x1 OR x2) AND NOT (x1 AND x2). Combine three perceptrons:
graph LR
x1["x1"] --> OR["OR neuron"]
x1 --> NAND["NAND neuron"]
x2["x2"] --> OR
x2 --> NAND
OR --> AND["AND neuron"]
NAND --> AND
AND --> out["output"]
def xor_network(x1, x2):
or_neuron = Perceptron(2)
or_neuron.weights = [1.0, 1.0]
or_neuron.bias = -0.5
nand_neuron = Perceptron(2)
nand_neuron.weights = [-1.0, -1.0]
nand_neuron.bias = 1.5
and_neuron = Perceptron(2)
and_neuron.weights = [1.0, 1.0]
and_neuron.bias = -1.5
hidden1 = or_neuron.predict([x1, x2])
hidden2 = nand_neuron.predict([x1, x2])
output = and_neuron.predict([hidden1, hidden2])
return output
print("\n=== XOR Gate (multi-layer network) ===")
for inputs, expected in xor_data:
result = xor_network(inputs[0], inputs[1])
print(f" {inputs} -> {result} (expected {expected})")All four cases correct. Stacking perceptrons into layers creates decision boundaries that no single perceptron can produce.
#Step 5: Train a Two-Layer Network
Step 4 hand-wired the weights. That works for XOR, but not for real problems where you don't know the right weights in advance. The fix: replace the step function with sigmoid and learn the weights automatically through backpropagation.
class TwoLayerNetwork:
def __init__(self, learning_rate=0.5):
import random
random.seed(0)
self.w_hidden = [[random.uniform(-1, 1), random.uniform(-1, 1)] for _ in range(2)]
self.b_hidden = [random.uniform(-1, 1), random.uniform(-1, 1)]
self.w_output = [random.uniform(-1, 1), random.uniform(-1, 1)]
self.b_output = random.uniform(-1, 1)
self.lr = learning_rate
def sigmoid(self, x):
import math
x = max(-500, min(500, x))
return 1.0 / (1.0 + math.exp(-x))
def forward(self, inputs):
self.inputs = inputs
self.hidden_outputs = []
for i in range(2):
z = sum(w * x for w, x in zip(self.w_hidden[i], inputs)) + self.b_hidden[i]
self.hidden_outputs.append(self.sigmoid(z))
z_out = sum(w * h for w, h in zip(self.w_output, self.hidden_outputs)) + self.b_output
self.output = self.sigmoid(z_out)
return self.output
def train(self, training_data, epochs=10000):
for epoch in range(epochs):
total_error = 0
for inputs, target in training_data:
output = self.forward(inputs)
error = target - output
total_error += error ** 2
d_output = error * output * (1 - output)
saved_w_output = self.w_output[:]
hidden_deltas = []
for i in range(2):
h = self.hidden_outputs[i]
hd = d_output * saved_w_output[i] * h * (1 - h)
hidden_deltas.append(hd)
for i in range(2):
self.w_output[i] += self.lr * d_output * self.hidden_outputs[i]
self.b_output += self.lr * d_output
for i in range(2):
for j in range(len(inputs)):
self.w_hidden[i][j] += self.lr * hidden_deltas[i] * inputs[j]
self.b_hidden[i] += self.lr * hidden_deltas[i]net = TwoLayerNetwork(learning_rate=2.0)
net.train(xor_data, epochs=10000)
for inputs, expected in xor_data:
result = net.forward(inputs)
predicted = 1 if result >= 0.5 else 0
print(f" {inputs} -> {result:.4f} (rounded: {predicted}, expected {expected})")Two key differences from Step 4. First, sigmoid replaces the step function -- it's smooth, so gradients exist. Second, the train method propagates error backward from output to hidden layer, adjusting every weight proportionally to its contribution to the error. That's backpropagation in 20 lines.
This is the bridge to Lesson 03. The math behind d_output and hidden_deltas is the chain rule applied to the network graph. We'll derive it properly there.
#Use It
Everything you just built from scratch exists in one import:
from sklearn.linear_model import Perceptron as SkPerceptron
import numpy as np
X = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([0, 0, 0, 1])
clf = SkPerceptron(max_iter=100, tol=1e-3)
clf.fit(X, y)
print([clf.predict([x])[0] for x in X])Five lines. Your 30-line Perceptron class does the same thing. The sklearn version adds convergence checks, multiple loss functions, and sparse input support -- but the core loop is identical: weighted sum, step function, weight update on error.
The real gap shows up at scale. What changes in production networks:
- The step function becomes sigmoid, ReLU, or other smooth activations
- Weights are learned automatically via backpropagation (Lesson 03)
- Layers get deeper: 3, 10, 100+ layers
- The same principle holds: each layer creates new features from the previous layer's outputs
A single perceptron can only draw straight lines. Stack them, and you can draw any shape.
#Ship It
This lesson produces:
outputs/skill-perceptron.md- a skill covering when single-layer vs multi-layer architectures are needed
#Exercises
- Train a perceptron on a NAND gate (the universal gate - any logic circuit can be built from NAND). Verify its weights and bias form a valid decision boundary.
- Modify the Perceptron class to track the decision boundary (w1x1 + w2x2 + b = 0) at each epoch. Print how the line shifts during training on the AND gate.
- Build a 3-input perceptron that outputs 1 only when at least 2 of the 3 inputs are 1 (a majority vote function). Is this linearly separable? Why?
#Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Perceptron | "A fake neuron" | A linear classifier: dot product of inputs and weights, plus bias, through a step function |
| Weight | "How important an input is" | A multiplier that scales each input's contribution to the decision |
| Bias | "The threshold" | A constant that shifts the decision boundary, letting the perceptron fire even with zero inputs |
| Activation function | "The thing that squishes values" | A function applied after the weighted sum - step function for perceptrons, sigmoid/ReLU for modern networks |
| Linearly separable | "You can draw a line between them" | A dataset where a single hyperplane can perfectly separate the classes |
| XOR problem | "The thing perceptrons can't do" | Proof that single-layer networks cannot learn non-linearly-separable functions |
| Decision boundary | "Where the classifier switches" | The hyperplane w*x + b = 0 that divides input space into two classes |
| Multi-layer perceptron | "A real neural network" | Perceptrons stacked in layers, where each layer's output feeds the next layer's input |
#Further Reading
- Frank Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain" (1958) -- the original paper that started it all
- Minsky & Papert, "Perceptrons" (1969) -- the book that proved XOR was unsolvable by single-layer networks and killed perceptron research for a decade
- Michael Nielsen, "Neural Networks and Deep Learning", Chapter 1 (http://neuralnetworksanddeeplearning.com/) -- free online, best visual explanation of how perceptrons compose into networks
Run this lesson in the Code Lab
A real JupyterLab, with this lesson's code files (and a ready-to-run notebook). The Quick Lab opens instantly in your browser. The Cloud Lab adds a terminal, saved files and the full toolchain.
julia main.jl
# Perceptron + 1-hidden-layer MLP in Julia. Single-layer Rosenblatt
# perceptron for AND/OR/NOT, then a hand-wired XOR network to show
# why the perceptron fails on XOR, then a trained 2-2-1 sigmoid MLP
# with manual backpropagation.
# Stdlib only. Sources:
# https://en.wikipedia.org/wiki/Perceptron
# https://docs.julialang.org/en/v1/manual/types/#Composite-Types
using Random
using Printf
mutable struct Perceptron
weights::Vector{Float64}
bias::Float64
lr::Float64
end
Perceptron(n_inputs::Int; lr::Float64=0.1) =
Perceptron(zeros(Float64, n_inputs), 0.0, lr)
function predict(p::Perceptron, inputs::Vector{Float64})::Int
return sum(p.weights .* inputs) + p.bias >= 0 ? 1 : 0
end
function train!(p::Perceptron, data::Vector{Tuple{Vector{Float64}, Int}}; epochs::Int=100)
for epoch in 1:epochs
errors = 0
for (inputs, target) in data
pred = predict(p, inputs)
err = target - pred
if err != 0
errors += 1
p.weights .+= p.lr * err .* inputs
p.bias += p.lr * err
end
end
if errors == 0
println("Converged at epoch $epoch")
return
end
end
println("Did not converge after $epochs epochs")
end
function test_gate(name::String, n_inputs::Int, data::Vector{Tuple{Vector{Float64}, Int}})
println("=== $name ===")
p = Perceptron(n_inputs)
train!(p, data)
println(" Weights: $(p.weights), Bias: $(p.bias)")
for (inputs, expected) in data
result = predict(p, inputs)
status = result == expected ? "OK" : "WRONG"
println(" $inputs -> $result (expected $expected) $status")
end
println()
end
# Hand-wired XOR via OR + NAND + AND. Demonstrates that a 2-layer
# network of perceptrons can compute XOR even though a single one cannot.
function xor_network(x1::Float64, x2::Float64)::Int
or_neuron = Perceptron(2)
or_neuron.weights = Float64[1.0, 1.0]
or_neuron.bias = -0.5
nand_neuron = Perceptron(2)
nand_neuron.weights = Float64[-1.0, -1.0]
nand_neuron.bias = 1.5
and_neuron = Perceptron(2)
and_neuron.weights = Float64[1.0, 1.0]
and_neuron.bias = -1.5
h1 = predict(or_neuron, Float64[x1, x2])
h2 = predict(nand_neuron, Float64[x1, x2])
return predict(and_neuron, Float64[h1, h2])
end
# Tiny trained MLP: 2 inputs -> 2 hidden sigmoid neurons -> 1 sigmoid output.
mutable struct TwoLayerNetwork
w_hidden::Matrix{Float64} # 2x2
b_hidden::Vector{Float64} # 2
w_output::Vector{Float64} # 2
b_output::Float64
lr::Float64
# caches for backprop
last_input::Vector{Float64}
hidden_out::Vector{Float64}
output::Float64
end
function TwoLayerNetwork(; lr::Float64=2.0, seed::Int=0)
rng = MersenneTwister(seed)
return TwoLayerNetwork(
rand(rng, 2, 2) .* 2 .- 1,
rand(rng, 2) .* 2 .- 1,
rand(rng, 2) .* 2 .- 1,
rand(rng) * 2 - 1,
lr,
Float64[],
zeros(Float64, 2),
0.0,
)
end
sigmoid(x::Float64)::Float64 = 1.0 / (1.0 + exp(-clamp(x, -500.0, 500.0)))
function forward!(net::TwoLayerNetwork, inputs::Vector{Float64})::Float64
net.last_input = inputs
for i in 1:2
z = net.w_hidden[i, 1] * inputs[1] + net.w_hidden[i, 2] * inputs[2] + net.b_hidden[i]
net.hidden_out[i] = sigmoid(z)
end
z_out = net.w_output[1] * net.hidden_out[1] + net.w_output[2] * net.hidden_out[2] + net.b_output
net.output = sigmoid(z_out)
return net.output
end
function backward!(net::TwoLayerNetwork, target::Float64)
err = target - net.output
d_output = err * net.output * (1 - net.output)
saved_w_output = copy(net.w_output)
hidden_deltas = zeros(Float64, 2)
for i in 1:2
h = net.hidden_out[i]
hidden_deltas[i] = d_output * saved_w_output[i] * h * (1 - h)
end
for i in 1:2
net.w_output[i] += net.lr * d_output * net.hidden_out[i]
end
net.b_output += net.lr * d_output
for i in 1:2, j in 1:2
net.w_hidden[i, j] += net.lr * hidden_deltas[i] * net.last_input[j]
end
for i in 1:2
net.b_hidden[i] += net.lr * hidden_deltas[i]
end
end
function train!(net::TwoLayerNetwork, data::Vector{Tuple{Vector{Float64}, Float64}};
epochs::Int=10000)
for epoch in 0:(epochs - 1)
total_err = 0.0
for (inputs, target) in data
out = forward!(net, inputs)
total_err += (target - out) ^ 2
backward!(net, target)
end
if epoch % 2000 == 0
@printf(" Epoch %d, error: %.4f\n", epoch, total_err)
end
end
end
function main()
and_data = Tuple{Vector{Float64}, Int}[
(Float64[0, 0], 0),
(Float64[0, 1], 0),
(Float64[1, 0], 0),
(Float64[1, 1], 1),
]
or_data = Tuple{Vector{Float64}, Int}[
(Float64[0, 0], 0),
(Float64[0, 1], 1),
(Float64[1, 0], 1),
(Float64[1, 1], 1),
]
not_data = Tuple{Vector{Float64}, Int}[
(Float64[0], 1),
(Float64[1], 0),
]
xor_data = Tuple{Vector{Float64}, Int}[
(Float64[0, 0], 0),
(Float64[0, 1], 1),
(Float64[1, 0], 1),
(Float64[1, 1], 0),
]
test_gate("AND Gate", 2, and_data)
test_gate("OR Gate", 2, or_data)
test_gate("NOT Gate", 1, not_data)
println("=== XOR Gate (single perceptron - will fail) ===")
p_xor = Perceptron(2)
train!(p_xor, xor_data; epochs=1000)
for (inputs, expected) in xor_data
result = predict(p_xor, inputs)
status = result == expected ? "OK" : "WRONG"
println(" $inputs -> $result (expected $expected) $status")
end
println()
println("=== XOR Gate (multi-layer network - works) ===")
for (inputs, expected) in xor_data
result = xor_network(inputs[1], inputs[2])
status = result == expected ? "OK" : "WRONG"
println(" $inputs -> $result (expected $expected) $status")
end
println()
println("=== XOR Gate (trained 2-layer network with backpropagation) ===")
xor_train = Tuple{Vector{Float64}, Float64}[
(Float64[0, 0], 0.0),
(Float64[0, 1], 1.0),
(Float64[1, 0], 1.0),
(Float64[1, 1], 0.0),
]
net = TwoLayerNetwork(lr=2.0)
train!(net, xor_train; epochs=10000)
println()
for (inputs, expected) in xor_train
result = forward!(net, inputs)
predicted = result >= 0.5 ? 1 : 0
@printf(" %s -> %.4f (rounded: %d, expected %d)\n", inputs, result, predicted, Int(expected))
end
end
if abspath(PROGRAM_FILE) == @__FILE__
main()
end
class Perceptron:
def __init__(self, n_inputs, learning_rate=0.1):
self.weights = [0.0] * n_inputs
self.bias = 0.0
self.lr = learning_rate
def predict(self, inputs):
total = sum(w * x for w, x in zip(self.weights, inputs))
total += self.bias
return 1 if total >= 0 else 0
def train(self, training_data, epochs=100):
for epoch in range(epochs):
errors = 0
for inputs, target in training_data:
prediction = self.predict(inputs)
error = target - prediction
if error != 0:
errors += 1
for i in range(len(self.weights)):
self.weights[i] += self.lr * error * inputs[i]
self.bias += self.lr * error
if errors == 0:
print(f"Converged at epoch {epoch + 1}")
return
print(f"Did not converge after {epochs} epochs")
def test_gate(name, n_inputs, data):
print(f"=== {name} ===")
p = Perceptron(n_inputs)
p.train(data)
print(f" Weights: {p.weights}, Bias: {p.bias}")
for inputs, expected in data:
result = p.predict(inputs)
status = "OK" if result == expected else "WRONG"
print(f" {inputs} -> {result} (expected {expected}) {status}")
print()
and_data = [
([0, 0], 0),
([0, 1], 0),
([1, 0], 0),
([1, 1], 1),
]
or_data = [
([0, 0], 0),
([0, 1], 1),
([1, 0], 1),
([1, 1], 1),
]
not_data = [
([0], 1),
([1], 0),
]
xor_data = [
([0, 0], 0),
([0, 1], 1),
([1, 0], 1),
([1, 1], 0),
]
test_gate("AND Gate", 2, and_data)
test_gate("OR Gate", 2, or_data)
test_gate("NOT Gate", 1, not_data)
print("=== XOR Gate (single perceptron - will fail) ===")
p_xor = Perceptron(2)
p_xor.train(xor_data, epochs=1000)
for inputs, expected in xor_data:
result = p_xor.predict(inputs)
status = "OK" if result == expected else "WRONG"
print(f" {inputs} -> {result} (expected {expected}) {status}")
print()
def xor_network(x1, x2):
or_neuron = Perceptron(2)
or_neuron.weights = [1.0, 1.0]
or_neuron.bias = -0.5
nand_neuron = Perceptron(2)
nand_neuron.weights = [-1.0, -1.0]
nand_neuron.bias = 1.5
and_neuron = Perceptron(2)
and_neuron.weights = [1.0, 1.0]
and_neuron.bias = -1.5
hidden1 = or_neuron.predict([x1, x2])
hidden2 = nand_neuron.predict([x1, x2])
return and_neuron.predict([hidden1, hidden2])
print("=== XOR Gate (multi-layer network - works) ===")
for inputs, expected in xor_data:
result = xor_network(inputs[0], inputs[1])
status = "OK" if result == expected else "WRONG"
print(f" {inputs} -> {result} (expected {expected}) {status}")
print()
class TwoLayerNetwork:
def __init__(self, learning_rate=0.5):
import random
random.seed(0)
self.w_hidden = [[random.uniform(-1, 1), random.uniform(-1, 1)] for _ in range(2)]
self.b_hidden = [random.uniform(-1, 1), random.uniform(-1, 1)]
self.w_output = [random.uniform(-1, 1), random.uniform(-1, 1)]
self.b_output = random.uniform(-1, 1)
self.lr = learning_rate
def sigmoid(self, x):
import math
x = max(-500, min(500, x))
return 1.0 / (1.0 + math.exp(-x))
def forward(self, inputs):
self.inputs = inputs
self.hidden_outputs = []
for i in range(2):
z = sum(w * x for w, x in zip(self.w_hidden[i], inputs)) + self.b_hidden[i]
self.hidden_outputs.append(self.sigmoid(z))
z_out = sum(w * h for w, h in zip(self.w_output, self.hidden_outputs)) + self.b_output
self.output = self.sigmoid(z_out)
return self.output
def train(self, training_data, epochs=10000):
for epoch in range(epochs):
total_error = 0
for inputs, target in training_data:
output = self.forward(inputs)
error = target - output
total_error += error ** 2
d_output = error * output * (1 - output)
saved_w_output = self.w_output[:]
hidden_deltas = []
for i in range(2):
h = self.hidden_outputs[i]
hd = d_output * saved_w_output[i] * h * (1 - h)
hidden_deltas.append(hd)
for i in range(2):
self.w_output[i] += self.lr * d_output * self.hidden_outputs[i]
self.b_output += self.lr * d_output
for i in range(2):
for j in range(len(inputs)):
self.w_hidden[i][j] += self.lr * hidden_deltas[i] * inputs[j]
self.b_hidden[i] += self.lr * hidden_deltas[i]
if epoch % 2000 == 0:
print(f" Epoch {epoch}, error: {total_error:.4f}")
print("=== XOR Gate (trained 2-layer network with backpropagation) ===")
net = TwoLayerNetwork(learning_rate=2.0)
net.train(xor_data, epochs=10000)
print()
for inputs, expected in xor_data:
result = net.forward(inputs)
predicted = 1 if result >= 0.5 else 0
print(f" {inputs} -> {result:.4f} (rounded: {predicted}, expected {expected})")
Checkpoint
5 questions on this lesson
Answer each one and see straight away whether you were right and why. Your first answer to a question counts. You are not signed in, so this is practice and nothing is saved.
Reusable outputs from this lesson. Copy them into your own work.
---
name: skill-perceptron
description: Understand the perceptron pattern and when to use single-layer vs multi-layer architectures
version: 1.0.0
phase: 3
lesson: 1
tags: [perceptron, neural-networks, classification, deep-learning]
---
# The Perceptron Pattern
A perceptron computes a weighted sum of inputs plus a bias, then applies a step function to produce a binary output. It is the fundamental unit of neural networks.
```
output = step(w1*x1 + w2*x2 + ... + wn*xn + bias)
```
## When a single perceptron is enough
- The problem is linearly separable: a straight line (or hyperplane) can divide the two classes
- Logic gates: AND, OR, NOT, NAND
- Simple threshold decisions: "is the score above X?"
- Binary classifiers on data that clusters into two non-overlapping regions
## When you need multiple layers
- The problem is not linearly separable: no single line can separate the classes
- XOR and parity problems
- Any task requiring "this but not that" reasoning (combinations of conditions)
- Real-world classification: images, text, audio - almost always non-linear
## Decision checklist
1. Plot or inspect your data. Can you draw a single straight boundary between classes?
- Yes: single perceptron works
- No: you need at least two layers
2. Can the problem be decomposed into AND/OR of simpler linear decisions?
- This decomposition tells you the minimum network structure
- XOR = (A OR B) AND (NOT (A AND B)) = 3 perceptrons in 2 layers
3. For problems with more than two classes, you need one output node per class
## The training rule
```
error = expected - predicted
weight_new = weight_old + learning_rate * error * input
bias_new = bias_old + learning_rate * error
```
If the prediction is correct, nothing changes. If wrong, weights shift to reduce the error. This only works for single-layer perceptrons. Multi-layer networks require backpropagation.
## Common mistakes
- Trying to learn non-linear patterns with a single perceptron (it will never converge)
- Setting the learning rate too high (weights oscillate) or too low (training takes forever)
- Forgetting the bias term (without it, the decision boundary must pass through the origin)
- Confusing perceptron convergence (guaranteed for linearly separable data) with general neural network convergence (not guaranteed)
The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.