Simple Machine Learning
A field journal · v.01
← Back
FiledSeptember 16, 2026·5 min read·By The Editor

Epoch vs Batch vs Iteration

An epoch is one full pass through your training data. A batch is how much the model sees before updating. An iteration is one update. One picture settles all three, plus how many epochs you actually need.

trainingbeginnerintuition

An epoch is one complete pass through your training data. That is the whole definition, and if it is all you came for you can stop here.

The confusion is never really about the word itself — it is about how it relates to the two words that always appear beside it. So:

The three words, in one picture

Figure 1 · how the three words nest
How epochs, batches and iterations nestA row of 1000 training examples is divided into 10 batches of 100. Processing one batch is one iteration; getting through all ten is one epoch.THE WHOLE TRAINING SET — 1,000 examples100100100100100100100100100100one BATCH100 examples= one ITERATION(weights update once)all ten batches = one EPOCH1,000 examples ÷ batch size 100 = 10 iterations per epochtrain for 5 epochs → 50 iterations → 50 weight updates
One epoch is one complete pass over your data. A batch is the handful the model looks at before updating its weights. An iteration is one such update. They are not competing terms — they are three different-sized rulers laid against the same process.
iterations per epoch=training examplesbatch size\text{iterations per epoch} = \frac{\text{training examples}}{\text{batch size}}

With 1,000 examples and a batch size of 100: ten iterations make an epoch. Train for five epochs and the model has updated its weights 50 times, having seen each example five times.

Epoch
One complete pass through the entire training set. The unit you think in when deciding how long to train.
Batch
The group of examples the model looks at before it updates its weights. Also called a minibatch.
Iteration
One weight update — one batch processed. Also called a step.
In plain English

They measure the same process with different rulers. An iteration is one step. An epoch is a lap. A batch is how much you carry per step. Nobody is contradicting anybody.

Why batches exist at all

You might reasonably ask why the model does not just look at all 1,000 examples and then make one very well-informed update.

Two reasons. The first is memory — real datasets do not fit in a GPU. The second is more interesting: more updates beat better updates. Ten decent steps get you further than one perfect step, because after each update the landscape looks different and the next step can be aimed afresh.

This is what the "stochastic" in stochastic gradient descent means: each batch gives a noisy estimate of the true direction downhill, and you take far more steps to make up for it. The arithmetic behind a single one of those steps is worked out in full in gradient descent by hand.

What batch size actually changes

Small batch (8–32)
Small batch (8–32)A loss curve descending over time with small noise.training steps →
Noisy, jumpy, but updates constantly and often generalises better. The noise itself acts a little like regularization.
Medium (64–256)
Medium (64–256)A loss curve descending over time with medium noise.training steps →
The usual answer. Enough smoothing to be stable, enough updates to move. Start here.
Large (1024+)
Large (1024+)A loss curve descending over time with large noise.training steps →
Beautifully smooth and fast per example — but far fewer updates, and it often lands in a worse solution.
  • Small batches give noisy gradients and frequent updates. The noise is not purely bad — it helps the model skip past shallow dips that a smoother path would settle into.
  • Large batches give a clean, accurate gradient and very few updates per epoch. Each step is well-aimed, but there are so few of them that progress per epoch can actually be worse.
In plain English

Doubling the batch size roughly halves the number of updates per epoch. If you double it, you usually have to raise the learning rate to compensate — otherwise the model covers half the ground it did before.

The practical default: start at 32 or 64. Raise it if training is slow and you have memory to spare; lower it if the model will not fit or refuses to generalise.

How many epochs?

This is the question people actually mean when they search for this. The honest answer: you do not decide in advance — you watch the validation loss and stop when it turns.

  • Validation loss still falling → keep going.
  • Validation loss flattening → you are close.
  • Validation loss rising while training loss keeps falling → stop. You are now memorising. That is the classic overfitting signature, explained in bias vs variance.

Stopping at that turn has a name — early stopping — and it is the cheapest regularizer in existence. There is no formula for the right number of epochs because the right number depends on your data, model and learning rate, all of which change.

In plain English

Watching training loss alone tells you nothing about when to stop. Training loss falls essentially forever. The signal is always in the validation curve.

The trap: epochs are not a fixed amount of work

An epoch is a proportion, not a quantity. Ten epochs on 1,000 examples is 10,000 example-views; ten epochs on 10 million examples is 100 million. "Train for 10 epochs" means something completely different for each.

This is why very large models are often trained for a single epoch, or even less. With a large enough corpus, one pass is plenty of updates — and showing the model the same text twice starts to encourage memorisation.

Quick reference

  • One epoch = one full pass over the data.
  • Iterations per epoch = dataset size ÷ batch size.
  • Batch size 32–64 is a sane default; raise it for speed, lower it for memory or generalisation.
  • Double the batch size, consider doubling the learning rate.
  • Choose epochs by watching validation loss, not by picking a number in advance.
  • Epochs measure passes, not work. On a huge dataset, one epoch may be all you need.
← Archive— end of essay —Front page →