Simple Machine Learning
A field journal · v.01
← Back
FiledSeptember 16, 2026·5 min read·By The Editor

The Dot Product, Explained

Multiply matching entries, add them up, get one number. It measures how much two vectors agree — and it is what neural networks spend nearly all their time computing.

linear-algebraintuitionbeginner

The dot product takes two lists of numbers and returns one number. Multiply the matching entries, add up the results, done.

It is worth learning properly for one reason: it is what neural networks spend essentially all of their time computing. Every neuron, every attention head, every embedding comparison is a dot product underneath.

The arithmetic

Figure 1 · it is a shopping bill
A dot product as a shopping billTwo lists of three numbers, multiplied pairwise — 2 times 4, 1 times 9, 3 times 2 — then added to give 23.shopping listhow many of each213price listwhat each one costs$4$9$2×××8+9+6multiply pairs= 23the billTHE DOT PRODUCTmultiply matching entries, add the results, get one number
Two lists of the same length go in. One number comes out. You already do this every time you work out what a basket costs — multiply each quantity by its price, add up the results. That is the entire operation.

Written out:

a⋅b=a1b1+a2b2+a3b3=(2)(4)+(1)(9)+(3)(2)=23\mathbf{a} \cdot \mathbf{b} = a_1 b_1 + a_2 b_2 + a_3 b_3 = (2)(4) + (1)(9) + (3)(2) = 23

Or for lists of any length:

a⋅b=∑i=1naibi\mathbf{a} \cdot \mathbf{b} = \sum_{i=1}^{n} a_i b_i
In plain English

The big Σ just means "add up all of these." The i underneath counts through the positions, 1 to n. The whole line says: walk both lists together, multiply each pair, total it up. That is the notation for a shopping bill.

Both lists must be the same length, because every entry needs a partner. That single requirement is why matrix dimensions have to line up, and why you spend so much time reshaping tensors.

What the number means

Arithmetic is one thing; intuition is another. Geometrically, the dot product measures how much two vectors agree.

Same direction
Same directionTwo arrows from a shared origin, 0 degrees apart.
Dot product is large and positive. These two vectors are saying the same thing.
At right angles
At right anglesTwo arrows from a shared origin, 90 degrees apart.90°
Dot product is exactly zero. Nothing in common — the formal word is orthogonal.
Opposite directions
Opposite directionsTwo arrows from a shared origin, 150 degrees apart.150°
Dot product is negative. These two actively disagree.

The formula that connects the two views:

a⋅b=∥a∥ ∥b∥cos⁡θ\mathbf{a} \cdot \mathbf{b} = \lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert \cos\theta
In plain English

The double bars mean "length of". So: multiply the two lengths, then scale by how aligned they are. cos θ is 1 when the arrows point the same way, 0 at right angles, and −1 when opposed. The dot product is length times length times agreement.

So the sign alone tells you something useful:

Positive
The two vectors point broadly the same way. The bigger the number, the stronger the agreement.
Zero
They are at right angles. No shared direction at all — orthogonal.
Negative
They point broadly opposite ways. They actively disagree.

This is why cosine similarity — the dot product with both lengths divided out, leaving only the cos⁡θ\cos\theta part — is the standard way to ask whether two embeddings mean similar things. Strip out magnitude, keep direction, compare.

Why neural networks are made of this

Here is the part that makes the whole subject click.

Figure 3 · what a layer actually computes
A neural network layer as a stack of dot productsThree inputs on the left connect to four neurons on the right. Each neuron takes the dot product of the whole input with its own row of weights.x₁x₂x₃••••inputsneuronsone dot productof every input withits own weight row12 connections = 12 multiplications = 4 dot products
A neuron is not a mysterious unit. It holds one row of weights, takes the dot product of that row with everything feeding into it, and passes the single resulting number onward. A layer is just several of those at once — which is exactly what matrix multiplication is.

A neuron holds a row of weights. It takes the dot product of that row with its inputs, adds a bias, and passes the result through an activation function. One number in, one number out.

A layer is several neurons doing that at once against the same inputs. Stack their weight rows into a grid and you have a matrix, and doing all those dot products simultaneously is precisely matrix multiplication:

y=Wx+b\mathbf{y} = W\mathbf{x} + \mathbf{b}
In plain English

Every entry of the output y is one dot product: one row of W against the input x. Matrix multiplication is not a new operation to learn — it is a table of dot products, computed in one go.

That is the entire reason GPUs matter. A GPU is a machine for doing enormous numbers of multiply-and-add operations in parallel, and a dot product is exactly multiply-and-add. When people say a model is "500 billion parameters," they are describing how many numbers take part in these sums.

Three places it shows up by name

  • Similarity search. Embeddings are vectors; comparing them is a dot product. This is how retrieval finds the paragraph relevant to your question — see tokens and embeddings.
  • Attention. A transformer decides which words to pay attention to by taking dot products between queries and keys. High score, more attention.
  • Projection. "How much of vector a lies along direction b" is a dot product. That is what a layer is doing: measuring the input against each learned direction in turn.

Getting it right

A few things that bite:

  • Lengths must match. Different-length vectors have no dot product. Most shape errors in deep learning code are this rule being enforced.
  • It is not elementwise multiplication. a * b in NumPy gives you a list; a @ b gives you the single dot-product number. Confusing the two is a classic silent bug.
  • Order does not matter for the dot product — a⋅b=b⋅a\mathbf{a} \cdot \mathbf{b} = \mathbf{b} \cdot \mathbf{a} — but it very much does for matrix multiplication.
  • Magnitude leaks in. A long vector produces big dot products with everything. If you want similarity rather than raw score, normalise first.

The short version

  • Multiply matching entries, add them up, get one number.
  • Sign tells you agreement: positive = aligned, zero = unrelated, negative = opposed.
  • a⋅b=∥a∥∥b∥cos⁡θ\mathbf{a} \cdot \mathbf{b} = \lVert a \rVert \lVert b \rVert \cos\theta — length times length times alignment.
  • A neuron is one dot product. A layer is a stack of them, also known as matrix multiplication.
  • Normalise before comparing, or magnitude dominates.
← Archive— end of essay —Front page →