1.1 — Vectors

Date: 2026-02-20 | Block: 1 — Linear Algebra

The idea in plain English

A vector is an arrow — something with a direction and a size (magnitude). When you say "walk 3 blocks east and 4 blocks north," that instruction is a vector: [3, 4]. The numbers tell you how far to go in each direction.

The intuition

Think of a vector as an arrow drawn on paper. The arrow [3, 4] points 3 units right and 4 units up. Crucially, the vector is the arrow itself — not where it starts. The same [3, 4] arrow means the same thing no matter where you draw it from.

      ↑
  4   |     * ← tip
      |    /
      |   /  ← this arrow is the vector [3,4]
      |  /
      | /
      +──────→
           3

In ML, vectors can have thousands of dimensions — a 28×28 image becomes a 784-number vector. You can't draw it, but the math works identically.

The math

Vector: just a list of numbers — v = [v₁, v₂, ..., vₙ]

Addition (element by element):

[1, 2] + [3, 1] = [4, 3]

Geometrically: follow the first arrow, then the second.

Scalar multiplication (multiply every element by the same number):

3 × [1, 2] = [3, 6]     ← same direction, 3× longer
-1 × [1, 2] = [-1, -2]  ← flipped to opposite direction

Magnitude (L2 norm) — the length of the arrow, by Pythagoras:

‖v‖ = √(v₁² + v₂² + ... + vₙ²)

Normalising — make the arrow length = 1 while keeping direction:

v̂ = v / ‖v‖

A worked example

v = [3, 4]
‖v‖ = √(3² + 4²) = √(9 + 16) = √25 = 5
v̂ = [3/5, 4/5] = [0.6, 0.8]
Check: √(0.6² + 0.8²) = √(0.36 + 0.64) = √1 = 1 ✓

Why this matters for ML

Data points are vectors. A 28×28 image is a 784-dimensional vector. A document represented by word counts is a vector. All of ML is operations on these vectors.

L2 regularisation works by penalising the magnitude of the weight vector: ‖w‖². It's literally saying "keep the arrow short."

Normalisation is the key step in cosine similarity — used in NLP, recommendation systems, and anywhere you care about direction but not magnitude.

The one thing to remember

A vector is an arrow: it has a direction and a size. Everything else in linear algebra is built on this.