Lengths and Distances #
The length of a vector is given by its norm.
The distance between two points (vectors) is the norm of their difference.
Distance quantifies how far two vectors (data points) are from each other.
\[ d(\mathbf{x},\mathbf{y}) = \lVert \mathbf{x} - \mathbf{y} \rVert \]Key Idea: Length measures size of a single vector. Distance measures separation between two vectors. Distance = norm applied to difference.
Why it matters #
- many ML algorithms depend on distances in feature space
Length of a Vector #
From analytic geometry lectures:
The length of a vector is its norm.
For vector:
\[ x = (x_1, x_2, \dots, x_n) \]Length (L2 norm):
\[ \lVert x \rVert = \sqrt{x_1^2 + x_2^2 + \cdots + x_n^2} \]Interpretation of Length #
- Measures distance from origin
- Indicates magnitude of vector
- Independent of direction
Examples:
- Small length → close to origin
- Large length → far from origin
Distance Between Two Vectors #
Distance is defined as:
\[ d(x, y) = \lVert x - y \rVert \]Steps:
- Subtract vectors
- Compute norm
Expanded Formula #
For vectors:
\[ x = (x_1, x_2, \dots, x_n), \quad y = (y_1, y_2, \dots, y_n) \]Distance:
\[ d(x,y) = \sqrt{(x_1 - y_1)^2 + \cdots + (x_n - y_n)^2} \]Geometric Interpretation #
- Distance = straight-line (Euclidean) distance
- Represents shortest path between two points
In 2D:
- forms a triangle
- follows Pythagoras theorem
Connection to Inner Product #
From lectures:
\[ \lVert x \rVert = \sqrt{x^T x} \]Distance can be written as:
\[ \lVert x - y \rVert = \sqrt{(x - y)^T (x - y)} \]Important Properties of Distance #
1. Non-negativity #
\[ d(x,y) \geq 0 \]2. Identity #
\[ d(x,y) = 0 \iff x = y \]3. Symmetry #
\[ d(x,y) = d(y,x) \]4. Triangle Inequality #
\[ d(x,z) \leq d(x,y) + d(y,z) \]Other Distance Measures #
L1 Distance #
\[ d(x,y) = \sum |x_i - y_i| \]Infinity Distance #
\[ d(x,y) = \max |x_i - y_i| \]Why Distance Matters (ML Insight) #
From lectures and applications:
Distances are used in:
- k-NN (nearest neighbour)
- clustering algorithms
- similarity measures
- recommendation systems
Key idea:
- closer points → more similar
- farther points → less similar
Distance and Feature Space #
In ML:
- each vector = data point
- space = feature space
Distance determines:
- grouping of data
- classification boundaries
Example #
Given:
\[ x = (1,2), \quad y = (4,6) \] \[ d(x,y) = \sqrt{(1-4)^2 + (2-6)^2} \] \[ = \sqrt{9 + 16} = 5 \]Connection to Norms #
Key relation:
\[ \text{Length} = \lVert x \rVert \] \[ \text{Distance} = \lVert x - y \rVert \]So distance is derived from norm.
Hidden Exam Pattern #
From lectures:
- Distance is combined with:
- norm
- inner product
- angles
👉 rarely asked in isolation
Common Mistakes #
- Forgetting subtraction step
- Mixing L1 and L2 distance
- Missing square root
- Arithmetic errors in expansion
Strategy to Prepare #
- Practice distance calculations
- Understand geometric meaning
- Link with norm and inner product
- Solve ML-style problems
Quick Summary Table #
| Concept | Formula | Meaning |
|---|---|---|
| Length | \lVert x \rVert | Distance from origin |
| Distance | \lVert x - y \rVert | Distance between points |
| L2 | sqrt(sum of squares) | Euclidean |
| L1 | sum of absolute values | Manhattan |
References #
- Lecture slides (Analytic Geometry)
- Course handout