Basis and Rank #
A basis is a minimal set of linearly independent vectors that spans a space.
The dimension of a space is the number of vectors in a basis.
Key Idea: Basis = independence + spanning. Rank tells us how many independent directions exist in a matrix.
A basis must satisfy two conditions ⭐
- Vectors must be linearly independent
- Vectors must span the space
This means:
- No redundancy (independence)
- Full coverage (spanning)
Why Basis Matters #
- Represents space efficiently
- Removes redundancy
- Helps define coordinates
- Used in ML for feature representation
Dimension #
Dimension is the number of vectors in a basis.
Examples:
- 2D space → dimension = 2
- 3D space → dimension = 3
Rank #
The rank of a matrix is the number of linearly independent columns.
- dimension of its column space (number of independent columns)
- measures the number of linearly independent columns (or rows) of a matrix.
Interpretation #
- High rank → richer information
- Low rank → redundancy
| Condition | Result |
|---|---|
| \( \text{rank}(A) < \text{rank}([A \mid \mathbf{b}]) \) | No solution (Inconsistent system) |
| \( \text{rank}(A) = \text{rank}([A \mid \mathbf{b}]) \) , no free variables | Unique solution |
| \( \text{rank}(A) = \text{rank}([A \mid \mathbf{b}]) \) , free variables exist | Infinitely many solutions |
Key idea:
The rank measures the number of independent equations.
Free variables indicate degrees of freedom in the solution.
Note ⭐ #
- Rank = number of pivot elements in REF / RREF
- Pivot columns are independent
- Non-pivot columns are dependent
How to Find Rank ⭐ #
Step-by-step:
- Convert matrix to REF
- Count pivot positions
- Number of pivots = rank
Column Space and Basis #
Column space is formed by:
- All linear combinations of columns of A
Basis of column space:
- Pivot columns of A
Important:
- Take pivot columns from ORIGINAL matrix
- Not from REF
Null Space Connection #
\[ A x = 0 \]- Solutions form null space
- Free variables → dimension of null space
Solution of Linear Systems ⭐ #
\[ A x = b \]System is consistent if:
\[ \text{rank}(A) = \text{rank}([A \mid b]) \]Interpretation of Solutions #
| Condition | Result |
|---|---|
| \( \text{rank}(A) < \text{rank}([A \mid b]) \) | No solution |
| \( \text{rank}(A) = \text{rank}([A \mid b]) \) , no free variables | Unique solution |
| \( \text{rank}(A) = \text{rank}([A \mid b]) \) , free variables exist | Infinite solutions |
Geometric Interpretation #
- Rank = number of independent directions
- Basis = coordinate system
Examples:
- Rank 1 → line
- Rank 2 → plane
- Rank 3 → space
Basis from Matrix ⭐ #
Steps:
- Convert to REF
- Identify pivot columns
- Select corresponding original columns
These columns form a basis.
Example #
\[ A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix} \]Second column is multiple of first.
Rank = 1
Basis = first column
Important Theorems ⭐ #
- Pivot columns form a basis
- Rank = number of pivots
- Rank ≤ min(m, n)
- Rank determines dimension
Hidden Exam Pattern #
From lectures:
- Rank, basis, and null space are combined
- Same matrix used across multiple sub-questions
Common Mistakes #
- Taking pivot columns from REF instead of original
- Miscounting pivots
- Ignoring free variables
- Confusing span with basis
Strategy to Prepare #
- Practice REF thoroughly
- Identify pivots correctly
- Link rank with null space
- Solve system-based questions
Quick Summary Table #
| Concept | Meaning |
|---|---|
| Basis | Independent + spanning |
| Rank | Number of independent columns |
| Pivot columns | Basis vectors |
| Free variables | Null space dimension |
References #
Lecture slides (Linear Systems, Vector Spaces)
Webinar explanations on rank and solutions
Course handout