Linear Independence #
A set of vectors is linearly independent if none of them can be written as a linear combination of the others.
\[ c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0} \;\Rightarrow\; c_1=\cdots=c_k=0 \]Independence means each vector adds new information.
Why it matters #
- Detects redundancy
- Connects to rank and basis
If one vector can already be formed using others, it does not add anything new.
Key Idea: Linear independence tells us whether vectors carry unique information or are redundant. If even one vector can be expressed using others, the set is dependent.
Intuition (From Lectures) #
Vectors represent directions in space.
If a vector lies in the span of others, it adds no new direction.
Independent vectors create new dimensions. From lecture explanation:
- Think of vectors as “features”
- Independent vectors → new feature
- Dependent vectors → repeated feature
Formal Definition #
\[ c_1 v_1 + c_2 v_2 + \cdots + c_k v_k = 0 \] \[ c_1 = c_2 = \cdots = c_k = 0 \]This means:
- only trivial solution exists
- no vector depends on others
Linear Dependence #
\[ \exists \; c_i \neq 0 \text{ such that } \sum c_i v_i = 0 \] In lectures, this was linked to solving:
\[ A x = 0 \]If non-trivial solution exists → dependent
Matrix Interpretation #
\[ A = [v_1 \; v_2 \; \cdots \; v_k] \]Columns independent ⇔ full rank.
From lecture:
- pivot columns → independent
- non-pivot columns → dependent
Rank Connection #
\[ \text{rank}(A) = k \Rightarrow \text{independent} \] \[ \text{rank}(A) < k \Rightarrow \text{dependent} \]Lecture insight:
Rank tells how many “useful” columns exist.
Anything beyond rank is redundant.
#
Dimension Relationship (VERY IMPORTANT) #
You cannot have more independent vectors than the dimension.
Case 1 #
\[ k > n \Rightarrow \text{always dependent} \]Lecture intuition:
Space has limited directions.
Extra vectors must reuse existing ones.
Case 2 #
\[ k = n \Rightarrow \text{check independence} \]Check using:
- determinant
- rank
Case 3 #
\[ k < n \Rightarrow \text{can be independent} \]But still check:
- vectors might lie in same direction
Visual Intuition #
flowchart LR A[2D Space] --> B[2 Independent Vectors] B --> C[Full Span] A --> D[3 Vectors] D --> E[Dependent]
Interpretation:
- 2 vectors → enough for plane
- extra vector → redundant
Examples #
Dependent #
\[ (1,2), (2,4) \]Second vector is multiple → no new direction
Independent #
\[ (1,0), (0,1) \]Both directions unique
Quick Summary #
| Case | Result |
|---|---|
| k > n | Dependent |
| k = n | Check |
| k < n | Can be independent |
REF Method #
- Convert to REF
- Find pivot columns
- Check pivots
Geometric Interpretation #
Independent → different directions
Dependent → same plane/line
Basis Connection #
Basis = independent + spanning
Example #
\[ v_1 = (1,2), \quad v_2 = (2,4) \] \[ v_2 = 2 v_1 \]Dependent.
Exam Focus #
- REF method
- Rank relation
- Null space connection
References #
- Lecture slides
- Course handout