Systems of Linear Equations #
A system of linear equations can be written compactly as:
\[ A\mathbf{x}=\mathbf{b} \]
This represents:
- a linear transformation applied to an unknown vector (\mathbf{x})
- producing an output vector (\mathbf{b})
Key components #
Coefficient matrix (A) #
(A) contains the coefficients of the variables.
\[ A \in \mathbb{R}^{m\times n} \]
Variable vector (\mathbf{x}) #
(\mathbf{x}) contains the unknowns.
\[ \mathbf{x}= \begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_n \end{bmatrix} \in \mathbb{R}^{n} \]
Constant vector (\mathbf{b}) #
(\mathbf{b}) contains constants on the right-hand side.
\[ \mathbf{b}= \begin{bmatrix} b_1\\ b_2\\ \vdots\\ b_m \end{bmatrix} \in \mathbb{R}^{m} \]
When does a solution exist? #
The type of solution depends on (A).
Unique solution #
If (A) is square and invertible, then the solution is unique.
\[ \det(A)\neq 0 \]
In that case:
\[ \mathbf{x}=A^{-1}\mathbf{b} \]
A unique solution requires (A) to be invertible (non-singular).
No solution or infinitely many solutions #
If (A) is not invertible (singular), then the system can have:
- no solution (inconsistent), or
- infinitely many solutions (multiple degrees of freedom)
\[ \det(A)=0 \]
Homogeneous systems #
A homogeneous system is:
\[ A\mathbf{x}=\mathbf{0} \]
- It always has the trivial solution (\mathbf{x}=\mathbf{0}).
- It has a non-trivial solution ((\mathbf{x}\neq \mathbf{0})) if and only if (A) is singular.
\[ A\mathbf{x}=\mathbf{0}\text{ has a non-trivial solution } \iff \det(A)=0 \]
Interpretation #
Linear combination view #
The equation (A\mathbf{x}=\mathbf{b}) means (\mathbf{b}) is a linear combination of the columns of (A).
If (A=[\mathbf{a}_1\ \mathbf{a}_2\ \dots\ \mathbf{a}_n]), then:
\[ A\mathbf{x}=\mathbf{b} \iff x_1\mathbf{a}_1+x_2\mathbf{a}_2+\cdots+x_n\mathbf{a}_n=\mathbf{b} \]
Geometric view #
Each equation corresponds to a hyperplane in (n)-dimensional space.
The solution set is where all hyperplanes intersect:
- one point (unique solution)
- no intersection (no solution)
- an intersection line/plane/subspace (infinitely many solutions)
Methods of solving (A\mathbf{x}=\mathbf{b}) #
Inverse method #
\[ \mathbf{x}=A^{-1}\mathbf{b} \]
Mainly for theory; numerically expensive and unstable for large systems.
Gaussian elimination (recommended) #
Solve by row-reducing the augmented matrix:
\[ [A\mid \mathbf{b}] \]
This leads to REF/RREF and reveals whether the system is:
- inconsistent
- uniquely solvable
- underdetermined (free variables)
Cramer’s rule #
Uses determinants to solve each variable (only for square systems with (\det(A)\neq 0)).
Useful for small (n), but not used for large systems.
LU decomposition #
Factor:
\[ A=LU \]
where:
- (L) is lower triangular
- (U) is upper triangular
Solve using forward + backward substitution.
Consistency of Linear Systems #
Consider a system of linear equations written in matrix form:
\[ A\mathbf{x} = \mathbf{b} \]- (A) → Coefficient matrix
- ([A | b]) → Augmented matrix
Notes #
- Consistent vs inconsistent systems
- Unique vs infinitely many solutions
- Geometric interpretation (lines/planes)