Matrices #
Matrices are the core data structure of linear algebra and the workhorse of machine learning.
Almost every ML model can be described as a sequence of matrix operations.
Matrix #
A matrix is a rectangular array of numbers arranged in rows and columns.
\[ A \in \mathbb{R}^{m \times n} \]
An ( m \times n ) matrix has:
- ( m ) rows
- ( n ) columns
Example of a matrix:
\[ A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \]
Vectors are just special cases of matrices (with one row or one column).
Equality of Matrices #
Two matrices are equal if:
- They have the same dimensions
- Every corresponding entry is equal
If either condition fails, the matrices are not equal.
Matrix Operations #
Matrix addition #
Matrices can be added only if they have the same dimensions.
\[ A \in \mathbb{R}^{m \times n}, \quad B \in \mathbb{R}^{m \times n} \]
Addition is performed element-wise.
Properties #
Commutative
\[ A + B = B + A \]
Associative
\[ (A + B) + C = A + (B + C) \]
Scalar multiplication #
Multiplying a matrix by a scalar multiplies every entry of the matrix.
Properties #
\[ c(A + B) = cA + cB \]
\[ (c + k)A = cA + kA \]
\[ (ck)A = c(kA) \]
\[ 1A = A \]
Matrix multiplication (very important) #
Matrix multiplication combines rows of the first matrix with columns of the second.
If:
- ( A \in \mathbb{R}^{m \times n} )
- ( B \in \mathbb{R}^{n \times p} )
then:
\[ AB \in \mathbb{R}^{m \times p} \]
Matrix multiplication represents composition of linear transformations.
Matrix multiplication is not commutative in general.
\[ AB \neq BA \]
Even if ( AB = 0 ), it does not imply
( A = 0 ) or ( B = 0 ).
Vector #
A vector is a matrix with exactly one row or one column.
\[ \text{Row vector: } 1 \times n \qquad \text{Column vector: } m \times 1 \]
In machine learning #
Vectors are used to represent:
- Data points
- Feature sets
- Model parameters
Machine learning models transform input vectors into output vectors using matrices.
Scalar #
A scalar is a single numerical value representing magnitude only.
\[ \alpha \in \mathbb{R} \]
Scalars do not have direction — only size.
Geometric intuition #
A scalar is like a number on a number line.
It tells you how much, not which way.
In machine learning #
Scalars commonly represent:
- Learning rates
- Bias terms
- Loss values