Mathematical Foundation

May 28, 2026

Mathematical Foundations for Machine Learning #

Machine Learning is built on mathematical principles that allow models to:

• represent data
• learn patterns
• optimise performance

flowchart LR
    DATA[Data]
    MATH[Math Models]
    OPT[Optimisation]
    MODEL[Trained Model]

    DATA --> MATH
    MATH --> OPT
    OPT --> MODEL

ML requires core mathematical tools to understand how ML algorithms work internally. Algebra deals with relationships between variables and quantities, while Calculus focuses on change and optimization.

Calculus

Calculus #

Calculus is:

• the mathematical framework for understanding and controlling how quantities change
• the mathematics of change and accumulation

It helps answer:

• How fast is something changing right now?
• What happens when inputs change slightly?
• Where is something maximum or minimum?

It answers two big questions:

• How fast is something changing right now? → derivatives (differentiation)
• How much has accumulated over an interval? → integrals (integration)

flowchart TD
  A[Calculus] --> B[Limits]
  B --> C[Continuity]
  B --> D[Derivatives]
  B --> E[Integrals]
  D --> F[Optimisation: maxima/minima]
  D --> G[ML: gradients & learning]
  E --> H[Accumulation: area/total change]

• Vector Calculus
  • Differentiation of Univariate Functions
  • Partial Differentiation and Gradients
  • Gradients of Vector-Valued and Matrix Functions
  • Useful Gradient Identities
  • Backpropagation and Automatic Differentiation
  • Higher-order derivatives
  • Taylor’s series
  • Maxima and Minima
• Continuous Optimisation
  • Optimisation using Gradient Descent
  • Constrained Optimisation
  • Lagrange Multipliers
  • Convex Optimisation
• Nonlinear Optimisation
  • Challenges in Gradient-Based Optimisation
  • Stochastic Gradient Descent (SGD)
  • Momentum-Based Learning
  • Adaptive Methods: AdaGrad, RMSProp, Adam
  • Tuning Hyperparameters and Preprocessing