My AI Notes #

Learning how machines learn! My working notes as I learn AI.

flowchart LR
    AI[Artificial Intelligence]
    ML[Machine Learning]
    DL[Deep Learning]
    FM[Foundation Models]
    LLM[LLM Models]

    AI --> ML
    ML --> DL
    DL --> FM
    FM --> LLM

    style AI fill:#E1F5FE
    style ML fill:#C8E6C9
    style DL fill:#90CAF9
    style FM fill:#64B5F6
    style LLM fill:#FFCCBC

• Mathematical Foundations for Machine Learning
• Statistical Methods
• Machine Learning
• Deep Neural Networks

• AI Foundation
  • AI Stages: ANI, AGI, ASI
  • AI Stack
  • AI Pipeline
  • AI Learning Resources
• Machine Learning
  • Supervised Learning
  • Unsupervised Learning
  • Semi-Supervised Learning
  • Reinforcement Learning
  • ML Workflow
  • Regression(Linear Models)
  • Ordinary Least Squares
  • Cost Function
  • Gradient Descent
  • Classification(Linear Models)
  • Decision Tree
  • Instance-based Learning
  • Support Vector Machine
  • Bayesian Learning
  • Ensemble Learning
  • Evaluation/Comparison
• Generative AI
  • Foundation Models
  • LLM - Model
  • AI Agents
  • Retrieval-Augmented Generation (RAG)
• Deep Learning
  • Neural Networks
  • Artificial Neuron and Perceptron
  • LNN for Regression
  • Gradient Descent Algorithm
  • LNN for Classification
  • Deep Feedforward Neural Networks (DFNN) for Classification
  • Convolutional Neural Networks
  • Recurrent Neural Networks
  • Deep Recurrent Neural Networks
  • Attention Mechanism
  • Transformer
  • Optimisation of Deep models
  • Regularisation for Deep models
• Mathematical Foundation
  • Linear Algebra
    • Linear Systems
      • Systems of Linear Equations
      • Matrices
      • Matrix Transposition
      • Solving Linear Systems
      • Forward and Backward Substitution
      • Inverse Matrix
      • Convex Combination
    • Vector Spaces
      • Basis and Rank
      • Linear Independence
      • Norm
      • Inner Products and Dot Product
      • Lengths and Distances
      • Angles and Orthogonality
      • Orthonormal Basis
      • Feature Space
      • Cauchy–Schwarz
    • Matrix Decompositions
      • Special Matrices
      • Characteristic Polynomial
      • Determinant and Trace
      • Eigenvalues and Eigenvectors
      • Diagonalization
      • Cholesky Decomposition
      • Singular Value Decomposition (SVD)
      • Eigen Decomposition
      • Matrix Approximation
      • Eigenvalues and Eigenvectors II
    • Dimensionality reduction and PCA
      • Principal Component Analysis (PCA)
      • PCA Theory
      • PCA in Practice
      • Latent Variable Perspective
      • Mathematical Preliminaries of SVM
      • Nonlinear SVM and Kernels
  • Calculus
    • Vector Calculus
      • Differentiation of Univariate Functions
      • Partial Differentiation and Gradients
      • Gradients of Vector-Valued and Matrix Functions
      • Useful Gradient Identities
      • Backpropagation and Automatic Differentiation
      • Higher-order derivatives
      • Taylor’s series
      • Maxima and Minima
    • Continuous Optimisation
      • Optimisation using Gradient Descent
      • Constrained Optimisation
      • Lagrange Multipliers
      • Convex Optimisation
    • Nonlinear Optimisation
      • Challenges in Gradient-Based Optimisation
      • Stochastic Gradient Descent (SGD)
      • Momentum-Based Learning
      • Adaptive Methods: AdaGrad, RMSProp, Adam
      • Tuning Hyperparameters and Preprocessing
• Statistics
  • Formula Sheet
  • Stats Formula Sheet
  • Basic Statistics
  • Basic Probability
  • Hypothesis Testing
  • Prediction & Forecasting
  • Gaussian Mixture model & Expectation Maximization
  • Conditional Probability & Bayes’ Theorem
    • Conditional Probability
    • Bayes’ Theorem
    • Naïve Bayes
  • Probability Distributions
    • Random Variables
    • Common Probability Distributions

• Machine Learning → The broad field where systems learn patterns from data to make predictions or decisions.
• Neural Networks → A subset of machine learning that uses interconnected artificial neurons to model complex relationships.
• Deep Learning → A subset of neural networks that uses many hidden layers to learn high-level features from large datasets.
• Foundation Models → Large deep learning models trained on massive datasets and reused across many tasks using transfer learning.
• LLMs (Large Language Models) → A specialised type of foundation model focused on understanding and generating human language.

flowchart TD
AI["Artificial<br/>Intelligence"]
ML["Machine<br/>Learning"]
NN["Neural<br/>Networks"]
DL["Deep<br/>Learning"]
FM["Foundation<br/>Models"]
LLM["LLM<br/>Models"]

AI --> ML
ML --> NN
NN --> DL
DL --> FM
FM --> LLM

LR["Linear<br/>Regression"]
DT["Decision<br/>Trees"]
ML --> LR
ML --> DT

MLP["MLP"]
CNN["CNN"]
NN --> MLP
NN --> CNN

CNNDL["CNN<br/>(deep)"]
RNN["RNN"]
DL --> CNNDL
DL --> RNN

BERT["BERT"]
CLIP["CLIP"]
FM --> BERT
FM --> CLIP

GPT["GPT"]
LLAMA["LLaMA"]
LLM --> GPT
LLM --> LLAMA

TEXT["Text"]
IMAGE["Images"]
AUDIO["Audio"]
VIDEO["Video"]
LLM --> TEXT
LLM --> IMAGE
LLM --> AUDIO
LLM --> VIDEO

style AI fill:#90CAF9,stroke:#1E88E5,color:#000
style ML fill:#90CAF9,stroke:#1E88E5,color:#000
style NN fill:#90CAF9,stroke:#1E88E5,color:#000

style DL fill:#CE93D8,stroke:#8E24AA,color:#000
style FM fill:#CE93D8,stroke:#8E24AA,color:#000

style LLM fill:#C8E6C9,stroke:#2E7D32,color:#000
style LR fill:#C8E6C9,stroke:#2E7D32,color:#000
style DT fill:#C8E6C9,stroke:#2E7D32,color:#000
style MLP fill:#C8E6C9,stroke:#2E7D32,color:#000
style CNN fill:#C8E6C9,stroke:#2E7D32,color:#000
style CNNDL fill:#C8E6C9,stroke:#2E7D32,color:#000
style RNN fill:#C8E6C9,stroke:#2E7D32,color:#000
style BERT fill:#C8E6C9,stroke:#2E7D32,color:#000
style CLIP fill:#C8E6C9,stroke:#2E7D32,color:#000
style GPT fill:#C8E6C9,stroke:#2E7D32,color:#000
style LLAMA fill:#C8E6C9,stroke:#2E7D32,color:#000
style TEXT fill:#C8E6C9,stroke:#2E7D32,color:#000
style IMAGE fill:#C8E6C9,stroke:#2E7D32,color:#000
style AUDIO fill:#C8E6C9,stroke:#2E7D32,color:#000
style VIDEO fill:#C8E6C9,stroke:#2E7D32,color:#000