Hypothesis Testing
#
Hypothesis testing is a statistical decision-making method used to decide whether sample evidence is strong enough to reject an initial assumption about a population.
It connects probability, sampling distributions, confidence intervals, significance levels, and decision rules.
Key takeaway:
Hypothesis testing is not about proving something with certainty.
It is about asking:
If the null hypothesis were true, how surprising would this sample result be?
December 14, 2025Foundation Model
#
AI models trained on massive datasets to perform a wide range of tasks with minimal fine-tuning.
are large deep learning neural networks
are large AI models trained on massive and diverse datasets (text, images, audio, or multiple modalities).
Contain millions or billions of parameters.
designed to perform a broad range of general tasks
designed for general-purpose intelligence, not a single task.
acts as base models for building specialised AI applications
LLM – Large Language Model
#
Large Language Models (LLMs) are advanced AI systems designed to process, understand, and generate human-like text.
They learn language by analysing massive amounts of text data, discovering patterns in:
grammar
meaning
context
relationships between words and sentences
Built on Deep Learning
Implemented using Neural Networks
Based on Transformers
Often combined with tools like:
- Retrieval (RAG)
- Agents
- External APIs
- Memory systems
What makes an LLM special?
#
- Built using deep neural networks
- Trained on very large datasets (books, articles, code, web text)
- Can perform many tasks without task-specific training
- General-purpose language understanding, not single-task models
LLMs are based on the Transformer Architecture, which allows models to understand context and long-range dependencies in text.
December 15, 2025AI Agents
#
Also referred to as Agentic AI.
AI agents are intelligent systems that can plan, make decisions, and take actions to achieve goals with minimal human intervention.
A common use case is task automation
for example booking travel based on a user’s request.
AI agents typically build on Generative AI and use Large Language Models (LLMs) as the reasoning core.
Agents often interact with tools (APIs, databases, calendars) to complete multi-step workflows.
Retrieval-Augmented Generation (RAG)
#
Retrieval-Augmented Generation (RAG) is a system design pattern that improves an LLM’s answers by:
- Retrieving relevant information from an external knowledge source, and then
- Augmenting the LLM prompt with that retrieved context before generating the final response.
RAG helps an LLM look things up first, then answer using evidence.
Why RAG is Useful
#
RAG is commonly used when:
- Your knowledge is in private documents (PDFs, policies, internal wiki)
- You need up-to-date information (things not in the model’s training data)
- You want fewer hallucinations by grounding answers in retrieved sources
- You want traceability (show “where the answer came from”)
RAG does not change the model weights.
It changes what the model sees at inference time by adding retrieved context.
Decision Tree
#
A decision tree classifies an example by asking a sequence of questions about its attributes until it reaches a leaf (final decision).
Key takeaway:
A decision tree grows by repeatedly splitting the training data into purer subsets using an impurity measure
(Entropy / Gini / Classification Error).
- Information Theory
- Entropy Based Decision Tree Construction
- Avoiding Overfitting
- Minimum Description Length
- Handling Continuous valued attributes, missing attributes
Decision trees need a way to measure:
“How mixed are the class labels at a node?”
Prediction & Forecasting
#
Prediction and forecasting use statistical models to estimate unknown or future values.
In this module, the focus is on correlation, regression, and time series forecasting.
Key takeaway:
Prediction estimates a value using a model.
Forecasting is prediction where the order of time matters.
- Correlation
- Regression
- Time series analysis
- Components of time series data
- Moving average and weighted moving average
- AR model
- ARMA model
- ARIMA model
- SARIMA and SARIMAX
- VAR and VARMAX
- Simple exponential smoothing
Prediction vs Forecasting ☆
#
| Concept | Meaning | Example |
|---|
| Prediction | Estimate an unknown output | Predict house price from area and rooms |
| Forecasting | Predict future values using time order | Forecast sales for next month |
All forecasting is prediction, but not all prediction is forecasting.
Overall Workflow
#
flowchart LR
A[Data] --> B[Explore Pattern]
B --> C[Choose Model]
C --> D[Train or Fit]
D --> E[Validate]
E --> F[Predict or Forecast]
F --> G[Interpret Error]
style A fill:#E1F5FE
style B fill:#C8E6C9
style C fill:#FFF9C4
style D fill:#EDE7F6
style E fill:#C8E6C9
style F fill:#E1F5FE
style G fill:#FFF9C4
Correlation ☆
#
Correlation measures the direction and strength of linear relationship between two variables.
Gaussian Mixture Model & Expectation Maximization
#
A Gaussian Mixture Model represents data as a weighted combination of multiple Gaussian distributions.
It is commonly used for soft clustering and density estimation.
Key takeaway:
K-means gives hard cluster membership.
GMM gives probabilities of belonging to each cluster.
- Gaussian Mixture Model
- soft clustering
- mixing coefficients
- latent variables
- likelihood and log-likelihood
- Expectation-Maximization algorithm
- E-step and M-step
- responsibilities
- convergence
Motivation ☆
#
Many real datasets are not described well by one Gaussian distribution.
Instance-based Learning
#
Instance-based learning is a family of methods that do not build one explicit global model during training. Instead, they store training examples and delay most of the work until a new query arrives.
When a new point must be classified or predicted, the algorithm compares it with previously seen examples, finds the most relevant neighbours, and uses them to produce the answer.
Instance-based Learning covers three linked ideas:
May 8, 2026Support Vector Machine (SVM)
#
Support Vector Machine (SVM) is a supervised machine learning algorithm used for:
- Classification (most common)
- Regression (SVR – Support Vector Regression)
It connects many earlier ideas:
- classification and decision boundaries
- linear classifiers
- margins
- optimisation
- constrained optimisation
- kernels for non-linear data
SVM is a discriminative classifier.
That means it does not try to model how each class is generated.
Instead, it tries to find the best separating boundary between classes.