Machine Learning on Arshad Siddiqui

Supervised Learning

Sat, 03 Jan 2026 10:29:52 +0100

Supervised Learning #

Trained using labelled data.
Each example in the training set includes the correct output.
The algorithm learns to generalise and make predictions on unseen data.
Generally more accurate than unsupervised methods.
Requires human intervention for labelling and setup.
Widely used due to its accuracy and efficiency.
Produces highly accurate results when trained on good-quality labelled data.

Classification #

Output is discrete (e.g. Yes/No, Spam/Not Spam).
Used for categorising data into predefined classes.
Support Vector Machine (SVM) is a common classifier (a linear classifier with margin-based separation).

Semi-Supervised Learning

Sat, 03 Jan 2026 10:29:52 +0100

Semi-Supervised Learning #

A combination of labelled and unlabelled data.
Useful when labelling large datasets is expensive or time-consuming.
Works well with high-volume datasets (e.g. millions of images).
Only a small fraction of data is labelled (e.g. a few thousand).
The algorithm learns from both labelled examples and structure in unlabelled data.
Ideal for medical imaging where labelled data is limited.
For example, a radiologist can label a small set of medical scans,
and the model uses that to learn from thousands of unlabelled scans.
Helps improve accuracy and generalisation with minimal manual effort.

Home | Machine Learning

Reinforcement Learning

Mon, 01 Jan 0001 00:00:00 +0000

Reinforcement Learning (RL) #

RL is learning by trial and error.

Reinforcement Learning (RL) is a type of machine learning where an autonomous agent learns to make decisions by interacting with an environment.

Instead of being told the correct answer, the agent:

takes actions
observes outcomes
receives rewards or penalties
gradually learns a strategy that maximises long-term reward

Reinforcement Learning teaches an agent how to act, not what to predict.

ML Workflow

Mon, 01 Jan 0001 00:00:00 +0000

Machine learning Workflow #

Data is the foundation of any machine learning system. Quality of data matters more than model complexity.

Role of Data #

Data determines:

What patterns the model can learn
How well it generalises
Whether bias or noise is introduced

Bad data → bad model (even with perfect algorithms).

Data Preprocessing, wrangling #

Raw data is never ready for training.

Data Issues

Noise
- For objects, noise is an extraneous object
- For attributes, noise refers to modification of original values
- Use Log or Z Transfer to convert to mean
Outliers
- Data objects with characteristics that are considerably different than most of the other data objects in the data set
- Handle: Use IQR method
- Find Lower and Upper Bound and replace Outlier with Lower or Upper Bound
Missing Values
- Eliminate data objects or variables
- Handle: Estimate missing values
  - Mean, Median or Mode
  - Prefer Median if there are missing outliers
- Ignore the missing value during analysis
Duplicate Data
- Major issue when merging data from heterogeneous sources
Inconsistent Codes
- Find all Unique and transfer all inconsistent to

Data Preprocessing techniques

Regression (Linear)

Mon, 01 Jan 0001 00:00:00 +0000

Linear Regression #

Linear Regression is a supervised ML method used to predict a numerical target by fitting a model that is linear in its parameters.

In ML , linear models are a core baseline: they’re fast, often surprisingly strong, and usually easy to interpret.

Key takeaway: Linear Regression learns parameters by minimising a squared-error cost. You can solve it directly (closed form) or iteratively (gradient descent), and you can extend it using basis functions and regularisation.

Ordinary Least Squares

Sat, 21 Feb 2026 00:00:00 +0000

Direct solution method - Ordinary Least Squares and the Line of Best Fit #

Revision:
OLS is the direct method for linear regression. It finds the best-fit line by minimising the sum of squared residuals without iterative updates.

Direct Method vs Iterative Method ☆ #

Linear regression parameters can be found in two main ways.

Method	Main idea	When used
Ordinary Least Squares	Compute the best parameters directly	Small or moderate datasets
Gradient Descent	Start with parameters and update repeatedly	Large datasets or many features

flowchart LR
 A["Linear Regression"] --> B["Direct Solution<br/>OLS"]
 A --> C["Iterative Solution<br/>Gradient Descent"]

 B --> B1["Normal Equation"]
 B --> B2["No learning rate"]
 B --> B3["One-shot solution"]

 C --> C1["Learning rate"]
 C --> C2["Repeated updates"]
 C --> C3["Stops after convergence"]

 style A fill:#E1F5FE,stroke:#5b7db1,color:#000
 style B fill:#C8E6C9,stroke:#5f8f6a,color:#000
 style C fill:#FFF9C4,stroke:#b59b3b,color:#000
 style B1 fill:#EDE7F6,stroke:#8a6fb3,color:#000
 style B2 fill:#EDE7F6,stroke:#8a6fb3,color:#000
 style B3 fill:#EDE7F6,stroke:#8a6fb3,color:#000
 style C1 fill:#EDE7F6,stroke:#8a6fb3,color:#000
 style C2 fill:#EDE7F6,stroke:#8a6fb3,color:#000
 style C3 fill:#EDE7F6,stroke:#8a6fb3,color:#000

Why It Is Called “Least Squares” ☆ #

OLS is called least squares because it chooses parameters that make the squared residual errors as small as possible.

Cost Function

Sat, 21 Feb 2026 00:00:00 +0000

Cost Function #

Revision:
A cost function converts model error into a single number. Training means changing the model parameters until this number becomes as small as possible.

Why Cost Function Matters in ML ☆ #

A machine learning model needs a way to decide whether one set of parameters is better than another.

For linear regression, every possible value of the parameters gives a different line. The cost function tells us which line is better by measuring how far the predictions are from the true values.

Gradient Descent

Sat, 21 Feb 2026 00:00:00 +0000

Gradient Descent for Linear Regression #

Revision:
Gradient descent is the step-by-step method for reducing the cost function when a direct closed-form solution is not convenient.

Where Gradient Descent Fits in ML ☆ #

Gradient descent is used when we want the model to learn parameters by repeatedly improving them.

For linear regression, it adjusts the slope and intercept until the prediction error becomes small.

flowchart LR
 A["Initial Parameters"] --> B["Make Predictions"]
 B --> C["Compute Cost"]
 C --> D["Compute Gradient"]
 D --> E["Update Parameters"]
 E --> B

 style A fill:#E1F5FE,stroke:#5b7db1,color:#000
 style B fill:#C8E6C9,stroke:#5f8f6a,color:#000
 style C fill:#FFF9C4,stroke:#b59b3b,color:#000
 style D fill:#EDE7F6,stroke:#8a6fb3,color:#000
 style E fill:#C8E6C9,stroke:#5f8f6a,color:#000

Core Idea ☆ #

The gradient tells us the direction in which the cost increases fastest.

Classification (Linear)

Mon, 01 Jan 0001 00:00:00 +0000

Linear models for Classification #

categorises data by finding a linear boundary (hyperplane) that separates classes
calculating a weighted sum of input features plus bias

flowchart TD
T["Linear<br/>classification<br/>models"] --> P["Perceptron"]
T --> LR["Logistic<br/>regression"]
T --> SVM["Linear<br/>SVM"]

P -->|uses| STEP["Step<br/>activation"]
LR -->|uses| SIG["Sigmoid<br/>+ log loss"]
SVM -->|uses| HNG["Hinge<br/>loss"]

style T fill:#90CAF9,stroke:#1E88E5,color:#000

style P fill:#C8E6C9,stroke:#2E7D32,color:#000
style LR fill:#C8E6C9,stroke:#2E7D32,color:#000
style SVM fill:#C8E6C9,stroke:#2E7D32,color:#000

style STEP fill:#CE93D8,stroke:#8E24AA,color:#000
style SIG fill:#CE93D8,stroke:#8E24AA,color:#000
style HNG fill:#CE93D8,stroke:#8E24AA,color:#000

Discriminant Functions
Decision Theory
Probabilistic Discriminative Classifiers
Logistic Regression

Logistic Regression #

Supervised machine learning algorithm
Binary classification algorithm
requires data to be linearly separable
predicts the probability that an input belongs to a specific class
uses Sigmoid function to convert inputs into a probability value between 0 and 1

Key takeaway: Logistic regression predicts $P(y=1\mid x)$ using a sigmoid of a linear score $z=w\cdot x+b$, then learns $w,b$ by maximising likelihood (equivalently minimising log-loss).

Decision Tree

Mon, 01 Jan 0001 00:00:00 +0000

Decision Tree #

A decision tree classifies an example by asking a sequence of questions about its attributes until it reaches a leaf (final decision).

Key takeaway: A decision tree grows by repeatedly splitting the training data into purer subsets using an impurity measure (Entropy / Gini / Classification Error).

Information Theory
Entropy Based Decision Tree Construction
Avoiding Overfitting
Minimum Description Length
Handling Continuous valued attributes, missing attributes

Information Theory #

Decision trees need a way to measure: “How mixed are the class labels at a node?”

Instance-based Learning

Mon, 01 Jan 0001 00:00:00 +0000

Instance-based Learning #

Instance-based learning is a family of methods that do not build one explicit global model during training. Instead, they store training examples and delay most of the work until a new query arrives.

When a new point must be classified or predicted, the algorithm compares it with previously seen examples, finds the most relevant neighbours, and uses them to produce the answer.

Instance-based Learning covers three linked ideas:

Support Vector Machine

Fri, 08 May 2026 00:00:00 +0000

Support Vector Machine (SVM) #

Support Vector Machine (SVM) is a supervised machine learning algorithm used for:

Classification (most common)
Regression (SVR – Support Vector Regression)

It connects many earlier ideas:

classification and decision boundaries
linear classifiers
margins
optimisation
constrained optimisation
kernels for non-linear data

SVM is a discriminative classifier.

That means it does not try to model how each class is generated.

Instead, it tries to find the best separating boundary between classes.

Bayesian Learning

Mon, 01 Jan 0001 00:00:00 +0000

Bayesian Learning #

Bayesian Learning is a probabilistic approach to machine learning.

Instead of only asking, “Which output should the model predict?”, Bayesian Learning asks:

Given the data we have observed, how likely is each hypothesis, class, or parameter value?

This makes Bayesian Learning useful when uncertainty matters.

It is especially important in classification, probabilistic modelling, generative models, and situations where we want to combine prior knowledge with observed data.

Ensemble Learning

Mon, 01 Jan 0001 00:00:00 +0000

Ensemble Learning #

Ensemble Learning is a machine learning approach where we combine multiple models to produce a stronger final prediction.

Instead of depending on one model, an ensemble uses a group of models and combines their outputs.

The main idea is simple:

Many weak or moderately good models can work together to produce a better and more stable model.

Key takeaway:
Ensemble Learning improves prediction by combining several models.

Unsupervised Learning

Mon, 01 Jan 0001 00:00:00 +0000

Unsupervised Learning #

Unsupervised Learning is used when we have input data but no target labels.

The model is not told the correct answer. Instead, it tries to discover hidden structure in the data.

K-means Clustering and variants
Review of EM algorithm
GMM based Soft Clustering
Applications

Supervised vs Unsupervised Learning #

Aspect	Supervised Learning	Unsupervised Learning
Data contains target label?	Yes	No
Learns from	Input-output pairs	Input features only
Main goal	Predict output	Discover structure
Example task	Classification, regression	Clustering
Example algorithm	Logistic regression, decision tree	K-means, GMM

Works on unlabelled raw data.
The algorithm discovers hidden patterns without prior knowledge of outcomes.
Requires no human intervention during training.
Does not make direct predictions — it groups or organises data instead.
Carries a higher risk because there’s no ground truth to verify results.
Common techniques include Clustering, Association, and Dimensionality Reduction.

The most common example is clustering, where similar records are grouped together.

Evaluation/Comparison

Mon, 01 Jan 0001 00:00:00 +0000

Machine Learning Model Evaluation/Comparison #

Comparing Machine Learning Models #

Emerging requirements e.g., bias, fairness, interpretability of ML models #

Home | Machine Learning

ML Pipeline

Tue, 21 Apr 2026 00:00:00 +0000

Machine Learning Pipeline: Preprocessing & Models #

This page explains both data preprocessing and model development concepts in a clear, structured way to support understanding.

A complete ML pipeline includes preprocessing, feature engineering, feature selection, and model training.

1. Data Preprocessing Overview #

Raw data is often:

Noisy
Incomplete
Inconsistent

Preprocessing ensures data is suitable for machine learning.

2. Missing Values #

Why they occur

Sensor errors
Data collection issues

Methods