AI

Convolutional Neural Networks

April 19, 2026
AI, Deep Learning
Deep Learning, Cnn, Convolutional Neural Networks, Computer Vision, Feature Maps, Receptive Field, Padding, Pooling

Convolutional Neural Networks (CNN) #

Convolutional Neural Networks (CNNs) are specialised neural networks designed for data with spatial structure, especially images. They became the standard model for computer vision because they preserve spatial locality, reuse the same pattern detector across the image, and build representations hierarchically. In practical terms, a CNN starts by learning simple features such as edges and corners, then combines them into textures, shapes, object parts, and finally full semantic categories.

Gaussian Mixture Model & Expectation Maximization

AI, ML, Stats
AI, ML, Stats, GMM, Expectation Maximization, Clustering

Gaussian Mixture Model & Expectation Maximization #

A Gaussian Mixture Model represents data as a weighted combination of multiple Gaussian distributions.

It is commonly used for soft clustering and density estimation.

Key takeaway:
K-means gives hard cluster membership.

GMM gives probabilities of belonging to each cluster.

  • Gaussian Mixture Model
  • soft clustering
  • mixing coefficients
  • latent variables
  • likelihood and log-likelihood
  • Expectation-Maximization algorithm
  • E-step and M-step
  • responsibilities
  • convergence

Motivation ☆ #

Many real datasets are not described well by one Gaussian distribution.

Instance-based Learning

AI, ML
AI, ML

Instance-based Learning #

Instance-based learning is a family of methods that do not build one explicit global model during training. Instead, they store training examples and delay most of the work until a new query arrives.

When a new point must be classified or predicted, the algorithm compares it with previously seen examples, finds the most relevant neighbours, and uses them to produce the answer.

Instance-based Learning covers three linked ideas:

Part-of-Speech Tagging and Hidden Markov Models

AI, ML
Natural Language Processing, NLP, Part-of-Speech Tagging, POS, Markov Chains, Hidden Markov Models, HMM

Part-of-Speech Tagging and Hidden Markov Models #

Part-of-Speech tagging assigns a grammatical category to each word in a sequence. Because many words can play different grammatical roles, a tagger must use surrounding context rather than examine each word independently.

Learning Objectives #

  • Identify common English word classes and Penn Treebank tags.
  • Explain why POS tagging is a sequence-labelling problem.
  • Describe the Markov assumption.
  • Distinguish a Markov Chain from a Hidden Markov Model.
  • Explain how an HMM represents POS tagging.

Big Picture #

flowchart TD
    A["Word Sequence"] --> B["Use Context"]
    B --> C["Infer Hidden Tags"]
    C --> D["Tagged Sequence"]

    style A fill:#E1F5FE
    style B fill:#C8E6C9
    style C fill:#FFF9C4
    style D fill:#EDE7F6

1. What Is Part-of-Speech Tagging? ☆ #

A part of speech describes the grammatical role played by a word in a sentence. POS tagging assigns one tag to every word in a sequence.

Deep CNN Architectures

April 19, 2026
AI, Deep Learning
Deep Learning, Cnn, Deep CNN Architectures, LeNet, AlexNet, VGG, NiN, GoogLeNet, ResNet, Transfer Learning

Deep CNN Architectures #

Once the basic ideas of convolution, pooling, channels, and classifier heads are understood, the next step is to study how successful CNN architectures are designed in practice. The history of deep CNNs is not just a list of famous models. It is a progression of design ideas: smaller filters, more depth, better optimisation, bottlenecks, multi-scale processing, residual connections, and transfer learning.

Key takeaway:
Deep CNN architectures evolved by solving specific problems one by one: LeNet established the template, AlexNet proved deep learning could dominate large-scale vision, VGG simplified the design, NiN introduced powerful 1 × 1 ideas, GoogLeNet made multi-scale processing efficient, and ResNet solved the optimisation problem of very deep networks.

CNN Pipeline

April 22, 2026
AI, Deep Learning
Deep Learning, Cnn, Keras

CNN Pipeline: Preprocessing & Models #

  • Understand CNN concepts deeply
  • Build CNN models step-by-step
  • Apply CNNs in assignments using Keras

Think of CNN as a pipeline: Image → Features → Patterns → Prediction

1. Image Representation #

\[ X \in \mathbb{R}^{H \times W \times C} \]
  • H = Height
  • W = Width
  • C = Channels

2. Convolution Operation #

\[ Z(i,j) = \sum_{m,n} X(i+m, j+n) \cdot K(m,n) \]
  • Sliding filter extracts features
  • Produces feature maps

3. Stride & Padding #

\[ Output = \frac{N - F + 2P}{S} + 1 \]

4. Activation (ReLU) #

\[ ReLU(x) = max(0, x) \]

5. Pooling #

  • Max Pooling → strongest feature
  • Average Pooling → smooth

6. Global Average Pooling #

\[ y_k = \frac{1}{HW} \sum_{i,j} x_{i,j,k} \]

7. Loss Function #

\[ L = - \sum y \log(\hat{y}) \]

8. CNN Architecture #

graph LR
A[Input Image] --> B[Conv]
B --> C[ReLU]
C --> D[Pooling]
D --> E[Conv Layers]
E --> F[Flatten / GAP]
F --> G[Dense]
G --> H[Output]

9. Training #

  • Forward pass
  • Loss computation
  • Backpropagation
  • Weight update

10. Keras Implementation #

Model #

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten

model = Sequential()

model.add(Conv2D(32, (3,3), activation='relu', input_shape=(64,64,3)))
model.add(MaxPooling2D((2,2)))

model.add(Conv2D(64, (3,3), activation='relu'))
model.add(MaxPooling2D((2,2)))

model.add(Flatten())

model.add(Dense(128, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

Compile #

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

Train #

model.fit(X_train, y_train, epochs=10, batch_size=32)

Predict #

pred = model.predict(X_test)

11. Tips #

  • Normalize images
  • Use small filters
  • Avoid too many dense layers

12. Summary #

CNN = Automatic feature extractor + classifier

Support Vector Machine

May 8, 2026
AI, ML
AI, ML, Support Vector Machine, SVM, Classification, Kernel Trick, Maximum Margin

Support Vector Machine (SVM) #

Support Vector Machine (SVM) is a supervised machine learning algorithm used for:

  • Classification (most common)
  • Regression (SVR – Support Vector Regression)

It connects many earlier ideas:

  • classification and decision boundaries
  • linear classifiers
  • margins
  • optimisation
  • constrained optimisation
  • kernels for non-linear data

SVM is a discriminative classifier.

That means it does not try to model how each class is generated.

Instead, it tries to find the best separating boundary between classes.

Recurrent Neural Networks

April 19, 2026
AI, Deep Learning
Deep Learning, RNN, Recurrent Neural Networks, Sequence Modelling, BPTT, Encoder Decoder, Teacher Forcing, Time Series

Recurrent Neural Networks #

Recurrent Neural Networks (RNNs) are neural networks designed for sequential data, where the order of inputs matters and the model must use information from earlier time steps to interpret later ones. Unlike a feedforward network, an RNN does not process each input in isolation. It carries a hidden state from one time step to the next, so the network can build a running summary of what it has seen so far.

Statistical, ML and Neural Models of POS Tagging

AI, ML
Natural Language Processing, NLP, Hidden Markov Models, Forward Algorithm, Viterbi Algorithm, POS Tagging

Statistical, ML and Neural Models of POS Tagging #

‘HMM Inference: Forward and Viterbi Algorithms’ covers the portion:

  • Forward Algorithm
  • Viterbi Algorithm
  • HMM inference for POS tagging

The complete Topic also includes:

  • Maximum Entropy Markov Models
  • Bidirectionality
  • Neural-network models for POS tagging

HMM Inference: Forward and Viterbi Algorithms #

Hidden Markov Models create two closely related inference problems:

  • Likelihood: How probable is an observed sequence under the model?
  • Decoding: Which hidden-state sequence most probably generated the observations?

The Forward Algorithm solves the likelihood problem, while the Viterbi Algorithm solves the decoding problem. Both use dynamic programming and a trellis, but they combine paths differently.

Deep Recurrent Neural Networks

April 19, 2026
AI, Deep Learning
Deep Learning, Deep RNN, LSTM, GRU, Bidirectional RNN, Stacked RNN, Sequence Modelling

Deep Recurrent Neural Networks #

Vanilla RNNs introduce the hidden-state idea, but they struggle on longer and more complex sequences because gradients can vanish across time. Deep recurrent models extend the RNN idea in two important ways:

  1. make the recurrent architecture richer, for example by stacking multiple recurrent layers or using information from both directions,
  2. use gates and memory cells to control what should be remembered, forgotten, updated, and exposed.

This is why practical recurrent modelling usually moves from a simple RNN to stacked RNNs, bidirectional RNNs, GRUs, or LSTMs.