Gradients of Vector-Valued and Matrix Functions
#
Covers gradients when outputs or parameters are vectors/matrices.
If f: R^n -> R^m, the derivative is the Jacobian.
[
J =
\begin{bmatrix}
\frac{\partial f_1}{\partial x_1} & \dots & \frac{\partial f_1}{\partial x_n} \
\vdots & \ddots & \vdots \
\frac{\partial f_m}{\partial x_1} & \dots & \frac{\partial f_m}{\partial x_n}
\end{bmatrix}
]For scalar f(x):
[
H = \nabla^2 f
]Hessian captures curvature.
December 15, 2025Generative AI
#
Generative Artificial Intelligence (GenAI) refers to a class of AI systems that can generate new content such as text, images, audio, video, or code, rather than only making predictions or classifications.
GenAI systems learn patterns and representations from large datasets and use them to produce novel outputs that resemble the data they were trained on.
How Generative AI Differs from Traditional AI
#
| Traditional AI | Generative AI |
|---|
| Predicts or classifies | Generates new content |
| Task-specific models | General-purpose models |
| Fixed outputs | Open-ended outputs |
| Often rule-based | Data-driven and probabilistic |
Core Idea of Generative AI
#
Instead of learning “what label to assign”, Generative AI learns “how data is structured” and then creates new data following that structure.
Reinforcement Learning (RL)
#
RL is learning by trial and error.
Reinforcement Learning (RL) is a type of machine learning where an autonomous agent learns to make decisions by interacting with an environment.
Instead of being told the correct answer, the agent:
- takes actions
- observes outcomes
- receives rewards or penalties
- gradually learns a strategy that maximises long-term reward
Reinforcement Learning teaches an agent how to act, not what to predict.
Useful Gradient Identities
#
[
\nabla (a^T x) = a
]
[
\nabla (x^T A x) = (A + A^T)x
]If A symmetric:
[
\nabla (x^T A x) = 2Ax
]These are heavily used in optimisation.
Home | Vector Calculus
Deep Reinforcement Learning
#
Deep Reinforcement Learning (DRL) studies how an agent learns to make a sequence of decisions by interacting with an environment and receiving feedback through rewards.
Reinforcement learning provides the framework for sequential decision-making. Deep learning extends this framework with powerful function approximators that can handle large or complex state and action spaces.
Deep Reinforcement Learning = Reinforcement Learning + Deep Neural Networks
Classical reinforcement learning foundations → progresses towards → value-based deep learning → policy-gradient methods → model-based approaches → imitation learning.
Inner Products and Dot Product
#
An inner product maps two vectors to a single scalar.
It allows us to measure:
- similarity
- vector length
- projections
- orthogonality
flowchart TD
T["Inner<br/>products<br/>(types)"] --> DOT["Euclidean<br/>Dot product"]
T --> WIP["Weighted<br/>inner product"]
T --> FN["Function-space<br/>(integral)"]
T --> HERM["Complex<br/>Hermitian"]
T --> MAT["Matrix<br/>inner product<br/>(Frobenius)"]
DOT --> Rn["Vectors in<br/>
<span>
\( \mathbb{R}^n \)
</span>
"]
WIP --> SPD["SPD matrix<br/>W"]
FN --> L2["L2 space<br/>functions"]
HERM --> Cn["Vectors in<br/>C^n"]
MAT --> Mnm["Matrices<br/>R^{m×n}"]
style T fill:#90CAF9,stroke:#1E88E5,color:#000
style DOT fill:#C8E6C9,stroke:#2E7D32,color:#000
style WIP fill:#C8E6C9,stroke:#2E7D32,color:#000
style FN fill:#C8E6C9,stroke:#2E7D32,color:#000
style HERM fill:#C8E6C9,stroke:#2E7D32,color:#000
style MAT fill:#C8E6C9,stroke:#2E7D32,color:#000
style Rn fill:#CE93D8,stroke:#8E24AA,color:#000
style SPD fill:#CE93D8,stroke:#8E24AA,color:#000
style L2 fill:#CE93D8,stroke:#8E24AA,color:#000
style Cn fill:#CE93D8,stroke:#8E24AA,color:#000
style Mnm fill:#CE93D8,stroke:#8E24AA,color:#000
Definition
#
For vectors
\( \mathbf{a}, \mathbf{b} \in \mathbb{R}^n \)
Natural Language Processing
#
Natural Language Processing (NLP) studies how computers can analyse, understand, represent, and generate human language.
It combines ideas from linguistics, computer science, machine learning, and deep learning to work with text and language-based information.
Natural Language Processing = Linguistics + Computation + Machine Learning
The learning path begins with language understanding and vector representations, progresses through language modelling, tagging, and parsing, and then moves towards transformers, knowledge graphs, Retrieval-Augmented Generation, and modern NLP applications.
Backpropagation and Automatic Differentiation
#
Backpropagation applies the chain rule:
- efficiently across a computational graph.
- repeatedly.
Chain rule:
[
\frac{dL}{dx} = \frac{dL}{dy} \cdot \frac{dy}{dx}
]
flowchart LR
x --> y
y --> L
Automatic differentiation computes exact derivatives efficiently using computational graphs.
Home | Vector Calculus