ML

Useful Gradient Identities #

[ \nabla (a^T x) = a ] [ \nabla (x^T A x) = (A + A^T)x ]

If A symmetric:

[ \nabla (x^T A x) = 2Ax ]

These are heavily used in optimisation.

Home | Vector Calculus

Deep Reinforcement Learning

AI, ML

Deep Reinforcement Learning, Reinforcement Learning, AI, ML

Deep Reinforcement Learning #

Deep Reinforcement Learning (DRL) studies how an agent learns to make a sequence of decisions by interacting with an environment and receiving feedback through rewards.

Reinforcement learning provides the framework for sequential decision-making. Deep learning extends this framework with powerful function approximators that can handle large or complex state and action spaces.

Deep Reinforcement Learning = Reinforcement Learning + Deep Neural Networks

The learning path begins with classical reinforcement learning foundations and progresses towards value-based deep learning, policy-gradient methods, model-based approaches, and imitation learning.

Inner Products and Dot Product

AI, ML

Machine Learning, Mathematics, Linear Algebra

Inner Products and Dot Product #

An inner product maps two vectors to a single scalar.

It allows us to measure:

similarity
vector length
projections
orthogonality

flowchart TD
T["Inner<br/>products<br/>(types)"] --> DOT["Euclidean<br/>Dot product"]
T --> WIP["Weighted<br/>inner product"]
T --> FN["Function-space<br/>(integral)"]
T --> HERM["Complex<br/>Hermitian"]
T --> MAT["Matrix<br/>inner product<br/>(Frobenius)"]

DOT --> Rn["Vectors in<br/>
<span>
  \( \mathbb{R}^n \)
  </span>

"]
WIP --> SPD["SPD matrix<br/>W"]
FN --> L2["L2 space<br/>functions"]
HERM --> Cn["Vectors in<br/>C^n"]
MAT --> Mnm["Matrices<br/>R^{m×n}"]

style T fill:#90CAF9,stroke:#1E88E5,color:#000

style DOT fill:#C8E6C9,stroke:#2E7D32,color:#000
style WIP fill:#C8E6C9,stroke:#2E7D32,color:#000
style FN fill:#C8E6C9,stroke:#2E7D32,color:#000
style HERM fill:#C8E6C9,stroke:#2E7D32,color:#000
style MAT fill:#C8E6C9,stroke:#2E7D32,color:#000

style Rn fill:#CE93D8,stroke:#8E24AA,color:#000
style SPD fill:#CE93D8,stroke:#8E24AA,color:#000
style L2 fill:#CE93D8,stroke:#8E24AA,color:#000
style Cn fill:#CE93D8,stroke:#8E24AA,color:#000
style Mnm fill:#CE93D8,stroke:#8E24AA,color:#000

Definition #

For vectors
\( \mathbf{a}, \mathbf{b} \in \mathbb{R}^n \)

Natural Language Processing

AI, ML

Natural Language Processing, NLP, AI, ML

Natural Language Processing #

Natural Language Processing (NLP) studies how computers can analyse, understand, represent, and generate human language.

It combines ideas from linguistics, computer science, machine learning, and deep learning to work with text and language-based information.

Natural Language Processing = Linguistics + Computation + Machine Learning

The learning path begins with language understanding and vector representations, progresses through language modelling, tagging, and parsing, and then moves towards transformers, knowledge graphs, Retrieval-Augmented Generation, and modern NLP applications.

Artificial and Computational Intelligence

AI, ML

Deep Reinforcement Learning

Artificial and Computational Intelligence #

Modular Structure #

1. Introduction #

Artificial Intelligence foundations
Overview of modern AI
AI application domains

2. Introduction to Intelligent Agents #

Notion of agents and environments
Rationality
Nature of environments
Structure of agents

3. Problem-Solving Agents Using Search #

Problem formulation
Uninformed and informed search algorithms
Heuristics
Greedy Best-First Search
A* Search and optimality of A*
Heuristic accuracy and algorithm performance
Admissible heuristics from relaxed problems
Pattern databases and experience
Learning heuristics
Local search and optimisation
Hill Climbing
Local Beam Search
Genetic Algorithms
Ant Colony Optimisation
Neural Architecture Search
Neuroevolution

4. Game Playing #

Minimax Algorithm
Alpha-Beta Pruning
Monte Carlo Tree Search
Stochastic Games

5. Knowledge Representation Using Logic #

Propositional and Predicate Logic
TT-Entail and theorem proving
Logic representation of intelligent agents
Proof by resolution
DPLL Algorithm
Agents based on Propositional Logic
Unification
Forward Chaining
Backward Chaining
Resolution

6. Multi-Agent Decision Making #

Properties of multi-agent environments
Multi-agent planning
Non-Cooperative Game Theory
Cooperative Game Theory
Collective decision making

7. Probabilistic Representation and Reasoning #

Representing knowledge in uncertain domains
Semantics of Bayesian Networks
Exact inference in Bayesian Networks
Approximate inference in Bayesian Networks

8. Probabilistic Reasoning Over Time #

Time and uncertainty
Inference in temporal models
Hidden Markov Models
Learning HMMs
Dynamic Bayesian Networks

9. Ethics in AI #

Explainable AI
Logically Explained Networks
Explainable Bayesian Networks

Experiments #

#	Experiment
1	Implement Uninformed Search Algorithms such as BFS and DFS
2	Implement the A* Algorithm for Informed Search
3	Implement Local Search Techniques using a Genetic Algorithm
4	Implement the Minimax Algorithm for Adversarial Search in game playing
5	Represent knowledge using logic and perform reasoning using Prolog
6	Experiment with Bayesian Networks and exact inference
7	Experiment with the application of a Hidden Markov Model in Natural Language Processing

Practical Tools #

Programming languages: Python, Prolog
Tools and libraries: Jupyter, NumPy, SciPy, Pandas, pgmpy, NLTK
Environments: Google Colab, SWI-Prolog Online

Book References #

Primary Textbook #

Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 4th Edition, Pearson Education, 2020.

Reference Books #

Ryszard S. Michalski, Jaime G. Carbonell and Tom M. Mitchell, Machine Learning: An Artificial Intelligence Approach, Elsevier, 2014.
Dan W. Patterson, Introduction to AI and Expert Systems, Prentice Hall of India, New Delhi, 2010.
Elaine Rich and Kevin Knight, Artificial Intelligence, 2nd Edition, Tata McGraw Hill Publishing Company, New Delhi, 2003.

Home | Artificial Intelligence