Deep Reinforcement Learning

Deep Reinforcement Learning #

Deep Reinforcement Learning (DRL) studies how an agent learns to make a sequence of decisions by interacting with an environment and receiving feedback through rewards.

Reinforcement learning provides the framework for sequential decision-making. Deep learning extends this framework with powerful function approximators that can handle large or complex state and action spaces.

Deep Reinforcement Learning = Reinforcement Learning + Deep Neural Networks

The learning path begins with classical reinforcement learning foundations and progresses towards value-based deep learning, policy-gradient methods, model-based approaches, and imitation learning.

Natural Language Processing

Natural Language Processing #

Natural Language Processing (NLP) studies how computers can analyse, understand, represent, and generate human language.

It combines ideas from linguistics, computer science, machine learning, and deep learning to work with text and language-based information.

Natural Language Processing = Linguistics + Computation + Machine Learning

The learning path begins with language understanding and vector representations, progresses through language modelling, tagging, and parsing, and then moves towards transformers, knowledge graphs, Retrieval-Augmented Generation, and modern NLP applications.

AI Stages: ANI, AGI, ASI

July 4, 2024

AI Development Stages: ANI → AGI → ASI #

Artificial Intelligence is often described in three stages, based on capability and scope:

• ANI: Task-specific intelligence (today’s AI)
• AGI: Human-level general intelligence (future goal)
• ASI: Beyond human intelligence (theoretical)

ANI — Artificial Narrow Intelligence #

• also called Weak AI
• designed to perform one specific task
• Operates within a predefined environment
• Cannot generalise beyond its training
• Most AI systems today are ANI

examples

Artificial Intelligence

Artificial Intelligence #

Artificial Intelligence (AI) is concerned with building systems that can perceive, reason, learn, decide and act in ways that achieve useful goals.

The foundations of AI, its major application areas, four classic ways of thinking about intelligence, important milestones in its development, and some of the risks that accompany increasingly capable AI systems.

Learning Objectives #

• explain what intelligence and artificial intelligence mean in practical terms
• identify the major disciplines that contributed to AI
• distinguish thinking humanly, acting humanly, thinking rationally and acting rationally
• explain the idea behind the Turing Test
• describe why the rational-agent view is central to modern AI
• recognise important application areas and risks of AI

Big Picture #

flowchart TD
    A[Artificial Intelligence] --> B[Human-like]
    A --> C[Rational]
    B --> D[Think like humans]
    B --> E[Act like humans]
    C --> F[Think rationally]
    C --> G[Act rationally]

    style A fill:#C8E6C9
    style B fill:#E1F5FE
    style C fill:#FFF9C4
    style D fill:#EDE7F6
    style E fill:#EDE7F6
    style F fill:#C8E6C9
    style G fill:#C8E6C9

1. Agent
2. Explore (all possible solutions)
3. Environment
4. Sensor → i/p → Percept
5. Actuator
6. Action
7. Initial State: Source
8. Transition Model
9. Random-isation
10. Learned Info → FACT → Knowledge Base
11. Backtracking

• State Space Transition Diagram / Search Tree
• Performance Measure → Numerical Measure (Minimise or Maximise)

M2 → Search M4 → Game M5 → Knowledge → Fact → KB → Inference

Basic Statistics

Basic Statistics #

Statistics: describes data (what you see).
Probability: models uncertainty (what you don’t know yet).

• Summarise a dataset using central tendency and variability
• Explain core probability ideas using simple examples
• Apply the axioms of probability
• Distinguish mutually exclusive vs independent events

flowchart TD
    A[Dataset] --> B[Central Tendency]
    A --> C[Variability]
    B --> B1[Mean]
    B --> B2[Median]
    B --> B3[Mode]
    C --> C1[Range]
    C --> C2[Variance]
    C --> C3[Standard Deviation]
    C --> C4[IQR]

Measures of Central Tendency #

Central tendency tells you where the “middle” of the data is. Describes a set of scores with a single number that describes the PERFORMANCE of the group.

ML and DL System Performance

ML and DL System Performance #

Machine learning system optimisation begins with measurement. Before changing an algorithm, adding processors, or moving work to a GPU, we need to understand what is slow, which resource is limiting performance, and how performance changes as the workload grows.

This page covers:

• time and space complexity
• throughput and latency
• the relationship between workload, throughput, and latency
• the main measurements used to describe system performance

Learning Objectives #

By the end of this page, you should be able to:

Reinforcement Learning

Reinforcement Learning #

Reinforcement Learning (RL) is a learning approach in which an agent improves its behaviour by interacting with an environment and observing the rewards produced by its actions.

Unlike supervised learning, the agent is not given a correct action label for every situation. It must discover useful behaviour through trial, feedback, and repeated interaction.

Why Reinforcement Learning? #

RL is useful when decisions affect what happens next and the quality of an action may only become clear later.

Understanding and Generation

NLP - Understanding and Generation #

• The Study of Language.
• Applications of Natural Language Understanding.
• Evaluating Language Understanding Systems.
• Different Levels of Language Analysis.
• Organisation of Natural Language Understanding Systems.

Learning Objectives #

• Explain what Natural Language Processing studies and how it relates to artificial intelligence and linguistics.
• Describe why human language is difficult for computers to process.
• Recognise the main applications and stages of an NLP pipeline.
• Distinguish morphological, lexical, syntactic, semantic, pragmatic, and discourse analysis.
• Explain the relationship between natural language understanding and natural language generation.
• Identify suitable ways to evaluate different NLP systems.

Big Picture #

flowchart TD
    A[Human Language] --> B[Acquire and Prepare Text or Speech]
    B --> C[Analyse Language Structure]
    C --> D[Represent Meaning and Context]
    D --> E[Model or Reason]
    E --> F[Understand, Predict or Generate]
    F --> G[Evaluate and Improve]

    style A fill:#E1F5FE
    style B fill:#C8E6C9
    style C fill:#FFF9C4
    style D fill:#EDE7F6
    style E fill:#E1F5FE
    style F fill:#C8E6C9
    style G fill:#FFF9C4

1. What Is Natural Language Processing? ☆ #

Natural language processing (NLP) is the discipline of building machines that can manipulate human language - or data that resembles human language - in the way that it is written, spoken, and organised.

Basic Probability

Basic Probability #

Probability models uncertainty: what you don’t know yet, but want to reason about.

Key takeaway: Probability is a number between 0 and 1 that measures how likely an event is. The whole topic is about defining events clearly and applying a few core rules consistently.

Probability quantifies uncertainty: a number between 0 and 1.

• 0 means: impossible
• 1 means: certain

Terminology #

Random experiment #

A random experiment is an action whose outcome is not known in advance.

Neural Networks

Neural Networks #

• A network of artificial neurons inspired by how neurons function in the human brain.
• At its core - a mathematical model designed to process and learn from data.
• Neural networks form the foundation of Deep Learning (involves training large and complex networks on vast amounts of data).

flowchart LR
 subgraph subGraph0["Input Layer"]
        I1(("Input 1"))
        I2(("Input 2"))
        I3(("Input 3"))
  end
 subgraph subGraph1["Hidden Layer"]
        H1(("Hidden 1"))
        H2(("Hidden 2"))
        H3(("Hidden 3"))
  end
 subgraph subGraph2["Output Layer"]
        O(("Output"))
  end
    I1 --> H1 & H2 & H3
    I2 --> H1 & H2 & H3
    I3 --> H1 & H2 & H3
    H1 --> O
    H2 --> O
    H3 --> O

    style I1 fill:#C8E6C9
    style I2 fill:#C8E6C9
    style I3 fill:#C8E6C9
    style H1 stroke:#2962FF,fill:#BBDEFB
    style H2 fill:#BBDEFB
    style H3 fill:#BBDEFB
    style O fill:#FFCDD2
    style subGraph0 stroke:none,fill:transparent
    style subGraph1 stroke:none,fill:transparent
    style subGraph2 stroke:none,fill:transparent

Structure of a Neural Network #

A typical neural network has three main layers: