Natural Language Processing

Understanding and Generation

AI, Natural Language Processing
AI, NLP, Natural Language Understanding, Natural Language Generation

NLP - Understanding and Generation #

  • The Study of Language.
  • Applications of Natural Language Understanding.
  • Evaluating Language Understanding Systems.
  • Different Levels of Language Analysis.
  • Organisation of Natural Language Understanding Systems.

Learning Objectives #

  • Explain what Natural Language Processing studies and how it relates to artificial intelligence and linguistics.
  • Describe why human language is difficult for computers to process.
  • Recognise the main applications and stages of an NLP pipeline.
  • Distinguish morphological, lexical, syntactic, semantic, pragmatic, and discourse analysis.
  • Explain the relationship between natural language understanding and natural language generation.
  • Identify suitable ways to evaluate different NLP systems.

Big Picture #

flowchart TD
    A[Human Language] --> B[Acquire and Prepare Text or Speech]
    B --> C[Analyse Language Structure]
    C --> D[Represent Meaning and Context]
    D --> E[Model or Reason]
    E --> F[Understand, Predict or Generate]
    F --> G[Evaluate and Improve]

    style A fill:#E1F5FE
    style B fill:#C8E6C9
    style C fill:#FFF9C4
    style D fill:#EDE7F6
    style E fill:#E1F5FE
    style F fill:#C8E6C9
    style G fill:#FFF9C4

1. What Is Natural Language Processing? ☆ #

Natural language processing (NLP) is the discipline of building machines that can manipulate human language - or data that resembles human language - in the way that it is written, spoken, and organised.

Vector Semantics and Embedding

AI, Natural Language Processing
AI, NLP, Vector Semantics, Word Embeddings

NLP - Vector Semantics and Embedding #

  • Lexical semantics and word meaning.
  • Lemmas, senses, and semantic relationships.
  • Distributional hypothesis.
  • Vector semantics and word embeddings.
  • Document and word vectors.
  • Dot product and cosine similarity.
  • Term Frequency–Inverse Document Frequency (TF-IDF).
  • Prediction-based word embeddings and self-supervision.
  • Word2Vec using Skip-gram with Negative Sampling and CBOW.
  • Embedding matrices, context-window choices, analogies, visualisation, and bias.
  • GloVe and global word–word co-occurrence statistics.

Learning Objectives #

  • Explain lexical semantics and distinguish a lemma from a word sense.
  • Compare synonymy, similarity, relatedness, antonymy, and connotation.
  • Explain the distributional hypothesis and its role in modelling meaning.
  • Describe how words and documents can be represented as vectors.
  • Construct and interpret word–document and word–context matrices.
  • Calculate dot product and cosine similarity between vectors.
  • Explain why raw word frequency can be misleading.
  • Calculate TF, IDF, and TF-IDF weights.
  • Explain how Word2Vec learns embeddings from a prediction task.
  • Construct positive and negative Skip-gram training pairs.
  • Explain how sigmoid, negative sampling, and gradient descent train SGNS.
  • Compare Skip-gram with CBOW.
  • Explain how context-window size affects the relationships captured.
  • Interpret word analogies and two-dimensional embedding visualisations.
  • Explain how GloVe combines global counts with learned dense vectors.
  • Recognise how social biases can be encoded in word embeddings.

Big Picture #

flowchart TD
    A[Words and Documents] --> B[Observe Their Context]
    B --> C[Represent Them as Vectors]
    C --> D[Compare Vector Directions]
    D --> E[Estimate Semantic Similarity]
    E --> F[Search, Classify, Retrieve or Generate]

    style A fill:#E1F5FE
    style B fill:#C8E6C9
    style C fill:#FFF9C4
    style D fill:#EDE7F6
    style E fill:#E1F5FE
    style F fill:#C8E6C9

1. Lexical Semantics ☆ #

Lexical semantics is the linguistic study of word meaning and the relationships between word meanings.