Understanding and Generation

Mon, 01 Jan 0001 00:00:00 +0000

NLP - Understanding and Generation #

The Study of Language.
Applications of Natural Language Understanding.
Evaluating Language Understanding Systems.
Different Levels of Language Analysis.
Organisation of Natural Language Understanding Systems.

Learning Objectives #

Explain what Natural Language Processing studies and how it relates to artificial intelligence and linguistics.
Describe why human language is difficult for computers to process.
Recognise the main applications and stages of an NLP pipeline.
Distinguish morphological, lexical, syntactic, semantic, pragmatic, and discourse analysis.
Explain the relationship between natural language understanding and natural language generation.
Identify suitable ways to evaluate different NLP systems.

Big Picture #

flowchart TD
 A[Human Language] --> B[Acquire and Prepare Text or Speech]
 B --> C[Analyse Language Structure]
 C --> D[Represent Meaning and Context]
 D --> E[Model or Reason]
 E --> F[Understand, Predict or Generate]
 F --> G[Evaluate and Improve]

 style A fill:#E1F5FE
 style B fill:#C8E6C9
 style C fill:#FFF9C4
 style D fill:#EDE7F6
 style E fill:#E1F5FE
 style F fill:#C8E6C9
 style G fill:#FFF9C4

1. What Is Natural Language Processing? ☆ #

Natural language processing (NLP) is the discipline of building machines that can manipulate human language - or data that resembles human language - in the way that it is written, spoken, and organised.

Vector Semantics and Embedding

Mon, 01 Jan 0001 00:00:00 +0000

NLP - Vector Semantics and Embedding #

Lexical semantics and word meaning.
Lemmas, senses, and semantic relationships.
Distributional hypothesis.
Vector semantics and word embeddings.
Document and word vectors.
Dot product and cosine similarity.
Term Frequency–Inverse Document Frequency (TF-IDF).
Prediction-based word embeddings and self-supervision.
Word2Vec using Skip-gram with Negative Sampling and CBOW.
Embedding matrices, context-window choices, analogies, visualisation, and bias.
GloVe and global word–word co-occurrence statistics.

Learning Objectives #

Explain lexical semantics and distinguish a lemma from a word sense.
Compare synonymy, similarity, relatedness, antonymy, and connotation.
Explain the distributional hypothesis and its role in modelling meaning.
Describe how words and documents can be represented as vectors.
Construct and interpret word–document and word–context matrices.
Calculate dot product and cosine similarity between vectors.
Explain why raw word frequency can be misleading.
Calculate TF, IDF, and TF-IDF weights.
Explain how Word2Vec learns embeddings from a prediction task.
Construct positive and negative Skip-gram training pairs.
Explain how sigmoid, negative sampling, and gradient descent train SGNS.
Compare Skip-gram with CBOW.
Explain how context-window size affects the relationships captured.
Interpret word analogies and two-dimensional embedding visualisations.
Explain how GloVe combines global counts with learned dense vectors.
Recognise how social biases can be encoded in word embeddings.

Big Picture #

flowchart TD
 A[Words and Documents] --> B[Observe Their Context]
 B --> C[Represent Them as Vectors]
 C --> D[Compare Vector Directions]
 D --> E[Estimate Semantic Similarity]
 E --> F[Search, Classify, Retrieve or Generate]

 style A fill:#E1F5FE
 style B fill:#C8E6C9
 style C fill:#FFF9C4
 style D fill:#EDE7F6
 style E fill:#E1F5FE
 style F fill:#C8E6C9

1. Lexical Semantics ☆ #

Lexical semantics is the linguistic study of word meaning and the relationships between word meanings.

Natural Language Processing on Arshad Siddiqui

Understanding and Generation

NLP - Understanding and Generation #

Learning Objectives #

Big Picture #

1. What Is Natural Language Processing? ☆ #

Vector Semantics and Embedding

NLP - Vector Semantics and Embedding #

Learning Objectives #

Big Picture #

1. Lexical Semantics ☆ #