NN and Neural Language Modelling

Mon, 01 Jan 0001 00:00:00 +0000

Neural Networks and Neural Language Modelling #

Neural networks learn useful representations and nonlinear relationships directly from data. In language modelling, they replace discrete N-gram identities with learned word embeddings and use these representations to predict the next word.

Learning Objectives #

Explain the computation performed by a neural unit.
Describe why hidden layers and nonlinear activations are needed.
Explain how feed-forward networks support NLP classification.
Trace the flow through a feed-forward neural language model.
Compare N-gram and neural language models.

Big Picture #

flowchart TD
 A["Context Words"] --> B["One-hot Inputs"]
 B --> C["Embedding Lookup"]
 C --> D["Combined Context"]
 D --> E["Hidden Layer"]
 E --> F["Softmax"]
 F --> G["Next-word Probabilities"]

 style A fill:#E1F5FE
 style B fill:#C8E6C9
 style C fill:#FFF9C4
 style D fill:#EDE7F6
 style E fill:#E1F5FE
 style F fill:#C8E6C9
 style G fill:#FFF9C4

1. Neural Network Units ☆ #

A neural unit receives input values, multiplies them by learned weights, adds a bias, and applies an activation function.

Neural Language Models on Arshad Siddiqui

NN and Neural Language Modelling

Neural Networks and Neural Language Modelling #

Learning Objectives #

Big Picture #

1. Neural Network Units ☆ #