N-gram Language Modelling #
This page is a structured learning template. Replace the comments with clear explanations, examples, formulas, diagrams, and practical insights while keeping the Hugo shortcodes intact.
Learning Objectives #
- Explain the Markov assumption used by n-gram language models.
- Calculate sequence probabilities using unigram, bigram, and trigram models.
- Explain the zero-probability problem and the role of smoothing.
- Evaluate a language model using appropriate metrics such as perplexity.
Chapter Map #
| Section | Topic | Status |
|---|---|---|
| 1 | N-grams | ☐ |
| 2 | Generalisation and Zeros | ☐ |
| 3 | Smoothing | ☐ |
| 4 | The Web and Stupid Backoff | ☐ |
| 5 | Evaluating Language Models | ☐ |
Big Picture #
1. N-grams ☆ #
Definition #
Intuition #
Key Concepts #
Formula or Model #
Worked Example #
Why It Matters in NLP #
Key Points to Remember #
2. Generalisation and Zeros ☆ #
Definition #
Intuition #
Key Concepts #
Formula or Model #
Worked Example #
Why It Matters in NLP #
Key Points to Remember #
3. Smoothing ☆ #
Definition #
Intuition #
Key Concepts #
Formula or Model #
Worked Example #
Why It Matters in NLP #
Key Points to Remember #
4. The Web and Stupid Backoff ☆ #
Definition #
Intuition #
Key Concepts #
Formula or Model #
Worked Example #
Why It Matters in NLP #
Key Points to Remember #
5. Evaluating Language Models ☆ #
Definition #
Intuition #
Key Concepts #
Formula or Model #
Worked Example #
Why It Matters in NLP #
Key Points to Remember #
Practical Exploration #
Build an n-gram language model and compare unsmoothed and smoothed predictions.
# Add a minimal, well-commented Python example here.
Comparison Table #
| Concept or Model | Main Idea | Strength | Limitation | Typical Use |
|---|---|---|---|---|
Common Mistakes #
Practice Questions #
Key Takeaways #
Understanding Checklist #
- I can explain N-grams without referring to notes.
- I can explain Generalisation and Zeros without referring to notes.
- I can explain Smoothing without referring to notes.
- I can explain The Web and Stupid Backoff without referring to notes.
- I can explain Evaluating Language Models without referring to notes.