https://arshadhs.github.io/tags/vector-semantics/Recent content in Vector Semantics on Arshad SiddiquiHugoen-ushttps://arshadhs.github.io/docs/ai/050-nlp/020-vector-semantics-and-embedding/Mon, 01 Jan 0001 00:00:00 +0000https://arshadhs.github.io/docs/ai/050-nlp/020-vector-semantics-and-embedding/<h1 id="nlp---vector-semantics-and-embedding"> NLP - Vector Semantics and Embedding <a class="anchor" href="#nlp---vector-semantics-and-embedding">#</a> </h1> <ul> <li>Lexical semantics and word meaning.</li> <li>Lemmas, senses, and semantic relationships.</li> <li>Distributional hypothesis.</li> <li>Vector semantics and word embeddings.</li> <li>Document and word vectors.</li> <li>Dot product and cosine similarity.</li> <li>Term Frequency–Inverse Document Frequency (TF-IDF).</li> <li>Prediction-based word embeddings and self-supervision.</li> <li>Word2Vec using Skip-gram with Negative Sampling and CBOW.</li> <li>Embedding matrices, context-window choices, analogies, visualisation, and bias.</li> <li>GloVe and global word–word co-occurrence statistics.</li> </ul> <h2 id="learning-objectives"> Learning Objectives <a class="anchor" href="#learning-objectives">#</a> </h2> <ul> <li>Explain lexical semantics and distinguish a lemma from a word sense.</li> <li>Compare synonymy, similarity, relatedness, antonymy, and connotation.</li> <li>Explain the distributional hypothesis and its role in modelling meaning.</li> <li>Describe how words and documents can be represented as vectors.</li> <li>Construct and interpret word–document and word–context matrices.</li> <li>Calculate dot product and cosine similarity between vectors.</li> <li>Explain why raw word frequency can be misleading.</li> <li>Calculate TF, IDF, and TF-IDF weights.</li> <li>Explain how Word2Vec learns embeddings from a prediction task.</li> <li>Construct positive and negative Skip-gram training pairs.</li> <li>Explain how sigmoid, negative sampling, and gradient descent train SGNS.</li> <li>Compare Skip-gram with CBOW.</li> <li>Explain how context-window size affects the relationships captured.</li> <li>Interpret word analogies and two-dimensional embedding visualisations.</li> <li>Explain how GloVe combines global counts with learned dense vectors.</li> <li>Recognise how social biases can be encoded in word embeddings.</li> </ul> <!-- ## Map | Section | Topic | Status | |---|---|---| | 1 | Lexical Semantics | | 2 | Lemmas and Word Senses | | 3 | Semantic Relationships | | 4 | Distributional Hypothesis | | 5 | Vector Semantics and Word Embeddings | | 6 | Documents as Vectors | | 7 | Words as Vectors | | 8 | Dot Product | | 9 | Cosine Similarity | | 10 | TF-IDF | | 11 | Count-Based and Prediction-Based Embeddings | --> <h2 id="big-picture"> Big Picture <a class="anchor" href="#big-picture">#</a> </h2> <pre class="mermaid">flowchart TD A[Words and Documents] --&gt; B[Observe Their Context] B --&gt; C[Represent Them as Vectors] C --&gt; D[Compare Vector Directions] D --&gt; E[Estimate Semantic Similarity] E --&gt; F[Search, Classify, Retrieve or Generate] style A fill:#E1F5FE style B fill:#C8E6C9 style C fill:#FFF9C4 style D fill:#EDE7F6 style E fill:#E1F5FE style F fill:#C8E6C9</pre> <h2 id="1-lexical-semantics-"> 1. Lexical Semantics ☆ <a class="anchor" href="#1-lexical-semantics-">#</a> </h2> <p><strong>Lexical semantics</strong> is the linguistic study of word meaning and the relationships between word meanings.</p>