https://arshadhs.github.io/tags/transformer/Recent content in Transformer on Arshad SiddiquiHugoen-ushttps://arshadhs.github.io/docs/ai/deep-learning/090-transformer/Mon, 01 Jan 0001 00:00:00 +0000https://arshadhs.github.io/docs/ai/deep-learning/090-transformer/<h1 id="transformer"> Transformer <a class="anchor" href="#transformer">#</a> </h1> <p>A transformer is a neural network architecture that uses attention as its main mechanism for processing sequences.</p> <p>Unlike RNNs, transformers do not process tokens one by one.</p> <p>They process many tokens in parallel and use self-attention to learn relationships between tokens.</p> <ul> <li> <p>is an architecture of neural networks</p> </li> <li> <p>based on the multi-head attention mechanism</p> </li> <li> <p>text is converted to numerical representations called tokens, and each token is converted into a vector via lookup from a word embedding table</p>