GeekCoding101 make your way to geek

Transformer Models: Self-Attention, LLM Basics and Efficiency

13 posts on transformer, newest first.

13posts
2024first post

A transformer is a neural network architecture, introduced in the 2017 paper “Attention Is All You Need”. It uses self-attention to weigh how each token in a sequence relates to every other token. It is the design behind today’s large language models.

The posts below explain it step by step:

  • what self-attention actually does
  • how the original paper reads, section by section
  • the core ideas around LLMs, such as parameters, embeddings and prompts
  • how transformer models are made smaller and served in production

Where to start

If you are new to transformers, read these in order:

  1. Why is the Transformer Model Called an “AI Revolution”? The first edition of the Daily AI Insight series. It explains why the architecture matters.
  2. 7 Key Insights on the Self-Attention Mechanism in AI Magic. Where attention came from, how relevance scores link words, real-world uses and limits.
  3. What Is an Embedding? The Bridge From Text to the World of Numbers. How text becomes the numbers a transformer works on.

Reading “Attention Is All You Need”

A walk through the paper itself, with notes taken along the way.

PostWhat you get
Diving into “Attention is All You Need”: My Transformer Journey Begins!Why this paper by Vaswani et al. is the starting point, and the plan for reading it
Terms Used in “Attention is All You Need”A table of key terms from the paper, in English and Chinese, with links to further reading
Transformers Demystified - Day 2 - Unlocking the Genius of Self-Attention and AI’s Greatest BreakthroughThe Introduction and Background sections, explained

LLM building blocks

Short Daily AI Insight explainers on the ideas that sit around the transformer.

  • What Are Parameters? Why Are “Bigger” Models Often “Smarter”? Weights, biases and what changes during training.
  • Parameters vs. Inference Speed: Why Is Your Phone’s AI Model ‘Slimmer’ Than GPT-4? The trade-off between model size and speed.
  • What Is Prompt Engineering and How to “Train” AI with a Single Sentence? Crafting inputs that guide a model to the result you want.
  • The Hallucination Problem in Generative AI: Why Do Models “Make Things Up”? Noisy data, architecture limits, prompts, and ways to reduce made-up answers.

Making transformer models smaller and serving them

Once a model works, the next questions are how to make it cheaper and how to run it.

PostTopic
Quantization: How to Unlock Incredible Efficiency on AI ModelsCompressing models by lowering precision, e.g. from FP32 floats to INT8 integers
Knowledge Distillation: How Big Models Train Smaller OnesSoft targets, temperature, the distillation loss, the main types, LLM examples like DistilBERT and DeepSeek-R1 distills, and how it compares with quantization and pruning
Ray Serve Guide: Autoscaling, FastAPI & OllamaDeployments, autoscaling, batching, multi-model pipelines, and when to choose Ray Serve over FastAPI or Ollama

FAQ

What is a transformer model? It is a neural network architecture that processes a whole sequence at once and uses self-attention to decide which tokens matter to each other. It was introduced in “Attention Is All You Need” and is the basis of modern LLMs.

What is self-attention in simple terms? For each word (token), the model scores how relevant every other word in the input is. It then mixes their information using those scores. The self-attention post walks through this with examples.

Is knowledge distillation the same as quantization? No. Distillation trains a smaller “student” model to imitate a larger “teacher”. Quantization keeps the model but stores its numbers at lower precision. Both make models cheaper to run, and they can be combined.

Where should I start if I’m new to transformers? Start with “Why is the Transformer Model Called an ‘AI Revolution’?”, then the self-attention post, then the embedding post. After that, move on to the “Attention Is All You Need” reading notes.

Do I need to sign up to read these posts? No. Every GeekCoding101 post is free to read, with no sign-up.