ChatGPT Explained: How Large Language Models Work
4 posts on chatgpt, newest first.
ChatGPT runs on large language models built on the Transformer architecture. It splits your text into tokens, turns those tokens into embeddings (lists of numbers), and predicts a reply one token at a time. That same predict-the-next-token approach is also why it sometimes “makes things up.” The posts below walk through each step in plain English, about five minutes each.
Start here: a four-step reading path
If you’re new to the topic, read the posts in this order. Each one builds on the last.
| Step | Post | What you’ll get from it |
|---|---|---|
| 1 | Why is the Transformer Model Called an “AI Revolution”? | The architecture behind ChatGPT, and why it changed AI. This is the first edition of Daily AI Insight. |
| 2 | Discovering the Joy of Tokens: AI’s Language Magic Unveiled | How text gets chopped into tokens before a model ever sees it, kept down-to-earth. |
| 3 | What Is an Embedding? The Bridge From Text to the World of Numbers | How words become numbers a model can work with. AI doesn’t comprehend words or syntax; it only works with numbers. |
| 4 | The Hallucination Problem in Generative AI: Why Do Models “Make Things Up”? | Why answers can be confidently wrong: noisy training data, architecture limits and prompts. Also covers how hallucination misleads users and ways to reduce it. |
- 8 min read
Why Do LLMs Hallucinate? Causes, Examples and Fixes
Why generative AI and LLMs hallucinate: next-token prediction, gaps in training data and vague prompts, with examples and practical ways to reduce it.
- 7 min read
What Is an Embedding? Text to Vectors, Explained
An embedding turns text into vectors so AI can compare meaning. How Word2Vec, GloVe and BERT build them, and how dot product and cosine similarity work.
- 8 min read
Tokens in AI: How LLM Tokenization Works
What tokens are, how LLMs split text with word, character and subword (BPE) tokenization, and why tokens shape context windows, cost and speed.
- 7 min read
Transformer Model Explained: Why It Was an AI Revolution
How the Transformer and self-attention work, why they replaced RNNs and LSTMs, and their limits. A plain-English guide to the model behind LLMs.
Key ideas in one place
- Transformer: the neural network design ChatGPT’s models are built on. It uses attention to weigh how words in a passage relate to each other.
- Token: a chunk of text, such as a word, part of a word or a punctuation mark. Models read and write in tokens, not letters or whole sentences.
- Embedding: a vector of numbers that represents a token’s meaning, so related words end up close together.
- Hallucination: an answer that sounds right but isn’t supported by facts. It’s a side effect of generating the most likely next token rather than looking things up.
About Daily AI Insight
Every post on this page belongs to Daily AI Insight, a GeekCoding101 series that unpacks AI one bite-sized topic at a time. It started on December 3, 2024. To keep going after these four, follow the daily-ai-insight tag below.
FAQ
How does ChatGPT actually work? It breaks your prompt into tokens and converts them into embeddings. A Transformer-based model then predicts the most likely next token, over and over, until the reply is complete. Steps 1–3 in the reading path above cover each part.
Why does ChatGPT make things up? The model generates plausible text rather than checking facts. Gaps or noise in training data, limits of the architecture, and vague prompts can all lead to confident but wrong answers. The hallucination post explains the causes and ways to reduce them.
What’s the difference between a token and an embedding? A token is the piece of text. An embedding is the list of numbers that represents that piece so the model can compute with it.
Do I need a technical background to follow these posts? No. The Daily AI Insight posts are written for beginners and take about five minutes each.
Is GeekCoding101 free? Yes. Every post is free to read and there is no sign-up.