GeekCoding101 make your way to geek

ChatGPT Explained: How Large Language Models Work

4 posts on chatgpt, newest first.

4posts
2024first post

ChatGPT runs on large language models built on the Transformer architecture. It splits your text into tokens, turns those tokens into embeddings (lists of numbers), and predicts a reply one token at a time. That same predict-the-next-token approach is also why it sometimes “makes things up.” The posts below walk through each step in plain English, about five minutes each.

Start here: a four-step reading path

If you’re new to the topic, read the posts in this order. Each one builds on the last.

StepPostWhat you’ll get from it
1Why is the Transformer Model Called an “AI Revolution”?The architecture behind ChatGPT, and why it changed AI. This is the first edition of Daily AI Insight.
2Discovering the Joy of Tokens: AI’s Language Magic UnveiledHow text gets chopped into tokens before a model ever sees it, kept down-to-earth.
3What Is an Embedding? The Bridge From Text to the World of NumbersHow words become numbers a model can work with. AI doesn’t comprehend words or syntax; it only works with numbers.
4The Hallucination Problem in Generative AI: Why Do Models “Make Things Up”?Why answers can be confidently wrong: noisy training data, architecture limits and prompts. Also covers how hallucination misleads users and ways to reduce it.

Key ideas in one place

  • Transformer: the neural network design ChatGPT’s models are built on. It uses attention to weigh how words in a passage relate to each other.
  • Token: a chunk of text, such as a word, part of a word or a punctuation mark. Models read and write in tokens, not letters or whole sentences.
  • Embedding: a vector of numbers that represents a token’s meaning, so related words end up close together.
  • Hallucination: an answer that sounds right but isn’t supported by facts. It’s a side effect of generating the most likely next token rather than looking things up.

About Daily AI Insight

Every post on this page belongs to Daily AI Insight, a GeekCoding101 series that unpacks AI one bite-sized topic at a time. It started on December 3, 2024. To keep going after these four, follow the daily-ai-insight tag below.

FAQ

How does ChatGPT actually work? It breaks your prompt into tokens and converts them into embeddings. A Transformer-based model then predicts the most likely next token, over and over, until the reply is complete. Steps 1–3 in the reading path above cover each part.

Why does ChatGPT make things up? The model generates plausible text rather than checking facts. Gaps or noise in training data, limits of the architecture, and vague prompts can all lead to confident but wrong answers. The hallucination post explains the causes and ways to reduce them.

What’s the difference between a token and an embedding? A token is the piece of text. An embedding is the list of numbers that represents that piece so the model can compute with it.

Do I need a technical background to follow these posts? No. The Daily AI Insight posts are written for beginners and take about five minutes each.

Is GeekCoding101 free? Yes. Every post is free to read and there is no sign-up.