GeekCoding101 make your way to geek

Daily AI Insight: AI and LLM Basics, One Topic at a Time

17 posts on daily-ai-insight, newest first.

17posts
2024first post

Daily AI Insight is a GeekCoding101 series of short explainers on how modern AI and large language models work, with one topic per post. It starts with the Transformer and works through self-attention, parameters, tokens, embeddings, prompting, fine-tuning, model compression and serving. Most posts take about five minutes to read.

The series ran in December 2024. The fundamentals it covers have not changed. If you are new to LLMs, follow the reading order below. If you are looking for one concept, jump to the topic table.

Where to start: a suggested reading order

The series builds on itself, so reading in publishing order (oldest first) works well:

  1. Why is the Transformer Model Called an “AI Revolution”? The first edition and the starting point.
  2. 7 Key Insights on the Self-Attention Mechanism in AI Magic. How relevance scores link words, and the limits of attention.
  3. What Are Parameters? Why Are “Bigger” Models Often “Smarter”? Weights, biases and why size matters.
  4. Parameters vs. Inference Speed: Why Is Your Phone’s AI Model ‘Slimmer’ Than GPT-4? The follow-up on size versus speed.
  5. What Is Prompt Engineering and How to “Train” AI with a Single Sentence?
  6. Discovering the Joy of Tokens: AI’s Language Magic Unveiled.
  7. What Is an Embedding? The Bridge From Text to the World of Numbers.
  8. Fine-Tuning Models: Unlocking the Extraordinary Potential of AI.
  9. Pretraining vs. Fine-Tuning: What’s the Difference?
  10. Empower Your AI Journey: Foundation Models Explained.
  11. Discover the Power of Zero-Shot and Few-Shot Learning.
  12. The Hallucination Problem in Generative AI: Why Do Models “Make Things Up”?
  13. Weight Initialization: Unleashing AI Performance Excellence.
  14. Knowledge Distillation: How Big Models Train Smaller Ones.
  15. Quantization: How to Unlock Incredible Efficiency on AI Models.
  16. Ray Serve Guide: Autoscaling, FastAPI & Ollama.
  17. OpenAI Unveils o3 and o3 Mini with Stunning ARC-AGI Performance. A news post from December 2024, not a tutorial.

Browse by topic

TopicPostsWhat you’ll learn
How Transformers workTransformer “AI Revolution”; Self-Attention; Parameters; Parameters vs. Inference SpeedWhat a Transformer is, how attention scores relate words, what parameters are and why bigger models are slower
From text to numbersTokens; EmbeddingsHow text is split into tokens and turned into vectors a model can process
Prompting and generalizationPrompt Engineering; Zero-Shot and Few-Shot Learning; HallucinationGuiding a model with input alone, transfer learning, and why models make things up
Training stagesFoundation Models; Pretraining vs. Fine-Tuning; Fine-Tuning Models; Weight InitializationFoundation vs. task models, masked and causal language modeling, specializing a pre-trained model, and how starting weights affect training
Making models smaller and fasterKnowledge Distillation; QuantizationSoft targets, temperature and distillation loss; reducing precision from FP32 to INT8; how the two compare with pruning
Serving modelsRay Serve GuideDeployments, autoscaling, batching, multi-model pipelines, and when to pick Ray Serve over FastAPI or Ollama
NewsOpenAI o3 and o3 miniThe December 2024 announcement and its ARC-AGI results

Most in-depth reads

Most posts are 3–6 minute explainers. Two go much further:

FAQ

What is Daily AI Insight? It is a GeekCoding101 series of short explainers on AI and large language models, one topic per post. It covers Transformers and self-attention through fine-tuning, distillation, quantization and model serving.

Where should a beginner start? Start with Why is the Transformer Model Called an “AI Revolution”?, the first edition. Then follow the reading order above. Each post builds on the ones before it.

What is the difference between pretraining and fine-tuning? Pretraining teaches a model general language patterns from a very large dataset. Fine-tuning then trains that pre-trained model further on specific data so it specializes in a task or domain. Pretraining vs. Fine-Tuning: What’s the Difference? covers both stages, including masked and causal language modeling.

How is knowledge distillation different from quantization? Distillation trains a smaller “student” model to imitate a larger “teacher.” Quantization keeps the same model but stores its numbers at lower precision, such as INT8 instead of FP32. Both make models cheaper to run, and they can be combined.

Is it free to read? Yes. Every GeekCoding101 post is free to read, with no sign-up.