Machine Learning Notes and Tutorials
25 posts on machine-learning, newest first.
(Post titles below link to each post’s existing URL from the list.)
Machine Learning Notes and Tutorials
This page collects every GeekCoding101 post tagged machine-learning. They are hands-on explainers of core ideas like knowledge distillation, quantization, embeddings and pretraining vs fine-tuning, plus day-by-day notes from Andrew Ng’s Supervised Machine Learning course. If you are new, start with the course notes. If you want one concept explained, jump to the explainer table below.
Core concepts, one post each
| Concept | Post | What you’ll learn |
|---|---|---|
| Knowledge distillation | Knowledge Distillation: How Big Models Train Smaller Ones | Soft targets, temperature, the distillation loss, the main types, LLM examples like DistilBERT and DeepSeek-R1 distills, and how it compares with quantization and pruning |
| Pretraining vs fine-tuning | Pretraining vs. Fine-Tuning: What’s the Difference? | What each stage does, masked and causal language modeling, key differences and why the stages are separate |
| Quantization | Quantization: How to Unlock Incredible Efficiency on AI Models | Compressing models by reducing precision, e.g. from FP32 floats to INT8 integers |
| Embeddings | What Is an Embedding? The Bridge From Text to the World of Numbers | How language is turned into numbers a model can work with |
| Parameters | What Are Parameters? Why Are “Bigger” Models Often “Smarter”? | Weights and biases, and how they adjust during training |
| Weight initialization | Weight Initialization: Unleashing AI Performance Excellence | Why the starting values of weights shape neural network training |
| Foundation models | Empower Your AI Journey: Foundation Models Explained | Foundation models vs task models, with GPT-4 and BERT as examples |
| Prompt engineering | What Is Prompt Engineering and How to “Train” AI with a Single Sentence? | Writing input prompts that guide generative AI to the result you want |
Transformers and “Attention is All You Need”
A walkthrough of the paper by Vaswani et al. that introduced the Transformer:
- Diving into “Attention is All You Need”: My Transformer Journey Begins! Why this paper matters and the plan for reading it.
- Terms Used in “Attention is All You Need” A glossary of the paper’s key terms with English and Chinese translations.
- Transformers Demystified - Day 2 - Unlocking the Genius of Self-Attention and AI’s Greatest Breakthrough The paper’s Introduction and Background sections.
Serving models in production
Ray Serve Guide: Autoscaling, FastAPI & Ollama covers how Ray Serve works:
- deployments
- autoscaling
- batching
- multi-model pipelines
It also explains when to choose Ray Serve over FastAPI or Ollama.
Supervised Machine Learning course notes (Andrew Ng)
These are day-by-day notes from working through Supervised Machine Learning: Regression and Classification, in reading order:
- Supervised Machine Learning - Day 1: why the series started, coming from building an AI-powered knowledge base chatbot
- Supervised Machine Learning – Day 2 & 3 - On My Way To Becoming A Machine Learning Person
- Mastering Multiple Features & Vectorization: Supervised Machine Learning – Day 4 and 5: multiple features and vectorization, with NumPy
- Supervised Machine Learning – Day 6: a review day
- Master Feature Scaling & Gradient Descent: Supervised Machine Learning – Day 7: why feature scaling matters and how to check that gradient descent is converging
- Master Learning Rate and Feature Engineering: Supervised Machine Learning – Day 8: choosing the learning rate and an introduction to feature engineering
- Master Gradient Descent and Binary Classification: Supervised Machine Learning – Day 9
- Grinding Through Logistic regression: Exploring Supervised Machine Learning – Day 10: decision boundary, logistic loss, cost function and gradient descent for logistic regression
- Overfitting! Unlocking the Last Key Concept in Supervised Machine Learning – Day 11, 12: overfitting vs underfitting, and fixing them with more data, simpler models or regularization
There is also a side note from the same period: Finished Machine Learning for Absolute Beginners - Level 1.
Azure AI notes
- Honored to Pass AI-102! Notes on earning the Microsoft Azure AI Engineer Associate certificate.
- Install Azure-Cli on Mac covers fixing azure-cli install failures on macOS: the Homebrew OpenSSL and Python permission errors, and the
chownandbrew unlinkcommands that worked.
AI news
Groundbreaking News: OpenAI Unveils o3 and o3 Mini with Stunning ARC-AGI Performance covers the models announced on December 20, 2024, at the end of OpenAI’s 12-day campaign.
- 13 min read
Attention Is All You Need, Explained: Intro & Background
A line-by-line walkthrough of the Introduction and Background of "Attention Is All You Need": why RNNs hit a wall and how self-attention replaced them.
- 7 min read
Attention Is All You Need: Key Terms in English & Chinese
A glossary of Transformer terms from "Attention Is All You Need" (Q/K/V, multi-head attention, positional encoding, BLEU) with Chinese translations.
- 5 min read
Attention Is All You Need Abstract Explained (Day 1)
The abstract of “Attention Is All You Need” (Vaswani et al., 2017) makes one bold claim. You can build a sequence-to-sequence model out of attention alone, with no recurrence and no convolution, and…
- 5 min read
OpenAI o3 on ARC-AGI: Scores, Cost, Limits
OpenAI's o3 scored 75.7% on ARC-AGI's Semi-Private set and 87.5% at 172x compute. What ARC-AGI tests, what it cost, and why it isn't AGI.
- 9 min read
Ray Serve Guide: Autoscaling, Batching, FastAPI & Ollama
What Ray Serve is, how deployments, autoscaling, batching and model composition work, with code, and when to pick Ray Serve over FastAPI or Ollama.
- 8 min read
Model Quantization: FP32 to INT8, PTQ vs QAT
What model quantization is, how FP32 weights map to INT8 with a scale and zero point, PTQ vs QAT vs dynamic quantization, memory math and trade-offs.
- 7 min read
Weight Initialization Explained: Xavier vs He
What weight initialization is, why it prevents vanishing and exploding gradients, and when to use Xavier, He or orthogonal init, with PyTorch code.
- 14 min read
Knowledge Distillation: How Big Models Train Smaller Ones
What knowledge distillation is, how soft targets, temperature and the distillation loss work, the main types, LLM examples like DistilBERT and DeepSeek-R1 distills, and how it compares with quantization and pruning.
- 7 min read
Foundation Models vs Task Models Explained
What foundation models and task models are, how they differ in data, training and scale, and when to prompt, fine-tune or distill for a specific task.
- 7 min read
Pretraining vs Fine-Tuning in LLMs: Key Differences
Pretraining vs fine-tuning: what each stage does, masked vs causal language modeling, full vs LoRA fine-tuning, comparison table and when to fine-tune.
- 7 min read
What Is an Embedding? Text to Vectors, Explained
An embedding turns text into vectors so AI can compare meaning. How Word2Vec, GloVe and BERT build them, and how dot product and cosine similarity work.
- 8 min read
What Is Prompt Engineering? With Examples
Prompt engineering is writing inputs that steer an AI model to the output you want. Core techniques, before/after examples, limits and when to fine-tune.
- 7 min read
What Are Model Parameters? Why Bigger AI Is Often Smarter
Parameters are the weights and biases a model learns in training. See how parameter count affects LLM performance, memory and cost, and when smaller wins.
- 6 min read
How I Passed AI-102: Prep & Exam Day
My AI-102 (Azure AI Engineer Associate) story: the training labs, MS Learn and notes I used, online exam day with OnVUE, and a 768 pass.
- 4 min read
Install Azure CLI on Mac: Fix Homebrew Permission Errors
brew install azure-cli failing on macOS? Fix the Homebrew OpenSSL and Python permission errors with chown and brew unlink, then verify az works.
- 6 min read
Overfitting vs Underfitting & Regularization - ML Day 11-12
My Day 11-12 notes from Andrew Ng's Supervised ML course: overfitting vs underfitting, how to fix them, and regularization with λ, formulas and Python.
- 6 min read
Logistic Regression: Decision Boundary & Cost – Day 10
Day 10 notes from Andrew Ng's ML course: the logistic regression decision boundary, why squared error fails, logistic loss vs cost, and gradient descent.
- 5 min read
Gradient Descent & Logistic Regression – ML Day 9
Day 9 of Andrew Ng's Supervised ML course: gradient descent for one-variable linear regression, binary classification, logistic regression and sigmoid.
- 6 min read
Choosing the Learning Rate & Feature Engineering – Day 8
How to pick a learning rate from a learning curve, plus feature scaling and feature engineering: Day 8 notes from Andrew Ng's Supervised ML course.
- 8 min read
Feature Scaling & Gradient Descent Convergence (ML Day 7)
Day 7 notes from Andrew Ng's ML course: why feature scaling speeds up gradient descent, mean vs z-score normalization, and how to check convergence.
- 5 min read
Types of Machine Learning: Beginner Notes (Level 1 Course)
Beginner ML notes: supervised, unsupervised, semi-supervised and reinforcement learning, plus overfitting vs underfitting, from a Level 1 course.
- 5 min read
Gradient Descent for Multiple Linear Regression: ML Day 6
Gradient descent for multiple linear regression updates every weight wⱼ by subtracting the learning rate times the average of (prediction − target) × xⱼ⁽ⁱ⁾, and updates b the same way but without the…
- 5 min read
Multiple Linear Regression & Vectorization – ML Day 4-5
Notes from Andrew Ng's Supervised ML course, days 4–5: multiple linear regression, dot product and NumPy vectorization, plus a self-quiz.
- 6 min read
Supervised ML Day 2 & 3: Gradient Descent and Learning Rate
Gradient descent minimizes a cost function J(w, b) by repeatedly moving w and b a small step against their partial derivatives, with the step size set by the learning rate α. If α is too small, it…
- 8 min read
Supervised Learning Day 1: Regression vs Classification
Supervised machine learning trains a model on labeled examples, meaning inputs (x) paired with the right answers (y), so it can predict y for inputs it hasn’t seen. Day 1 of Andrew Ng’s Supervised…
FAQ
What’s the difference between knowledge distillation and quantization? Knowledge distillation trains a smaller “student” model to imitate a larger “teacher” model. Quantization keeps the same model but stores its numbers at lower precision, for example INT8 instead of FP32. Both make models cheaper to run. The knowledge distillation post compares them side by side, along with pruning.
What’s the difference between pretraining and fine-tuning? Pretraining teaches a model general patterns from a large amount of data, using objectives such as masked or causal language modeling. Fine-tuning then adapts that pretrained model to a specific task with a smaller, targeted dataset. See Pretraining vs. Fine-Tuning: What’s the Difference? for details.
What is Ray Serve used for? Ray Serve is a model-serving library built on Ray. It handles deployments, autoscaling, request batching and multi-model pipelines. The Ray Serve guide explains how it works and when it is a better fit than FastAPI or Ollama.
I’m new to machine learning. Where should I start? Start with Supervised Machine Learning - Day 1 and follow the course notes in order. Then read the concept explainers as each topic comes up.
Are these posts free? Yes. Every post on GeekCoding101 is free to read, with no sign-up.