GeekCoding101 make your way to geek

Machine Learning Notes and Tutorials

25 posts on machine-learning, newest first.

25posts
2024first post

(Post titles below link to each post’s existing URL from the list.)

Machine Learning Notes and Tutorials

This page collects every GeekCoding101 post tagged machine-learning. They are hands-on explainers of core ideas like knowledge distillation, quantization, embeddings and pretraining vs fine-tuning, plus day-by-day notes from Andrew Ng’s Supervised Machine Learning course. If you are new, start with the course notes. If you want one concept explained, jump to the explainer table below.

Core concepts, one post each

ConceptPostWhat you’ll learn
Knowledge distillationKnowledge Distillation: How Big Models Train Smaller OnesSoft targets, temperature, the distillation loss, the main types, LLM examples like DistilBERT and DeepSeek-R1 distills, and how it compares with quantization and pruning
Pretraining vs fine-tuningPretraining vs. Fine-Tuning: What’s the Difference?What each stage does, masked and causal language modeling, key differences and why the stages are separate
QuantizationQuantization: How to Unlock Incredible Efficiency on AI ModelsCompressing models by reducing precision, e.g. from FP32 floats to INT8 integers
EmbeddingsWhat Is an Embedding? The Bridge From Text to the World of NumbersHow language is turned into numbers a model can work with
ParametersWhat Are Parameters? Why Are “Bigger” Models Often “Smarter”?Weights and biases, and how they adjust during training
Weight initializationWeight Initialization: Unleashing AI Performance ExcellenceWhy the starting values of weights shape neural network training
Foundation modelsEmpower Your AI Journey: Foundation Models ExplainedFoundation models vs task models, with GPT-4 and BERT as examples
Prompt engineeringWhat Is Prompt Engineering and How to “Train” AI with a Single Sentence?Writing input prompts that guide generative AI to the result you want

Transformers and “Attention is All You Need”

A walkthrough of the paper by Vaswani et al. that introduced the Transformer:

  1. Diving into “Attention is All You Need”: My Transformer Journey Begins! Why this paper matters and the plan for reading it.
  2. Terms Used in “Attention is All You Need” A glossary of the paper’s key terms with English and Chinese translations.
  3. Transformers Demystified - Day 2 - Unlocking the Genius of Self-Attention and AI’s Greatest Breakthrough The paper’s Introduction and Background sections.

Serving models in production

Ray Serve Guide: Autoscaling, FastAPI & Ollama covers how Ray Serve works:

  • deployments
  • autoscaling
  • batching
  • multi-model pipelines

It also explains when to choose Ray Serve over FastAPI or Ollama.

Supervised Machine Learning course notes (Andrew Ng)

These are day-by-day notes from working through Supervised Machine Learning: Regression and Classification, in reading order:

  1. Supervised Machine Learning - Day 1: why the series started, coming from building an AI-powered knowledge base chatbot
  2. Supervised Machine Learning – Day 2 & 3 - On My Way To Becoming A Machine Learning Person
  3. Mastering Multiple Features & Vectorization: Supervised Machine Learning – Day 4 and 5: multiple features and vectorization, with NumPy
  4. Supervised Machine Learning – Day 6: a review day
  5. Master Feature Scaling & Gradient Descent: Supervised Machine Learning – Day 7: why feature scaling matters and how to check that gradient descent is converging
  6. Master Learning Rate and Feature Engineering: Supervised Machine Learning – Day 8: choosing the learning rate and an introduction to feature engineering
  7. Master Gradient Descent and Binary Classification: Supervised Machine Learning – Day 9
  8. Grinding Through Logistic regression: Exploring Supervised Machine Learning – Day 10: decision boundary, logistic loss, cost function and gradient descent for logistic regression
  9. Overfitting! Unlocking the Last Key Concept in Supervised Machine Learning – Day 11, 12: overfitting vs underfitting, and fixing them with more data, simpler models or regularization

There is also a side note from the same period: Finished Machine Learning for Absolute Beginners - Level 1.

Azure AI notes

  • Honored to Pass AI-102! Notes on earning the Microsoft Azure AI Engineer Associate certificate.
  • Install Azure-Cli on Mac covers fixing azure-cli install failures on macOS: the Homebrew OpenSSL and Python permission errors, and the chown and brew unlink commands that worked.

AI news

Groundbreaking News: OpenAI Unveils o3 and o3 Mini with Stunning ARC-AGI Performance covers the models announced on December 20, 2024, at the end of OpenAI’s 12-day campaign.

FAQ

What’s the difference between knowledge distillation and quantization? Knowledge distillation trains a smaller “student” model to imitate a larger “teacher” model. Quantization keeps the same model but stores its numbers at lower precision, for example INT8 instead of FP32. Both make models cheaper to run. The knowledge distillation post compares them side by side, along with pruning.

What’s the difference between pretraining and fine-tuning? Pretraining teaches a model general patterns from a large amount of data, using objectives such as masked or causal language modeling. Fine-tuning then adapts that pretrained model to a specific task with a smaller, targeted dataset. See Pretraining vs. Fine-Tuning: What’s the Difference? for details.

What is Ray Serve used for? Ray Serve is a model-serving library built on Ray. It handles deployments, autoscaling, request batching and multi-model pipelines. The Ray Serve guide explains how it works and when it is a better fit than FastAPI or Ollama.

I’m new to machine learning. Where should I start? Start with Supervised Machine Learning - Day 1 and follow the course notes in order. Then read the concept explainers as each topic comes up.

Are these posts free? Yes. Every post on GeekCoding101 is free to read, with no sign-up.