Transformers are changing the AI landscape, and it all began with the groundbreaking paper “Attention is All You Need.” Today, I explore the Introduction and Background sections of the paper,…
Below is a comprehensive table of key terms used in the paper “Attention is All You Need,” along with their English and Chinese translations. Where applicable, links to external resources are…
Today marks the beginning of my adventure into one of the most groundbreaking papers in AI for transformer: “Attention is All You Need” by Vaswani et al. If you’ve ever been curious about how modern…
On December 20, 2024, OpenAI concluded its 12-day “OpenAI Christmas Gifts” campaign by revealing two groundbreaking models: o3 and o3 mini. At the same time, the ARC Prize organization announced…
How Ray Serve works: deployments, autoscaling, batching and multi-model pipelines, and when to choose Ray Serve over FastAPI or Ollama.
Quantization is a transformative AI optimization technique that compresses models by reducing precision from high-bit floating-point numbers (e.g., FP32) to low-bit integers (e.g., INT8). This…
Weight Initialization in AI plays a crucial role in ensuring effective neural network training. It determines the starting values for connections (weights) in a model, significantly influencing…
What knowledge distillation is, how soft targets, temperature and the distillation loss work, the main types, LLM examples like DistilBERT and DeepSeek-R1 distills, and how it compares with…