An open community for learning, writing, and tinkering on the infrastructure behind modern AI — inference engines, training systems, ml stacks, and everything in between.
A from-scratch tour of what's actually inside an LLM: how a transformer turns tokens into predictions, what Query, Key, and Value really mean, and how generating text one token at a time builds the KV cache — the growing pool of memory that makes inference so expensive.
The word "mask" means at least four unrelated things in deep learning — what a token can see, what counts toward the loss, what is hidden to create a task, and what is randomly dropped. One field guide to all of them, with why each exists and what breaks without it.
You don't need a massive tech budget or a cluster of high-end GPUs to train your own AI. LoRA allows developers to fine-tune giant models right on a standard laptop. Here is the zero-jargon, first-principles explanation of the clever shortcut that leveled the playing field.
A neural network never sees a word, an image, or a sound — only a list of numbers. Starting from that one fact and a single neuron, this guide builds the whole machine: how any input becomes numbers, why weights, biases, and activations each exist, and how neurons stack into layers and layers into a model.
The word "mask" means at least four unrelated things in deep learning — what a token can see, what counts toward the loss, what is hidden to create a task, and what is randomly dropped. One field guide to all of them, with why each exists and what breaks without it.
You don't need a massive tech budget or a cluster of high-end GPUs to train your own AI. LoRA allows developers to fine-tune giant models right on a standard laptop. Here is the zero-jargon, first-principles explanation of the clever shortcut that leveled the playing field.
A from-scratch tour of what's actually inside an LLM: how a transformer turns tokens into predictions, what Query, Key, and Value really mean, and how generating text one token at a time builds the KV cache — the growing pool of memory that makes inference so expensive.
A neural network never sees a word, an image, or a sound — only a list of numbers. Starting from that one fact and a single neuron, this guide builds the whole machine: how any input becomes numbers, why weights, biases, and activations each exist, and how neurons stack into layers and layers into a model.
Inspect attention patterns layer-by-layer for any Hugging Face model. Click any head to see its causal mask, induction behavior, and sink tokens.
LiveEstimate tokens/sec for any model, precision, batch size, and GPU combination — memory-bound roofline only, no kernel-quality wishful thinking.
LivePer-GPU VRAM breakdown for training and inference — params, gradients, optimizer state, activations, KV cache, with ZeRO/FSDP sharding.
LiveRun a small set of evaluations against any inference endpoint and get back a structured scorecard — quality, latency, cost, and refusal rate side by side.
BetaGenerate a structured model card from a checkpoint and an evaluation log — covers intended use, training data summary, evals, limitations, and ethics.
BetaSide-by-side timings for Triton, CUDA, and PyTorch implementations of the same kernel — attention, layernorm, GEMM, custom — across shapes and dtypes.
SoonAsk questions, share what you built, run a poll, or just discuss ML systems.
Open the Forum →Real-time chat for the working day. Quick questions, debugging help, paper club, and the occasional argument about whether MoE is overrated.
Join the server →One email per publication. No spam, unsubscribe anytime.