LogoCookLLM Docs
LogoCookLLM Docs
HomeCookLLM

Principles

Tokenization
Tokenization BasicsBPE AlgorithmGPT TokenizersBPE Training Engineering
Model Architecture
Transformer LM
From token ids to logitsEmbedding and LM Head
Attention Mechanisms
From Self-Attention to GQAAttention Sink
Position Encoding
Position Encoding BasicsRoPE Math DerivationRoPE ImplementationLength Extrapolation
GPU Programming Basics
GPU Architecture BasicsTensor LayoutTriton Basics: Vector Add
FlashAttention
Flash Attention PrinciplesFrom Naive Implementation to Auto-TuningBlock Pointers and Multi-Dim SupportCausal Masking OptimizationGrouped Query AttentionBackward Pass Implementation
Distributed Training
Data ParallelismZeRO OptimizerFully Sharded Data ParallelTensor ParallelismPipeline ParallelismMulti-Dimensional Hybrid Parallelism

Hands-on Training

Overview
Pretraining
Pretraining DataTokenizer TrainingModel ArchitectureData PipelineTraining LoopMonitoring and Validation
X (Twitter)

CookLLM

Deeply learn the core technologies and practical applications of large language models

👋 Course Overview

Welcome to CookLLM! This is a systematic course focused on large language model (LLM) technology, taking you from principles to practice so you truly understand LLMs.

Why CookLLM?

I used to be a CV algorithm engineer. The essence of CV work is constantly resolving corner cases and bad cases, optimizing along two dimensions: performance and speed. In mature businesses, improving metrics by 1% over a year is already considered good. From a tech evolution standpoint, this field has entered a phase of incremental refinement.

Large models represent general capability. They will inevitably roll into one field after another in the future; it is only a matter of time. I do not want to carve patterns in the “stock”; I want to create waves in the “increment.”

In this wave, I do not want to be a bystander. I want to be a participant.

So I started self-studying LLM technology. But I soon found this path is not easy:

  • Scattered resources - knowledge is spread across blogs, papers, and videos, with no systematic learning path
  • Hard papers - the math is intimidating, and after reading you still do not know how to implement
  • Uneven quality - some tutorials contain mistakes, wasting a lot of time

I spent a lot of time exploring, organizing, and practicing, and gradually formed my own learning system.

CookLLM is me organizing that system and sharing it with people who want to deeply understand LLMs, just like me.

What You Will Get From This Course

  • More than calling APIs - deeply understand the principles behind each algorithm
  • Runnable code - every concept has accompanying code you can clone and run
  • Structured knowledge - from fundamentals to advanced topics, step by step

Course Structure

Principles

Deeply understand LLM core algorithms: tokenization, Attention, GPU programming, Flash Attention

Hands-on Training

Build and train a small language model from scratch: Tokenizer, Dataset, Model, Training

More course content is cooking... See our roadmap to learn what is coming next!

Who This Is For

  • Developers transitioning to AI - have programming fundamentals and want to enter the LLM field
  • Algorithm engineers seeking deeper understanding - not satisfied with calling APIs and want to know how things work underneath
  • Learners who want to build models from scratch - want to train a model themselves, not just use others'

Membership Benefits

As a paid member, you will get:

  • ✅ Full access to all course chapters
  • ✅ Advanced technical deep dives
  • ✅ Complete code implementations and project templates
  • ✅ Continuous updates with the latest content

Get Help

  • 📕 Follow us on Xiaohongshu
  • 🐦 Follow us on X (Twitter)
  • 💬 Join our Discord community
  • 📧 Contact support via support@cookllm.com

Resource Platforms

  • 🤗 Hugging Face — datasets and models
  • 🟣 ModelScope — datasets and models
  • 📊 SwanLab — training experiment logs

Tokenization

Deeply understand LLM tokenization, from BPE to GPT implementations

Table of Contents

👋 Course Overview
Why CookLLM?
What You Will Get From This Course
Course Structure
Who This Is For
Membership Benefits
Get Help
Resource Platforms