Principles

Tokenization Basics BPE Algorithm GPT Tokenizers BPE Training Engineering

Model Architecture

Attention Mechanisms

Position Encoding

Position Encoding Basics RoPE Math Derivation RoPE Implementation Length Extrapolation

GPU Programming Basics

GPU Architecture Basics Tensor Layout Triton Basics: Vector Add

Flash Attention Principles From Naive to Auto-Tuning Block Pointers and Multi-Dim Support Causal Masking Optimization Grouped Query Attention Backward Pass

Hands-on Training

环境搭建

使用 uv 安装 cookllm-bento 依赖，并检查 GPU 与配置文件

👨‍🍳

Content is cooking...

We're preparing high-quality content for you. Stay tuned!

Table of Contents

1. 准备代码仓库

3. 创建虚拟环境

4. 配置环境变量

5. 检查 GPU 和 PyTorch

6. 理解项目入口