路线图
探索我们的未来计划和即将推出的课程模块
Overall progress10/26 completed
38%
Backlog
9Modern Transformer tricks doclow
KV Cache & text generation docmedium
Evaluation metrics doclow
RLHF: PPO training doc and codemedium
DPO (Direct Preference Optimization) doc and codemedium
GRPO (Group Relative Policy Optimization) doc and codemedium
Build a mini vLLM inference enginehigh
Multimodal model doc and codemedium
LLM Agent framework doc and codemedium
In Progress
7Introduce training parallelism modulehigh
Introduce modern LLM model architectures designshigh
Normalization doc (BatchNorm → RMSNorm)medium
Activation functions doc (ReLU → SwiGLU)medium
Optimizer theory doc (Adam → Muon)medium
Training techniques doc (LR schedule, mixed precision)medium
SFT (Supervised Fine-Tuning) doc and codemedium
Done
10Introduce LLM tokenizerhigh
Training tokenizer from scratchhigh
Flash Attention doc and codehigh
GPU programming doc and codehigh
RoPE position encoding doc and codehigh
build the cookllm-bento codebasehigh
Training: tokenizer hands-on (RustBPE + tiktoken)high
Training: BentoLM architecture design & implementationhigh
Training: pretrain pipeline (data, loop, monitoring)high
Training: pretrain inference & text generationmedium