> For the complete documentation index, see [llms.txt](https://chenzihong.gitbook.io/llm-everything/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://chenzihong.gitbook.io/llm-everything/readme.md).

# 前言

![](/files/71uFgmDifEdC5Pzwy8Ge)

## LLM-Everything

**从零开始，系统掌握大语言模型的一切。**

[![GitBook](https://img.shields.io/static/v1?message=Documented%20on%20GitBook\&logo=gitbook\&logoColor=ffffff\&label=%20\&labelColor=5c5c5c\&color=3F89A1)](https://chenzihong.gitbook.io/llm-everything) [![知乎](https://img.shields.io/static/v1?message=%E7%9F%A5%E4%B9%8E%E4%B8%93%E6%A0%8F\&logo=zhihu\&logoColor=ffffff\&label=%20\&labelColor=5c5c5c\&color=0084FF)](https://www.zhihu.com/column/c_1931824303218885390)

***

### ✨ 为什么是这个项目？

市面上不缺 LLM 教程，但缺的是**真正讲明白**的。

* 🎯 **不复制粘贴** — 每篇文章精心打磨，用生动的方式拆解复杂概念
* 🔨 **从零实现代码** — 不只讲理论，带你亲手写出来，在实战中理解原理
* 🗺️ **体系化路线** — 从基础到前沿，完整的学习路径，不再迷路

***

### 📚 知识地图

#### 🎚️ 基础部分

**🐍 Python 基础**

* [logging 模块](/llm-everything/basics/python-basics/logging.md)
* [import 模块](/llm-everything/basics/python-basics/import.md)
* [multiprocessing 模块](/llm-everything/basics/python-basics/multiprocessing.md)

**🐘 机器学习基础**

* 文本表示模型
  * [Bag-of-Words](/llm-everything/basics/machine-learning-basics/feature-extraction/text-representation-models/bag-of-words.md)
  * [Topic Model](/llm-everything/basics/machine-learning-basics/feature-extraction/text-representation-models/topic-model.md)
  * [Static Word Embeddings](/llm-everything/basics/machine-learning-basics/feature-extraction/text-representation-models/static-word-embeddings.md)

**🪿 深度学习基础**

* 🚧 持续更新中...

**🐬 LLM 基础**

* [思考模式切换](/llm-everything/basics/llm-basics/switch-thinking.md)
* [为什么现在的LLM都是decoder-only架构](/llm-everything/basics/llm-basics/why-decoder-only.md)

#### 🐬 Prompt Engineering

* [Tree of Thoughts](/llm-everything/prompt-engineering/tree-of-thoughts.md)

#### 🦖 Transformer 架构

> 逐模块拆解 Transformer

* [Tokenizer](/llm-everything/transformer/tokenizer.md)
* [Embeddings](/llm-everything/transformer/embeddings.md)
  * [ELMo](/llm-everything/transformer/embeddings/elmo.md) 
  * [BERT](/llm-everything/transformer/embeddings/bert.md)
  * [GPT](/llm-everything/transformer/embeddings/gpt.md)
* [Positional Encoding](/llm-everything/transformer/positional-encoding.md)
* [Self Attention](/llm-everything/transformer/self-attention.md)
* [Multi-Head Attention](/llm-everything/transformer/multi-head-attention.md)
* [Add & Norm](/llm-everything/transformer/add-and-norm.md)
* [FeedForward](/llm-everything/transformer/feedforward.md)
* [Linear & Softmax](/llm-everything/transformer/linear-and-softmax.md)
* [Decoding Strategy](/llm-everything/transformer/decoding-strategy.md)

#### 🎄 LLM 训练

**显存需求**

* [LLM 精度问题](/llm-everything/train/llm-vram-needs/llm-precision.md)
* [训练需要多少显存](/llm-everything/train/llm-vram-needs/vram_needs_for_llm_training.md)

**分布式并行**

* [数据并行](/llm-everything/train/distributed-training-parallelism/data-parallelism.md) 
* [模型并行](/llm-everything/train/distributed-training-parallelism/model-parallelism.md) 
* [优化器并行](/llm-everything/train/distributed-training-parallelism/optimizer-parallelism.md)
* [异构系统并行](/llm-everything/train/distributed-training-parallelism/heterogeneous-system-parallelism.md)

**训练流程**

* **数据准备**
  * [课程学习](/llm-everything/train/data-preparation/curriculum-learning.md)
* [预训练](/llm-everything/train/pre-train.md)
  * [数据工程](/llm-everything/train/pre-train/data-engineering.md)
  * [超参配置](/llm-everything/train/pre-train/hyper-param.md)
  * [长文本拓展](/llm-everything/train/pre-train/long-text-extension.md)
  * [评估](/llm-everything/train/pre-train/evaluation_and_engineering.md)
* [监督微调](/llm-everything/train/sft.md)
  * [数据工程](/llm-everything/train/sft/data_engineering.md)
  * [参数高效微调](/llm-everything/train/sft/peft.md)
  * [训练策略与稳定性](/llm-everything/train/sft/training-strategy.md)
  * [评估](/llm-everything/train/sft/evaluation.md)
* [强化学习](/llm-everything/train/reinforce-learning.md)
  * [RLHF基础 & PPO](/llm-everything/train/reinforce-learning/rlhf-basics-and-ppo.md)
  * [DPO](/llm-everything/train/reinforce-learning/dpo.md)
  * [GRPO](/llm-everything/train/reinforce-learning/grpo.md)
  * [DAPO](/llm-everything/train/reinforce-learning/dapo.md)

#### 🐒 MoE（混合专家模型）

* [专家并行](/llm-everything/moe/expert-parallelism.md)

#### 🪿 LLM 应用

* [信息检索](/llm-everything/llm-application/info-retrieval.md)
  * [相似度度量](/llm-everything/llm-application/info-retrieval/similarity.md)
  * [文本表示方法](/llm-everything/llm-application/info-retrieval/text-representation.md)
  * [词向量](/llm-everything/llm-application/info-retrieval/word-vector.md)
  * [跨模态相似度](/llm-everything/llm-application/info-retrieval/cross-modal-similarity.md)
  * [大规模向量检索](/llm-everything/llm-application/info-retrieval/large-scale-retrieval.md)
* [RAG](/llm-everything/llm-application/rag.md)
  * [RAG 完整链路](/llm-everything/llm-application/rag/end-to-end-rag-pipeline.md)
  * [Agentic RAG](/llm-everything/llm-application/rag/agentic-rag.md)
  * [Graph RAG](/llm-everything/llm-application/rag/graph-rag.md)
* [AutoResearch](/llm-everything/llm-application/autoresearch.md)

#### 🐢 多模态大模型

* 多模态大模型基础
* 核心组件
  * 视觉编码器
  * 模态连接器
    * [QFormer](/llm-everything/multi-modal-llm/components/connector/qformer.md)
  * 视觉分词器
* 工程实现
  * 最常用的数据集格式—— [WebDataset](/llm-everything/multi-modal-llm/engineering/webdataset.md)

#### 🦔 LLM Infra

* [K8s](/llm-everything/llm-infra/k8s.md)
  * [Kubernetes 自动扩缩容的三种玩法：HPA、VPA 和 KEDA](/llm-everything/llm-infra/k8s/auto-scale.md)

***

### 🤝 参与贡献

本项目正在快速迭代中，欢迎：

* 🐛 提 Issue 指出错误或疑问
* 🔀 提 PR 补充内容
* ⭐ 觉得有用就给个 Star，这是最大的鼓励

***

**如果这个项目帮到了你，请点个 ⭐ Star 支持一下！**