# 前言

![](/files/71uFgmDifEdC5Pzwy8Ge)

## LLM-Everything

**从零开始，系统掌握大语言模型的一切。**

[![GitBook](https://img.shields.io/static/v1?message=Documented%20on%20GitBook\&logo=gitbook\&logoColor=ffffff\&label=%20\&labelColor=5c5c5c\&color=3F89A1)](https://chenzihong.gitbook.io/llm-everything) [![知乎](https://img.shields.io/static/v1?message=%E7%9F%A5%E4%B9%8E%E4%B8%93%E6%A0%8F\&logo=zhihu\&logoColor=ffffff\&label=%20\&labelColor=5c5c5c\&color=0084FF)](https://www.zhihu.com/column/c_1931824303218885390)

***

### ✨ 为什么是这个项目？

市面上不缺 LLM 教程，但缺的是**真正讲明白**的。

* 🎯 **不复制粘贴** — 每篇文章精心打磨，用生动的方式拆解复杂概念
* 🔨 **从零实现代码** — 不只讲理论，带你亲手写出来，在实战中理解原理
* 🗺️ **体系化路线** — 从基础到前沿，完整的学习路径，不再迷路

***

### 📚 知识地图

#### 🎚️ 基础部分

**🐍 Python 基础**

* [logging 模块](/llm-everything/basics/python-basics/logging.md)
* [import 模块](/llm-everything/basics/python-basics/import.md)
* [multiprocessing 模块](/llm-everything/basics/python-basics/multiprocessing.md)

**🐘 机器学习基础**

* 文本表示模型
  * [Bag-of-Words](/llm-everything/basics/machine-learning-basics/feature-extraction/text-representation-models/bag-of-words.md)
  * [Topic Model](/llm-everything/basics/machine-learning-basics/feature-extraction/text-representation-models/topic-model.md)
  * [Static Word Embeddings](/llm-everything/basics/machine-learning-basics/feature-extraction/text-representation-models/static-word-embeddings.md)

**🪿 深度学习基础**

* 🚧 持续更新中...

**🐬 LLM 基础**

* [思考模式切换](/llm-everything/basics/llm-basics/switch-thinking.md)
* [为什么现在的LLM都是decoder-only架构](/llm-everything/basics/llm-basics/why-decoder-only.md)

#### 🐬 Prompt Engineering

* [Tree of Thoughts](/llm-everything/prompt-engineering/tree-of-thoughts.md)

#### 🦖 Transformer 架构

> 逐模块拆解 Transformer

* [Tokenizer](/llm-everything/transformer/tokenizer.md)
* [Embeddings](/llm-everything/transformer/embeddings.md)
  * [ELMo](/llm-everything/transformer/embeddings/elmo.md) 
  * [BERT](/llm-everything/transformer/embeddings/bert.md)
  * [GPT](/llm-everything/transformer/embeddings/gpt.md)
* [Positional Encoding](/llm-everything/transformer/positional-encoding.md)
* [Self Attention](/llm-everything/transformer/self-attention.md)
* [Multi-Head Attention](/llm-everything/transformer/multi-head-attention.md)
* [Add & Norm](/llm-everything/transformer/add-and-norm.md)
* [FeedForward](/llm-everything/transformer/feedforward.md)
* [Linear & Softmax](/llm-everything/transformer/linear-and-softmax.md)
* [Decoding Strategy](/llm-everything/transformer/decoding-strategy.md)

#### 🎄 LLM 训练

**显存需求**

* [LLM 精度问题](/llm-everything/train/llm-vram-needs/llm-precision.md)
* [训练需要多少显存](/llm-everything/train/llm-vram-needs/vram_needs_for_llm_training.md)

**分布式并行**

* [数据并行](/llm-everything/train/distributed-training-parallelism/data-parallelism.md) 
* [模型并行](/llm-everything/train/distributed-training-parallelism/model-parallelism.md) 
* [优化器并行](/llm-everything/train/distributed-training-parallelism/optimizer-parallelism.md)
* [异构系统并行](/llm-everything/train/distributed-training-parallelism/heterogeneous-system-parallelism.md)

**训练流程**

* **数据准备**
  * [课程学习](/llm-everything/train/data-preparation/curriculum-learning.md)
* [预训练](/llm-everything/train/pre-train.md)
  * [数据工程](/llm-everything/train/pre-train/data-engineering.md)
  * [超参配置](/llm-everything/train/pre-train/hyper-param.md)
  * [长文本拓展](/llm-everything/train/pre-train/long-text-extension.md)
  * [评估](/llm-everything/train/pre-train/evaluation_and_engineering.md)
* [监督微调](/llm-everything/train/sft.md)
  * [数据工程](/llm-everything/train/sft/data_engineering.md)
  * [参数高效微调](/llm-everything/train/sft/peft.md)
  * [训练策略与稳定性](/llm-everything/train/sft/training-strategy.md)
  * [评估](/llm-everything/train/sft/evaluation.md)
* [强化学习](/llm-everything/train/reinforce-learning.md)
  * [RLHF基础 & PPO](/llm-everything/train/reinforce-learning/rlhf-basics-and-ppo.md)
  * [DPO](/llm-everything/train/reinforce-learning/dpo.md)
  * [GRPO](/llm-everything/train/reinforce-learning/grpo.md)
  * [DAPO](/llm-everything/train/reinforce-learning/dapo.md)

#### 🐒 MoE（混合专家模型）

* [专家并行](/llm-everything/moe/expert-parallelism.md)

#### 🪿 LLM 应用

* [信息检索](/llm-everything/llm-application/info-retrieval.md)
  * [相似度度量](/llm-everything/llm-application/info-retrieval/similarity.md)
  * [文本表示方法](/llm-everything/llm-application/info-retrieval/text-representation.md)
  * [词向量](/llm-everything/llm-application/info-retrieval/word-vector.md)
  * [跨模态相似度](/llm-everything/llm-application/info-retrieval/cross-modal-similarity.md)
  * [大规模向量检索](/llm-everything/llm-application/info-retrieval/large-scale-retrieval.md)
* [AutoResearch](/llm-everything/llm-application/autoresearch.md)
* [RAG](/llm-everything/llm-application/rag.md)
* [Graph RAG](/llm-everything/llm-application/graph-rag.md)

#### 🐢 多模态大模型

* [QFormer](/llm-everything/multi-modal-llm/components/connector/qformer.md)

***

### 🤝 参与贡献

本项目正在快速迭代中，欢迎：

* 🐛 提 Issue 指出错误或疑问
* 🔀 提 PR 补充内容
* ⭐ 觉得有用就给个 Star，这是最大的鼓励

***

**如果这个项目帮到了你，请点个 ⭐ Star 支持一下！**


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://chenzihong.gitbook.io/llm-everything/readme.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.