Notes from Professor Hung-yi Lee’s (李宏毅) course “Machine Learning in the Era of Generative AI (2025)” at National Taiwan University.
What Can Generative AI Do?
Professor Lee starts with a demo: an AI avatar of himself, generated entirely by AI. The face is generated, the script is written by ChatGPT from a slide image, and the voice is synthesized by Breezy Voice (MediaTek) using a reference audio clip. The final video is produced by Heygen.
The real bottleneck isn’t generating the lecture video - it’s creating the slides. He tested ChatGPT Deep Research to generate a 13,000-word course outline, then used Gamma to turn it into slides. The result? Technically functional, but shallow. AI can generate content, but the hard part is knowing what to say.
How Language Models Work
A language model is a function that takes tokens as input and outputs the probability distribution for the next token. That’s it. The magic is in how this simple mechanism produces intelligent behavior.
Tokens: Text is split into chunks (tokens). GPT-4 uses ~100,000 possible tokens. “Hello world” might be 2 tokens; “Hello World” (capitalized) might be 3.
Layers: The function f is actually many smaller functions stacked together. Each layer transforms the input slightly. More layers = more transformation steps = more complex reasoning possible.
Why Depth Matters (A Simple Analogy)
Imagine each layer is a lookup table. To add three single-digit numbers (A + B + C) in one step, you need a table with 10 × 10 × 10 = 1,000 entries.
But if you split it into two steps:
- Step 1: A + B → B’ (100 entries)
- Step 2: B’ + C → result (190 entries)
Total: 290 entries instead of 1,000. Deep learning doesn’t make problems more complex - it makes them simpler by breaking them into steps.
Test-Time Scaling: When Depth Isn’t Enough
What if the problem needs more steps than you have layers? Let the model “think” - generate intermediate reasoning tokens before the final answer.
This is called Test-Time Scaling (or as Professor Lee puts it: “深度不够长度来凑” - when depth isn’t enough, use length).
OpenAI’s o1 model uses this. Stanford’s s1 paper showed that longer thinking = higher accuracy. Their method? When the model tries to output an end token, replace it with “wait” and force it to keep going.
Transformer Architecture
Modern LLMs use Transformer architecture, which has two types of layers:
- Self-attention: Considers all input tokens to produce output
- Feed-forward: Processes each token independently
The name “Transformer”? According to the original authors in a 2024 interview: “We just thought it sounded cool.”
Limitation: Self-attention scales poorly with input length. For very long inputs (like extended reasoning), alternatives like Mamba are being explored.
Architecture vs Parameters
A neural network has two parts:
- Architecture: Designed by humans (the “天资” - innate talent)
- Parameters: Learned from data (the “后天努力” - acquired through effort)
Transformer is the architecture. The billions of numbers that make it work are the parameters.
Training: Pre-training and Alignment
Pre-training: Learn from massive internet text. Predict the next token, over and over. This creates a “base model” - good at continuing text, but not at following instructions.
Alignment: Transform the base model into an assistant. Two approaches:
- SFT (Supervised Fine-Tuning): Train on human-written conversations
- RLHF (Reinforcement Learning from Human Feedback): Learn from human preferences
The result: a model that answers questions instead of just completing text.
Giving AI New Abilities
Three approaches, from least to most invasive:
1. Prompting: Give instructions in the context. The model’s parameters don’t change - it’s like an employee following company rules at work but reverting to normal behavior at home.
2. Fine-tuning: Modify the parameters. Risky - you might break existing abilities. Professor Lee’s demo: fine-tuning GPT-4o-mini to be a TA named “小金” worked, but the model started giving bizarre answers to unrelated questions.
3. Model Editing: Surgically modify specific parameters. Like implanting a “thought stamp” directly into the model’s brain.
4. Model Merging: Combine parameters from different models. One model writes code, another speaks Chinese - merge them to get both abilities without needing the original training data.
Key Takeaways
- LLMs are next-token predictors, but this simple mechanism produces complex behavior
- Depth (layers) and length (thinking tokens) both enable more complex reasoning
- Fine-tuning is powerful but dangerous - it can break existing abilities
- The field is moving toward surgical interventions (editing) and model combination (merging)
本文整理自台湾大学李宏毅教授的「生成式AI时代下的机器学习(2025)」课程。
生成式AI能做什么？
李老师用一个AI分身开场：脸是生成的，讲稿是ChatGPT根据投影片图片写的，声音是Breezy Voice（联发科）用参考音档合成的，最后用Heygen生成视频。
真正的瓶颈不是生成讲课视频，而是做投影片。他测试用ChatGPT Deep Research生成了一万三千字的课程大纲，再用Gamma转成投影片。结果？技术上可行，但内容浅薄。AI能生成内容，但难的是知道该说什么。
语言模型如何运作
语言模型是一个函数：输入tokens，输出下一个token的概率分布。就这样。神奇之处在于这个简单机制如何产生智能行为。
Tokens：文字被切成小块（tokens）。GPT-4使用约10万种可能的tokens。“Hello world”可能是2个tokens；“Hello World”（大写）可能是3个。
层（Layers）：函数f其实是很多小函数叠在一起。每一层稍微转换输入。层数越多 = 转换步骤越多 = 能处理越复杂的推理。
为什么深度重要（简单比喻）
想象每一层是一个查表。要一步算出三个个位数相加（A + B + C），需要10 × 10 × 10 = 1000笔数据的表格。
但如果拆成两步：
- 第一步：A + B → B’（100笔）
- 第二步：B’ + C → 结果（190笔）
总共：290笔而不是1000笔。深度学习不是让问题变复杂，而是把问题拆解成简单的步骤。
测试时间缩放：深度不够怎么办
如果问题需要的步骤比层数还多怎么办？让模型「思考」——在最终答案前生成中间推理的tokens。
这叫做Test-Time Scaling（或者李老师说的：「深度不够长度来凑」）。
OpenAI的o1模型用这个方法。Stanford的s1论文显示：想得越长，正确率越高。他们的方法？当模型要输出结束符号时，直接换成”wait”，强迫它继续讲下去。
Transformer架构
现代LLM使用Transformer架构，有两种层：
- Self-attention：考虑所有输入tokens来产生输出
- Feed-forward：独立处理每个token
为什么叫”Transformer”？根据原作者2024年的访谈：「我们只是觉得这名字很酷。」
限制：Self-attention在输入很长时效率差。对于很长的输入（如延长推理），正在探索Mamba等替代方案。
架构 vs 参数
神经网络有两部分：
- 架构：人类设计的（「天资」）
- 参数：从数据学习的（「后天努力」）
Transformer是架构。让它运作的数十亿个数字是参数。
训练：预训练与对齐
预训练：从大量网络文字学习。一直预测下一个token。这产生「基础模型」——擅长接续文字，但不擅长遵循指令。
对齐：把基础模型变成助手。两种方法：
- SFT（监督式微调）：用人写的对话训练
- RLHF（人类反馈强化学习）：从人类偏好学习
结果：模型会回答问题，而不只是接续文字。
赋予AI新能力
三种方法，从最轻到最重：
1. 提示（Prompting）：在上下文中给指令。模型参数不变——就像员工在公司遵守规定，回家就恢复原样。
2. 微调（Fine-tuning）：修改参数。有风险——可能破坏原有能力。李老师的示范：微调GPT-4o-mini变成助教「小金」成功了，但模型开始对无关问题给出奇怪答案（问谁是美国总统，回答李宏毅）。
3. 模型编辑（Model Editing）：精准修改特定参数。像直接在模型脑中植入「思想钢印」。
4. 模型合并（Model Merging）：合并不同模型的参数。一个模型会写程序，另一个会说中文——合并后两种能力都有，不需要原始训练数据。
重点整理
- LLM是下一个token预测器，但这个简单机制产生复杂行为
- 深度（层数）和长度（思考tokens）都能实现更复杂的推理
- 微调很强大但危险——可能破坏原有能力
- 领域正朝向精准介入（编辑）和模型组合（合并）发展