Hung-yi Lee ML 2025 Lecture 10: Model Editing - AI Microsurgery

Notes from Professor Hung-yi Lee’s (李宏毅) course “Machine Learning in the Era of Generative AI (2025)” at National Taiwan University.

What is Model Editing?

Model Editing aims to surgically modify specific knowledge in a language model without:

• Full retraining
• Affecting unrelated knowledge
• Breaking the model’s other capabilities

Think of it as “AI microsurgery” - precisely editing neural network parameters to change specific facts.

The Three Pillars of Model Editing

1. Reliability

The edit must actually work. If you want to change “The Space Needle is in Seattle” to “The Space Needle is in Taipei”, the model must output “Taipei” when asked.

2. Locality

Unrelated knowledge should NOT be affected. Editing the Space Needle’s location shouldn’t change:

• What the Eiffel Tower’s location is
• Who the US President is
• Any other unrelated facts

3. Generalization

The edit should work for paraphrased questions:

• “Where is the Space Needle?” → Taipei
• “The Space Needle is located in…” → Taipei
• “What city contains the Space Needle?” → Taipei

Why Not Just Fine-tune?

Standard fine-tuning has problems for knowledge editing:

• Often breaks other model capabilities
• Requires preparing “locality” data (things you don’t want changed)
• Expensive for single fact changes
• Model may not believe the new information

Method 1: In-Context Knowledge Editing (IKE)

No parameter changes required!

Simply provide the new knowledge in the prompt:

New information: The current US President is Trump.
Question: Who is the US President?

Pros: Simple, no training needed Cons: Uses context window, model may not always follow

Method 2: ROME (Rank-One Model Editing)

A surgical approach that directly modifies model parameters.

Key Insight

Knowledge in Transformers is stored in feed-forward network layers. ROME identifies WHERE knowledge is stored and modifies just that part.

The Process

Step 1: Locate the knowledge

• Find which layer/position stores the target knowledge
• Use causal tracing: mask input tokens and see which positions affect output

Step 2: Compute the edit The goal: Make W_head × k* = v*

Where:

• W_head is the weight matrix to edit
• k* is the key (input representation)
• v* is the desired output representation

The Math (Simplified):

W_head = W + (v* - W×k*) × k*ᵀ × (C⁻¹)

This is a rank-one update (hence the name) that:

• Changes output for k* to v*
• Minimally affects other inputs

Why Rank-One?

• Minimal change to the weight matrix
• Preserves other knowledge
• Has closed-form solution (no gradient descent needed!)

Method 3: Hypernetworks (MEND)

Train a neural network to edit other neural networks!

The Concept

Instead of manually computing edits:

1. Train an “editor model” on many editing examples
2. Editor learns to output parameter changes
3. Apply editor to new editing tasks

Training Process

1. Prepare editing examples (input, old output, new output)
2. Train editor to produce parameter deltas
3. Editor learns patterns of successful edits

Advantage: Once trained, can edit quickly without optimization Disadvantage: Requires training the editor first

The “Three-Body Problem” Analogy

Professor Lee uses the “Thought Seal” (思想钢印) from the Three-Body Problem novel:

In the story, a scientist invents a way to directly edit human beliefs by modifying neurons. Model Editing is similar - we’re directly editing the “beliefs” stored in neural network parameters.

Practical Considerations

When to Use Model Editing?

• Updating outdated facts (e.g., current president)
• Correcting specific errors
• Removing sensitive information

Limitations

• Editing too many facts can still break the model
• Generalization is hard to guarantee
• Some knowledge is distributed across many parameters

Key Takeaways

1. Model Editing ≠ Fine-tuning: It’s surgical, not broad training

2. Three criteria matter: Reliability, Locality, Generalization

3. ROME is elegant: Closed-form solution, rank-one update

4. Knowledge has location: Feed-forward layers store factual associations

5. Trade-offs exist: More edits = higher risk of breaking things