Paper: 2603.24586 Authors: Aditya Mittal, Ryan Shar, Zichu Wu, Shyam Agarwal, Tongshuang Wu, Chris Donahue, Ameet Talwalkar, Wayne Chi, Valerie Chen Categories: cs.SE, cs.CL
The Gap
Existing LLM-as-judge evaluations test models on clean, complete prompts with clear intent. But real coding happens in messy contexts: partial information, ambiguous requirements, interactive back-and-forth. Prior work measures alignment on synthetic benchmarks (HumanEval, MBPP) where the “right answer” is obvious. No one has systematically measured how LLM judges perform in realistic developer workflows where preferences are subjective and context-dependent.
The gap: We don’t know if LLM judges actually predict what human developers prefer when context is partial and intent is fuzzy.
Problem: LLM judges evaluated on clean benchmarks
|
v
Assumption: Real coding = messy context + ambiguous intent
|
v
Method: TRACE framework tests judges in 3 realistic modalities
|
v
Evidence: Best judges underperform humans by 12-23%
|
v
Conclusion: 35 systematic biases identified across quality dimensions
The Increment
One sentence: Before this paper, we assumed LLM judges were good enough for code evaluation; after, we know they systematically misalign with human preferences on most code quality dimensions in realistic settings.
Core Mechanism
TRACE operates in three stages. First, it collects human preference data across three interaction modalities: chat-based programming (conversational code generation), IDE autocompletion (inline suggestions), and instructed code editing (targeted modifications). For each modality, developers compare code pairs and indicate preferences.
Second, TRACE feeds the same pairs to LLM judges and measures prediction accuracy against human choices. The framework tests 13 different models as judges, from GPT-4 to smaller open-source alternatives.
Third, TRACE automatically extracts rubric items—specific evaluation criteria like “code length,” “explanation verbosity,” “error handling”—and measures how much weight humans vs. judges place on each item. This reveals systematic biases: dimensions where judges consistently disagree with humans.
Input: Code pairs + partial context
|
+---> Human annotators ---> Preference labels
|
+---> LLM judges (13 models) ---> Predicted preferences
|
v
Alignment measurement: Accuracy gap (12-23%)
|
v
Rubric extraction: Weight each quality dimension
|
v
Output: 35 systematic bias sources identified
Think of TRACE as a taste test panel for code evaluation. Imagine a food critic competition where both professional critics (humans) and AI food reviewers (LLM judges) taste the same dishes under realistic conditions—not in a sterile lab, but in a busy restaurant with incomplete menus and vague descriptions. TRACE is the scoring system that not only measures who predicts diner preferences better, but also breaks down exactly why they disagree: one group cares more about presentation, another about portion size, a third about ingredient complexity. The framework maps each judge’s “palate”—their implicit rubric—and shows where AI critics systematically diverge from human taste.
Key Concepts
-
Rubric extraction: Traditional evaluation uses predefined checklists (readability, efficiency, correctness). TRACE flips this: it observes how humans and models actually make decisions, then reverse-engineers the implicit criteria they’re using. If humans consistently prefer shorter explanations but judges prefer longer ones, TRACE detects “explanation length” as a rubric item with opposite weights. It’s like watching someone shop for clothes repeatedly and inferring their style rules, rather than asking them to fill out a style questionnaire.
-
Interaction modality: Not all coding is the same. Chatting with an AI assistant (“write me a function to parse JSON”) has different context than seeing an autocomplete suggestion mid-typing or editing existing code per instructions. Each modality has different information availability, time pressure, and user expectations. TRACE tests judges across all three because a model that works well for chat might fail at autocompletion—the “right” code depends on the interaction context.
-
Alignment gap: This isn’t just accuracy—it’s the delta between how well judges predict human preferences versus how well humans predict each other. If humans agree 85% of the time and the best judge hits 62%, the alignment gap is 23 percentage points. This gap matters because it represents systematic misunderstanding, not random noise. A 23% gap means judges are confidently wrong about what developers want, which compounds when judges are