VeoPlace: Teaching Vision-Language Models to Place Chips Like Human Designers

Paper: 2603.28733 Authors: Ikechukwu Uchendu, Swati Goel, Karly Hou, Ebrahim Songhori, Kuang-Huei Lee, Joe Wenjie Jiang, Vijay Janapa Reddi, Vincent Zhuang Categories: cs.LG

The Gap

Chip floorplanning—arranging macro blocks on a silicon canvas—is fundamentally a spatial reasoning task. Human experts excel at it because they see the chip layout and reason about component relationships visually. Recent ML approaches (reinforcement learning like MaskPlace, graph neural networks) treat it as an abstract optimization problem, encoding the chip as graphs or sequences. They learn from scratch, ignoring that foundation models already possess strong visual reasoning capabilities. The gap: no one has asked whether VLMs, trained on billions of images, can understand chip layouts as visual problems without domain-specific training.

Problem: ML placers ignore spatial reasoning
   |
   v
Observation: Humans use visual intuition
   |
   v
Hypothesis: VLMs have visual reasoning built-in
   |
   v
Method: VLM proposes regions -> Base placer executes -> Evolve proposals
   |
   v
Evidence: 32% wirelength reduction, generalizes across placers
   |
   v
Conclusion: Visual reasoning transfers to chip design

The Increment

One sentence: Before—ML placers learned chip placement from scratch as abstract optimization; after—off-the-shelf VLMs guide placement through visual reasoning without any training.

Core Mechanism

VeoPlace has three components working in a loop. First, a VLM (GPT-4V or Gemini) receives a visual rendering of the current chip layout—macros drawn as colored rectangles on a canvas. The VLM proposes which macro to place next and suggests a subregion of the canvas (e.g., “place macro M3 in the top-left quadrant”). Second, a base placer (could be any existing placer like MaskPlace or DREAMPlace) executes this constrained placement—it only searches within the VLM’s suggested region. Third, an evolutionary optimizer maintains a population of VLM prompts, mutates them (changing phrasing, adding constraints), evaluates resulting placements, and keeps the best-performing prompts for the next generation.

   [Chip Layout Image]
          |
          v
   +-------------+
   |    VLM      |  "Place M3 in top-left,
   | (GPT-4V)    |   near power grid"
   +-------------+
          |
          v
   [Region Constraint: x<0.5, y<0.5]
          |
          v
   +-------------+
   | Base Placer |  Searches only in
   | (MaskPlace) |  constrained region
   +-------------+
          |
          v
   [New Layout + Wirelength Score]
          |
          v
   +------------------+
   | Evolutionary     |  Mutate prompts,
   | Optimizer (CMA)  |  keep best ones
   +------------------+
          |
          v
   [Next Generation Prompts] ---> Loop back to VLM

Think of VeoPlace as a design studio with three roles. The VLM is the creative director who looks at the current design mockup and sketches rough zones (“put the memory blocks here, logic there”). The base placer is the junior designer who takes those sketches and works out exact coordinates within the suggested zones. The evolutionary optimizer is the studio manager who reviews all the creative director’s sketches over time, notices which phrasing leads to better designs (“when we say ‘cluster near power’ we get 10% better results”), and coaches the creative director to use more effective language. The key insight: the creative director doesn’t need chip design training—their general visual reasoning (learned from millions of images) transfers directly because chip layouts are just spatial arrangements.

Key Concepts

• Vision-Language Model as Spatial Reasoner: VLMs like GPT-4V were trained on diverse images with captions—photos, diagrams, maps, floor plans. They learned to parse spatial relationships: “the red object is left of the blue one,” “these items cluster in the corner.” A chip layout is just another spatial diagram. When you show a VLM a chip canvas with colored rectangles (macros) and ask “where should this new block go?”, it applies the same spatial reasoning it uses for arranging furniture in a room photo. No chip-specific training needed—the visual reasoning is already there. The paper’s bet: spatial reasoning is domain-agnostic.

• Evolutionary Prompt Optimization: The VLM’s suggestions depend heavily on how you phrase the prompt. “Place macro M3” might yield different results than “Place M3 near the power grid to minimize wire length.” But which phrasing works best? VeoPlace treats prompts as genes in an evolutionary algorithm. Start with a population of prompt variations. Evaluate each by running the full placement pipeline and measuring wirelength. Mutate the best prompts (change words, add constraints, swap phrasing). Repeat for generations. This searches the space of natural language instructions to find phrasings that guide the VLM toward better placements. It’s like A/B testing prompts, but with genetic algorithms instead of manual trials.

• Constrained Placement as Hierarchical Decomposition: Instead of placing all macros at once (a massive search space), VeoPlace decomposes the problem. The VLM proposes a subregion for one macro at a time. The base placer only searches within that subregion—drastically reducing the search space for each decision. This is hierarchical: high-level spatial reasoning (VLM) constrains low-level optimization (placer). It’s like solving a jigsaw puzzle by first sorting pieces into corner/edge/middle groups (high-level), then fitting pieces within each group (low-level). The constraint makes each sub-problem tractable.

Framework Shift

Before (MaskPlace, GNN placers):        After (VeoPlace):

  [Graph/Sequence Encoding]               [Visual Chip Layout]
           |                                       |
           v                                       v
  +-------------------+                   +--------------+
  | Train RL/GNN      |                   | VLM (GPT-4V) |
  | from scratch      |                   | zero-shot    |
  | on chip data      |                   +--------------+
  +-------------------+                           |
           |                              [Region Proposal]
           v                                       |
  [Learned Policy]                                 v
           |                              +----------------+
           v                              | Base Placer    |
  [Place All Macros]                      | (constrained)  |
                                          +----------------+
                                                  |
                                                  v
                                          [Evolve Prompts]

Abstract optimization                    Visual reasoning
Learn from scratch                       Transfer from foundation model
Monolithic placement                     Hierarchical decomposition

From training domain-specific models to leveraging pre-trained visual reasoning, the core shift is treating chip layout as a visual problem that foundation models already understand.

Expert Assessment

Problem choice: Real gap. The field has been stuck in a local optimum—throwing more compute at training specialized models while ignoring that foundation models have relevant capabilities. The observation that human designers use visual reasoning is obvious in hindsight but genuinely unexplored in ML-based EDA. This sits at the intersection of two trends: foundation models eating specialized tasks, and EDA desperately needing better automation.

Method maturity: Clever insight with straightforward execution. The evolutionary prompt optimization is a bit brute-force (CMA-ES over prompt space is expensive), but it works. A simpler approach might be gradient-based prompt tuning or few-shot learning with good examples, but the authors chose robustness over elegance. The real contribution is the observation that VLMs can do this at all—the engineering around it is competent but not groundbreaking.

Experimental integrity: Baselines are fair (MaskPlace is state-of-the-art learning-based), and the generalization test to DREAMPlace (analytical placer) is strong evidence. The 32% wirelength reduction is impressive, but I’d want to see compute cost comparisons—evolutionary search isn’t free. One red flag: no ablation on VLM choice beyond GPT-4V vs Gemini. What about open-source VLMs? The paper hints at “strong visual reasoning” being key but doesn’t isolate which VLM capabilities matter. Also, the evolutionary optimization section is thin—how many generations? How sensitive to hyperparameters?

Writing quality: The intro and related work are solid. The method section rushes through evolutionary optimization details—this is where I’d want pseudocode and hyperparameter tables. The results section is thorough but could use failure case analysis. Which benchmarks does VeoPlace struggle with, and why? The discussion hand-waves about “future work” without addressing obvious questions: Does this scale to modern chips with thousands of macros? What about timing constraints beyond wirelength?

Verdict: Weak accept—the core idea is novel and results are strong, but the paper feels like an early-stage exploration rather than a mature system. It opens a door (VLMs for EDA) but doesn’t fully walk through it.

Takeaways

Practitioners can steal the hierarchical decomposition pattern: use a high-level reasoner (doesn’t have to be a VLM—could be a heuristic, a coarse model, or human input) to propose constraints, then let a specialized optimizer work within those constraints. This reduces search space and makes optimization tractable. The specific technique: evolutionary prompt optimization is broadly applicable—if you’re using LLMs/VLMs in a pipeline and prompt engineering matters, treat prompts as parameters to optimize rather than hand-tuning them. Finally, the meta-lesson: before training a specialized model, check if a foundation model already has the capability you need. Visual reasoning, spatial understanding, and common-sense physics are increasingly “free” in large VLMs—exploit that before reinventing the wheel.