DyTopo: Dynamic Topology Routing for Multi-Agent Reasoning via Semantic Matching

Paper: 2602.06039 Categories: cs.AI, cs.CL, cs.MA

Abstract

Multi-agent systems powered by Large Language Models (LLMs) have shown remarkable capabilities in complex reasoning tasks. However, existing approaches typically rely on fixed communication topologies (e.g., chain, tree, or fully-connected graphs), which fail to adapt to the varying semantic requirements of different reasoning steps. We propose DyTopo (Dynamic Topology), a framework that dynamically constructs agent communication topologies based on semantic matching between agent expertise and task requirements.

Key Contributions

• Dynamic Topology Construction: Adaptively builds communication graphs based on semantic relevance rather than predefined static structures
• Semantic Matching Module: Learns to match agent capabilities with sub-task requirements using embedding-based similarity
• Routing Mechanism: Efficiently routes information between agents based on semantic affinity scores
• Scalability: Maintains efficiency as the number of agents increases through sparse connectivity

The Problem with Static Topologies

Current multi-agent reasoning systems use fixed communication patterns:

• Chain topology: Sequential information flow, limited parallelism
• Tree topology: Hierarchical but rigid structure
• Fully-connected: $O(n^2)$ communication overhead, information overload
• Star topology: Central bottleneck, single point of failure

These static structures cannot adapt to:

1. Varying complexity across reasoning steps
2. Different expertise requirements for sub-problems
3. Dynamic information dependencies

DyTopo Framework

Architecture Overview

The framework consists of three main components:

1. Agent Pool: Specialized agents with different expertise domains
2. Semantic Router: Dynamically determines communication paths
3. Topology Constructor: Builds task-specific communication graphs

Semantic Matching

For each reasoning step, compute semantic affinity between agents:

$S_{ij} = \frac{\mathbf{e}_i^T \mathbf{q}_j}{\|\mathbf{e}_i\| \|\mathbf{q}_j\|}$

where $\mathbf{e}_i$ is the expertise embedding of agent $i$ and $\mathbf{q}_j$ is the query embedding for sub-task $j$.

Dynamic Routing

The routing decision uses a learned threshold:

$A_{ij} = \mathbb{1}[S_{ij} > \tau + \sigma(\mathbf{W}[\mathbf{e}_i; \mathbf{q}_j])]$

where $\tau$ is a base threshold and $\sigma(\cdot)$ is a learned adaptive component.

Topology Construction

The communication graph $G_t$ at step $t$ is constructed as:

$G_t = (V, E_t), \quad E_t = \{(i, j) : A_{ij} = 1\}$

This creates sparse, task-relevant connections rather than dense all-to-all communication.

Methodology

Training Objective

DyTopo is trained end-to-end with a composite loss:

$\mathcal{L} = \mathcal{L}_{task} + \lambda_1 \mathcal{L}_{sparse} + \lambda_2 \mathcal{L}_{coverage}$

• $\mathcal{L}_{task}$: Task-specific reasoning accuracy
• $\mathcal{L}_{sparse}$: Encourages sparse connectivity for efficiency
• $\mathcal{L}_{coverage}$: Ensures all relevant expertise is accessible

Agent Specialization

Each agent maintains:

• Domain-specific knowledge embeddings
• Reasoning style preferences
• Historical performance on sub-task types

Results

Benchmark Performance

DyTopo demonstrates consistent improvements across reasoning benchmarks:

Method	GSM8K	MATH	HotpotQA	StrategyQA
Chain-of-Agents	78.2	42.1	65.3	71.8
Tree-of-Agents	79.5	43.8	67.1	73.2
Fully-Connected	80.1	44.2	66.8	72.9
DyTopo	82.7	47.3	70.5	76.4

Efficiency Analysis

Communication overhead comparison (relative to fully-connected):

Topology	Messages	Latency	Accuracy
Fully-Connected	100%	100%	Baseline
Chain	15%	180%	-2.1%
Tree	25%	120%	-1.3%
DyTopo	35%	85%	+2.6%

DyTopo achieves better accuracy with fewer messages and lower latency.

Ablation Studies

Key findings from ablations:

• Semantic matching contributes +1.8% accuracy over random routing
• Adaptive threshold improves +0.9% over fixed threshold
• Sparsity regularization reduces messages by 40% with minimal accuracy loss

Why This Matters

The key insight: optimal agent communication depends on task semantics. By learning to route information based on semantic relevance:

1. Reduces irrelevant information exchange
2. Enables parallel processing of independent sub-tasks
3. Scales efficiently with agent pool size
4. Adapts to varying task complexity

Takeaways

1. Dynamic over static: Adaptive topologies outperform fixed communication structures for multi-agent reasoning
2. Semantic routing: Matching agent expertise to task requirements improves both efficiency and accuracy
3. Sparse is sufficient: Dense all-to-all communication is unnecessary and often harmful
4. Scalability: Semantic-based routing maintains efficiency as systems grow
5. End-to-end learning: Joint optimization of routing and reasoning yields best results