KoRe: Compressing Knowledge Graphs into Discrete Tokens for LLMs

Paper: 2605.20170 Authors: Davide Cavicchini, Fausto Giunchiglia, Jacopo Staiano Categories: cs.CL

The Gap

LLMs encode knowledge in their parameters — opaque, hard to update, prone to hallucinations. Knowledge Graphs (KGs) offer structured, editable knowledge but current integration methods (RAG, fine-tuning) either flood the context window with verbose triples or require expensive retraining. The field has two incompatible paradigms: parametric (LLMs) vs symbolic (KGs), with no efficient bridge that preserves both interpretability and computational economy.

Problem: LLM knowledge is opaque + KG integration is expensive
   |
   v
Assumption: KG structure can be compressed without losing semantics
   |
   v
Method: Encode 1-hop subgraphs -> discrete tokens -> inject into LLM
   |
   v
Evidence: 10x token reduction + competitive accuracy on 3 benchmarks
   |
   v
Conclusion: Compact KG representations ground LLMs efficiently

The Increment

One sentence: Before — choose between parametric opacity or context-window bloat; after — inject structured knowledge as compact discrete tokens without retraining.

Core Mechanism

KoRe operates in three stages. First, given an entity mention in text, it extracts the relevant 1-hop subgraph from a KG (the entity plus all directly connected neighbors and relations). Second, it encodes this subgraph into a fixed-length discrete representation using a learned encoder that maps graph structure to a sequence of token IDs from a learned codebook. Third, these “knowledge tokens” are prepended to the input text and fed into a frozen LLM backbone.

The encoder is trained separately on a KG corpus to learn compact representations that preserve relational structure. During inference, the LLM sees both the original text and the compressed KG context, but the KG arrives as a handful of discrete tokens rather than hundreds of verbose triples. The LLM’s parameters remain frozen — no fine-tuning required.

Input text: "Einstein won the Nobel Prize"
     |
     v
Extract 1-hop subgraph from KG:
  Einstein --won--> Nobel Prize
  Einstein --born--> 1879
  Einstein --field--> Physics
     |
     v
Encode subgraph -> [K1, K2, K3] (3 discrete tokens)
     |
     v
Prepend to text: [K1, K2, K3] + "Einstein won the Nobel Prize"
     |
     v
Feed to frozen LLM -> prediction

Think of KoRe as a compression algorithm for knowledge. Imagine you’re packing for a trip: instead of bringing entire books (verbose KG triples), you bring index cards with key facts (discrete tokens). The encoder is your packing strategy — it learns which facts to keep and how to arrange them compactly. The LLM is your brain at the destination — it reads the index cards alongside the main text and makes decisions. The codebook is your set of standardized card templates, learned from many packing sessions. You never repack your brain (no LLM retraining), just hand it better-organized cards.

Key Concepts

• 1-hop subgraph: For a given entity, this is the immediate neighborhood in the knowledge graph — the entity itself, all entities directly connected to it, and the relations between them. If “Einstein” is the entity, a 1-hop subgraph includes “Nobel Prize” (connected by “won”), “Physics” (connected by “field”), “1879” (connected by “born”), but not “Relativity” if it’s two hops away through “Physics”. This local slice captures the most relevant context without exponential explosion.

• Discrete token encoding: Instead of feeding raw graph triples as text (“Einstein won Nobel Prize. Einstein born 1879…”), the method learns a vocabulary of abstract tokens (like K1, K2, K3) where each token represents a compressed pattern in the graph structure. A learned encoder maps subgraphs to sequences of these tokens. The LLM treats them like any other token in its vocabulary, but they carry dense relational information. It’s like replacing “the quick brown fox” with a single symbol that the model has learned means exactly that phrase.

• Frozen backbone: The LLM’s weights are never updated during KoRe integration. This is crucial — it means you can plug KoRe into any existing LLM without retraining costs. The knowledge tokens are simply prepended to the input, and the LLM processes them through its existing attention mechanisms. The model learns to “read” these tokens during its original pretraining (or through the encoder’s training), but the LLM itself remains unchanged.

Framework Shift

Before (RAG / Fine-tuning):          After (KoRe):

Text input                           Text input
    |                                    |
    v                                    v
Retrieve KG triples                  Extract 1-hop subgraph
    |                                    |
    v                                    v
"Einstein won Nobel Prize.           Encode -> [K1, K2, K3]
 Einstein born 1879.                     |
 Einstein field Physics..."              v
 (100+ tokens)                       Prepend (3 tokens)
    |                                    |
    v                                    v
Stuff into context window            Feed to frozen LLM
    |                                    |
    v                                    v
LLM (maybe fine-tuned)               LLM (frozen)

From verbose symbolic injection to compact learned compression, the core shift is trading interpretability for efficiency while preserving grounding.

Expert Assessment

Problem choice: Real gap. The parametric vs symbolic divide has plagued the field since BERT met Freebase. RAG is clunky, fine-tuning is expensive, and hallucinations remain unsolved. This sits squarely in the “grounding LLMs” trajectory, which is hot but crowded.

Method maturity: Clever compression idea, but the execution feels undercooked. Why 1-hop only? The paper doesn’t justify this hard boundary — it’s likely a computational convenience dressed as a design choice. The encoder architecture (details sparse in the abstract) probably matters more than admitted. Also, “learned codebook” raises questions: how big? how trained? what if the KG schema changes?

Experimental integrity: Three benchmarks is decent but not exhaustive. The 10x token reduction is impressive, but “competitive performance” is weasel-wording — does it match RAG or fall short? Baselines matter: if they’re comparing against naive triple concatenation, that’s a strawman. Need to see ablations (what if you use 2-hops? what if you don’t compress?) and failure cases (when does compression lose critical info?).

Writing quality: The abstract oversells. “Consistently proven beneficial” (about KGs) is vague. “Compact discrete knowledge tokens” sounds like jargon invented to sound novel. The methodology section (not in abstract) likely buries key details. If I were reviewing, I’d ask: show me the encoder architecture, show me failure modes, show me what happens when the KG is incomplete.

Verdict: weak accept — Solid incremental contribution with practical value, but needs deeper analysis of when and why compression works.

Takeaways

Steal the compression framing: If you’re integrating any structured data (databases, ontologies, schemas) into LLMs, ask: can I learn a compact encoding instead of dumping raw records? The 1-hop subgraph idea generalizes — local context often suffices, and you can train a small encoder to compress it.

Discrete tokens as interface: This paper shows you can inject non-textual information into LLMs without fine-tuning by treating it as a learned vocabulary. Applicable beyond KGs: imagine discrete tokens for code ASTs, protein structures, or UI state.

Watch the tradeoff: Compression always loses information. The paper doesn’t deeply explore what gets lost (nuanced relations? rare entities?). Before adopting this approach, profile your task: if you need exhaustive reasoning over multi-hop paths, compression might hurt. If you need quick grounding on common entities, it’s a win.