G2Rec: Unifying Global Graph Context and Semantic Tokenization for Generative Recommendation

Paper: 2606.20554 Authors: Ruizhong Qiu, Yinglong Xia, Dongqi Fu, Hanqing Zeng, Ren Chen, Xiangjun Fan, Hong Li, Hong Yan, Hanghang Tong Categories: cs.IR, cs.AI

The Gap

Existing generative recommendation systems tokenize items into discrete IDs to predict the next interaction. But they face two separate walls:

1. Graph-based context integration (e.g., graph serialization, Graph Neural Networks) either cannot scale to millions of nodes or only captures local neighborhoods (e.g., 2-hop in GNNs).
2. Semantic tokenization (e.g., using item descriptions) is done heuristically—no learning signal ties the token structure to user behavior patterns.

The result: the model sees either a narrow view of user co-engagement or semantically meaningful but behaviorally misaligned tokens. The paper asks: can we get both—a global, scalable graph perspective and a semantic token space learned with supervision from user behavior?

+----------------------------------------------------+
| Problem: lack of holistic + supervised context     |
|   - Graph methods: local horizon, scalability hit  |
|   - Semantic tokenization: no behavioral signal    |
+----------------------------------------------------+
         |
         v
+----------------------------------------------------+
| Assumption: unify global graph diffusion           |
| with token-level supervision to form interest      |
| prototypes that guide both token and prediction    |
+----------------------------------------------------+
         |
         v
+----------------------------------------------------+
| Method: G2Rec                                      |
|   - Build user co-engagement graph (large-scale)   |
|   - Diffuse via Personalized PageRank to get       |
|     interest prototypes per user                   |
|   - Supervise tokenization with prototypes         |
|   - Train generative model with prototype tokens   |
+----------------------------------------------------+
         |
         v
+----------------------------------------------------+
| Evidence: online deployment + 3 public datasets    |
|   - Outperforms SR+GNN, BERT4Rec, P5, etc.        |
|   - Ablation shows each component necessary        |
+----------------------------------------------------+
         |
         v
+----------------------------------------------------+
| Conclusion: global graph + supervised tokenization |
| is feasible and superior for generative rec        |
+----------------------------------------------------+

The Increment

One sentence: Before G2Rec, no generative recommendation system could simultaneously inject holistic user co-engagement context and semantically grounded token learning; after G2Rec, we have a scalable way to do both, leading to better next-item prediction.

Core Mechanism

G2Rec has four main blocks that connect in a loop:

1. User Co‑engagement Graph Construction – From user–item interaction logs, build a bipartite graph, then project it into a user–user graph (two users are connected if they co-interact with the same item). This graph is huge (billions of edges), so they sample a fixed-size neighborhood per user and use a distributed storage system (e.g., a key-value store) to serve mini-batches.

2. Graph Diffusion for Interest Prototypes – Apply Personalised PageRank (PPR) on the sampled user graph. For each user, the diffusion scores over all other users become a soft “interest prototype” vector. This vector captures not just direct neighbors but the whole reachable graph (global context) because PPR approximates a full-diffusion limit.

3. Supervised Semantic Tokenization – Each item originally has a textual description. They build a VQ‑VAE (Vector Quantised Variational Autoencoder) that encodes the description into a sequence of discrete tokens (codebook lookup). The key twist: the VQ‑VAE is trained not only to reconstruct the text, but also to make the token sequence *predictive of the user’s interest prototypes. That is, the loss includes a term that matches the token embedding to the PPR prototype.

4. Generative Model with Prototype‑aware Tokens – The final generative model (e.g., a Transformer decoder) takes the user’s history, which is now a sequence of prototype‑enhanced item tokens, and predicts the next item tokens. Because the tokens already carry global behavioral semantics, the generative model learns faster and generalises better.

Data flow:
Raw logs → user‑item interactions → user‑user graph (distributed, sampled) → PPR → interest prototype per user → item text + prototype → VQ‑VAE training → token embeds → Transformer next‑item prediction.

                 +-----------------------------+
                 | User-Item Interaction Logs  |
                 +--------------+--------------+
                                |
                                v
                 +-----------------------------+
                 | User Co-engagement Graph    |
                 | (user-user edges via common |
                 |  items; distributed storage)|
                 +--------------+--------------+
                                |
                    (sample neighborhoods)
                                v
                 +-----------------------------+
                 | Personalized PageRank       |
                 | (global diffusion scores)   |
                 +--------------+--------------+
                                |
                     (interest prototype vector)
                                |
                 +-----------------------------+
                 | Item Text Descriptions      |
                 +------+----------------------+
                        |
                        v
                 +-----------------------------+
                 | VQ-VAE with prototype loss  |
                 | (semantic tokenization +    |
                 |  behavioral alignment)      |
                 +-----------------------------+
                                |
                     (item token sequences)
                                |
                 +-----------------------------+
                 | Transformer Next-Item Pred  |
                 | (generative recommendation) |
                 +-----------------------------+

Structural Metaphor: Library System with Readership Social Graph

Imagine a giant public library. Each patron (user) checks out books (items). The library wants to predict what a patron will borrow next.

Old way: The librarian only looks at the last 3 books a patron borrowed (local sequence), and maybe the book’s genre (semantic description), but never knows that Patron A and Patron B always read the same authors.
G2Rec way: The librarian first builds a “readership social graph”: if two patrons have borrowed the same book, they get a connection. Then the librarian doesn’t just look at direct friends; they use a “reputation diffusion” algorithm (PPR) to find who are the most influential readers across the entire graph. This gives each patron a “taste profile” (interest prototype) that summarises what the whole community is reading.

Next, instead of just using the book’s title as a token (raw text), the librarian creates a set of “community‑aware labels” (supervised tokenization). For example, a fantasy novel might be tokenized not just as “fantasy”, but as “fantasy + high‑influence‑reader‑cluster‑5” because the prototype says this book is popular among a certain graph community. This label is learned in a way that helps predict the next book—it’s not just descriptive, it’s predictive.

Finally, the generative model (the librarian’s prediction engine) uses these enriched tokens to forecast. It works better because the tokens already encode the global social knowledge, not just the local history.

Key Concepts

• Personalised PageRank (PPR): A graph diffusion method that answers “if I start at user X and randomly walk, stopping with probability α, how likely am I to end at each other user?”. The resulting scores are a global, soft clustering of users based on co‑engagement. It scales because it can be approximated via random walks and used in mini‑batches.

• Vector Quantised Variational Autoencoder (VQ‑VAE): A neural network that compresses a continuous input (here, item text embeddings) into a discrete code (a token from a codebook). The codebook is a set of embedding vectors; each input is mapped to the nearest codebook vector. G2Rec adds an extra loss that forces the chosen code to also align with the PPR prototype of the current user—so the token is not just a compression of text, but a bridge between text and global user behaviour.

• Interest Prototype: A fixed‑length vector per user, derived from PPR scores, that represents the user’s position in the global co‑engagement space. It is not a ground‑truth interest label (none exist), but a relational signature learned from the graph. The model uses this prototype to supervise tokenization and also as a conditioning signal in the generative model.

Framework Shift

Before (mainstream approach):
The generative model sees only local context (e.g., last 10 items) and independent semantic tokens (from item descriptions only, no behavioral signal).

[Before]                            [After]
User History:                       User History:
+---+---+---+                       +---+---+---+
| A | B | C |  (item IDs)          |a1 |b2 |c3 | (prototype-aware tokens)
+---+---+---+                       +---+---+---+
    |  (no graph)                          |
    v                                      v
+-------------------+                +-------------------+
| Sequence Model    |                | User Co-engage    |
| (Transformer)     |                | Graph (global)    |
+-------------------+                +---------+---------+
    |                                          |
    v                                          v
Next-item ID: D                       PPR -> Prototype -> Supervised VQ-VAE
                                                          |
                                                          v
                                                  Sequence Model
                                                  |
                                                  v
                                          Next-item token: d4

One sentence: From *local‑history + independent semantic tokens to global‑graph + behaviorally‑supervised tokens, the core shift is using graph diffusion to create context that directly shapes the token vocabulary.

Expert Assessment

Problem choice: Real gap. Industrial recommendation struggles to capture long‑tail user interests without either missing global patterns (GNNs) or ignoring behavior signals (tokens). The paper targets a concrete pain point in production systems. It sits at the intersection of graph learning and sequence modeling, which is where the field is going.

Method maturity: Clever integration, not brute force. The PPR + VQ‑VAE combination is elegant: it turns a graph algorithm into a supervision signal for token learning without needing ground‑truth interest labels. Simpler alternatives? You could try to directly use item‑co‑occurrence statistics, but they lack the diffusion’s long‑range reach. No obvious overlooked baseline.

Experimental integrity: Strong. They compare against diverse baselines (SR‑GNN, BERT4Rec, P5, UniSRec) on three public datasets and report online A/B test results. Ablation studies confirm each component matters. One minor concern: the VQ‑VAE training adds complexity; a sensitivity study on codebook size would be nice, but not a red flag.

Writing quality: The paper is dense but clear. The weakest section is the graph construction details—it says “distributed sampling” but doesn’t specify the sampling algorithm or the data skew handling. Rewriting Section 3.1 with a concrete example of how they build the user–user graph from a billion‑edge bipartite graph would elevate the paper.

Verdict: Strong accept — a genuine contribution that solves a known industrial problem with a novel, well‑validated method.

Takeaways

1. Use graph diffusion as a surrogate for user interest labels. If you have an interaction graph but no explicit interest labels, PPR (or any global diffusion) gives you a soft clustering that can supervise any downstream token learner. That trick transfers to product recommendation, social network prediction, etc.

2. Supervised tokenization with a graph‑derived target: Instead of training a VQ‑VAE only on reconstruction loss, add a loss that aligns token embeddings with user interest prototypes. This turns tokens into carriers of behavioral context—adopt this for any task where you have both a graph and a sequence.

3. Scalable graph service: The paper shows you don’t need to load the whole graph into GPU memory—sample neighborhoods once and serve them as key‑value pairs. This engineering pattern is directly applicable to any large‑scale graph‑enhanced ML system.