TikStance: Teaching AI to Read Political Stance in TikTok's Multimodal Conversation Trees

Paper: 2607.15240 Authors: Yazhi Zhang, Fuqiang Niu, Bowen Zhang Categories: cs.CL

The Gap

Political conversations have migrated from Twitter threads and news comment sections to short-video platforms like TikTok. But the datasets we use to train and evaluate stance detection models are stuck in the old world. Here’s the state of play:

• SemEval datasets (2016–2019): Text-only, single target per instance, no conversational context.
• VAST (2022): Scaled up targets but still text-only, no audiovisual signal, no hierarchical threading.
• Multi-target datasets: Emerging, but on Twitter/Reddit — not short-video platforms.
• Multimodal stance: Some work on memes (image + text), but nothing that captures video’s audiovisual richness alongside structured conversation trees.

The fundamental problem: TikTok comments don’t exist in isolation. A reply to a reply carries different meaning than a top-level comment. The video itself — its visuals, tone, creator — shapes stance. Yet no dataset captures all three dimensions simultaneously.

[Problem]
  Short-video political discourse is growing fast
  |
  v
[Gap]
  Existing datasets = text-only OR single-target OR flat structure
  |
  v
[Assumption]
  Multimodal + hierarchical context should improve stance detection
  |
  v
[Method]
  TikStance: 161 videos, 13,876 comments, parent-linked trees
  + multi-annotator labeling (3 annotators, re-annotation on disagreement)
  |
  v
[Evidence]
  Inter-annotator agreement: alpha = 0.743 / 0.723 / 0.722
  23.3% comments are nested replies
  Target-dependent stance distributions vary by depth
  |
  v
[Conclusion]
  Dataset enables multimodal, hierarchical, multi-target stance research

The Increment

One sentence: Before this paper, studying political stance on short-video platforms meant ignoring how comments nest inside each other, how the video itself shapes meaning, or both — now there’s a dataset that captures all three.

Core Mechanism

TikStance is a dataset, not a model — so the “mechanism” here is the data construction pipeline and annotation design. The dataset is organized around discussion units: each unit contains one host video (with metadata and audiovisual content) linked to a parent-linked comment tree. This tree structure preserves the nesting that happens naturally in TikTok’s UI — a reply to a reply is a child node, not a flat entry.

For each discussion unit, annotators label video-to-target stance (what stance does the video itself express toward Trump/Biden/Harris?) and comment-to-target stance (what stance does each comment express?). The stance scheme has three classes: Favor, Against, None. Three independent annotators label each item. When disagreement arises, the item goes through a re-annotation round. The final agreement scores — Krippendorff’s alpha around 0.72–0.74 — indicate reasonable but not perfect consensus, which is expected for subjective political content.

[Data Construction Pipeline]

  TikTok API  -->  Video Collection (161 videos)
       |              |
       |              v
       |         Comment Crawling (tree structure preserved)
       |              |
       v              v
  Metadata +    Parent-linked comment tree
  Audio/Video       |
                    v
              Annotation Layer
              |         |         |
              v         v         v
           Annotator  Annotator  Annotator
              |         |         |
              v         v         v
           [Agree?] --yes--> Final Label
              |
             no
              v
         Re-annotation round
              |
              v
         Final Label (majority vote)

  Stance classes: Favor | Against | None
  Targets: Trump | Biden | Harris
  Levels: Video-to-target, Comment-to-target

The radio station metaphor: Imagine TikTok as a political talk radio station. Each video is a broadcast — the host’s tone, body language, and visuals all convey stance before a single word is spoken. The comments section is the call-in queue, but here’s the twist: callers can respond to *other callers, not just the host. So you get chains like “I disagree with what the first caller said about Biden’s health — here’s why.” That’s the nested reply structure. TikStance captures the full picture: what the broadcast says (video stance), what each caller says (comment stance), and the conversation topology that shows who’s responding to whom. Without the tree structure, you’d just hear a jumble of voices. Without the video, you’d miss the host’s sarcasm or visual emphasis. Without multi-target labels, you couldn’t tell if someone supporting Harris was also opposing Trump or simply ignoring him.

Key Concepts

• Stance Detection: Think of it as the difference between *what someone says and what they believe. Sentiment analysis tells you if text is positive or negative. Stance detection tells you if the speaker is for or against a specific target. “This stock just crashed” is negative sentiment, but if the speaker shorted the stock, they’re actually in favor of the crash. In political discourse, stance is the real signal — sarcasm, irony, and rhetorical questions make sentiment unreliable. The three-class scheme (Favor/Against/None) is the minimum viable taxonomy; “None” captures genuine neutrality, off-topic comments, and cases where stance is genuinely ambiguous.

• Hierarchical Conversation Trees: In flat datasets, a comment like “Exactly!” is meaningless. In a tree, you know it’s a reply to “Trump’s tariff policy is disastrous” — now “Exactly!” carries clear stance information. This nesting is why 23.3% of TikStance comments are nested replies: they depend on their parent for meaning. The paper’s finding that stance distribution varies by conversation depth supports this — deeper comments have different characteristics than top-level ones.

• Multimodal Context: Text alone misses how a TikTok creator delivers a message. A video of someone reading Trump quotes with a deadpan face might be satire. A reaction video with shocked expressions signals stance without words. TikStance preserves this audiovisual signal alongside the text, enabling models that can learn to “read the room” — something impossible with text-only datasets.

Framework Shift

Before (mainstream approach):              After (this paper):

  [Single Target]                          [Multiple Targets]
        |                                        |
  [Text Only]                              [Video + Text + Metadata]
        |                                        |
  [Flat Comments]                          [Parent-linked Tree]
        |                                        |
  [Independent Labels]                     [Hierarchical Labels]
        |                                        |
  [Single Annotator or                     [3 Annotators + 
   Unverified Quality]                      Re-annotation + alpha]

From flat, text-only, single-target stance analysis to multimodal, tree-structured, multi-target analysis — the core shift is treating political discourse as it actually exists on short-video platforms rather than forcing it into a format convenient for NLP pipelines.

Expert Assessment

Problem choice: Real gap, well-motivated. Short-video platforms are where political discourse actually happens for younger demographics, and the research infrastructure hasn’t caught up. The multi-target design (Trump/Biden/Harris in the 2024 cycle) is timely and practically relevant. This isn’t manufactured — it’s genuinely underserved.

Method maturity: This is a dataset paper, not a methods paper, so “maturity” applies to the annotation protocol rather than a model. The triple-annotation with re-annotation on disagreement is standard but solid. The inter-annotator agreement (alpha ~0.72–0.74) is acceptable for subjective political stance but not spectacular — the authors could have explored *why certain targets have different agreement levels. No baseline models are trained, which limits our ability to assess whether the dataset’s structure actually helps downstream tasks.

Experimental integrity: The descriptive statistics are honest — they show target-dependent variation in stance distributions and the significance of nested replies. But there’s a notable gap: no baseline experiments. The paper claims the dataset “supports research in” various areas but doesn’t demonstrate it by training even a simple model. We’re asked to take on faith that the multimodal and hierarchical structure matters. The 161-video scale is small, though the 13,876 comments partially compensate.

Writing quality: Clear and functional but uninspired. The related work section is thin — a deeper comparison with existing multimodal stance datasets would strengthen the motivation. The descriptive analysis section could be expanded; showing *how stance shifts between parent and child comments would make the case for hierarchical structure more compelling than aggregate statistics alone. The abstract is well-written.

Verdict: weak accept — the gap is real and the dataset fills it, but the absence of any downstream experiments weakens the contribution. A simple baseline showing that hierarchical context improves stance accuracy would have pushed this to strong accept.

Takeaways

For dataset builders: The parent-linked tree annotation design is transferable — anyone building conversational datasets for platforms with threaded comments (Reddit, YouTube, even email chains) should consider preserving the hierarchy rather than flattening it.

For stance detection researchers: If you’re working on political stance, this dataset is a new benchmark to test whether your models can handle multimodal input and hierarchical context. The multi-target design also lets you study whether stance toward one political figure predicts stance toward another.

For practitioners: The annotation protocol — triple labeling with re-annotation on disagreement and Krippendorff’s alpha reporting — is a template for building reliable datasets in subjective domains. The alpha values (~0.72) suggest that expecting human agreement beyond this range on political stance may be unrealistic, which has implications for model evaluation metrics.