DEBISS: Building a Multi-Layered Spoken Debate Corpus for Real Argumentation

Paper: 2603.05459 Authors: Klaywert Danillo Ferreira de Souza, David Eduardo Pereira, Cláudio E. C. Campelo, Larissa Lucena Vasconcelos Categories: cs.CL, cs.DB

The Gap

Existing debate corpora fall into two camps: written debates from online forums (Reddit CMV, Kialo) that miss prosodic cues and spontaneous speech patterns, or parliamentary/competitive debates (Hansard, IQ2) that follow rigid formats unlike everyday argumentation. The few spoken debate datasets focus on multi-party interactions, making speaker attribution and argument flow analysis messy. None bridge the gap between naturalistic spoken language and structured argument analysis with quality metrics.

Problem: Debate corpora either written OR spoken, never both with full NLP stack
   |
   v
Assumption: Individual debates can be semi-structured to balance naturalness + analyzability
   |
   v
Method: Collect spoken monologues -> Multi-layer annotation (ASR + diarization + argument mining + quality)
   |
   v
Evidence: DEBISS corpus with 4 annotation layers enables multiple downstream tasks
   |
   v
Conclusion: Spoken + structured + quality-assessed debates fill the corpus gap

The Increment

One sentence: Before DEBISS, you chose between written debates with argument structure or spoken debates with prosody—now you get both plus quality assessment in individual semi-structured formats.

Core Mechanism

DEBISS isn’t a new algorithm—it’s a corpus construction methodology. The authors collected individual spoken debates where one person argues a position in a semi-structured format (not free-form rambling, not rigid tournament rules). Each debate recording flows through four annotation pipelines in sequence.

First, automatic speech recognition converts audio to text transcripts. Second, speaker diarization segments identify when the debater speaks versus silence or other speakers. Third, argument mining annotators label claims, premises, and argumentative relations following established schemes. Fourth, quality assessors rate debater performance on dimensions like coherence, persuasiveness, and logical soundness.

Raw Audio Recording
   |
   v
[ASR Engine] --> Transcript Text
   |
   v
[Diarization] --> Speaker Segments (who spoke when)
   |
   v
[Argument Mining] --> Claims + Premises + Relations
   |
   v
[Quality Assessment] --> Performance Scores
   |
   v
DEBISS Entry (4-layer annotation)

Think of DEBISS like preparing a musical score from a live jazz performance. The raw audio is the performance itself—spontaneous, with all the timing and emotional inflection. ASR is like transcribing the notes onto staff paper. Diarization marks which instrument plays when (though here it’s mostly one “instrument”—the debater). Argument mining is the harmonic analysis—identifying chord progressions, motifs, and how musical phrases relate. Quality assessment is the critic’s review—rating technique, creativity, and overall impact. Each layer adds interpretive structure without losing the original performance’s richness.

Key Concepts

• Semi-structured debate format: Imagine a podcast interview versus a scripted speech versus a shouting match. Semi-structured sits in the middle—the debater knows the topic and has time to prepare, but delivers arguments spontaneously without reading a script. There’s a loose framework (opening position, supporting points, conclusion) but no rigid turn-taking or time limits enforced mid-speech. This captures how people actually argue when they’re trying to be coherent but not performing for judges. Example: You’re explaining to your family why you’re switching careers—you’ve thought it through, you have reasons, but you’re not reading from notes.

• Argument mining annotation: This isn’t sentiment analysis (“positive/negative”) or topic modeling (“this paragraph is about economics”). It’s identifying the logical skeleton of reasoning. A claim is a statement the debater wants you to believe (“Remote work increases productivity”). A premise is evidence or reasoning supporting that claim (“Studies show 25% fewer distractions at home”). Relations show how premises support claims or how claims attack other claims. Think of it like diagramming sentences in grammar class, but for logic instead of syntax. The goal: make implicit reasoning explicit so machines can trace argument flow.

• Multi-layer annotation: Each debate isn’t just transcribed or just argument-mined—it has all four layers simultaneously. This is like having a movie with the film, the screenplay, the director’s commentary, and the critic’s review all aligned. Researchers can study how prosodic features (from audio) correlate with argument strength (from mining) and persuasiveness (from quality scores). Or train models that predict quality from transcript features. The layers aren’t independent datasets—they’re synchronized views of the same debate, enabling cross-layer analysis impossible with single-annotation corpora.

Framework Shift

Before (mainstream approach):        After (this paper):

Written debates (Reddit/Kialo):      Spoken individual debates:
  Text only                            Audio + Transcript
  Argument structure                   Argument structure
  No prosody                           Prosody preserved
  Multi-party chaos                    Single speaker clarity
                                       Quality metrics
     OR
                                   All layers synchronized:
Spoken debates (Parliament):         [Audio Layer]
  Audio + Transcript                      |
  No argument mining                 [Text Layer]
  Rigid format                            |
  No quality metrics               [Argument Layer]
                                          |
                                   [Quality Layer]

From fragmented single-purpose corpora to integrated multi-layer debate datasets, the core shift is treating spoken argumentation as a phenomenon requiring simultaneous acoustic, linguistic, logical, and evaluative analysis.

Expert Assessment

Problem choice: This is a real infrastructure gap, not manufactured. The field has plenty of written argument corpora and plenty of speech corpora, but almost nothing that combines both with argument structure and quality labels. It’s unglamorous work—corpus building doesn’t win best paper awards—but it’s foundational. The timing is right as argument mining matures beyond written text.

Method maturity: There’s no methodological innovation here, which is fine—this is a resource paper. The authors apply existing tools (ASR, diarization, argument annotation schemes) in a new combination. The semi-structured format is a reasonable design choice, though they don’t justify why this format over others. The annotation process seems standard but details are sparse—inter-annotator agreement numbers would strengthen credibility.

Experimental integrity: This isn’t an experimental paper, so there are no baselines or ablations to scrutinize. The red flag is the abstract’s vagueness—“broad range of NLP task annotations” sounds impressive but what’s actually in the corpus? How many debates? What languages? What topics? The paper needs to show corpus statistics, annotation guidelines, and sample entries. Without seeing the data, it’s hard to assess quality.

Writing quality: The abstract reads like a grant proposal, not a technical paper. “The process of debating is essential in our daily lives” is filler. The authors should cut the motivation paragraph and immediately state: “We present DEBISS, a corpus of X debates with Y hours of audio, annotated for speech recognition, diarization, argument structure, and quality.” The related work section (not in this abstract) likely needs expansion to position this against existing debate corpora more precisely.

Verdict: weak accept — Addresses a real gap with solid execution, but the paper needs to show its work with detailed corpus statistics and annotation protocols before it’s truly useful.

Takeaways

If you’re building a specialized corpus in any domain, steal the multi-layer annotation strategy. Don’t just collect data for one task—think about what complementary annotations would enable cross-layer research. For speech + NLP work, the semi-structured format is clever: structured enough to analyze systematically, natural enough to capture real language phenomena. The quality assessment layer is underrated—most corpora stop at descriptive annotations, but adding evaluative judgments (even subjective ones with inter-rater reliability) opens up new research questions about what makes communication effective, not just what it contains.