Paper: 2604.21852 Authors: Research Team Categories: cs.CL, cs.AI

Problem

Current event extraction benchmarks are limited:

  • Focus on narrow domains (news, finance)
  • Incomplete coverage of event elements
  • Lack of Why and How questions
  • Small scale, limited diversity

EVENT5Ws Benchmark

Comprehensive open-domain event extraction:

5W Framework

  1. Who: Participants in the event
  2. What: Action or occurrence
  3. When: Temporal information
  4. Where: Spatial/locational context
  5. Why: Motivation and cause (often missing!)

Dataset Characteristics

PropertyValue
Event Types200+
Documents50,000+
Languages10+
DomainsNews, Social, Scientific, Legal

Evaluation Metrics

  • Element extraction (precision, recall, F1)
  • Event coreference across documents
  • Temporal reasoning accuracy
  • Causal chain completion

Key Findings

State-of-the-art models struggle with:

  • Why extraction: 45% accuracy vs 82% for What
  • Coreference: 68% vs 89% for entity linking
  • Cross-document events: 52% vs 79% for single-doc

Takeaways

  • Open-domain event extraction requires broader coverage
  • Why and How are under-explored but crucial
  • Cross-document event linking is challenging
  • Need better models for causal and temporal reasoning

论文: 2604.21852 作者: 研究团队 分类: cs.CL, cs.AI

问题

当前事件抽取基准测试有限：

  • 专注于狭窄领域（新闻、金融）
  • 事件元素覆盖不完整
  • 缺少Why和How问题
  • 规模小，多样性有限

EVENT5Ws基准

综合开放域事件抽取：

5W框架

  1. Who：事件参与者
  2. What：动作或发生
  3. When：时间信息
  4. Where：空间/位置上下文
  5. Why：动机和原因（经常缺失！）

数据集特征

属性值
事件类型200+
文档50,000+
语言10+
领域新闻、社交、科学、法律

评估指标

  • 元素抽取（精确率、召回率、F1）
  • 跨文档事件共指
  • 时间推理准确率
  • 因果链补全

关键发现

最先进模型在以下方面存在困难：

  • Why抽取：45%准确率 vs What的82%
  • 共指：68% vs 实体链接的89%
  • 跨文档事件：52% vs 单文档的79%

要点总结

  • 开放域事件抽取需要更广泛覆盖
  • Why和How探索不足但至关重要
  • 跨文档事件链接具有挑战性
  • 需要更好的因果和时间推理模型