Paper: 2604.21856 Authors: Research Team Categories: cs.CV, cs.LG
Problem
Video understanding models struggle with temporal dynamics:
- Most models treat time as fixed, missing speed semantics
- Cannot generate slow-motion or accelerate videos
- Temporal super-resolution is underexplored
- Self-supervised learning from videos is challenging
Self-Supervised Temporal Learning
A unified framework for learning temporal flow:
Key Capabilities
-
Speed-Conditioned Generation
- Input: video + target speed
- Output: video at specified speed
- Learn inherent motion dynamics
-
Temporal Super-Resolution
- Interpolate between frames
- Generate high-FPS from low-FPS input
- Preserve semantic consistency
-
Slow-Motion Synthesis
- Generate intermediate frames
- Maintain action coherence
- Handle complex motions
Training Objective
Self-supervised objective using temporal consistency:
- Forward prediction: predict future frames
- Backward prediction: reconstruct past frames
- Speed prediction: predict video speed from content
Results
| Capability | Previous SOTA | Seeing Fast & Slow |
|---|---|---|
| Speed Control Accuracy | 67% | 89% |
| Temporal Super-Res | 72% FID | 68% FID |
| Slow-Motion Quality | 2.1 (MOS) | 4.3 (MOS) |
Takeaways
- Self-supervised learning can capture temporal semantics
- Speed is a natural supervisory signal for video models
- Temporal understanding enables creative video manipulation
- Unified framework beats task-specific approaches
论文: 2604.21856 作者: 研究团队 分类: cs.CV, cs.LG
问题
视频理解模型在时间动态方面存在困难：
- 大多数模型将时间视为固定的，错过速度语义
- 无法生成慢动作或加速视频
- 时间超分辨率探索不足
- 视频自我监督学习具有挑战性
自我监督时间学习
学习时间流的统一框架：
关键能力
-
速度条件生成
- 输入：视频 + 目标速度
- 输出：指定速度的视频
- 学习固有运动动态
-
时间超分辨率
- 在帧之间插值
- 从低FPS输入生成高FPS
- 保持语义一致性
-
慢动作合成
- 生成中间帧
- 保持动作连贯性
- 处理复杂运动
训练目标
使用时间一致性的自我监督目标：
- 前向预测：预测未来帧
- 后向预测：重建过去帧
- 速度预测：从内容预测视频速度
实验结果
| 能力 | 之前SOTA | 快慢兼察 |
|---|---|---|
| 速度控制准确率 | 67% | 89% |
| 时间超分辨率 | 72% FID | 68% FID |
| 慢动作质量 | 2.1 (MOS) | 4.3 (MOS) |
要点总结
- 自我监督学习可以捕捉时间语义
- 速度是视频模型的自然监督信号
- 时间理解为创意视频操作提供可能
- 统一框架优于任务特定方法