Temporal Taskification: Making Time a First-Class Citizen in RL

Paper: 2604.21834 Authors: Research Team Categories: cs.LG, cs.AI

Problem

Current RL systems treat time as:

• A simple step counter
• Implicit in reward timing
• Not explicitly modeled

This leads to:

• Poor temporal generalization
• Inability to reason about time
• Slow learning in time-dependent tasks

Temporal Taskification

Making time explicit in RL:

Key Ideas

1. Temporal Abstraction

   • Multi-scale time representations
   • Hierarchical time horizons
   • Time-aware state encoding

2. Temporal Rewards

   • Time-dependent reward shaping
   • Deadline-aware learning
   • Temporal credit assignment

3. Time-Predictive Features

   • Learn how long actions take
   • Predict time to goal
   • Plan with time budgets

Architecture

• Temporal encoder network
• Time-conditioned policy
• Deadline predictor
• Temporal value function

Results

Task	Standard RL	Temporal Taskification
Navigation	78% success	94% success
Time-constrained	45% success	87% success
Training efficiency	Baseline	3x faster

Takeaways

• Time is underutilized in RL
• Temporal abstractions improve generalization
• Time-aware rewards enable deadline learning
• Better credit assignment through time modeling