SFT initialization
Fine-tune the action expert of Fast-WAM on demonstrations with flow matching. The video expert stays frozen in every stage.
Hindsight-Informed Training for World Action Models
1National University of Singapore
✉Corresponding author
World action models (WAMs) have shown promising generalization and data efficiency in robot manipulation by coupling action generation with predictions of future visual dynamics. Translating these capabilities into reliable task execution, however, remains a challenge for existing post-training methods. Offline supervised fine-tuning provides dense supervision on expert trajectories, but its flow-matching action expert is trained only on noised demonstration actions and never on the intermediate states produced by its own sampler; task-specific adaptation can also compromise pretrained capabilities. On-policy reinforcement learning trains on the policy’s own behavior, yet sparse task rewards make exploration and credit assignment difficult, while dense reward shaping still provides scalar evaluations rather than direct action targets.
We introduce HINT-WAM, the first on-policy self-distillation framework that supervises the action expert on its own sampling trajectories using hindsight, the future video of the demonstration, as privileged information available to the WAM itself. For each demonstration observation, the student generates its action through its own denoising trajectory, and at the states along this trajectory the same model acts as a teacher conditioned additionally on this hindsight, providing a conditional action distribution as a dense training target, while the student sees only deployment-time inputs and needs no environmental rewards or simulator interaction. To transfer this supervision to a flow-matching action expert, HINT-WAM distills the teacher into the student with an energy score, a distribution-level objective that matches the teacher’s action distribution from the student’s own samples rather than regressing its velocity field.
The teacher and the student are the same network. The teacher additionally reads hindsight: the future video of the demonstration being imitated. The student sees only deployment-time inputs and is trained on the states its own sampler visits.
Fine-tune the action expert of Fast-WAM on demonstrations with flow matching. The video expert stays frozen in every stage.
With the SFT model frozen, a lightweight encoder pools the demonstration's future latent frames into hindsight tokens that the action expert reads through zero-initialized, gated cross-attention adapters (~13M params).
The student rolls out its own ODE, so supervision lands on states it really visits. No simulator and no reward are needed, and nothing is back-propagated through the rollout. “On-policy” refers to the generation of actions: observations come from demonstrations, and the student generates the denoising trajectory of its own action chunk.
The student cannot see the future, so from its point of view the teacher's behavior is multi-modal. Regressing the teacher's velocity drives the student toward the teacher's conditional mean, which can be correct under none of the modes. Instead we match distributions with a strictly proper scoring rule that needs only one teacher sample and no action likelihood:
The first term pulls the student's samples toward the teacher's sample; the second pushes them apart and prevents collapse. Here $\hat{\mathbf a}=\mathbf a_\sigma-\sigma\,v_\theta(\mathbf a_\sigma,\sigma,o)$ is the one-pass implied action. The full objective adds a behavior-cloning anchor: $\mathcal{L}_B=\lambda_{\mathrm{dist}}\mathcal{L}_{\mathrm{dist}}+\lambda_{\mathrm{BC}}\mathcal{L}_{\mathrm{BC}}$.
Success rates (%). All baselines share the starting checkpoint, data, optimizer and budget unless noted. Fast-WAM is the released checkpoint without post-training.
| Method | LIBERO-90 | RoboTwin 2.0 | |
|---|---|---|---|
| Clean | Random. | ||
| Fast-WAM | 15.56 | 86.48 | 83.44 |
| SFT | 78.88 | – | – |
| Full SFT | 81.24 | 84.12 | 83.24 |
| BC control | 71.56 | 85.36 | 82.08 |
| Self-imitation | 79.62 | – | – |
| Shuffled hindsight | 80.28 | – | – |
| HINT-WAM (Ours) | 85.18 | 87.04 | 83.68 |
After post-training on LIBERO-90, we evaluate on the four original LIBERO suites that the released checkpoint was trained on.
| Method | New | Original suites | ||||
|---|---|---|---|---|---|---|
| LIBERO-90 | Spatial | Object | Goal | Long | Avg | |
| Fast-WAM | 15.56 | 96.8 | 99.6 | 95.8 | 94.0 | 96.55 |
| Without replay | ||||||
| SFT | 78.88 | 58.6 | 33.2 | 53.4 | 4.6 | 37.45 |
| Full SFT | 81.24 | 54.4 | 30.2 | 44.6 | 2.2 | 32.85 |
| HINT-WAM | 85.18 | 70.8 | 38.4 | 58.8 | 8.2 | 44.05 |
| With replay | ||||||
| SFT | 76.70 | 97.2 | 95.8 | 92.0 | 81.2 | 91.55 |
| Full SFT | 77.64 | 97.8 | 96.0 | 94.4 | 83.4 | 92.90 |
| HINT-WAM | 81.56 | 98.4 | 96.0 | 95.8 | 90.2 | 95.10 |
Because the student is distilled on its own sampling trajectories, its updates stay closer to its current behavior. With replay, HINT-WAM comes within 1.5 points of the released checkpoint on the original suites, with the largest gain on the long-horizon suite.
The off-policy variant replaces the student’s sampler states with demonstrated actions noised to the same points, i.e. the states SFT sees. The shuffled-hindsight baseline (80.3%) keeps on-policy states but removes the privileged information. Hindsight and on-policy states thus each improve SFT by less than 2% alone but add 6.3% together: the teacher’s correction is most useful on the denoising states the student actually reaches at inference.
Rollouts of the compared policies. On LIBERO, both start from the same initial state, and each card shows the rollout video above five keyframes.
SFT lifts the pot but carries it away from the stove and disturbs the frying pan. HINT-WAM grasps the handle and places it on the stove.
SFT picks up the white bowl but drifts toward the stove and stalls. HINT-WAM lifts the bowl and carries it to the top of the cabinet.
SFT drifts toward the bowl without completing the task, while HINT-WAM places the item in the drawer from the same initial state.
HINT-WAM succeeds where SFT fails.
Rollouts on tasks from the original LIBERO suites after post-training on LIBERO-90. Rows show the released Fast-WAM checkpoint (before post-training), SFT and HINT-WAM from the same initial state. Use the arrows to browse.
Clean-setting rollouts on RoboTwin 2.0, matched by episode index. Object placements are randomized per run, so the starting layout can differ between the two policies. Use the arrows to browse.
@article{dat2026hintwam,
title={HINT-WAM: Hindsight-Informed Training for World Action Models},
author={Phuong Tuan Dat and Qi Li and Xinchao Wang},
journal={arXiv preprint arXiv:2611.24789},
year={2026}
}