HINT-WAM

Hindsight-Informed Training for World Action Models

Phuong Tuan Dat1 Qi Li1 Xinchao Wang1,✉

1National University of Singapore

✉Corresponding author

HINT-WAM teaser: SFT versus HINT-WAM rollouts on two LIBERO-90 tasks, and a radar chart of success rates
HINT-WAM post-trains world action models with hindsight. Left: rollouts on two LIBERO-90 tasks from the same initial state, where SFT fails and HINT-WAM succeeds. Right: success rates (%) of HINT-WAM, the BC control, and full SFT on the new LIBERO-90 tasks, RoboTwin 2.0, and the four original LIBERO suites after post-training on LIBERO-90 without replay.
85.2%LIBERO-90 success
(SFT 78.9%)
+6.3points over SFT,
action expert only
0.2%extra parameters,
teacher-only
0environment rewards
needed

Abstract

World action models (WAMs) have shown promising generalization and data efficiency in robot manipulation by coupling action generation with predictions of future visual dynamics. Translating these capabilities into reliable task execution, however, remains a challenge for existing post-training methods. Offline supervised fine-tuning provides dense supervision on expert trajectories, but its flow-matching action expert is trained only on noised demonstration actions and never on the intermediate states produced by its own sampler; task-specific adaptation can also compromise pretrained capabilities. On-policy reinforcement learning trains on the policy’s own behavior, yet sparse task rewards make exploration and credit assignment difficult, while dense reward shaping still provides scalar evaluations rather than direct action targets.

We introduce HINT-WAM, the first on-policy self-distillation framework that supervises the action expert on its own sampling trajectories using hindsight, the future video of the demonstration, as privileged information available to the WAM itself. For each demonstration observation, the student generates its action through its own denoising trajectory, and at the states along this trajectory the same model acts as a teacher conditioned additionally on this hindsight, providing a conditional action distribution as a dense training target, while the student sees only deployment-time inputs and needs no environmental rewards or simulator interaction. To transfer this supervision to a flow-matching action expert, HINT-WAM distills the teacher into the student with an energy score, a distribution-level objective that matches the teacher’s action distribution from the student’s own samples rather than regressing its velocity field.

Method

The teacher and the student are the same network. The teacher additionally reads hindsight: the future video of the demonstration being imitated. The student sees only deployment-time inputs and is trained on the states its own sampler visits.

Overview of HINT-WAM: Stage 0 SFT, Stage A hindsight teacher, Stage B on-policy self-distillation
Overview. Stage 0 fine-tunes the action expert with flow matching. Stage A trains a hindsight encoder and gated cross-attention adapters on the frozen SFT model, giving the teacher. Stage B integrates the student's own ODE without gradients and matches its implied actions to a teacher sample with the energy score, plus a behavior-cloning anchor. Only the student is deployed.
Stage 0

SFT initialization

Fine-tune the action expert of Fast-WAM on demonstrations with flow matching. The video expert stays frozen in every stage.

Stage A

Hindsight teacher

With the SFT model frozen, a lightweight encoder pools the demonstration's future latent frames into hindsight tokens that the action expert reads through zero-initialized, gated cross-attention adapters (~13M params).

Stage B

On-policy self-distillation

The student rolls out its own ODE, so supervision lands on states it really visits. No simulator and no reward are needed, and nothing is back-propagated through the rollout. “On-policy” refers to the generation of actions: observations come from demonstrations, and the student generates the denoising trajectory of its own action chunk.

Why the energy score?

The student cannot see the future, so from its point of view the teacher's behavior is multi-modal. Regressing the teacher's velocity drives the student toward the teacher's conditional mean, which can be correct under none of the modes. Instead we match distributions with a strictly proper scoring rule that needs only one teacher sample and no action likelihood:

$$\mathcal{L}_{\mathrm{ES}}=\sum_{k=1}^{K}\bar w_k\Big[\tfrac{1}{M}\sum_{m}\|\hat{\mathbf a}_{m,k}-\mathbf a_T\|_2-\tfrac{1}{2M^2}\sum_{m,m'}\|\hat{\mathbf a}_{m,k}-\hat{\mathbf a}_{m',k}\|_2\Big]$$

The first term pulls the student's samples toward the teacher's sample; the second pushes them apart and prevents collapse. Here $\hat{\mathbf a}=\mathbf a_\sigma-\sigma\,v_\theta(\mathbf a_\sigma,\sigma,o)$ is the one-pass implied action. The full objective adds a behavior-cloning anchor: $\mathcal{L}_B=\lambda_{\mathrm{dist}}\mathcal{L}_{\mathrm{dist}}+\lambda_{\mathrm{BC}}\mathcal{L}_{\mathrm{BC}}$.

Results

Post-training on LIBERO-90 and RoboTwin 2.0

Success rates (%). All baselines share the starting checkpoint, data, optimizer and budget unless noted. Fast-WAM is the released checkpoint without post-training.

MethodLIBERO-90RoboTwin 2.0
CleanRandom.
Fast-WAM15.5686.4883.44
SFT78.88––
Full SFT81.2484.1283.24
BC control71.5685.3682.08
Self-imitation79.62––
Shuffled hindsight80.28––
HINT-WAM (Ours)85.1887.0483.68

Continual learning: avoid catastrophic forgetting

After post-training on LIBERO-90, we evaluate on the four original LIBERO suites that the released checkpoint was trained on.

MethodNewOriginal suites
LIBERO-90SpatialObjectGoalLongAvg
Fast-WAM15.5696.899.695.894.096.55
Without replay
SFT78.8858.633.253.44.637.45
Full SFT81.2454.430.244.62.232.85
HINT-WAM85.1870.838.458.88.244.05
With replay
SFT76.7097.295.892.081.291.55
Full SFT77.6497.896.094.483.492.90
HINT-WAM81.5698.496.095.890.295.10

Because the student is distilled on its own sampling trajectories, its updates stay closer to its current behavior. With replay, HINT-WAM comes within 1.5 points of the released checkpoint on the original suites, with the largest gain on the long-horizon suite.

Ablations and cost

Ablation: velocity matching vs energy score
Distillation objective. Gain over SFT with velocity matching (+0.1; 79.02%), kernel MMD (80.63%), action-space MSE (82.93%) or the energy score (85.18%).
Effect of the BC anchor
BC anchor. Distillation alone (−4.5) and BC alone (−7.3) both hurt; together they add +6.3.
Effect of number of student samples
Student samples $M$. Success improves monotonically to 85.2% at $M=4$. Action-space MSE with $M=4$ reaches only 82.9%, so the gain stems from the repulsion term rather than from the number of samples alone.
Effect of on-policy states: SFT, off-policy distillation, on-policy HINT-WAM
On-policy states. Distilling on demonstration states (off-policy) lifts SFT only from 78.9% to 80.7%; distilling on the student’s own sampling trajectories reaches 85.2%.

The off-policy variant replaces the student’s sampler states with demonstrated actions noised to the same points, i.e. the states SFT sees. The shuffled-hindsight baseline (80.3%) keeps on-policy states but removes the privileged information. Hindsight and on-policy states thus each improve SFT by less than 2% alone but add 6.3% together: the teacher’s correction is most useful on the denoising states the student actually reaches at inference.

Computational cost

Success versus training time
Training time. 1.28× SFT and 0.56× full SFT, with higher success. The deployed policy is the unmodified Fast-WAM.

Qualitative Comparison

Rollouts of the compared policies. On LIBERO, both start from the same initial state, and each card shows the rollout video above five keyframes.

Learning new tasks (LIBERO-90): SFT fails, HINT-WAM succeeds

“put the moka pot on the stove”

Keyframes, moka pot task

SFT lifts the pot but carries it away from the stove and disturbs the frying pan. HINT-WAM grasps the handle and places it on the stove.

“put the white bowl on top of the cabinet”

Keyframes, white bowl task

SFT picks up the white bowl but drifts toward the stove and stalls. HINT-WAM lifts the bowl and carries it to the top of the cabinet.

“put the butter at the back in the top drawer of the cabinet and close it”

Keyframes, task 3

SFT drifts toward the bowl without completing the task, while HINT-WAM places the item in the drawer from the same initial state.

“pick up the book and place it in the front compartment of the caddy”

Keyframes, task 81

HINT-WAM succeeds where SFT fails.

Continual learning: SFT forgets old skills, HINT-WAM retains them

Rollouts on tasks from the original LIBERO suites after post-training on LIBERO-90. Rows show the released Fast-WAM checkpoint (before post-training), SFT and HINT-WAM from the same initial state. Use the arrows to browse.

RoboTwin 2.0 (dual-arm): SFT fails, HINT-WAM succeeds

Clean-setting rollouts on RoboTwin 2.0, matched by episode index. Object placements are randomized per run, so the starting layout can differ between the two policies. Use the arrows to browse.

BibTeX

@article{dat2026hintwam,
  title={HINT-WAM: Hindsight-Informed Training for World Action Models},
  author={Phuong Tuan Dat and Qi Li and Xinchao Wang},
  journal={arXiv preprint arXiv:2611.24789},
  year={2026}
}