Research project · Probabilistic world models

Belief-Aware
Oracle Particle
Dynamics

Learning multimodal, object-centric transitions from structured 3D state histories.

Yin Kang et al. · Korea University

charles_kang@korea.ac.kr

Paper Code CV

Scene 3/4 dataset 96-frame simulation

↓

Motivation

Partial observation, plural futures

One visible state can imply more than one physical future.

Object motion under occlusion is only partially observed. A deterministic transition tends to average incompatible outcomes, while a short state window can omit the physical context needed for stable long-horizon prediction.

This project separates those challenges. A variational belief encoder summarizes full object histories; a gated Gaussian mixture transition retains multiple plausible 3D displacements at every step.

Infer what cannot be observed, then preserve what can happen next.

Dataset

Synthetic object dynamics

Controlled scenes make motion and occlusion measurable.

The Scene 3/4 batch contains completed Kubric/Blender simulations with PyBullet physics. Each clip begins with a static observation interval and continues into a dynamic rollout with four tracked balls.

900completed clips

96 framesper sequence

24 FPStemporal resolution

256 × 256render size

4 ballsper scene

Dataset sample 0000 showing four balls moving through a simulated room — Sample 0000Boundary interactions

Dataset sample 0010 showing multi-object trajectories in a simulated room — Sample 0010Multi-object motion

Dataset sample 0450 showing varied ball dynamics and occlusion — Sample 0450Trajectory variation

00–23

Static scan establishes object identity, geometry, and scene context.

24–95

Dynamic frames expose collisions, occlusion, and long-range trajectory changes.

Method

A distribution over motion

Structured state enters; multiple futures leave.

The transition model reasons directly over typed 3D object slots, independent of the image encoder and decoder path.

01
Oracle state

position · Δposition · quaternion · size · type
02
Belief encoder

A full-history posterior supplies 16-dimensional object context.
03
Particle transformer

Causal attention models temporal and inter-object interactions.
04
Gated GMM

Mixture weights, means, scales, and a motion gate define displacement.
05
Autoregressive rollout

Predicted state is fed back to test stability across the full horizon.

Experiment

Configuration at a glance

A compact model, evaluated across a long horizon.

Context: 8 observed frames
Forecast: 32-step rollout
Output: 5-component GMM
Backbone: 256 hidden · 4 layers
Attention: 8 heads
Objective: NLL + βKL + rollout

Evaluation includes mean, MAP, oracle-component, and sampled rollouts, plus teacher-forced mixture diagnostics.

Results

Run 0006 · Scene 4 validation

Inspect the rollout, not just one number.

0.291dynamic position L2

0.169MAP dynamic L2

0.189dynamic minADE · 16 samples

−4.410teacher-forced GMM NLL

Ground-truth and mean-predicted 3D object trajectories across 32 forecast steps — 01 / 3D rollout
Ground truth and mean prediction reveal object-wise drift across the complete forecast.

Position and velocity errors plotted across the forecast horizon — 02 / Horizon error
Per-step curves show where autoregressive error begins to compound.

Sixteen sampled future trajectories compared with ground truth — 03 / Sampled futures
Sixteen fixed stochastic rollouts expose diversity, coverage, and spread.

Teacher-forced Gaussian mixture displacement diagnostics — 04 / GMM diagnostics
Teacher-forced analysis inspects mixture weights, displacement components, and predictive scale.

My contribution

Research engineering

Research ideas carried through to working systems.

The statements below are publication placeholders and should be aligned with the final authorship breakdown before release.

Model design

Designed the typed object-state representation, belief pathway, and gated mixture transition interface.

Training systems

Built deterministic data splits, checkpointing, distributed training, gradient clipping, and experiment configuration.

Probabilistic evaluation

Implemented rollout modes, best-component analysis, sampled trajectory metrics, coverage, and GMM diagnostics.

Scientific tooling

Created reproducible 3D visualizations and metric artifacts for diagnosing long-horizon behavior.

Implementation

Reproducible by construction

One entry point, from local iteration to distributed training.

The trainer supports single-GPU and torchrun execution, with configuration and vocabulary stored beside each checkpoint.

train.sh

python train_oracle_particle_transition.py --config configs/oracle_particle_transition.json

# distributed
CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun \
  --nproc_per_node=4 \
  train_oracle_particle_transition.py \
  --config configs/oracle_particle_transition.json

Citation

Use, inspect, and build on the work.

Venue and publication details remain replaceable until the paper is released.

Yin Kang et al.charles_kang@korea.ac.krKorea University

@article{author2026oracle,
  title   = {Belief-Aware Oracle Particle Dynamics},
  author  = {Kang, Yin and others},
  journal = {[Venue]},
  year    = {2026}
}