Visualizations

See World Models in Action

Interactive demonstrations showing how World Models perceive, predict, and act. Understand the V-M-C architecture through visual exploration.

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                      World Model Architecture                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│    ┌──────────────┐      ┌──────────────┐      ┌──────────────┐ │
│    │              │      │              │      │              │ │
│    │   Vision     │ ──▶  │   Memory     │ ──▶  │  Controller  │ │
│    │   Model (V)  │      │   Model (M)  │      │     (C)      │ │
│    │              │      │              │      │              │ │
│    │    VAE       │      │   MDN-RNN    │      │   Linear     │ │
│    │              │      │              │      │              │ │
│    └──────────────┘      └──────────────┘      └──────────────┘ │
│          │                     │                     │          │
│          ▼                     ▼                     ▼          │
│    ┌──────────────┐      ┌──────────────┐      ┌──────────────┐ │
│    │   Latent     │      │   Hidden     │      │   Action     │ │
│    │   Vector z   │      │   State h    │      │   Output a   │ │
│    └──────────────┘      └──────────────┘      └──────────────┘ │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Vision Model (V)

Uses a VAE to compress 64x64 pixel frames into a 32-dimensional latent vector z, capturing essential visual features.

Memory Model (M)

An MDN-RNN that predicts future latent states, maintaining a 256-dimensional hidden state h for temporal context.

Controller (C)

A simple linear model that maps z and h to actions, trained using evolution strategies (CMA-ES).

Interactive Demonstrations

Vision Model

Variational Autoencoder (VAE)

See how the Vision Model compresses high-dimensional observations into a compact latent space.

Memory Model

MDN-RNN Predictions

Visualize how the Memory Model predicts future states based on current observations and actions.

Controller

Controller Decision Making

Watch how the Controller uses compressed representations to make decisions.

Full System

Full World Model Pipeline

Experience the complete V-M-C architecture working together in real-time.

Want to Learn More?

Dive deeper into the theory and implementation of World Models through our structured learning modules.

Start Learning Ask AI Assistant