You can move and change the state of the objects in the environment. The environment can be programatically generated.
We provide multiple human avatars as agents, with Path-planning, IK and Animations to generate realistic actions.
You can record activities across different modalities and using multiple camera views simulatenously.
You can add multiple agents in the environment, and control them concurrently.
VirtualHome: Simulating Household Activities via Programs
Presents VirtualHome and proposes a model to predict programs from videos or descriptions. The predicted program is finetuned using RL to be executable in the simulator.X. Puig*, K. Ra*, M. Boben*, J. Li, T. Wang, S. Fidler, A. Torralba.
Synthesizing Environment-Aware Activities via Activity Sketches
Represents activities via Sketches. Proposes a model to generate programs from sketches such that they are consistent with a target environment.A. Liao*, X. Puig*, M. Boben, A. Torralba, S.Fidler.
Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration
Presents a challenge where an agent has to infer a human-like agent's goals, and find the best way to achieve the goals efficiently.X. Puig, T. Shu, S. Li, Z. Wang, A. Liao, J. Tenenbaum, S. Fidler, A. Torralba.