To interact with the simulator, simply clone the VirtualHome API repository:

git clone

And download the executable for your platform:

After downloading, you should be able to use the simulator. From the VirtualHome API repository, create a communication with the simulation:

# cd into virtualhome repo
from simulation.unity_simulator importcomm_unity 
YOUR_FILE_NAME = "Your path to the simulator"
comm = unity_simulator.UnityCommunication(file_name=YOUR_FILE_NAME)

The simulator should have opened. Try getting an image of the current environment.

# Start the first environment
# Get an image of the first camera
success, image = comm.camera_image([0])

You should be all set. Look a overview to understand the different components in VirtualHome, and at API. For an overview of the commands available. The repository also contains a notebook to play with the simulator.

Build your own environment

If you wish, you can also modify VirtualHome at your will, and build your own executable for your research. Check out the Unity Repository below, with instructions on how to build your own executables

Download Activity Program Datasets

You can download the datasets of activity programs along with descriptions, used in Synthesizing Environment-Aware Activities via Activity Sketches (Liao et al.) in the following links