Environments

VirtualHome is composed of 50 custom designed environmenta where activities can be executed. Each environment is encoded as a dictionary, containing a node for every object and edges between them representing relationships. Each environment can be updated by modifying the corresponding dictionary. We call this dictionary EnvironmentGraph.

You can load each environment using unity_simulator.UnityCommunication.reset() function, specifying the matching environment id. You can retrieve the information of an environment using unity_simulator.UnityCommunication.environment_graph(). After loading an environment, you can modify it using unity_simulator.UnityCommunication.expand_scene().

Environments

Below are a few samples of the environments included in VirtualHome.

Environment ID	Apartment View
0
1
2
3
4
5
6

Environment Graph

The Environment Graph is the way to represent a environment in VirtualHome. It is represented as a dictionary of node, corresponding to objects, and edges, corresponding to relationships.

Here is an example:

{
   "nodes":[
      {
         "id":1,
         "class_name":"character",
         "states":[

         ],
         "properties":[

         ],
         "category":"Person"
      },
      {
         "id":2,
         "class_name":"kitchen",
         "states":[

         ],
         "properties":[

         ],
         "category":"Room"
      }
   ],
   "edges":[
      {
         "from_id":1,
         "to_id":2,
         "relation_type":"INSIDE"
      }
   ]
}

The structure of the Graph is documented below.

Environment Graph

EnvironmentGraph

Object Properties

nodes (list(Nodes)) – list of objects
edges (list(Edges)) – list of relationships between objects

Nodes

Object Properties

id (int) – id of object
class_name (str) – class_name of the object
category (str) – cateogry of the object
prefab_name (str) – name of the Unity game object used
states (list(str)) – states of the objects
properties (list(str)) – properties of the objects
bounding_box (list(BoundingBox)) – bounding box of the given object
obj_transform (ObjTransform) – 3D information of the given object

BoundingBox

Object Properties

center (list(float)) – center of the bounding box, in x,y,z
size (list(float)) – size of the bounding box, in x,y,z. The size is axis aligned

ObjTransform

Object Properties

position (list(float)) – x,y,z position of the game object. Does not necessarily match with BoundingBox center.
rotation (list(float)) – x,y,z,w representation a quaternion of the object rotation https://docs.unity3d.com/ScriptReference/Quaternion.html

Edges

Object Properties

from_id (int) – id of Node in the from relationship
to_id (int) – id of Node in the to relationship
relation_type (str) – relationship between the 2 objects.

States

Objects can have the following states

State	Meaning
OPEN	Indicates whether a container is open.
CLOSED	Indicates whether a container is closed.
ON	Indicates whether a light, electronic device is ON.
OFF	Indicates whether a light, electronic device is OFF.

Properties

Objects can have the following properties. Some have effect on the Uniy Simulator and some are just used to categorize the objects. We only indicate meaning for the former.

Property	Meaning
CAN_OPEN	Whether the object can be opened
CLOTHES
CONTAINERS
COVER_OBJECT
CREAM
CUTTABLE
DRINKABLE
EATABLE
GRABBABLE	Whether the object can be grabbed
HANGABLE
HAS_PAPER
HAS_PLUG
HAS_SWITCH	Whether the object can be turned on or off
LIEABLE
LOOKABLE
MOVABLE
POURABLE
READABLE
RECIPIENT
SITTABLE	Whether the agent can sit in this object
SURFACE	Whether the agent can place things on this object

Relationships

Objects can have different relationships between each other, we describe here their meaning

Relation Type	Meaning
ON	Object from_id is on top of object to_id.
INSIDE	Object from_id is inside of object to_id.
BETWEEN	Used for doors. Door connects with room to_id.
CLOSE	Object from_id is close to object to_id (< 1.5 metres).
FACING	Object to_id is visible from objects from_id and distance is < 5 metres. If object1 is a sofa or a chair it should also be turned towards object2.
HOLDS_RH	Character from_id holds object to_id with the right hand.
HOLDS_LH	Character from_id holds object to_id with the left hand.
SITTING	Character from_id is sitting in object to_id.

Environmental Manipulation

VirtualHome also supports environmental manipulation. The current supported features are to set the time to a pre defined state which accurate day/night lighting, and physics where objects in the environment experience gravity with semi-accurate collisions. To set the time you can call the comm.set_time() function and to activate gravity in the environment the comm.activate_physics() function can be called.

Set Time

The following example shows how to set the time in the environment:

# Set the time of the environment using hours, minutes and seconds
comm.set_time(hours=0, minutes=0, seconds=0, scaler=1)

set_time

The time of the environment will be set to accurately display lightning and corresponding actions which can be simulated depending on the time.

Arguments

hours, minutes, seconds

Modifier

Preconditions

Environment has to be in a active state

Postconditions

The time will be set for the environment and outside lighting will be affected by the time.

Example

comm.set_time(hours=6, minutes=30, seconds=0)

Activate Physics

The following example shows how to activate physics:

# Activate physics (gravity)
comm.activate_physics()

activate_physics

All interactble objects can be influenced by gravity and collisions.

Arguments

gravity

Modifier

Preconditions

Environment has to be in a active state

Postconditions

All interactble objects expereince gravatational force and can represent semi-acccurate collisions

Example

comm.activate_physics(gravity=-10)

Remove Terrain

The following example shows how to remove the outdoor terrain:

# Remove terrain
comm.remove_terrain()

remove_terrain

All outdoor objects will be removed.

Arguments

none

Modifier

Preconditions

Environment has to be in a active state

Postconditions

Outdoor terrain is removed

Example

comm.remove_terrain()