Environments

VirtualHome is composed of 7 scenes where activities can be executed. Each scene is encoded as a dictionary, containing a node for every object and edges between them representing relationships. Each environment can be updated by modifying the corresponding dictionary. We call this dictionary EnvironmentGraph.

You can load each environment using unity_simulator.UnityCommunication.reset() function, specifying the matching environment id. You can retrieve the information of an environment using unity_simulator.UnityCommunication.environment_graph(). After loading an environment, you can modify it using unity_simulator.UnityCommunication.expand_scene().

Scenes

Below is a list of the 7 scenes included in VirtualHome.

Scene Id Apartment View
0 ../_images/scene1rot.png
1 ../_images/scene2rot.png
2 ../_images/scene3rot.png
3 ../_images/scene4rot.png
4 ../_images/scene5rot.png
5 ../_images/scene6rot.png
6 ../_images/scene7rot.png

EnvironmentGraph

The EnvironmentGraph is the way to represent a scene in VirtualHome. It is represented as a dictionary of node, corresponding to objects, and edges, corresponding to relationships.

Here is an example:

{
   "nodes":[
      {
         "id":1,
         "class_name":"character",
         "states":[

         ],
         "properties":[

         ],
         "category":"Person"
      },
      {
         "id":2,
         "class_name":"kitchen",
         "states":[

         ],
         "properties":[

         ],
         "category":"Room"
      }
   ],
   "edges":[
      {
         "from_id":1,
         "to_id":2,
         "relation_type":"INSIDE"
      }
   ]
}

The structure of the Graph is documented below.

EnvironmentGraph

EnvironmentGraph
Object Properties:
 
  • nodes (list(Nodes)) – list of objects
  • edges (list(Edges)) – list of relationships between objects
Nodes
Object Properties:
 
  • id (int) – id of object
  • class_name (str) – class_name of the object
  • category (str) – cateogry of the object
  • prefab_name (str) – name of the Unity game object used
  • states (list(str)) – states of the objects
  • properties (list(str)) – properties of the objects
  • bounding_box (list(BoundingBox)) – bounding box of the given object
  • obj_transform (ObjTransform) – 3D information of the given object
BoundingBox
Object Properties:
 
  • center (list(float)) – center of the bounding box, in x,y,z
  • size (list(float)) – size of the bounding box, in width, height, depth
ObjTransform
Object Properties:
 
  • position (list(float)) – x,y,z position of the game object. Does not necessarily match with BoundingBox center.
  • rotation (list(float)) – pitch, yaw, roll
Edges
Object Properties:
 
  • from_id (int) – id of Node in the from relationship
  • to_id (int) – id of Node in the to relationship
  • relation_type (str) – relationship between the 2 objects.

States

Objects can have the following states

State Meaning
OPEN Indicates whether a container is open.
CLOSED Indicates whether a container is closed.
ON Indicates whether a light, electronic device is ON.
OFF Indicates whether a light, electronic device is OFF.

Properties

Objects can have the following properties. Some have effect on the Uniy Simulator and some are just used to categorize the objects. We only indicate meaning for the former.

Property Meaning
CAN_OPEN Whether the object can be opened
CLOTHES  
CONTAINERS  
COVER_OBJECT  
CREAM  
CUTTABLE  
DRINKABLE  
EATABLE  
GRABBABLE Whether the object can be grabbed
HANGABLE  
HAS_PAPER  
HAS_PLUG  
HAS_SWITCH Whether the object can be turned on or off
LIEABLE  
LOOKABLE  
MOVABLE  
POURABLE  
READABLE  
RECIPIENT  
SITTABLE Whether the agent can sit in this object
SURFACE Whether the agent can place things on this object

Relationships

Objects can have different relationships between each other, we describe here their meaning

Relation Type Meaning
ON Object from_id is on top of object to_id.
INSIDE Object from_id is inside of object to_id.
BETWEEN Used for doors. Door connects with room to_id.
CLOSE Object from_id is close to object to_id (< 1.5 metres).
FACING Object to_id is visible from objects from_id and distance is < 5 metres. If object1 is a sofa or a chair it should also be turned towards object2.
HOLDS_RH Character from_id holds object to_id with the right hand.
HOLDS_LH Character from_id holds object to_id with the left hand.
SITTING Character from_id is sitting in object to_id.