Actions

Structure of the Programs

Agents interact in VirtualHome via programs, lists of instructions that tell which actions each agent should do, and with which objects.

These commands are called using unity_simulator.UnityCommunication.render_script(), where a script is a list of strings of the form:

<char{id1}> {action1}

with {action1} being of the form

[action_name] # action with no args
[action_name] arg1 # action with a single arg
[action_name] arg1 arg2 # action with 2 args

And args correspond to objects, of the form

<object_name> (object_id)

An example of an action would be

<char0> [put] <glass> (3) <table> (2)

Multiple agents can do actions at the same time. You specify it as follows

<char0> [put] <glass> (3) <table> (2) | <char1> [walk] <kitchen> (5)

Actions available

Below is the list of actions available, notice that after an action is executed, the graph is updated, according to tue rules in Environments.

Walk

Walks to a room or object

Arguments

1

Modifier

Preconditions
  • character is not sitting

  • obj1 is reachable (not inside some closed container) or obj1 is a room.

  • obj1 is not grabbed

Postconditions
  • character will move at a distance close to the object, making sure it is visible.

  • if obj1 is a room, character will move to the center of the room

Example
  • [walk] <kitchen> (1)


Run

Runs to a room or object

Arguments

1

Modifier

Preconditions
  • character is not sitting

  • obj1 is reachable (not inside some closed container) or obj1 is a room.

  • obj1 is not grabbed

Postconditions
  • character will move at a distance close to the object, making sure it is visible.

  • if obj1 is a room, character will move to the center of the room

Example
  • [run] <kitchen> (1)


Walktowards

Walks some distance towards a room or object

Arguments

1

Modifier

Preconditions
  • character is not sitting

  • obj1 is reachable (not inside some closed container) or obj1 is a room.

  • obj1 is not grabbed

Postconditions
  • character will move at a distance close to the object, making sure it is visible.

  • if obj1 is a room, character will move to the center of the room

Example
  • [walktowards] <kitchen> (1)

  • [walktowards] <table> (1) :3:


Walkforward

Walks 1 meter forward.

Arguments

0

Modifier

Preconditions
  • character is not sitting

Postconditions
  • character will 1 meter forward according to current orientation.

Example
  • [walkforward]


TurnLeft

Turns 30 degrees left

Arguments

0

Modifier

Preconditions
  • character is not sitting

Postconditions
  • character is turned 30 degree counterclockwise

Example
  • [turnleft]


TurnRight

Turns 30 degrees right

Arguments

0

Modifier

Preconditions
  • character is not sitting

Postconditions
  • character is turned 30 degree clockwise

Example
  • [turnleft]


Sit

Sit on an object

Arguments

1

Modifier

Preconditions
  • character is not sitting

  • character is close to obj1

  • obj1 has property sittable

  • number of objects on obj1 must be less than: ‘couch’: 4, ‘bed’: 4, ‘chair’: 1, ‘loveseat’: 2, ‘sofa’: 4, ‘toilet’: 1, ‘pianobench’: 2, ‘bench’: 2

Postconditions
  • Add directed edges: character sitting obj1

Example
  • [sit] <chair> (1)


StandUp

Stand Up

Arguments

0

Modifier

Preconditions
  • character state is sitting

Postconditions
  • character remove state sitting

Example
  • [standup]


Grab

Grab an object

Arguments

1

Modifier

Preconditions
  • obj1 property is grabbable except water

  • exists edge character close obj1

  • character is close to obj1

  • obj1 is reachable (not inside some closed container)

  • No edge character holds_rh any object or no edge character holds_lh any_object. Character has at least one free hand.

Postconditions
  • Add directed edges: character holds_rh or hold_lh, obj1

  • obj1 is not anymore on a surface or inside a container

Example
  • [grab] <apple> (1)


Open

Open an object

Arguments

1

Modifier

Preconditions
  • obj1 property is opennable and state is closed.

  • character is close to obj1

  • obj1 is reachable (not inside some closed container)

  • No edge character holds_rh any object or no edge character holds_lh any_object. Character has at least one free hand.

Postconditions
  • obj1 state is open

Example
  • [open] <fridge> (1)


Close

Close an object

Arguments

1

Modifier

Preconditions
  • obj1 property is opennable and state is open.

  • character is close to obj1

  • obj1 is reachable (not inside some closed container)

  • No edge character holds_rh any object or no edge character holds_lh any_object. Character has at least one free hand.

Postconditions
  • obj1 state is closed.

Example
  • [close] <fridge> (1)


Put

Put an object on some other object

Arguments

2

Modifier

Preconditions
  • exists edge character holds_lh obj1 or character holds_rh obj1

  • exists edge character close obj2

Postconditions
  • remove directed edges: character holds_lh obj1 or character holds_rh obj1

  • add directed edges: obj1 on obj2

Example
  • [putback] <apple> (1) <table> (1)


PutIn

Put an object inside some other object

Arguments

2

Modifier

Preconditions
  • exists edge character holds_lh obj1 or character holds_rh obj1

  • exists edge character close obj2

  • obj2 is not closed

Postconditions
  • remove directed edges: character holds_lh obj1 or character holds_rh obj1

  • add directed edges: obj1 inside obj2

Example
  • [putin] <apple> (1) <fridge> (1)


SwitchOn

Turn an object on

Arguments

1

Modifier

Preconditions
  • obj1 has property switch

  • obj1 state is off

  • exists edge character close obj1

Postconditions
  • obj1 state is on

Example
  • [switchon] <stove> (1)


SwitchOff

Turn an object off

Arguments

1

Modifier

Preconditions
  • obj1 has property switch

  • obj1 state is on

  • exists edge character close obj1

Postconditions
  • obj1 state is off

Example
  • [switchoff] <stove> (1)


Drink

Drink from an object

Arguments

1

Modifier

Preconditions
  • obj1 property is drinkable or recipient

  • exists edge character close obj1

Postconditions

Example
  • [drink] <waterglass> (1)


Touch

Touch an object

Arguments

1

Modifier

Preconditions
  • character is close to obj1

  • obj1 is reachable (not inside some closed container)

Postconditions

Example
  • [touch] <stove> (1)


LookAt

Look at an object

Arguments

1

Modifier

Preconditions
  • character is facing obj1

  • obj1 is reachable (not inside some closed container)

Postconditions

Example
  • [lookat] <stove> (1)