Actions

Structure of the Programs

Agents interact in VirtualHome via programs, lists of instructions that tell which actions each agent should do, and with which objects.

These commands are called using unity_simulator.UnityCommunication.render_script(), where a script is a list of strings of the form:

<char{id1}> {action1}

with {action1} being of the form

[action_name] # action with no args
[action_name] arg1 # action with a single arg
[action_name] arg1 arg2 # action with 2 args

And args correspond to objects, of the form

<object_name> (object_id)

An example of an action would be

<char0> [put] <glass> (3) <table> (2)

Multiple agents can do actions at the same time. You specify it as follows

<char0> [put] <glass> (3) <table> (2) | <char1> [walk] <kitchen> (5)

Actions available

Below is the list of actions available, notice that after an action is executed, the graph is updated, according to tue rules in Environments.

Walk

Walks to a room or object

Arguments:

1

Modifier:
Preconditions:
  • character is not sitting
  • obj1 is reachable (not inside some closed container) or obj1 is a room.
  • obj1 is not grabbed
Postconditions:
  • character will move at a distance close to the object, making sure it is visible.
  • if obj1 is a room, character will move to the center of the room
Example:
  • [walk] <kitchen> (1)


Run

Runs to a room or object

Arguments:

1

Modifier:
Preconditions:
  • character is not sitting
  • obj1 is reachable (not inside some closed container) or obj1 is a room.
  • obj1 is not grabbed
Postconditions:
  • character will move at a distance close to the object, making sure it is visible.
  • if obj1 is a room, character will move to the center of the room
Example:
  • [run] <kitchen> (1)


Walktowards

Walks some distance towards a room or object

Arguments:

1

Modifier:
Preconditions:
  • character is not sitting
  • obj1 is reachable (not inside some closed container) or obj1 is a room.
  • obj1 is not grabbed
Postconditions:
  • character will move at a distance close to the object, making sure it is visible.
  • if obj1 is a room, character will move to the center of the room
Example:
  • [walktowards] <kitchen> (1)
  • [walktowards] <table> (1) :3:


Walkforward

Walks 1 meter forward.

Arguments:

0

Modifier:
Preconditions:
  • character is not sitting
Postconditions:
  • character will 1 meter forward according to current orientation.
Example:
  • [walkforward]


TurnLeft

Turns 30 degrees left

Arguments:

0

Modifier:
Preconditions:
  • character is not sitting
Postconditions:
  • character is turned 30 degree counterclockwise
Example:
  • [turnleft]


TurnRight

Turns 30 degrees right

Arguments:

0

Modifier:
Preconditions:
  • character is not sitting
Postconditions:
  • character is turned 30 degree clockwise
Example:
  • [turnleft]


Sit

Sit on an object

Arguments:

1

Modifier:
Preconditions:
  • character is not sitting
  • character is close to obj1
  • obj1 has property sittable
  • number of objects on obj1 must be less than: ‘couch’: 4, ‘bed’: 4, ‘chair’: 1, ‘loveseat’: 2, ‘sofa’: 4, ‘toilet’: 1, ‘pianobench’: 2, ‘bench’: 2
Postconditions:
  • Add directed edges: character sitting obj1
Example:
  • [sit] <chair> (1)


StandUp

Stand Up

Arguments:

0

Modifier:
Preconditions:
  • character state is sitting
Postconditions:
  • character remove state sitting
Example:
  • [standup]


Grab

Grab an object

Arguments:

1

Modifier:
Preconditions:
  • obj1 property is grabbable except water
  • exists edge character close obj1
  • character is close to obj1
  • obj1 is reachable (not inside some closed container)
  • No edge character holds_rh any object or no edge character holds_lh any_object. Character has at least one free hand.
Postconditions:
  • Add directed edges: character holds_rh or hold_lh, obj1
  • obj1 is not anymore on a surface or inside a container
Example:
  • [grab] <apple> (1)


Open

Open an object

Arguments:

1

Modifier:
Preconditions:
  • obj1 property is opennable and state is closed.
  • character is close to obj1
  • obj1 is reachable (not inside some closed container)
  • No edge character holds_rh any object or no edge character holds_lh any_object. Character has at least one free hand.
Postconditions:
  • obj1 state is open
Example:
  • [open] <fridge> (1)


Close

Close an object

Arguments:

1

Modifier:
Preconditions:
  • obj1 property is opennable and state is open.
  • character is close to obj1
  • obj1 is reachable (not inside some closed container)
  • No edge character holds_rh any object or no edge character holds_lh any_object. Character has at least one free hand.
Postconditions:
  • obj1 state is closed.
Example:
  • [close] <fridge> (1)


Put

Put an object on some other object

Arguments:

2

Modifier:
Preconditions:
  • exists edge character holds_lh obj1 or character holds_rh obj1
  • exists edge character close obj2
Postconditions:
  • remove directed edges: character holds_lh obj1 or character holds_rh obj1
  • add directed edges: obj1 on obj2
Example:
  • [putback] <apple> (1) <table> (1)


PutIn

Put an object inside some other object

Arguments:

2

Modifier:
Preconditions:
  • exists edge character holds_lh obj1 or character holds_rh obj1
  • exists edge character close obj2
  • obj2 is not closed
Postconditions:
  • remove directed edges: character holds_lh obj1 or character holds_rh obj1
  • add directed edges: obj1 inside obj2
Example:
  • [putin] <apple> (1) <fridge> (1)


SwitchOn

Turn an object on

Arguments:

1

Modifier:
Preconditions:
  • obj1 has property switch
  • obj1 state is off
  • exists edge character close obj1
Postconditions:
  • obj1 state is on
Example:
  • [switchon] <stove> (1)


SwitchOff

Turn an object off

Arguments:

1

Modifier:
Preconditions:
  • obj1 has property switch
  • obj1 state is on
  • exists edge character close obj1
Postconditions:
  • obj1 state is off
Example:
  • [switchoff] <stove> (1)


Drink

Drink from an object

Arguments:

1

Modifier:
Preconditions:
  • obj1 property is drinkable or recipient
  • exists edge character close obj1
Postconditions:
Example:
  • [drink] <waterglass> (1)


Touch

Touch an object

Arguments:

1

Modifier:
Preconditions:
  • character is close to obj1
  • obj1 is reachable (not inside some closed container)
Postconditions:
Example:
  • [touch] <stove> (1)


LookAt

Look at an object

Arguments:

1

Modifier:
Preconditions:
  • character is facing obj1
  • obj1 is reachable (not inside some closed container)
Postconditions:
Example:
  • [lookat] <stove> (1)