Environment Handlers¶
Minecraft is an extremely complex environment which provides players with visual, auditory, and informational observation of many complex data types. Furthermore, players interact with Minecraft using more than just embodied actions: players can craft, build, destroy, smelt, enchant, manage their inventory, and even communicate with other players via a text chat.
To provide a unified interface with which agents can obtain and perform
similar observations and actions as players, we have provided
first-class for support for this multi-modality in the environment:
the observation and action spaces of environments are
gym.spaces.Dict
spaces. These observation and action
dictionaries are comprised of individual fields we call handlers.
Note
In the documentation of every environment we provide a listing
of the exact gym.space
of the observations returned by and actions expected by the environment’s step function. We are slowly
building documentation for these handlers, and you can click those highlighted with blue for more information!
Spaces¶
Enum Spaces¶
Some observation and action spaces are Enum
types. Examples include
the equip observation
and
the equip action.
Observation and action spaces that are Enum
are encoded as strings by default (e.g. “none”,
“log”, and “sandstone#2”) when they are returned from env.step()
and env.reset()
, or
yielded from minerl.data.DataPipeline.batch_iter()
.
When building an action to pass into env.step(act)
, the Enum component of the action dict can
be encoded as either a string or an integer.
Tip
The Enum integer value that corresponds to each Enum string value can be accessed via
Enum.values_map[string_value]
. For example, to get the integer value corresponding to the
equip action “dirt” in MineRLObtainDiamond
or MineRLBasaltBuildVillageHouse
, you can
call env.action_space.spaces["equip"].values_map["dirt"]
.
Observations¶
Visual Observations - pov
, third-person
¶
- pov : Box(width, height, nchannels)
An RGB image observation of the agent’s first-person perspective.
- Type
np.uint8
- third-person : Box(width, height, nchannels)
An RGB image observation of the agent’s third-person perspective.
Warning
This observation is not yet supported by any environment.
- Type
np.uint8
- compass-observation : Box(1)
The current position of the minecraft:compass object from 0 (behind agent left) to 0.5 in front of agent to 1 (behind agent right)
Note
This observation uses the default Minecraft game logic which includes compass needle momentum. As such it may change even when the agent has stoped moving!
Equip Observations - equipped_items
¶
- equipped_items.mainhand.type : Enum('none', 'air', ..., 'other'))
This observation is an Enum type. See Enum Spaces for more information.
The type of the item that the player has equipped in the mainhand slot. If the mainhand slot is empty then the value is ‘air’. If the mainhand slot contains an item not inside this observation space, then the value is ‘other’.
- Type
np.int64
- Shape
[1]
Actions¶
Camera Control - camera
¶
- camera : Box(2) [delta_pitch, delta_yaw]
This action changes the orientation of the agent’s head by the corresponding number of degrees. When the
pov
observation is available, the camera changes its orientation pitch by the first component and its yaw by the second component. Bothdelta_pitch
anddelta_yaw
are limited to [-180, 180] inclusive- Type
np.float32
- Shape
[2]
- attack : Discrete(1) [attack]
This action causes the agent to attack.
- Type
np.float32
- Shape
[1]
Tool Control - equip
and use
¶
- equip : Enum('none', 'air', ..., 'other'))
This is action is an Enum type. See Enum Spaces for more information.
This action equips the first instance of the specified item from the agents inventory to the main hand if the specified item is present, otherwise does nothing.
air
matches any empty slot in an agent’s inventory and functions as an un-equip, or equip-nothing action.- Type
np.int64
- Shape
[1]
Note
equip 'none'
andequip 'other'
are both no-op actions. In other words, they leave the currently equipped item unchanged. However, in the MineRL dataset,other
takes on a special meaning.other
is the wildcard equip action that is recorded in the dataset whenever a player equipped an item that wasn’t included in this action space’s Enum.Warning
env.step(act) typically will not process the equip action for two ticks (i.e., you will not see the observation value equipped_items change until two more calls to env.step.)
This is due to a limitation with the current version of Malmo, our Minecraft backend.
- use : Discrete(1) [use]
This action is equivalent to right-clicking in Minecraft. It causes the agent to use the item it is holding in the mainhand slot, or to open doors or gates when it is facing an applicable Minecraft structure.
- Type
np.int64
- Shape
[1]