Wrappers

Index

DIAMBRA Arena comes with a large number of ready-to-use wrappers and examples showing how to apply them. They cover a wide spectrum of use cases, and also provide reference templates to develop custom ones. In order to activate wrappers, one needs to properly set the WrapperSettings class attributes and provide them as input to the environment creation method, as shown in the next code block. The class attributes are described in the next sections below.

from diambra.arena import EnvironmentSettings, WrapperSettings

# Settings specification
settings = EnvironmentSettings()

# Wrapper settings specification
wrapper_settings = WrapperSettings()
wrapper_settings.setting_1 = value_1
wrapper_settings.setting_2 = value_2

env = diambra.arena.make("doapp", settings, wrapper_settings)

Implementation examples and templates can be found in the code repository, here.

Use of these functionalities can be found in this and this examples.

Generic Wrappers

No Attack Buttons Combinations

NameTypeDefault Value(s)Value RangeDescription
no_attack_buttons_combinationsboolFalseTrue / FalseRestricts the number of attack actions to single buttons, removing attack buttons combinations
wrappers_settings.no_attack_buttons_combinations = True

NoOp Steps After Reset

NameTypeDefault Value(s)Value RangeDescription
no_op_maxint0[0, 12]Performs a maximum of N No-Action steps after episode reset
wrappers_settings.no_op_max = 6

Observation Wrappers

Warp Frame

DEPRECATED: For speed, consider using the environment setting frame_shape that performs the same operation on the engine side, as described in this section.

NameTypeDefault Value(s)Value RangeTarget Observation ElementDescription
frame_shapetuple of three int (H, W, C)(0, 0, 0)H, W: [0, 512]
C: 0 or 1
frameIf active, resizes the frame and/or converts it from RGB to grayscale.
Combinations:
(0, 0, 0) - Deactivated;
(H, W, 0) - RGB frame resized to H X W;
(0, 0, 1) - Grayscale frame;
(H, W, 1) - Grayscale frame resized to H X W;
Keeps observation element of type Box, changes its shape
wrappers_settings.frame_shape = (128, 128, 1)

Stack Frames With Optional Dilation

NameTypeDefault Value(s)Value RangeTarget Observation ElementDescription
stack_framesint1[1, 48]frameStacks latest N frames together along the third dimension.
Keeps observation element of type Box, changes its shape
dilationint1[1, 48]frameBuilds frame stacks adding one every N frames.
Keeps observation element of type Box
wrappers_settings.stack_frame = 4
wrappers_settings.dilation = 1

Add Last Action to Observation

NameTypeDefault Value(s)Value RangeTarget Observation ElementDescription
add_last_actionboolFalseTrue / FalseactionAdds latest action to the observation space dictionary, under the key action.
Adds an observation element of type Discrete or MultiDiscrete depending on the action space
wrappers_settings.add_last_action = True

Stack Actions

NameTypeDefault Value(s)Value RangeTarget Observation ElementDescription
stack_actionsint1[1, 48]actionStacks latest N actions together.
Changes observation element type from Discrete or MultiDiscrete (depending on the action space) to MultiDiscrete, having N elements, each with cardinality equal to the original set(s) of the action space
wrappers_settings.stack_actions = 12

For the Stack Actions wrapper to to be applied, the Add Last Action one must be activated.

Scale Observation

NameTypeDefault Value(s)Value RangeTarget Observation ElementDescription
scaleboolFalseTrue / FalseAllActivates observation scaling.
Affects observation elements as follows:
- Box: type kept, changes bounds to be between 0 and 1.
- Discrete (Binary): depends on process_discrete_binary (see below).
- Discrete: changed to MultiBinary through one-hot encoding, size equal to original Discrete cardinality.
- MultiDiscrete: changed to N-MultiBinary through N-times one-hot encoding, N equal to original MultiDiscrete size, encoding size equal to original MultiDiscrete element cardinality
exclude_image_scalingboolFalseTrue / FalseFrameIf True, prevents scaling to be applied on the game frame
process_discrete_binaryboolFalseTrue / FalseDiscrete BinaryControls how Discrete (Binary) observations are treated:
- False: not affected;
- True: changed to MultiBinary through one-hot encoding, size equal to 2
wrappers_settings.scale = True
wrappers_settings.exclude_image_scaling = True
wrappers_settings.process_discrete_binary = True

Role-relative Observation

NameTypeDefault Value(s)Value RangeTarget Observation ElementDescription
role_relativeboolFalseTrue / FalsePlayer specific observationRenames observation depending on the role played by the agent. For example: if the agent is playing with P1 role, P1 key becomes own and P2 key becomes opp, thus obs["P1"]["key_1"] becomes obs["own"]["key_1"] and obs["P2"]["key_1"] becomes obs["opp"]["key_1"]
wrappers_settings.role_relative = True

Flatten and Filter Observation

NameTypeDefault Value(s)Value RangeTarget Observation ElementDescription
flattenboolFalseTrue / FalseObservationActivates observation dictionary flattening, removing nested dictionaries and creating new keys joining original ones using “_” across nesting levels. For example: obs["P1"]["key_1"] becomes obs["P1_key_1"]
filter_keyslist of str[]-Observation except frameDefines the list of RAM states to keep in the observation space
wrappers_settings.flatten = True
wrappers_settings.filter_keys = ["stage", "P1_health", "P2_character"]

The two operations are applied simultaneously, thus for filtering to be applied, flattening must be activated.

Action Wrappers

Repeat Action

NameTypeDefault Value(s)Value RangeDescription
repeat_actionint1[1, 12]Keeps repeating the same action for N environment steps

In order to use this wrapper, the step_ratio setting must be set to equal to 1.

wrappers_settings.repeat_action = 4

Reward Wrappers

Normalize Reward

NameTypeDefault Value(s)Value RangeDescription
normalize_rewardboolFalseTrue / FalseActivates reward normalization. Divides reward value by the product between the normalization_factor value (see next row) and the delta between maximum and minium health bar value for the given game
normalization_factorfloat0.5[-inf, +inf]Defines the normalization factor multiplying the delta between maximum and minium health bar value for the given game
wrappers_settings.normalize_reward = True
wrappers_settings.normalization_factor = 0.5

Clip Reward

NameTypeDefault Value(s)Value RangeDescription
clip_rewardboolFalseTrue / FalseActivates reward clipping. Applies the sign function to the reward value
wrappers_settings.clip_reward = True

Custom Wrappers

NameTypeDefault Value(s)Value RangeDescription
wrapperslist of pairs (list) of [WrapperClass, kwargs][]-Applies in sequence the list of wrappers classes with their correspondend settings
wrappers_settings.wrappers = \
   [[CustomWrapper1, {"setting1_1": value1_1,"setting1_2": value1_2}], \
    [CustomWrapper2, {"setting2_1": value2_1,"setting2_2": value2_2}]]

The custom wrappers are applied after all the activated built-in ones, if any.