With the goal of supporting the exploration and research of techniques dealing with offline learning based on data (e.g. offline Reinforcement Learning, Imitation Learning, Behavioral Cloning, etc.), DIAMBRA Arena comes with two, easy-to-use, useful features: an environment wrapper allowing to record episodes (to be used, for example, to save human expert demonstrations), and a loader class allowing to easily access the stored dataset.
In order to activate the episode recording wrapper, one needs to properly set the RecordingSettings class attributes and provide them as input to the environment creation method, as shown in the next code block.
from diambra.arena import EnvironmentSettings, WrapperSettings, RecordingSettings
# Settings specification
settings = EnvironmentSettings()
# Wrapper settings specification
wrapper_settings = WrapperSettings()
# Recording settings specification
recording_settings = RecordingSettings()
recording_settings.setting_1 = value_1
recording_settings.setting_2 = value_2
env = diambra.arena.make("doapp", settings, wrappers_settings, recording_settings)
The class attributes are described in the following table:
| Name | Type | Default Value(s) | Description |
|---|---|---|---|
username | None U str | None | Provides an identifier to be associated with the recorded episode |
dataset_path | None U str | None | Specifies the path where to save recorded episodes |
episode_recording_settings.username = "user"
episode_recording_settings.dataset_path = "/home/user/DIAMBRA/"
Use of this functionality can be found in this example.
Implementation examples and templates can be found in the code repository, here.
By default, this wrapper acts before wrappers are applied. Thus, it will store the original, native, unprocessed observations (and reward and actions) as generated by the base environment. This guarantees a better generality and transferability of the generated dataset, but requires preprocessing at load time.
DIAMBRA Arena provides a simple dedicated class DiambraDataLoader demonstrating how to load and minimally process recorded episodes, it only requires the dataset folder path as input parameter and can be customized adding the additional processing operations required. The data loader class is created as follows:
from diambra.arena.utils.diambra_data_loader import DiambraDataLoader
data_loader = DiambraDataLoader(dataset_path)
Use of this functionality can be found in this example.
Implementation of this class can be found in the code repository, here.