Logger

For logging data out of the simulator using HDF5 files.

The logger provides an interface for easily saving time series data from many simulation trials, along with the parameters used for the simulation. Data is logged in HDF5 (Hierarchical Data Format) files.

Data is stored by trial, in a hierarchy like a file structure, as shown below. Values in < > are determined by what you actually log, but the params group and time dataset are always created.

log_file.h5
├── trial_<1>
│   ├── params
│   │   ├── <param_1>
│   │   ├── <param_2>
│   │   ├── ⋮
│   │   └── <param_n>
│   ├── system_info
│   │   ├── datetime_local
│   │   ├── gridsim_version
│   │   ├── ⋮
│   │   └── version
│   ├── time
│   ├── <aggregator_1>
│   ├── <aggregator_2>
│   ├── ⋮
│   └── <aggregator_n>
├── trial_<2>
│   └── ⋮
├── ⋮
└── trial_<n>

All values logged with log_param() and log_config() are saved in params.

Time series data is stored in datasets directly under the trial_<n> group. They are created by add_aggregator(), and new values are added by log_state(). Calling this method also adds a value to the time dataset, which corresponds to the World time at which the state was saved.

class gridsim.logger.Logger(world: gridsim.world.World, filename: str, trial_num: int, overwrite_trials: bool = False)

Logger to save data to an HDF5 file, from a single simulation trial.

Note that creating this only creates the Logger with which you can save data. You must use the methods below to actually save anything to the file with the Logger.

Parameters
  • world (World) – World whose simulation data you want to save.

  • filename (str) – Name of the HDF5 file to save data to (.hdf extension). If the file does not exist, it will be created. If it does exist, it will be appended to (with the overwriting caveat specified below). Using ~ to indicate the home directory in the path is supported. If the directory does not exist, it will be created (if possible).

  • trial_num (int) – Trial number under which to save the data.

  • overwrite_trials (bool, optional) – Whether to overwrite a trial’s data if it already exists, by default False

add_aggregator(name: str, func: Callable[[List[gridsim.robot.Robot]], numpy.ndarray])

Add an aggregator function that will map from the list of all Robots in the world to a 1D array of floats. This will be used for logging the state of the World; the output of the aggregator is one row in the HDF5 Dataset named with the name.

The function reduces the state of the Robots to a single or multiple values. It could map to one float per robot (such as a state variable of each Robot) or a single value (length 1 array, such as an average value over all Robots).

Because of Python’s dynamic typing, this does not validate whether the subclass of Robot has any parameters or functions that are called by the aggregator. The user is responsible for adding any necessary checks in the aggregator function.

Notes

The width of the aggregator table is set when this function is called, which is determined by the length of the output of func. If the length depends on the number of Robots, all Robots should be added to the World before adding any aggregators to the Logger.

The aggregator func will be applied to all robots in the world, regardless of type. However, if you have multiple types of Robots in your World, you can make an aggregator that applies to one type by filtering the robots by type within the func.

Parameters
  • name (str) – Key that will be used to identify the aggregator results in the HDF5 log file.

  • func (Callable[[List[Robot]], np.ndarray]) – Function that maps from a list of Robots to a 1D array to log some state of the Robots at the current time.

Raises

ValueError – If aggregator name uses a reserved name (see above) or if aggregator does not return a 1D numpy array.

get_trial() → int

Get the trial number that this Logger is logging

Returns

Number of the current trial being logged

Return type

int

log_config(config: gridsim.config_parser.ConfigParser, exclude: List[str] = [])

Save all of the parameters in the configuration.

Notes

Due to HDF5 limitations (and my own laziness), only the following datatypes can be saved in the HDF5 parameters:

  • string

  • integer

  • float

  • boolean

  • list of integers and/or floats

Parameters
  • config (ConfigParser) – Configuration loaded from a YAML file.

  • exclude (List[str], optional) – Names (keys) of any configuration parameters to exclude from the saved parameters. This can be useful for excluding an array of values that vary by condition, and you want to only include the single value used in this instance.

log_param(param_name: str, val: Union[str, int, float, bool, list, dict], sub_group: Optional[str] = None)

Save a single parameter value. This is useful for saving fixed parameters that are not part of your configuration file, and therefore not saved with log_config().

This has the same type restrictions for values as log_config().

Parameters
  • param_name (str) – Name/key of the parameter value to save

  • val (Union[str, int, float, bool, list, dict]) – Value of the parameter to save

  • sub_group (str) – Name of a sub-group of the “params” group in which to log the parameter. If not given, the parameter will be placed directly in the params group. You can specify multiple levels of sub-groups by concatenating names with /. e.g., sub/subsub

Warns

UserWarning – If user attempts to save invalid parameter (e.g., invalid data type)

log_state()

Save the output of all of the aggregator functions. If you have not added any aggregators with log_state(), nothing will be saved by this function.

The runs each previously-added aggregator function and appends the result to the respective HDF5 Dataset. It also saves the current time of the World to the time Dataset.

log_system_info()

Log system information for future validation and comparison. This saves the following information as individual datasets within the trial_#/system_info group:

  • system: System name (platform.system()) (e.g., ‘Linux’, ‘Windows’)

  • node: System node/host/network name (platform.node())

  • release: System release (platform.release) (e.g., kernel version)

  • version: System version (platform.version)

  • python_version: Python version (platform.python_version) (e.g., ‘3.8.2’)

  • gridsim_version: Currently installed Gridsim version

  • datetime_local: Local date and time when trial was run