Logger¶
For logging data out of the simulator using HDF5 files.
The logger provides an interface for easily saving time series data from many simulation trials, along with the parameters used for the simulation. Data is logged in HDF5 (Hierarchical Data Format) files.
Data is stored by trial, in a hierarchy like a file structure, as shown below. Values in <
>
are determined by what you actually log, but the params
group and time
dataset are always
created.
log_file.h5
├── trial_<1>
│ ├── params
│ │ ├── <param_1>
│ │ ├── <param_2>
│ │ ├── ⋮
│ │ └── <param_n>
│ ├── system_info
│ │ ├── datetime_local
│ │ ├── gridsim_version
│ │ ├── ⋮
│ │ └── version
│ ├── time
│ ├── <aggregator_1>
│ ├── <aggregator_2>
│ ├── ⋮
│ └── <aggregator_n>
├── trial_<2>
│ └── ⋮
├── ⋮
└── trial_<n>
All values logged with log_param()
and
log_config()
are saved in params
.
Time series data is stored in datasets directly under the trial_<n>
group. They are created by
add_aggregator()
, and new values are added by
log_state()
. Calling this method also adds a value to the time
dataset, which corresponds to the World
time at which the state was saved.
-
class
gridsim.logger.
Logger
(world: gridsim.world.World, filename: str, trial_num: int, overwrite_trials: bool = False)¶ Logger to save data to an HDF5 file, from a single simulation trial.
Note that creating this only creates the Logger with which you can save data. You must use the methods below to actually save anything to the file with the Logger.
- Parameters
world (World) – World whose simulation data you want to save.
filename (str) – Name of the HDF5 file to save data to (
.hdf
extension). If the file does not exist, it will be created. If it does exist, it will be appended to (with the overwriting caveat specified below). Using~
to indicate the home directory in the path is supported. If the directory does not exist, it will be created (if possible).trial_num (int) – Trial number under which to save the data.
overwrite_trials (bool, optional) – Whether to overwrite a trial’s data if it already exists, by default False
-
add_aggregator
(name: str, func: Callable[[List[gridsim.robot.Robot]], numpy.ndarray])¶ Add an aggregator function that will map from the list of all Robots in the world to a 1D array of floats. This will be used for logging the state of the World; the output of the aggregator is one row in the HDF5 Dataset named with the
name
.The function reduces the state of the Robots to a single or multiple values. It could map to one float per robot (such as a state variable of each Robot) or a single value (length 1 array, such as an average value over all Robots).
Because of Python’s dynamic typing, this does not validate whether the subclass of Robot has any parameters or functions that are called by the aggregator. The user is responsible for adding any necessary checks in the aggregator function.
Notes
The width of the aggregator table is set when this function is called, which is determined by the length of the output of
func
. If the length depends on the number of Robots, all Robots should be added to theWorld
before adding any aggregators to theLogger
.The aggregator
func
will be applied to all robots in the world, regardless of type. However, if you have multiple types of Robots in yourWorld
, you can make an aggregator that applies to one type by filtering the robots by type within thefunc
.- Parameters
name (str) – Key that will be used to identify the aggregator results in the HDF5 log file.
func (Callable[[List[Robot]], np.ndarray]) – Function that maps from a list of Robots to a 1D array to log some state of the Robots at the current time.
- Raises
ValueError – If aggregator
name
uses a reserved name (see above) or if aggregator does not return a 1D numpy array.
-
get_trial
() → int¶ Get the trial number that this Logger is logging
- Returns
Number of the current trial being logged
- Return type
int
-
log_config
(config: gridsim.config_parser.ConfigParser, exclude: List[str] = [])¶ Save all of the parameters in the configuration.
Notes
Due to HDF5 limitations (and my own laziness), only the following datatypes can be saved in the HDF5 parameters:
string
integer
float
boolean
list of integers and/or floats
- Parameters
config (ConfigParser) – Configuration loaded from a YAML file.
exclude (List[str], optional) – Names (keys) of any configuration parameters to exclude from the saved parameters. This can be useful for excluding an array of values that vary by condition, and you want to only include the single value used in this instance.
-
log_param
(param_name: str, val: Union[str, int, float, bool, list, dict], sub_group: Optional[str] = None)¶ Save a single parameter value. This is useful for saving fixed parameters that are not part of your configuration file, and therefore not saved with
log_config()
.This has the same type restrictions for values as
log_config()
.- Parameters
param_name (str) – Name/key of the parameter value to save
val (Union[str, int, float, bool, list, dict]) – Value of the parameter to save
sub_group (str) – Name of a sub-group of the “params” group in which to log the parameter. If not given, the parameter will be placed directly in the params group. You can specify multiple levels of sub-groups by concatenating names with
/
. e.g.,sub/subsub
- Warns
UserWarning – If user attempts to save invalid parameter (e.g., invalid data type)
-
log_state
()¶ Save the output of all of the aggregator functions. If you have not added any aggregators with
log_state()
, nothing will be saved by this function.The runs each previously-added aggregator function and appends the result to the respective HDF5 Dataset. It also saves the current time of the World to the
time
Dataset.
-
log_system_info
()¶ Log system information for future validation and comparison. This saves the following information as individual datasets within the
trial_#/system_info
group:system
: System name (platform.system()
) (e.g., ‘Linux’, ‘Windows’)node
: System node/host/network name (platform.node()
)release
: System release (platform.release
) (e.g., kernel version)version
: System version (platform.version
)python_version
: Python version (platform.python_version
) (e.g., ‘3.8.2’)gridsim_version
: Currently installed Gridsim versiondatetime_local
: Local date and time when trial was run