Input Parameters

-m / --monitor-template: defines the monitor template(s) to be processed. You can choose from the set of all implemented monitors (see Implemented Monitors).

-d / --dataset: defines the ID-dataset that the NN was trained on. You can choose from the set of integrated datasets (see Benchmarks) or implement your own (see Implementation of a dataset).

-mu / --monitor-by-user: If you choose to implement your own monitor, use this parameter instead of --monitor-template. It refers to the file your monitor is implemented in (see Implementation of a monitor).

-mu-name / --monitor-by-user-name: If your monitor in your given file is not named Monitor, you need to provide the class name here.

-mc / --monitor-config: (BETA) Configuration file to simplify the method of implementing a custom monitor and also for setting specified parameters (see Monitor Configuration (BETA)).

-nn / --neural-network: The NN given as either an ONNX or (preferrably because more stable) a Pytorch-model (torch.nn.Module saved with torch.save).

-op / --optimize: Boolean flag that determines whether Monitizer runs its optimization

-oo / --optimization-objective: The optimization configuration file. Monitizer provides a default optimization-configuration.ini. Refer to Optimization configuration on the details of how the config should look like.

-p` / ``--parameters: You can give fixed parameters for the monitor template if you do not want to optimize. This must be given as a list in the format `` “{‘PARAMETER_ONE’ : VALUE, ‘PARAMETER_TWO’ : VALUE}”, e.g. ``"{'temperature':99,'noise':0.9}".

-e / --evaluate: Boolean flag that determines whether Monitizer runs its evaluation.

-ec / --evaluation-criteria: Can be either short,``test``,``full``, or auroc. auroc is the only evalution that works on monitor templates without optimization. Any other evaluation must be performed on optimized monitors or on monitors with predefined values (see –parameters). short and test perform the evaluation on a subset of all OOD datasets. full uses all of them.

-ed / --evaluation-dataset: You can specify the OOD dataset that you want to evaluate on. Check out REFERENCE for possible dataset names you can use.

-c / --confidence-intervals: Boolean flag that determines whether Monitizer additionally outputs confidence intervals.

-o / --output: You can define the location where Monitizer stores its output (see Output).

-l / --output-latex: Whether the output results shall also immediately be printed as latex tabels (useful for paper writing :) ).

-s / --seed: Sets a seed to have reproducible results.

--log: You can specify where to store the log files. The logs are not stored if the location is not specified! CHECK