Metrics

class ignite_framework.metrics.base_metric.BaseMetric(metric_name, input_ref, started_ref='engine_run_started_ref', iteration_completed_ref='n_iteration_completed_ref', completed_ref='n_epoch_completed_ref', device=<ignite_framework.framework.framework.StateObjectsReference object>)

Base class for all Metrics.

Parameters:

• metric_name (str) – name of metric which will be complemented with the engines’s name as prefix separated with a _`. The engine’s name is derived from ‘input_ref`.
• input_ref (StateObjectsReference) – Reference of the output variable from which the metric is calculated, normally the output of a engine, e.g. state.trainer (i.e. state.engines.trainer)
• output_transform (callable, optional) – a callable that is used to transform the Engine’s process_function’s output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.
• device (str of torch.device, optional) – device specification in case of distributed computation usage. In most of the cases, it can be defined as “cuda:local_rank” or “cuda” if already set torch.cuda.set_device(local_rank). By default, if a distributed process group is initialized and available, device is set to cuda.

completed()

compute()

Computes the metric based on it’s accumulated state. This is called at the end of each epoch. :returns: the actual quantity of interest. :rtype: Any

Raises:NotComputableError – raised when the metric cannot be computed.

device

StateDevice is the device parameter defining the device the processes of the state object container should run on.

iteration_completed()

name = 'BaseMetric'

output

StateVariable is the basic state variable without any additional type checking or setting conditions.

Features:

• gets/sets/calls a _value and gets/sets/calls a list of callbacks which are callables triggered when _value is set

• overloaded suffixes: [‘_callbacks’, _ref’, ‘_every’, ‘_once’]

  • _ref: returns StateObjectsReference-instance of itself
  • _callbacks: returns list of callbacks (functions called when value is set)

• set-value type dependent __set__() overload:

  • CALLABLE_TYPES & tuple`([<CALLABLE_TYPES>, <dict>]): appended to its `_callbacks-list
  • any other type: sets its _value

• state shortcut auto-generation

reset()

Resets the metric to it’s initial state. This is called at the start of each epoch.

started()

update()

Updates the metric’s state using the passed batch output given by self._input_ref.caller_name. This is called once for each batch. :param output: the is the output from the engine’s process function.

ignite_framework.metrics.base_metric.reinit__is_reduced(func)

ignite_framework.metrics.base_metric.sync_all_reduce(*attrs)

class ignite_framework.metrics.metrics.AverageOutput(metric_name, input_ref, started_ref='engine_run_started_ref', iteration_completed_ref='n_iteration_completed_ref', completed_ref='n_epoch_completed_ref', batch_size_fn=<function AverageOutput.<lambda>>, device='', ignore_zero_division=True)

Averaged output over Metric’s period start_event till completed_event with measures taken at every iteration_completed_event.

compute()

Computes the metric based on it’s accumulated state. This is called at the end of each epoch. :returns: the actual quantity of interest. :rtype: Any

Raises:NotComputableError – raised when the metric cannot be computed.

device

StateDevice is the device parameter defining the device the processes of the state object container should run on.

get(*name_or_parts)

Get method for parameterized attribute names, especially for state object containers, but not only.

Main reason is to simply avoid unhandy commands, e.g. getattr(state.transitions, ‘{}_{}’.format(name, suffix) commands and simply write state.transitions.get(name, suffix).

get_bso_names()

name = 'AverageOutput'

output

StateVariable is the basic state variable without any additional type checking or setting conditions.

Features:

• gets/sets/calls a _value and gets/sets/calls a list of callbacks which are callables triggered when _value is set

• overloaded suffixes: [‘_callbacks’, _ref’, ‘_every’, ‘_once’]

  • _ref: returns StateObjectsReference-instance of itself
  • _callbacks: returns list of callbacks (functions called when value is set)

• set-value type dependent __set__() overload:

  • CALLABLE_TYPES & tuple`([<CALLABLE_TYPES>, <dict>]): appended to its `_callbacks-list
  • any other type: sets its _value

• state shortcut auto-generation

reset()

Resets the metric to it’s initial state. This is called at the start of each epoch.

set(*name_or_parts_and_value)

update()

Updates the metric’s state using the passed batch output given by self._input_ref.caller_name. This is called once for each batch. :param output: the is the output from the engine’s process function.

class ignite_framework.metrics.metrics.OutputMetric(metric_name, input_ref, caller_ref='', device='')

compute()

Computes the metric based on it’s accumulated state. This is called at the end of each epoch. :returns: the actual quantity of interest. :rtype: Any

Raises:NotComputableError – raised when the metric cannot be computed.

device

StateDevice is the device parameter defining the device the processes of the state object container should run on.

get(*name_or_parts)

Get method for parameterized attribute names, especially for state object containers, but not only.

Main reason is to simply avoid unhandy commands, e.g. getattr(state.transitions, ‘{}_{}’.format(name, suffix) commands and simply write state.transitions.get(name, suffix).

get_bso_names()

name = 'OutputMetric'

output

StateVariable is the basic state variable without any additional type checking or setting conditions.

Features:

• gets/sets/calls a _value and gets/sets/calls a list of callbacks which are callables triggered when _value is set

• overloaded suffixes: [‘_callbacks’, _ref’, ‘_every’, ‘_once’]

  • _ref: returns StateObjectsReference-instance of itself
  • _callbacks: returns list of callbacks (functions called when value is set)

• set-value type dependent __set__() overload:

  • CALLABLE_TYPES & tuple`([<CALLABLE_TYPES>, <dict>]): appended to its `_callbacks-list
  • any other type: sets its _value

• state shortcut auto-generation

reset()

Resets the metric to it’s initial state. This is called at the start of each epoch.

set(*name_or_parts_and_value)

update()

Updates the metric’s state using the passed batch output given by self._input_ref.caller_name. This is called once for each batch. :param output: the is the output from the engine’s process function.

class ignite_framework.metrics.metrics.RunningAverage(metric_name, input_ref, caller_ref='', alpha=0.98, device='')

Compute running average of a output, e.g. a metric ouput or the output of process function.

Note: a metric class as metric_output_ref is not handled here anymore due to the enhancement of the metric arguments to customize the callback variables (events). Nevertheless, this feature with default callback variables can easily be implemented if desired.

alpha

StateParameter is the basic state parameter without any additional type checking or setting conditions. StateParameters of any kind should be made available in state.params for e.g. hyper-parameter optimization or Tensorboard tracking.

Features:

• same as StateVariable
• state.params reference auto-generation as state.params.<bso_ctr_name>_<parameter_name>

compute()

Computes the metric based on it’s accumulated state. This is called at the end of each epoch. :returns: the actual quantity of interest. :rtype: Any

Raises:NotComputableError – raised when the metric cannot be computed.

device

StateDevice is the device parameter defining the device the processes of the state object container should run on.

get(*name_or_parts)

Get method for parameterized attribute names, especially for state object containers, but not only.

Main reason is to simply avoid unhandy commands, e.g. getattr(state.transitions, ‘{}_{}’.format(name, suffix) commands and simply write state.transitions.get(name, suffix).

get_bso_names()

name = 'RunningAverage'

output

StateVariable is the basic state variable without any additional type checking or setting conditions.

Features:

• gets/sets/calls a _value and gets/sets/calls a list of callbacks which are callables triggered when _value is set

• overloaded suffixes: [‘_callbacks’, _ref’, ‘_every’, ‘_once’]

  • _ref: returns StateObjectsReference-instance of itself
  • _callbacks: returns list of callbacks (functions called when value is set)

• set-value type dependent __set__() overload:

  • CALLABLE_TYPES & tuple`([<CALLABLE_TYPES>, <dict>]): appended to its `_callbacks-list
  • any other type: sets its _value

• state shortcut auto-generation

reset()

Resets the metric to it’s initial state. This is called at the start of each epoch.

set(*name_or_parts_and_value)

update()

Updates the metric’s state using the passed batch output given by self._input_ref.caller_name. This is called once for each batch. :param output: the is the output from the engine’s process function.