Quickunderstanding Feature Dev ============================== Here we are, after all the claiming that **this framework enhances Ignite to a rapid feature development tool**! I hope I'm mot wrong... personally, I experience implementing new fully integrated, shortcutted, auto-categorized features/transitions/`bso-containers` (you name it) with tons of individual events a.k.a. `state objects` feels exactly like implementing a simple standard python class... the framework base classes deliver the full integration for free. The only differences are: * You define your `state objects` (a.k.a. custom events) as simple class attribute and select the desired `state object` types which add even more functionalities for free. * There is a simple order for `__init__` when to e.g. initialize the base class and when to set the callbacks and so on. * For full integration simple naming rules for input & output attributes must be observed. In the following I will quickly discuss `demo features `_ but I would also recommend to have a look at already existing `bso-containers` and trainsitions which are recommonden below the demo code blocks. Based on the demos the `basic rules for feature implementation `_ (:doc:`examples/quickunderstanding_feature_dev_example`) are presented followed by `some more detailed sections `_. .. note:: The framework offers really quick (& dirty) hacks to get whatever idea up & running very quickly. You can declare this a feature of the framework & you are welcome to mess up what is needed to get fast results in your script. To guarantee a long-term stable architecture that keeps running even when messing up the application scripts, clear design rules for new features are mandatory and are part of the architecture like the implementation itself. In progress ----------- The "How-to-implement-feautres" guideline is just being written in this very moment meaning everything presented here is half cooked. Everybody is welcome to contribute, question, change & progress everything... if anything is not clear enough, I either haven't documented it clearly or haven't thought of or implemented it clearly enough. Simple feature demos -------------------- Below, two very simple examples are given to demonstrate a low-level feature (e.g. transition like a metric, or an output handler) handling a single task and a high level feature (e.g. pipeline or composition like a supervised trainer or a metrics comparision chart generator) handling multiple low-level features/tasks sequentially or in parallel. Underneath each example proper implementations of real features are listed for study and comparison. .. code-block:: python # STATE from ignite_framework.states import state # BASE CLASS from ignite_framework.transitions import BaseTransition # UTILS from ignite_framework.feature_dev_tools.state_objects import StateDevice, StateVariable from ignite_framework.feature_dev_tools.utils import get_caller_refs, create_bso_ctr_name_prefix_from_input_ref # SELECT THE BASE CLASS # NOTE: # - The base class will define the state container to which # the transition/feature is automatically attached to. # - Here `BaseTransition` will assign `DoubleTheInput` to `state.transitions` class DoubleTheInput(BaseTransition): # Set up state object (current Ignite: "custom event") output = StateVariable() device = StateDevice() def __init__(self, double_name, input_ref, caller_refs='', device=state.configs.default_configs.device_for_modules_ref): # ARGUMENT CHECKING # NOTE: # - mainly `input_ref(s)` and `caller_ref(s)` are reformatted # (e.g. as list) or set to default values etc. # - the `caller_ref(s)` default values is(are) normally the `input_ref(s)` caller_refs = get_caller_refs(caller_refs=caller_refs, input_refs=input_ref) # AUTOMATIC OR ASSISTED NAMING name = '_'.join([create_bso_ctr_name_prefix_from_input_ref(input_ref=input_ref), double_name]) # BASE CLASS INITIALIZATION # NOTE: Initializing the base class start framework support, therefore # to order when it's initialized MATTERS! super().__init__(name=name) # ASSIGN REFERENCES # Assigned input references which will be tracked by the framework self._input_ref = input_ref # NOTE: # - `device` could also be a fixed value, e.g. `gpu` or `cpu' # - passing in a reference synchronizes `self.device` with the ref, otherwise fixed value is set self.device = device # Everything else self._double_factor = 2 # APPEND METHODS TO CALLER REFS CALLBACKS # NOTE: The callback appending can be individualized as required self._set_callbacks(caller_refs=caller_refs) def run(self): self.output = self._double_factor * self._input_ref.caller_name def _set_callbacks(self, caller_refs): for caller_ref in caller_refs: # Append the methods as desired to the callbacks caller_ref.caller_name = ('short description is helpful', self.run) # NOTE: # - if bso-container has method `run` then run will be detected # and append when simply `self` is assigned, see below # - alternative implementations with automatic description # generation: `caller_ref.caller_name = self` Note that most code lines are comments, the real transition code is really short! Good examples of useful feature are e.g. :doc:`source_code/engines` (in comparison to `current Ignite Engine `_\), :doc:`source_code/base_metric`, :doc:`source_code/metrics` which may also look familiar. .. code-block:: python # BASE CLASS from ignite_framework.pipelines import BasePipeline # FEATURE CLASSES from ignite_framework.output_handlers import OutputHandler from ignite_framework.metrics import OutputMetric class CurrentIgniteMetricWithOutputHandler(BasePipeline): """ Demo pipeline dowing nothing but piping e.g. the engine output through an output handler and a metric. """ def __init__(self, name, input_ref, caller_refs='', metric_device=''): # No Auto or assisted naming # Argument checking caller_refs = get_caller_refs(caller_refs=caller_refs, input_refs=input_ref) # Initialize base class super().__init__(name=name, device=metric_device) # SAVE NAME OF PIPELINE BSO-CONTAINERS TO `self.composed_bso_ctrs` ordered dictionary # NOTE: The name of each bso-container must be related to the pipeline `name` self.composed_bso_ctrs['output_handler'] = OutputHandler(input_refs=input_ref, output_handler_name=name, caller_refs=caller_refs).name self.composed_bso_ctrs['metric'] = OutputMetric(metric_name=name + '_metric', input_ref=input_ref, caller_refs='') # All callbacks are set by `OutputHandler` and `OutputMetric` Basic feature implementation rules ---------------------------------- Try to use this as a checklist when implementing new features. That's why details of the checklist are discussed separately in successive sections: * **never use** ``Frame*`` **classes: feature development and framework development are strictly separated** * define feature scope and resulting ``Base*``-class: * single task feature: choose according base class to assign the feature to the correct `state container`, e.g. ``BaseMetric`` to assign to ``state.metrics``, ``BaseCharts`` for ``state.tensorboard`` or ``BaseOutputHandler`` for ``state.output_handlers`` etc. * multi task feature: choose ``BasePipeline`` or ``Base*Composition`` explained `here `_ * `state objects` defined as simple class attribute: * define `state objects` before ``__init__`` (overview) * only attributes references by other features should be defined as `state object` (otherwise regular instance attribute) * naming: * `output` reserved for feature output within the model dataflow path, i.e. value calculated from ``input_ref`` * currently no further restrictions * currently no further restrictions/rulse * ``__init__`` arguments and command sequence: * ``__init__`` arguments: * notes: * any arguments requiring a `state object reference` as value must have argument name with suffix `_ref` * default value placeholder is ``''``, **not** ``None`` * data references must be assigned during `__init__` * `bso container name`: * ``name``: requires manual `bso ctr name` setting, no assisted prefix addition, no automatic naming * ``name=''``: automatic naming, optional manual `bso ctr name` setting, no assisted prefix addition * ``*_name``: requires manual `bso ctr name` setting, with assisted prefix addition, no automatic naming * ``*_name=''``: automatic naming, optional manual `bso ctr name` setting with assisted prefix addition * `input_ref(s)`: * only input data originating from model output is named ``input_ref(s)`` (for dataflow analysis) * ``input_ref(s)`` must be assigned to ``self._input_ref(s)`` (for dataflow graph analysis) * ``__init__`` command sequence (when do what in ``__init__``)`: 1. arguments checking and reformating 2. automatic and/or assisted name creation ``*_name`` --> ``name`` (if required) 3. base class initialization with automatic/assisted/manual ``name`` (launches reference tracking and shortcut creation) 4. reference assignment, e.g. ``self._input_ref(s) input_ref(s)`` 5. everything else 6. set callbacks ``self._set_callbacks``: * default value: ``caller_ref(s)=''`` defaults to ``caller_ref(s)=input_ref(s)`` * ``output`` `state object`: * only output data originating from model output is set as ``ouput`` (for dataflow analysis) * successive `bso containers`/features may reference the output as ``SuccessiveFeature(.., input_ref=this_feature.output_ref,...)`` * `everything else unrestricted`: * any methods may be arbitrarily implemented * any methods may be arbitrarily appended to any callbacks (just appending needs to happen in ``self._set_callbacks()``) * any number of `state objects` allowed (but please don't trash ``state`` with useless `state objects`) Basic feature design rules -------------------------- The following rules try to guide multiple processes: * correct application feature implementation * guideline for standardizing and enhancing the process of application feature implementation * guideline for enhancing the framework features and architecture and prevent the framework from degradation. While coding an application or framework feature or refining the framework architecture, keep these aspects in mind: * a `bso container`/feature only performs exactly one task: * low-level, e.g.: transform output, calculate metric * high-level, e.g.: coordinate multiple low-level (or high-level) features, also just one task * a feature is encapsulated: * never holds other features, despite `state components` (e.g. dataloader in engine) * above means: no more passing in a output handler into a metric as in current Ignite * data transfer between feature exclusively with `state objects references` which are set at initialization * method calls between features exclusively with callbacks, despite assigned `state component` methods (e.g. trainer engine gets next batch form trainer dataloader) * above means: time sequential alignment of feature processing is managed exclusively with `state object callbacks` * internal methods of a feature can of course call its own internal methods * maximum user overview: * every feature is categorized in a `state container`, guaranteed by base classes and encapsulation * high individualization and provision of manual setting options * automation always builds on low-level features and should never exclude the possibility of manual individualizations * designed for complete user spectrum: researchers, engineers, feature developers, application developers * designed for any user level: from experts to beginners * experts works with low-level features (actually with any feature) * beginner works with high-level features * everybody benefits from ``state`` overview and feature encapsulation * **PYTHONIC** (hope it still is!) State objects types ------------------- T.B.A. Bso container types and base class selection -------------------------------------------- T.B.A. I'm not quite sure how clear the naming of the framework objects is, so just for recapitulation: * `Feature` here is often used as synonym for `base state object container` (`bso-container`) because it is more graphic. But it also is a little ambiguous w.r.t it's general meaning and features provided by the framework (e.g. shortcuts). Still `Feature Dev` is really about developing all kinds of features, not only transitions, metrics and output handlers, but also state plugins, new engines, useful analysis and visualization tools etc., all new features that are based on `bso containers`. * `Transitions` are the most common `bso containers`, e.g. engines, metrics, output handler, but not e.g. `state components`, `default configs` container etc.. This is why we are talking about `feature dev` and not of transition dev. * `State components`... The namings above also relate to the (super-)types of the feature classes: 3 main `state container` types all inherited from supertypeclass ``MetaStateContainer``: ``MetaBaseTransitionsContainer``, ``MetaStateComponentsContainer``, ``MetaStateValueContainer`` Transitions ----------- T.B.A. State Components ---------------- T.B.A. State value containers ---------------------- T.B.A. Pipelines & Compositions ........................ T.B.A. ContainerIDs ------------ T.B.A.