ToDo's
======

Next framework implmentation steps:

* parallel computation
* distributed computation
* output manager to identify and connect outputs automatically (always only optional), e.g.:
    * outputs: `Engine`, `Metric`, `OutputHandler`
    * inputs: `Metrics`, `OutputHandler`, `Chart`, `FileLogger`
    * standard naming & typing, e.g. `y`, `y_pred` (?!), 'loss',
* resource & performance optimization tools, e.g. freeing memory, `state object` type-checking-turn-off after first training iteration
* debugger mode features that can be turned on/off with ``state.configs.default_configs.debug_mode = True/False``

Here's a list of possible feature to implement, some of which may only be realizable due to the new framework architecture:

* maintenance features:
    * time sequential data processing analysis
    * resource loggers: ``CpuPsutilLogger``, ``GpuPynvmlLogger``, memory logger, all thread based not callback triggered
    * state object (general objects) finder tools
    * object finder tool
* pipelines:
    * hyperparameter tuning
* visualization:
    * state structure vis tool
    * data flow vis tool
    * time sequential data processing vis tool
    * overlay visuslisation of data flow & data process time sequence (evtl. 3D)
    * full state visualization with current values (in Tensorboard)

Some notes... just notes:

* parallel/distributed computing:
    * automatic distribution of engines/paths to (multi-)threads or (multi-)processes & distributed machines
    * `state` runs on main process whereas engines or paths managed by e.g. a process pool
    * engine/paths-cloning for each distributed machine which are all visualized in main-process-`state`
    * distributed engines/`state`\s update main-`state` which can be visualized "realtime" in TensorBoard
* output manager: standardized classification of outputs, i.e. from engines, metrics, output handlers etc., so successive `bso-containers` know how to handle the input references automatically, e.g. a engine outputs data labels and loss and the successive accuracy metric obviously chooses the labels when the user passes in the "full output reference". It's a simple task to automate but always really annoying for the user. Same with output handlers and charts... "auto-charts/loggers" should select the correct chart/log-class for given outputs, e.g. logging scalar, histogram etc.