digideep.utility package

Subpackages

• digideep.utility.visdom_engine package
  • Submodules
  • digideep.utility.visdom_engine.Instance module
  • digideep.utility.visdom_engine.WebServer module
  • digideep.utility.visdom_engine.Wrapper module
  • Module contents

Submodules

digideep.utility.filter module

class digideep.utility.filter.MovingAverage(size=1, window_size=10)

Bases: object

An implementation of moving average. It has an internal queue of the values.

Parameters:

• size (python:int) – The length of the value vector.
• window_size (python:int) – The window size for calculation of the moving average.

append(item)

data

max

mean

median

min

std

digideep.utility.logging module

class digideep.utility.logging.Logger

Bases: object

This is a helper class which is intended to simplify logging in a single file from different modules in a package. The Logger class uses a singleton [1] pattern.

It also provides multi-level logging each in a specific style. The levels are DEBUG, INFO, WARN, ERROR, FATAL.

Example

logger.set_log_level(2)
logger.info('This is a test of type INFO.')   # Will not be shown
logger.warn('This is a test of type WARN.')   # Will be shown
logger.fatal('This is a test of type FATAL.') # Will be shown

logger.set_log_level(3)
logger.info('This is a test of type INFO.')   # Will not be shown
logger.warn('This is a test of type WARN.')   # Will not be shown
logger.fatal('This is a test of type FATAL.') # Will be shown

logger.set_logfile('path_to_the_log_file')
# ... All logs will be stored in the specified file from now on.
#     They will be shown on the output as well.

Footnotes

[1]	https://gist.github.com/pazdera/1098129

debug(*args, sep=' ', end='\n', flush=False)

error(*args, sep=' ', end='\n', flush=False)

fatal(*args, sep=' ', end='\n', flush=False)

static getInstance()

Static access method.

info(*args, sep=' ', end='\n', flush=False)

set_log_level(log_level)

set_logfile(filename)

warn(*args, sep=' ', end='\n', flush=False)

digideep.utility.monitoring module

class digideep.utility.monitoring.Monitor

Bases: object

A very simple and lightweight implementation for a global monitoring tool. This class keeps track of a variable’s mean, standard deviation, minimum, maximum, and sum in a recursive manner.

>>> monitor.reset()
>>> for i in range(1000):
...   monitor('loop index', i)
...
>>> print(monitor)
>> loop index [1000x] = 499.5 (+-577.639 %95) in range{0 < 999}

Todo

Provide batched monitoring of variables.

Note

This class does not implement moving average. For a moving average implementation refer to MovingAverage.

discard_key(key)

dump()

get_meta_key(key)

pack_data()

pack_keys(keys)

reset()

set_meta_key(key, value)

set_output_file(path)

class digideep.utility.monitoring.WindowValue(value, window)

Bases: object

append(value)

get_max()

get_min()

get_num()

get_std()

get_sum()

get_win()

digideep.utility.plotting module

class digideep.utility.plotting.Plotter(**params)

Bases: object

append(y, x=None)

digideep.utility.profiling module

class digideep.utility.profiling.KeepTime(name)

Bases: object

add_name()

get_current_level()

get_full_path()

get_level()

pop_name()

set_level()

class digideep.utility.profiling.Profiler

Bases: object

This class provides a very simple yet light implementation of function profiling. It is very easy to use:

>>> profiler.reset()
>>> profiler.start("loop")
>>> for i in range(100000):
...   print(i)
...
>>> profiler.lapse("loop")
>>> print(profiler)
>> loop [1x, 27.1s]

Alternatively, you may use profiler with KeepTime:

>>> with KeepTime("loop2"):
...   for i in range(100000):
...     print(i)
...
>>> print(profiler)
>> loop2 [1x, 0.0s]

Note

The number of callings to start() and lapse() should be the same.

dump(meta={})

get_keys()

get_occurence(name)

get_time_average(name)

get_time_overall(name)

lapse(name)

reset()

set_output_file(path)

start(name)

digideep.utility.stats module

digideep.utility.timer module

class digideep.utility.timer.Timer(task, interval=1.0)

Bases: threading.Thread

Thread that executes a task every N seconds

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

setInterval(interval)

Set the number of seconds we sleep between executing our task

shutdown()

Stop this thread

task_exec()

The task done by this thread - override in subclasses

digideep.utility.toolbox module

digideep.utility.toolbox.count_parameters(model)

Counts the number of parameters in a PyTorch model.

digideep.utility.toolbox.dump_dict_as_json(filename, dic, sort_keys=False)

This function dumps a python dictionary in json format to a file.

Parameters:

• filename (path) – The address to the file.
• dic (dict) – The dictionary to be dumped in json format. It should be json-serializable.
• sort_keys (bool, False) – Will sort the dictionary by its keys before dumping to the file.

digideep.utility.toolbox.dump_dict_as_yaml(filename, dic)

digideep.utility.toolbox.get_class(addr)

Return a instance of a class by using only its name.

Parameters:addr (str) – The name of the class/function which should be in the format MODULENAME[.SUBMODULE1[.SUBMODULE2[...]]].CLASSNAME

digideep.utility.toolbox.get_module(addr)

Return a instance of a module by using only its name.

Parameters:addr (str) – The name of the module which should be in the format MODULENAME[.SUBMODULE1[.SUBMODULE2[...]]]

digideep.utility.toolbox.get_rng_state()

digideep.utility.toolbox.load_json_as_dict(filename)

digideep.utility.toolbox.load_yaml_as_dict(filename)

digideep.utility.toolbox.seed_all(seed, cuda_deterministic=False)

digideep.utility.toolbox.set_rng_state(states)

digideep.utility.toolbox.strict_update(dict_target, dict_source)

Module contents