oneflow.nn.MinMaxObserver

class oneflow.nn.MinMaxObserver(quantization_formula: str = 'google', quantization_bit: int = 8, quantization_scheme: str = 'symmetric', per_layer_quantization: bool = True)

Compute the quantization parameters of the input tensor.

First compute the max and min values of input tensor:

\[ \begin{align}\begin{aligned}& max\_value = max(input)\\& min\_value = min(input)\end{aligned}\end{align} \]

Then compute the scale and zero_point with the following equations:

if quantization_scheme == “symmetric”:

\[ \begin{align}\begin{aligned}& denom = 2^{quantization\_to\_bit - 1} - 1\\& scale = max(|max\_value|,|min\_value|) / denom\\& zero\_point = 0\end{aligned}\end{align} \]

elif quantization_scheme == “affine”:

\[ \begin{align}\begin{aligned}& denom = 2^{quantization\_to\_bit} - 1\\& scale = (max\_value - min\_value) / denom\\& zero\_point = -min\_value / scale\end{aligned}\end{align} \]

If per_layer_quantization is False, then the shape of scale and zero_point will be (input.shape[0],).

Parameters

  • input (oneflow.Tensor) – the input value(s), in oneflow.float32.

  • quantization_formula (str) – Support “google” or “cambricon”.

  • quantization_bit (int) – Quantize input to uintX / intX, X can be in range [2, 8]. Defaults to 8.

  • quantization_scheme (str) – “symmetric” or “affine”, quantize to signed / unsigned integer. Defaults to “symmetric”.

  • per_layer_quantization (bool) – True or False, means per-layer / per-channel quantization. Defaults to True.

Returns

The scale and zero_point of input tensor.

Return type

Tuple[oneflow.Tensor, oneflow.Tensor]

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> weight = (np.random.random((2, 3, 4, 5)) - 0.5).astype(np.float32)

>>> input_tensor = flow.tensor(
...    weight, dtype=flow.float32
... )

>>> quantization_bit = 8
>>> quantization_scheme = "symmetric"
>>> quantization_formula = "google"
>>> per_layer_quantization = True

>>> min_max_observer = flow.nn.MinMaxObserver(quantization_formula=quantization_formula, quantization_bit=quantization_bit,
... quantization_scheme=quantization_scheme, per_layer_quantization=per_layer_quantization)

>>> scale, zero_point = min_max_observer(
...    input_tensor, )
__init__(quantization_formula: str = 'google', quantization_bit: int = 8, quantization_scheme: str = 'symmetric', per_layer_quantization: bool = True) → None

Calls super().__setattr__(‘a’, a) instead of the typical self.a = a to avoid Module.__setattr__ overhead. Module’s __setattr__ has special handling for parameters, submodules, and buffers but simply calls into super().__setattr__ for all other attributes.

Methods

__call__(*args, **kwargs)

Call self as a function.

__delattr__(name)

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__(value, /)

Return self==value.

__format__(format_spec, /)

Default object formatter.

__ge__(value, /)

Return self>=value.

__getattr__(name)

__getattribute__(name, /)

Return getattr(self, name).

__getstate__()

__gt__(value, /)

Return self>value.

__hash__()

Return hash(self).

__init__([quantization_formula, …])

Calls super().__setattr__(‘a’, a) instead of the typical self.a = a to avoid Module.__setattr__ overhead.

__init_subclass__

This method is called when a class is subclassed.

__le__(value, /)

Return self<=value.

__lt__(value, /)

Return self<value.

__ne__(value, /)

Return self!=value.

__new__(**kwargs)

Create and return a new object.

__reduce__()

Helper for pickle.

__reduce_ex__(protocol, /)

Helper for pickle.

__repr__()

Return repr(self).

__setattr__(name, value)

Implement setattr(self, name, value).

__setstate__(state)

__sizeof__()

Size of object in memory, in bytes.

__str__()

Return str(self).

__subclasshook__

Abstract classes can override this to customize issubclass().

_apply(fn)

_get_backward_hooks()

Returns the backward hooks for use in the call function.

_get_name()

_load_from_state_dict(state_dict, prefix, …)

_maybe_warn_non_full_backward_hook(args, …)

_named_members(get_members_fn[, prefix, recurse])

_register_load_state_dict_pre_hook(hook[, …])

These hooks will be called with arguments: state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs, before loading state_dict into self.

_register_state_dict_hook(hook)

These hooks will be called with arguments: self, state_dict, prefix, local_metadata, after the state_dict of self is set.

_save_to_state_dict(destination, prefix, …)

_shallow_repr()

_to_memory_format(memory_format)

Casts the parameters and buffers in this module to another memory format.

add_module(name, module)

Adds a child module to the current module.

apply(fn)

Applies fn recursively to every submodule (as returned by .children()) as well as self.

buffers([recurse])

Returns an iterator over module buffers.

children()

Returns an iterator over immediate children modules.

cpu()

Moves all model parameters and buffers to the CPU.

cuda([device])

Moves all model parameters and buffers to the GPU.

double()

Casts all floating point parameters and buffers to double datatype.

eval()

Sets the module in evaluation mode.

extra_repr()

Set the extra representation of the module

float()

Casts all floating point parameters and buffers to float datatype.

forward(input)

get_parameter(target)

Return the parameter refenreced by target.

get_submodule(target)

Get submodule accroding to the name of submodule.

half()

Casts all floating point parameters and buffers to half datatype.

load_state_dict(state_dict[, strict])

Copies parameters and buffers from state_dict into this module and its descendants.

make_contiguous_params_group()

Get contiguous parameters group after creating the whole module.

modules()

Returns an iterator over all modules in the network.

named_buffers([prefix, recurse])

Returns an iterator over module buffers, yielding both the name of the buffer as well as the buffer itself.

named_children()

Returns an iterator over immediate children modules, yielding both the name of the module as well as the module itself.

named_modules([memo, prefix])

Returns an iterator over all modules in the network, yielding both the name of the module as well as the module itself.

named_parameters([prefix, recurse])

Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.

parameters([recurse])

Returns an iterator over module parameters.

register_backward_hook(hook)

Registers a backward hook on the module.

register_buffer(name, tensor[, persistent])

Adds a buffer to the module.

register_forward_hook(hook)

Registers a forward hook on the module.

register_forward_pre_hook(hook)

Registers a forward pre-hook on the module.

register_full_backward_hook(hook)

Registers a backward hook on the module.

register_parameter(name, param)

Adds a parameter to the module.

register_state_dict_pre_hook(hook)

These hooks will be called with arguments: self, prefix, and keep_vars before calling state_dict on self.

requires_grad_([requires_grad])

Change if autograd should record operations on parameters in this module.

state_dict([destination, prefix, keep_vars])

Returns a dictionary containing a whole state of the module.

to(*args, **kwargs)

Moves and/or casts the parameters and buffers.

to_consistent(*args, **kwargs)

This interface is no longer available, please use oneflow.nn.Module.to_global() instead.

to_empty(*, device)

Moves the parameters and buffers to the specified device without copying storage.

to_global([placement, sbp])

Convert the parameters and buffers to global.

to_local()

to_memory_format(memory_format)

train([mode])

Sets the module in training mode.

zero_grad([set_to_none])

Sets gradients of all model parameters to zero.

Attributes

_grad_t

alias of Union[Tuple[oneflow.Tensor, …], oneflow.Tensor]