oneflow.utils.global_view.global_mode

class oneflow.utils.global_view.global_mode(enabled, placement=None, sbp=None)

Create a scope to provide global information for the computation process within it.

It provides convinience for converting from local execution to global execution, especially for converting to ddp global execution.

1. Make the source op create the global tensor directly.

2. Make it legal for the “to(device)” API of the global tensor.

3. Make it legal to use “.device” to get the device type of the global tensor.

Note

Both placement and sbp are required if the global mode is enabled.

Parameters

• enabled (bool) – whether the global mode is enbaled.

• placement (oneflow.placement, optional) – the desired placement of the input. Default: None

• sbp (oneflow.sbp.sbp, list/tuple of oneflow.sbp.sbp, optional) – the desired sbp of the input or self-defined functions in order to specify SBP. Default: None

Example:

.. code-block:: python

class LinearEvalGraphWithDDP(flow.nn.Graph):
    def __init__(self):
        super().__init__()
        self.linear_dp = linear_dp

    def build(self, x):
        with global_mode(True, placement=P, sbp=B):
            device = self.linear_dp.weight.device

            x = x.to(device)

            out = self.linear_dp(x)

            # The local tensor will be converted to global
            sample = flow.randn(out.shape, device="cpu").to(device)
            out = out + sample * 100
            out = out - sample * 100

        return out

.. code-block:: python

with global_mode(False):
    # The tensor will be keeped as local.
    sample = flow.randn(out.shape, device="cpu").to(device)
    out = out + sample * 100
    out = out - sample * 100

__init__(*args, **kwargs)

Overloaded function.

1. __init__(self: oneflow._oneflow_internal.global_view.global_mode, arg0: bool) -> None

2. __init__(self: oneflow._oneflow_internal.global_view.global_mode, enabled: bool, placement: oneflow._oneflow_internal.placement, sbp: List[oneflow::Symbol<oneflow::SbpParallel>]) -> None

3. __init__(self: oneflow._oneflow_internal.global_view.global_mode, enabled: bool, placement: oneflow._oneflow_internal.placement, sbp: oneflow::Symbol<oneflow::SbpParallel>) -> None

Methods

__delattr__(name, /)	Implement delattr(self, name).
__dir__()	Default dir() implementation.
__enter__(self)
__eq__(value, /)	Return self==value.
__exit__(self, arg0, arg1, arg2)
__format__(format_spec, /)	Default object formatter.
__ge__(value, /)	Return self>=value.
__getattribute__(name, /)	Return getattr(self, name).
__gt__(value, /)	Return self>value.
__hash__()	Return hash(self).
__init__(*args, **kwargs)	Overloaded function.
__init_subclass__	This method is called when a class is subclassed.
__le__(value, /)	Return self<=value.
__lt__(value, /)	Return self<value.
__ne__(value, /)	Return self!=value.
__new__(**kwargs)	Create and return a new object.
__reduce__()	Helper for pickle.
__reduce_ex__(protocol, /)	Helper for pickle.
__repr__()	Return repr(self).
__setattr__(name, value, /)	Implement setattr(self, name, value).
__sizeof__()	Size of object in memory, in bytes.
__str__()	Return str(self).
__subclasshook__	Abstract classes can override this to customize issubclass().