oneflow.Tensor.local_to_global

Tensor.local_to_global(placement=None, sbp=None, *, check_meta=Ture) → Tensor

Creates a global tensor from a local tensor.

Note

This tensor must be local tensor.

Both placement and sbp are required.

The returned global tensor takes this tensor as its local component in the current rank.

There is no data communication usually, but when sbp is oneflow.sbp.broadcast, the data on rank 0 will be broadcast to other ranks.

Parameters

• placement (flow.placement, optional) – the desired placement of returned global tensor. Default: None

• sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional) – the desired sbp of returned global tensor. Default: None

Keyword Arguments

check_meta (bool, optional) – indicates whether to check meta information when createing global tensor from local tensor. Only can be set to False when the shape and dtype of the input local tensor on each rank are the same. If set to False, the execution of local_to_global can be accelerated. Default: True

>>> # Run on 2 ranks respectively
>>> import oneflow as flow
>>> input = flow.tensor([0., 1.], dtype=flow.float32) 
>>> output = input.local_to_global(placement=flow.placement("cpu", ranks=[0, 1]), sbp=[flow.sbp.split(0)], check_meta=False) 
>>> print(output.size()) 
>>> print(output) 

>>> # results on rank 0
oneflow.Size([4])
tensor([0., 1., 0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)

>>> # results on rank 1
oneflow.Size([4])
tensor([0., 1., 0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)