oneflow.Tensor.local_to_global

Tensor.local_to_global(placement=None, sbp=None, *, check_meta=True, copy=False)Tensor

Creates a global tensor from a local tensor.

Note

This tensor must be local tensor.

Both placement and sbp are required.

The returned global tensor takes this tensor as its local component in the current rank.

There is no data communication usually, but when sbp is oneflow.sbp.broadcast, the data on rank 0 will be broadcast to other ranks.

Warning

When the sbp is oneflow.sbp.broadcast, the data on the non-0 rank will be modified. If you want to keep the input local tensor unchanged, please set the arg copy to True.

Parameters
  • placement (flow.placement, optional) – the desired placement of returned global tensor. Default: None

  • sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional) – the desired sbp of returned global tensor. Default: None

Keyword Arguments
  • check_meta (bool, optional) – indicates whether to check meta information when createing global tensor from local tensor. Only can be set to False when the shape and dtype of the input local tensor on each rank are the same. If set to False, the execution of local_to_global can be accelerated. Default: True

  • copy (bool, optional) – When copy is set, the returned global tensor takes the replication of this tensor as its local component in the current rank. Default: False

>>> # Run on 2 ranks respectively
>>> import oneflow as flow
>>> input = flow.tensor([0., 1.], dtype=flow.float32) 
>>> output = input.local_to_global(placement=flow.placement("cpu", ranks=[0, 1]), sbp=[flow.sbp.split(0)], check_meta=False) 
>>> print(output.size()) 
>>> print(output) 
>>> # results on rank 0
oneflow.Size([4])
tensor([0., 1., 0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)
>>> # results on rank 1
oneflow.Size([4])
tensor([0., 1., 0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)