oneflow.Tensor.to_global

Tensor.to_global(placement=None, sbp=None, **kwargs) → Tensor

Creates a global tensor if this tensor is a local tensor, otherwise performs Tensor placement and/or sbp conversion.

Note

This tensor can be local tensor or global tensor.

• For local tensor

  Both placement and sbp are required.

  The returned global tensor takes this tensor as its local component in the current rank.

  There is no data communication usually, but when sbp is oneflow.sbp.broadcast, the data on rank 0 will be broadcast to other ranks.

• For global tensor

  At least one of placement and sbp is required.

  If placement and sbp are all the same as this tensor’s own placement and sbp, then returns this tensor own.

Parameters

• placement (flow.placement, optional) – the desired placement of returned global tensor. Default: None

• sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional) – the desired sbp of returned global tensor. Default: None

Keyword Arguments

• grad_sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional) – manually specify the sbp of this tensor’s grad tensor in the backward pass. If None, the grad tensor sbp will be infered automatically. It is only used if this tensor is a global tensor. Default: None

• check_meta (bool, optional) – indicates whether to check meta information. If set to True, check the input meta information on each rank. Default: True if this tensor is a local tensor, False if this tensor is a global tensor

For local tensor:

>>> # Run on 2 ranks respectively
>>> import oneflow as flow
>>> input = flow.tensor([0., 1.], dtype=flow.float32) 
>>> output = input.to_global(placement=flow.placement("cpu", ranks=[0, 1]), sbp=[flow.sbp.split(0)], check_meta=False) 
>>> print(output.size()) 
>>> print(output) 

>>> # results on rank 0
oneflow.Size([4])
tensor([0., 1., 0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)

>>> # results on rank 1
oneflow.Size([4])
tensor([0., 1., 0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)

For global tensor:

>>> # Run on 2 ranks respectively
>>> import oneflow as flow
>>> input = flow.tensor([0., 1.], dtype=flow.float32, placement=flow.placement("cpu", ranks=[0, 1]), sbp=[flow.sbp.broadcast]) 
>>> output = input.to_global(placement=flow.placement("cpu", ranks=[0, 1]), sbp=[flow.sbp.split(0)]) 
>>> print(output.size()) 
>>> print(output) 

>>> # results on rank 0
oneflow.Size([2])
tensor([0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)

>>> # results on rank 1
oneflow.Size([2])
tensor([0., 1.], placement=oneflow.placement(type="cpu", ranks=[0, 1]), sbp=(oneflow.sbp.split(dim=0),), dtype=oneflow.float32)