oneflow.comm.all_to_all

oneflow.comm.all_to_all(output_tensor_list, input_tensor_list)

Each process scatters list of input tensors to all processes in a group and return gathered list of tensors in output list.

Parameters
  • output_tensor_list (list[Tensor]) – List of tensors to be gathered one per rank.

  • input_tensor_list (list[Tensor]) – List of tensors to scatter one per rank.