oneflow

oneflow

Copyright 2020 The OneFlow Authors. All rights reserved.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

oneflow.abs()

Return the absolute value of each element in input tensor:math:y = |x| element-wise.

Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> x = flow.tensor(np.array([-1, 2, -3, 4]).astype(np.float32))
>>> flow.abs(x)
tensor([1., 2., 3., 4.], dtype=oneflow.float32)
oneflow.acos()

Returns a new tensor with the inverse cosine of the elements of input.

\[\text{out}_{i} = \arccos(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> arr = np.array([0.5, 0.6, 0.7])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.acos(input)
>>> output
tensor([1.0472, 0.9273, 0.7954], dtype=oneflow.float32)
oneflow.acosh()

Returns a new tensor with the inverse hyperbolic cosine of the elements of input.

\[\text{out}_{i} = \cosh^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x1 = flow.tensor(np.array([2, 3, 4]).astype(np.float32))
>>> out1 = flow.acosh(x1)
>>> out1
tensor([1.3170, 1.7627, 2.0634], dtype=oneflow.float32)
>>> x2 = flow.tensor(np.array([1.5, 2.6, 3.7]).astype(np.float32),device=flow.device('cuda'))
>>> out2 = flow.acosh(x2)
>>> out2
tensor([0.9624, 1.6094, 1.9827], device='cuda:0', dtype=oneflow.float32)
oneflow.adaptive_avg_pool1d(input, output_size)

Applies a 1D adaptive average pooling over an input signal composed of several input planes.

See oneflow.nn.AdaptiveAvgPool1d

Parameters
  • input – input tensor

  • output_size – the target output size (single integer)

oneflow.adaptive_avg_pool2d(input, output_size)

Applies a 2D adaptive average pooling over an input signal composed of several input planes.

See oneflow.nn.AdaptiveAvgPool2d

Parameters
  • input – input tensor

  • output_size – the target output size (single integer or double-integer tuple)

oneflow.adaptive_avg_pool3d(input, output_size)

Applies a 3D adaptive average pooling over an input signal composed of several input planes.

See oneflow.nn.AdaptiveAvgPool3d

Parameters
  • input – input tensor

  • output_size – the target output size (single integer or triple-integer tuple)

oneflow.add()

Computes the addition of input by other for each element, scalar and broadcast promotation are supported. The formula is:

\[out = input + other\]

For example:

>>> import numpy as np
>>> import oneflow as flow

# element-wise add
>>> x = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> y = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.add(x, y).numpy()
>>> out.shape
(2, 3)

# scalar add
>>> x = 5
>>> y = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.add(x, y).numpy()
>>> out.shape
(2, 3)

# broadcast add
>>> x = flow.tensor(np.random.randn(1,1), dtype=flow.float32)
>>> y = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.add(x, y).numpy()
>>> out.shape
(2, 3)
oneflow.addmm(beta=1, input, alpha=1, mat1, mat2, out=None)Tensor

Performs a matrix multiplication of the matrices mat1 and mat2. The matrix input is added to the final result.

If mat1 is a \((n \times m)\) tensor, mat2 is a \((m \times p)\) tensor, then input must be broadcastable with a \((n \times p)\) tensor and out will be a \((n \times p)\) tensor.

alpha and beta are scaling factors on matrix-vector product between mat1 and mat2 and the added matrix input respectively.

\[\text{out} = \beta\ \text{input} + \alpha\ (\text{mat1}_i \mathbin{@} \text{mat2}_i)\]

For inputs of type FloatTensor or DoubleTensor, arguments beta and alpha must be real numbers, otherwise they should be integers.

Parameters
  • beta (Number, optional) – multiplier for input (\(\beta\))

  • input (Tensor) – matrix to be added

  • alpha (Number, optional) – multiplier for \(mat1 @ mat2\) (\(\alpha\))

  • mat1 (Tensor) – the first matrix to be multiplied

  • mat2 (Tensor) – the second matrix to be multiplied

  • out (Tensor, optional) – the output tensor.

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> input = flow.tensor(np.array([[1,2,4],[5,11,9.1]]))
>>> mat1 = flow.tensor(np.array([[7.3,1.9,7.3],[10.2,1,5.5]]))
>>> mat2 = flow.tensor(np.array([[7.3,1.9,7.3],[10.2,1,5.5],[3.7,2.2,8.1]]))
>>> output = flow.addmm(input, mat1, mat2)
>>> output
tensor([[100.6800,  33.8300, 126.8700],
        [110.0100,  43.4800, 133.6100]], dtype=oneflow.float64)
>>> output.shape
oneflow.Size([2, 3])
>>> input2 = flow.tensor(np.array([1.7]))
>>> mat1 = flow.tensor(np.array([[1,2],[5,9.1],[7.7,1.4]]))
>>> mat2 = flow.tensor(np.array([[1,2,3.7],[5,9.1,6.8]]))
>>> output2 = flow.addmm(input2, mat1, mat2, alpha=1, beta=2)
>>> output2
tensor([[14.4000, 23.6000, 20.7000],
        [53.9000, 96.2100, 83.7800],
        [18.1000, 31.5400, 41.4100]], dtype=oneflow.float64)
>>> output2.shape
oneflow.Size([3, 3])
oneflow.arange(start: int = 0, end: Optional[int] = None, step: int = 1, dtype: oneflow._oneflow_internal.dtype = oneflow.int64, device: Optional[Union[oneflow._oneflow_internal.device, str]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[Union[oneflow._oneflow_internal.sbp.sbp, List[oneflow._oneflow_internal.sbp.sbp]]] = None, requires_grad: bool = False)

Returns a 1-D tensor of size \(\left\lfloor \frac{\text{end} - \text{start}}{\text{step}} \right\rfloor + 1\) with values from start to end with step step. Step is the gap between two values in the tensor.

\[\text{out}_{i+1} = \text{out}_i + \text{step}.\]
Parameters
  • start (int) – the starting value for the set of points. Default: 0.

  • end (int) – the ending value for the set of points

  • step (int) – the gap between each pair of adjacent points. Default: 1.

Keyword Arguments
  • dtype (flow.dtype, optional) – If dtype is not given, the dtype is inferred to be flow.int64.

  • device (flow.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow

>>> y = flow.arange(0, 5)
>>> y
tensor([0, 1, 2, 3, 4], dtype=oneflow.int64)
oneflow.arccosh()

Returns a new tensor with the inverse hyperbolic cosine of the elements of input.

\[\text{out}_{i} = \cosh^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x1 = flow.tensor(np.array([2, 3, 4]).astype(np.float32))
>>> out1 = flow.acosh(x1)
>>> out1
tensor([1.3170, 1.7627, 2.0634], dtype=oneflow.float32)
>>> x2 = flow.tensor(np.array([1.5, 2.6, 3.7]).astype(np.float32),device=flow.device('cuda'))
>>> out2 = flow.acosh(x2)
>>> out2
tensor([0.9624, 1.6094, 1.9827], device='cuda:0', dtype=oneflow.float32)
oneflow.arcsin()

Returns a new tensor with the arcsine of the elements of input.

\[\text{out}_{i} = \sin^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([-0.5,  0.8, 1.0,  -0.8]), dtype=flow.float32)
>>> output = flow.asin(input)
>>> output.shape
oneflow.Size([4])
>>> output
tensor([-0.5236,  0.9273,  1.5708, -0.9273], dtype=oneflow.float32)
>>> input1 = flow.tensor(np.array([[0.8, 1.0], [-0.6, -1.0]]), dtype=flow.float32)
>>> output1 = input1.asin()
>>> output1.shape
oneflow.Size([2, 2])
>>> output1
tensor([[ 0.9273,  1.5708],
        [-0.6435, -1.5708]], dtype=oneflow.float32)
oneflow.arcsinh()

Returns a new tensor with the inverse hyperbolic sine of the elements of input.

\[\text{out}_{i} = \sinh^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([2, 3, 4]), dtype=flow.float32)
>>> output = flow.asinh(input)
>>> output.shape
oneflow.Size([3])
>>> output
tensor([1.4436, 1.8184, 2.0947], dtype=oneflow.float32)

>>> input1 = flow.tensor(np.array([[-1, 0, -0.4], [5, 7, 0.8]]), dtype=flow.float32)
>>> output1 = input1.asinh()
>>> output1.shape
oneflow.Size([2, 3])
>>> output1
tensor([[-0.8814,  0.0000, -0.3900],
        [ 2.3124,  2.6441,  0.7327]], dtype=oneflow.float32)
oneflow.arctan()

Returns a new tensor with the arctangent of the elements of input.

\[\text{out}_{i} = \tan^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([0.5, 0.6, 0.7]), dtype=flow.float32)
>>> output = flow.atan(input)
>>> output.shape
oneflow.Size([3])
oneflow.arctanh()

Returns a new tensor with the inverse hyperbolic tangent of the elements of input.

\[\text{out}_{i} = \tanh^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> np_arr = np.array([0.5, 0.6, 0.7]).astype(np.float32)
>>> input = flow.tensor(np_arr, dtype=flow.float32)
>>> output = flow.atanh(input)
>>> output
tensor([0.5493, 0.6931, 0.8673], dtype=oneflow.float32)
oneflow.argmax(input, dim: Optional[int] = None, keepdim: bool = False)

The op computes the index with the largest value of a Tensor at specified axis.

Parameters
  • input (oneflow.Tensor) – Input Tensor

  • dim (int, optional) – dimension to be calculated. Defaults to the last dim (-1)

  • keepdim (bool optional) – whether the output tensor has dim retained or not. Ignored if dim=None.

Returns

A Tensor(dtype=int32) contains the index with the largest value of input

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input = np.array([[1, 3, 8, 7, 2],
...            [1, 9, 4, 3, 2]], dtype=np.float32)

>>> output = flow.argmax(flow.tensor(input))
>>> output
tensor(6, dtype=oneflow.int64)
>>> output = flow.argmax(flow.tensor(input), dim=1)
>>> output
tensor([2, 1], dtype=oneflow.int64)
oneflow.argsort(input, dim: int = - 1, descending: bool = False)

This operator sorts the input Tensor at specified dim and return the indices of the sorted Tensor.

Parameters
  • input (oneflow.Tensor) – The input Tensor.

  • dim (int, optional) – dimension to be sorted. Defaults to the last dim (-1).

  • descending (bool, optional) – controls the sorting order (ascending or descending).

Returns

The indices of the sorted Tensor.

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> x = np.array([[10, 2, 9, 3, 7],
...               [1, 9, 4, 3, 2]]).astype("float32")
>>> input = flow.Tensor(x)
>>> output = flow.argsort(input)
>>> output
tensor([[1, 3, 4, 2, 0],
        [0, 4, 3, 2, 1]], dtype=oneflow.int32)
>>> output = flow.argsort(input, descending=True)
>>> output
tensor([[0, 2, 4, 3, 1],
        [1, 2, 3, 4, 0]], dtype=oneflow.int32)
>>> output = flow.argsort(input, dim=0)
>>> output
tensor([[1, 0, 1, 0, 1],
        [0, 1, 0, 1, 0]], dtype=oneflow.int32)
oneflow.argwhere(input, dtype: Optional[oneflow._oneflow_internal.dtype] = oneflow.int32)

This operator finds the indices of input Tensor input elements that are non-zero.

It returns a list in which each element is a coordinate that points to a non-zero element in the condition.

Parameters
  • input (oneflow.Tensor) – The input Tensor.

  • dtype (Optional[flow.dtype], optional) – The data type of output. Defaults to None.

Returns

The result Tensor.

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> x = np.array([[0, 1, 0],
...            [2, 0, 2]]).astype(np.float32)

>>> input = flow.Tensor(x)
>>> output = flow.argwhere(input)
>>> output
tensor([[0, 1],
        [1, 0],
        [1, 2]], dtype=oneflow.int32)
oneflow.asin()

Returns a new tensor with the arcsine of the elements of input.

\[\text{out}_{i} = \sin^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([-0.5,  0.8, 1.0,  -0.8]), dtype=flow.float32)
>>> output = flow.asin(input)
>>> output.shape
oneflow.Size([4])
>>> output
tensor([-0.5236,  0.9273,  1.5708, -0.9273], dtype=oneflow.float32)
>>> input1 = flow.tensor(np.array([[0.8, 1.0], [-0.6, -1.0]]), dtype=flow.float32)
>>> output1 = input1.asin()
>>> output1.shape
oneflow.Size([2, 2])
>>> output1
tensor([[ 0.9273,  1.5708],
        [-0.6435, -1.5708]], dtype=oneflow.float32)
oneflow.asinh()

Returns a new tensor with the inverse hyperbolic sine of the elements of input.

\[\text{out}_{i} = \sinh^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([2, 3, 4]), dtype=flow.float32)
>>> output = flow.asinh(input)
>>> output.shape
oneflow.Size([3])
>>> output
tensor([1.4436, 1.8184, 2.0947], dtype=oneflow.float32)

>>> input1 = flow.tensor(np.array([[-1, 0, -0.4], [5, 7, 0.8]]), dtype=flow.float32)
>>> output1 = input1.asinh()
>>> output1.shape
oneflow.Size([2, 3])
>>> output1
tensor([[-0.8814,  0.0000, -0.3900],
        [ 2.3124,  2.6441,  0.7327]], dtype=oneflow.float32)
oneflow.atan()

Returns a new tensor with the arctangent of the elements of input.

\[\text{out}_{i} = \tan^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([0.5, 0.6, 0.7]), dtype=flow.float32)
>>> output = flow.atan(input)
>>> output.shape
oneflow.Size([3])
oneflow.atan2(input, other)

Element-wise arctangent of input{i}/other{i} with consideration of the quadrant. Returns a new tensor with the signed angles in radians between vector (other{i},input{i}) and vector (1, 0).

The shapes of input and other must be broadcastable.

Parameters
  • input (Tensor) – the first input tensor.

  • other (Tensor) – the second input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> x1 = flow.Tensor(np.array([1,2,3]))
>>> y1 = flow.Tensor(np.array([3,2,1]))
>>> x2 = flow.Tensor(np.array([1.53123589,0.54242598,0.15117185]))
>>> y2 = flow.Tensor(np.array([-0.21906378,0.09467151,-0.75562878]))
>>> x3 = flow.Tensor(np.array([1,0,-1]))
>>> y3 = flow.Tensor(np.array([0,1,0]))

>>> flow.atan2(x1,y1).numpy()
array([0.32175055, 0.7853982 , 1.2490457 ], dtype=float32)
>>> flow.atan2(x2,y2).numpy()
array([1.7128955, 1.3980033, 2.9441385], dtype=float32)
>>> flow.atan2(x3,y3).numpy()
array([ 1.5707964,  0.       , -1.5707964], dtype=float32)
oneflow.atanh()

Returns a new tensor with the inverse hyperbolic tangent of the elements of input.

\[\text{out}_{i} = \tanh^{-1}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> np_arr = np.array([0.5, 0.6, 0.7]).astype(np.float32)
>>> input = flow.tensor(np_arr, dtype=flow.float32)
>>> output = flow.atanh(input)
>>> output
tensor([0.5493, 0.6931, 0.8673], dtype=oneflow.float32)
oneflow.batch_gather()

Gather the element in batch dims.

Parameters
  • in (Tensor) – the input tensor.

  • indices (Tensor) – the indices tensor, its dtype must be int32/64.

For example:

Example 1:

>>> import oneflow as flow
>>> import numpy as np

>>> x = flow.Tensor(np.array([[1, 2, 3],
...                           [4, 5, 6]]))
>>> indices = flow.tensor(np.array([1, 0]).astype(np.int64))
>>> out = flow.batch_gather(x, indices)

tensor([[4., 5., 6.],
        [1., 2., 3.]], dtype=oneflow.float32)

Example 2:

>>> import oneflow as flow
>>> import numpy as np

>>> x = flow.Tensor(np.array([[[1, 2, 3], [4, 5, 6]],
...                           [[1, 2, 3], [4, 5, 6]]]))
>>> indices = flow.tensor(np.array([[1, 0],
...                                 [0, 1]]).astype(np.int64))
>>> out = flow.batch_gather(x, indices)

tensor([[[4., 5., 6.],
         [1., 2., 3.]],
        [[1., 2., 3.],
         [4., 5., 6.]]], dtype=oneflow.float32)
oneflow.bernoulli(input, *, generator=None, out=None)

This operator returns a Tensor with binaray random numbers (0 / 1) from a Bernoulli distribution.

Parameters
  • input (Tensor) – the input tensor of probability values for the Bernoulli distribution

  • generator – (Generator, optional) a pseudorandom number generator for sampling

  • out (Tensor, optional) – the output tensor.

Shape:
  • Input: \((*)\). Input can be of any shape

  • Output: \((*)\). Output is of the same shape as input

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> arr = np.array(
...    [
...        [1.0, 1.0, 1.0],
...        [1.0, 1.0, 1.0],
...        [1.0, 1.0, 1.0],
...    ]
... )
>>> x = flow.tensor(arr, dtype=flow.float32)
>>> y = flow.bernoulli(x)
>>> y
tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], dtype=oneflow.float32)
oneflow.broadcast_like(x, like_tensor, broadcast_axes: Optional[Sequence] = None)

This operator broadcast tensor x to like_tensor according to the broadcast_axes.

Parameters
  • x (Tensor) – The input Tensor.

  • like_tensor (Tensor) – The like Tensor.

  • broadcast_axes (Optional[Sequence], optional) – The axes you want to broadcast. Defaults to None.

Returns

Broadcasted input Tensor.

Return type

[Tensor]

For example:

>>> import oneflow as flow

>>> x = flow.randn(3, 1, 1)
>>> like_tensor = flow.randn(3, 4, 5)
>>> broadcast_tensor = flow.broadcast_like(x, like_tensor, broadcast_axes=[1, 2])
>>> broadcast_tensor.shape
oneflow.Size([3, 4, 5])
oneflow.cast()

The operation takes input tensor x and casts it to the output with dtype

Parameters
  • x (oneflow.Tensor) – A Tensor

  • dtype (flow.dtype) – Data type of the output tensor

Returns

A Tensor with specific dtype.

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> np_arr = np.random.randn(2, 3, 4, 5).astype(np.float32)
>>> input = flow.tensor(np_arr, dtype=flow.float32)
>>> output = flow.cast(input, flow.int8)
>>> np.array_equal(output.numpy(), np_arr.astype(np.int8))
True
oneflow.cat(inputs, dim=0)

Concatenate two or more Tensor s at specified dim.

Analogous to numpy.concatenate

Parameters
  • inputs – a list of Tensor

  • dim – a int.

Returns

A Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> input1 = flow.tensor(np.random.randn(2, 6, 5, 3), dtype=flow.float32)
>>> input2 = flow.tensor(np.random.randn(2, 6, 5, 3), dtype=flow.float32)
>>> input3 = flow.tensor(np.random.randn(2, 6, 5, 3), dtype=flow.float32)

>>> out = flow.cat([input1, input2, input3], dim=1)
>>> out.shape
oneflow.Size([2, 18, 5, 3])
oneflow.ceil()

Returns a new tensor with the ceil of the elements of input, the smallest integer greater than or equal to each element.

The equation is:

\[\text{out}_{i} = \left\lceil \text{input}_{i} \right\rceil = \left\lfloor \text{input}_{i} \right\rfloor + 1\]
Parameters

input (oneflow.Tensor) – A Tensor.

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = flow.tensor(np.array([0.1, -2, 3.4]).astype(np.float32))
>>> y = flow.ceil(x)
>>> y.shape
oneflow.Size([3])
>>> y
tensor([ 1., -2.,  4.], dtype=oneflow.float32)
>>> x = flow.tensor(np.array([[2.5, 4.6, 0.6],[7.8, 8.3, 9.2]]).astype(np.float32))
>>> y = x.ceil()
>>> y.shape
oneflow.Size([2, 3])
>>> y
tensor([[ 3.,  5.,  1.],
        [ 8.,  9., 10.]], dtype=oneflow.float32)
>>> x = flow.tensor(np.array([[[2.2, 4.4, 6.5],[7.1, 8.2, 9.3]],[[10.6,11.2,12.2],[13.5,14.8,15.9]]]).astype(np.float32))
>>> y = flow.ceil(x)
>>> y.shape
oneflow.Size([2, 2, 3])
>>> y
tensor([[[ 3.,  5.,  7.],
         [ 8.,  9., 10.]],

        [[11., 12., 13.],
         [14., 15., 16.]]], dtype=oneflow.float32)
oneflow.chunk(input, chunks, dim)

Splits a tensor into a specific number of chunks. Each chunk is a view of the input tensor. Last chunk will be smaller if the tensor size along the given dimension dim is not divisible by chunks.

Parameters
  • input (oneflow.Tensor) – The tensor to split.

  • chunks (int) – Number of chunks to return.

  • dim (int) – Dimension along which to split the tensor.

Returns

List of Tensors.

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> np_arr = np.random.randn(5, 3, 6, 9).astype(np.float32)
>>> input = flow.Tensor(np_arr)
>>> of_out = []
>>> of_out = flow.chunk(input, chunks=3, dim=2)
>>> chunks = 3
>>> of_out_shape = []
>>> for i in range(0, chunks):
...     of_out_shape.append(of_out[i].numpy().shape)
>>> of_out_shape
[(5, 3, 2, 9), (5, 3, 2, 9), (5, 3, 2, 9)]

>>> np_arr = np.random.randn(5, 3, 6, 9).astype(np.float32)
>>> input = flow.Tensor(np_arr)
>>> of_out = []
>>> of_out = flow.chunk(input, chunks=4, dim=3)
>>> chunks = 4
>>> of_out_shape = []
>>> for i in range(0, chunks):
...     of_out_shape.append(of_out[i].numpy().shape)
>>> of_out_shape
[(5, 3, 6, 2), (5, 3, 6, 2), (5, 3, 6, 2), (5, 3, 6, 3)]
oneflow.clamp()

Clamp all elements in input into the range [ min, max ] and return a resulting tensor:

\[\begin{split}y_i = \begin{cases} \text{min} & \text{if } x_i < \text{min} \\ x_i & \text{if } \text{min} \leq x_i \leq \text{max} \\ \text{max} & \text{if } x_i > \text{max} \end{cases}\end{split}\]

If input is of type FloatTensor or DoubleTensor, args min and max must be real numbers, otherwise they should be integers.

Parameters
  • input (Tensor) – the input tensor.

  • min (Number) – lower-bound of the range to be clamped to. Defaults to None.

  • max (Number) – upper-bound of the range to be clamped to. Defaults to None.

  • out (Tensor, optional) – the output tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> arr = np.array([0.2, 0.6, -1.5, -0.3])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.clamp(input, min=-0.5, max=0.5)
>>> output
tensor([ 0.2000,  0.5000, -0.5000, -0.3000], dtype=oneflow.float32)

>>> arr = np.array([0.2, 0.6, -1.5, -0.3])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.clamp(input, min=None, max=0.5)
>>> output
tensor([ 0.2000,  0.5000, -1.5000, -0.3000], dtype=oneflow.float32)

>>> arr = np.array([0.2, 0.6, -1.5, -0.3])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.clamp(input, min=-0.5, max=None)
>>> output
tensor([ 0.2000,  0.6000, -0.5000, -0.3000], dtype=oneflow.float32)
oneflow.clip()

Clamp all elements in input into the range [ min, max ] and return a resulting tensor:

\[\begin{split}y_i = \begin{cases} \text{min} & \text{if } x_i < \text{min} \\ x_i & \text{if } \text{min} \leq x_i \leq \text{max} \\ \text{max} & \text{if } x_i > \text{max} \end{cases}\end{split}\]

If input is of type FloatTensor or DoubleTensor, args min and max must be real numbers, otherwise they should be integers.

Parameters
  • input (Tensor) – the input tensor.

  • min (Number) – lower-bound of the range to be clamped to. Defaults to None.

  • max (Number) – upper-bound of the range to be clamped to. Defaults to None.

  • out (Tensor, optional) – the output tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> arr = np.array([0.2, 0.6, -1.5, -0.3])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.clamp(input, min=-0.5, max=0.5)
>>> output
tensor([ 0.2000,  0.5000, -0.5000, -0.3000], dtype=oneflow.float32)

>>> arr = np.array([0.2, 0.6, -1.5, -0.3])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.clamp(input, min=None, max=0.5)
>>> output
tensor([ 0.2000,  0.5000, -1.5000, -0.3000], dtype=oneflow.float32)

>>> arr = np.array([0.2, 0.6, -1.5, -0.3])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.clamp(input, min=-0.5, max=None)
>>> output
tensor([ 0.2000,  0.6000, -0.5000, -0.3000], dtype=oneflow.float32)
oneflow.cos()

Returns a new tensor with the cosine of the elements of input.

\[\text{out}_{i} = \cos(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> arr = np.array([1.4309,  1.2706, -0.8562,  0.9796])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.cos(input).numpy()
oneflow.cosh()

Returns a new tensor with the hyperbolic cosine of the elements of input.

\[\text{out}_{i} = \cosh(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> arr = np.array([ 0.1632,  1.1835, -0.6979, -0.7325])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.cosh(input).numpy()
>>> output
array([1.0133467, 1.7859949, 1.2535787, 1.2804903], dtype=float32)
oneflow.diag()

If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal. If input is a matrix (2-D tensor), then returns a 1-D tensor with diagonal elements of input.

Parameters
  • input (Tensor) – the input tensor.

  • diagonal (Optional[int], 0) – The diagonal to consider. If diagonal = 0, it is the main diagonal. If diagonal > 0, it is above the main diagonal. If diagonal < 0, it is below the main diagonal. Defaults to 0.

Returns

the output Tensor.

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> arr = np.array(
...     [
...        [1.0, 2.0, 3.0],
...        [4.0, 5.0, 6.0],
...        [7.0, 8.0, 9.0],
...     ]
... )

>>> input = flow.tensor(arr, dtype=flow.float32)
>>> flow.diag(input)
tensor([1., 5., 9.], dtype=oneflow.float32)
oneflow.div()

Computes the division of input by other for each element, scalar and broadcast promotation are supported. The formula is:

\[out = \frac{input}{other}\]
Parameters

For example:

>>> import numpy as np
>>> import oneflow as flow

# element-wise divide
>>> input = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.div(input,other).numpy()
>>> out.shape
(2, 3)

# scalar divide
>>> input = 5
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.div(input,other).numpy()
>>> out.shape
(2, 3)

# broadcast divide
>>> input = flow.tensor(np.random.randn(1,1), dtype=flow.float32)
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.div(input,other).numpy()
>>> out.shape
(2, 3)
oneflow.empty(*size, dtype: Optional[oneflow._oneflow_internal.dtype] = None, device: Optional[Union[oneflow._oneflow_internal.device, str]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[Union[oneflow._oneflow_internal.sbp.sbp, List[oneflow._oneflow_internal.sbp.sbp]]] = None, requires_grad: bool = False)

Returns a tensor filled with uninitialized data. The shape of the tensor is defined by the variable argument size.

Parameters
  • size (int... or oneflow.Size) – Defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple or oneflow.Size.

  • dtype (flow.dtype, optional) – The desired data type of returned tensor. Default: flow.float32.

  • device (torch.device, optional) – The desired device of returned local tensor. If None, uses the current device.

  • placement (flow.placement, optional) – The desired device of returned consistent tensor. If None, will construct local tensor.

  • sbp (flow.sbp or List[flow.sbp], optional) – The desired sbp of returned consistent tensor.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow
>>> y = flow.empty(4, 5)  # construct local empty tensor
>>> y.shape
oneflow.Size([4, 5])
>>> y.is_consistent
False
>>> placement = flow.placement("cpu", {0: [0]})
>>> y = flow.empty(4, 5, placement=placement, sbp=flow.sbp.broadcast)  # construct consistent empty tensor
>>> y.is_consistent
True
oneflow.eq(input, other)

Computes element-wise equality. The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters
Returns

  • A boolean tensor that is True where input is equal to other and False elsewhere

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> input = flow.tensor(np.array([2, 3, 4, 5]), dtype=flow.float32)
>>> other = flow.tensor(np.array([2, 3, 4, 1]), dtype=flow.float32)

>>> y = flow.eq(input, other)
>>> y
tensor([1, 1, 1, 0], dtype=oneflow.int8)
oneflow.equal(input, other)

Computes element-wise equality. The second argument can be a number or a tensor whose shape is broadcastable with the first argument.

Parameters
Returns

  • A boolean tensor that is True where input is equal to other and False elsewhere

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> input = flow.tensor(np.array([2, 3, 4, 5]), dtype=flow.float32)
>>> other = flow.tensor(np.array([2, 3, 4, 1]), dtype=flow.float32)

>>> y = flow.eq(input, other)
>>> y
tensor([1, 1, 1, 0], dtype=oneflow.int8)
oneflow.erf()

Computes the error function of each element. The error function is defined as follows:

\[\operatorname{erf}(x)=\frac{2}{\sqrt{\pi}} \int_{0}^{x} e^{-t^{2}} d t\]
Parameters

x (oneflow.Tensor) – A Tensor

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> x = flow.tensor(np.array([0, -1., 10.]), dtype=flow.float32)
>>> out = flow.erf(x)
>>> out.shape
oneflow.Size([3])
>>> out.numpy()
array([ 0.       , -0.8427008,  1.       ], dtype=float32)

>>> x = flow.tensor(np.array([[0, -1., 10.], [5, 7, 0.8]]), dtype=flow.float32)
>>> out = flow.erf(x)
>>> out.shape
oneflow.Size([2, 3])
>>> out.numpy()
array([[ 0.        , -0.8427008 ,  1.        ],
       [ 1.        ,  1.        ,  0.74210095]], dtype=float32)

>>> x = flow.tensor(np.array([[0, -1., 10.], [5, 7, 0.8], [2, 3, 4]]), dtype=flow.float32)
>>> out = x.erf()
>>> out.shape
oneflow.Size([3, 3])
>>> out.numpy()
array([[ 0.        , -0.8427008 ,  1.        ],
       [ 1.        ,  1.        ,  0.74210095],
       [ 0.9953223 ,  0.9999779 ,  1.        ]], dtype=float32)
oneflow.erfc()

Computes the complementary error function of each element of input. The complementary error function is defined as follows:

\[\operatorname{erfc}(x)=1-\frac{2}{\sqrt{\pi}} \int_{0}^{x} e^{-t^{2}} d t\]
Parameters

x (oneflow.Tensor) – A Tensor

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> x = flow.tensor(np.array([0, -1., 10.]), dtype=flow.float32)
>>> out = flow.erfc(x)
>>> out
tensor([1.0000e+00, 1.8427e+00, 2.8026e-45], dtype=oneflow.float32)

>>> x = flow.tensor(np.array([[0, -1., 10.], [5, 7, 0.8]]), dtype=flow.float32)
>>> out = flow.erfc(x)
>>> out
tensor([[1.0000e+00, 1.8427e+00, 2.8026e-45],
        [1.5375e-12, 4.1838e-23, 2.5790e-01]], dtype=oneflow.float32)
oneflow.exp()

This operator computes the exponential of Tensor.

The equation is:

\[out = e^x\]
Parameters

x (oneflow.Tensor) – A Tensor

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = flow.tensor(np.array([1, 2, 3]).astype(np.float32), dtype=flow.float32)
>>> y = flow.exp(x)
>>> y
tensor([ 2.7183,  7.3891, 20.0855], dtype=oneflow.float32)
oneflow.expand(input, *sizes)

This operator expand the input tensor to a larger size.

Passing -1 as the size for a dimension means not changing the size of that dimension.

Tensor can be also expanded to a larger number of dimensions and the new ones will be appended at the front.

For the new dimensions, the size cannot be set to -1.

Parameters
  • input (oneflow.Tensor) – The input Tensor.

  • *sizes (oneflow.Size or int) – The desired expanded size.

Returns

The result Tensor.

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = np.array([[[[0, 1]],
...               [[2, 3]],
...               [[4, 5]]]]).astype(np.int32)

>>> input = flow.Tensor(x)

>>> out = input.expand(1, 3, 2, 2)
>>> out.shape
oneflow.Size([1, 3, 2, 2])
oneflow.expm1()

Returns a new tensor with the exponential of the elements minus 1 of input.

The equation is:

\[y_{i} = e^{x_{i}} - 1\]
Parameters

input (oneflow.Tensor) – A Tensor.

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = flow.tensor(np.array([1, 2, 3]).astype(np.float32))
>>> y = flow.expm1(x)
>>> y.shape
oneflow.Size([3])
>>> y
tensor([ 1.7183,  6.3891, 19.0855], dtype=oneflow.float32)

>>> x = flow.tensor(np.array([[[2, 4, 6],[7, 8, 9]],[[10,11,12],[13,14,15]]]).astype(np.float32))
>>> y = flow.expm1(x)
>>> print(y.shape)
oneflow.Size([2, 2, 3])
>>> print(y.numpy())
[[[6.3890562e+00 5.3598152e+01 4.0242880e+02]
  [1.0956332e+03 2.9799580e+03 8.1020840e+03]]

 [[2.2025465e+04 5.9873141e+04 1.6275380e+05]
  [4.4241238e+05 1.2026032e+06 3.2690165e+06]]]
oneflow.eye(n, m=None, device: Union[str, oneflow._oneflow_internal.device] = 'cpu', requires_grad: bool = False)

This operator creates a 2-D Tensor with ones on the diagonal and zeros elsewhere.

Parameters
  • n (int) – the number of rows.

  • m (Optional[int], optional) – the number of colums with default being n. Defaults to None.

Keyword Arguments
  • device (flow.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

Returns

The result Blob with ones on the diagonal and zeros elsewhere.

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> out = flow.eye(3, 3)
>>> out
tensor([[1., 0., 0.],
        [0., 1., 0.],
        [0., 0., 1.]], dtype=oneflow.float32)
oneflow.flatten(input, start_dim: int = 0, end_dim: int = - 1)

Flattens a contiguous range of dims into a tensor.

Parameters
  • start_dim – first dim to flatten (default = 0).

  • end_dim – last dim to flatten (default = -1).

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> input = flow.Tensor(32, 1, 5, 5)
>>> output = input.flatten(start_dim=1)
>>> output.shape
oneflow.Size([32, 25])
oneflow.flip(input, dims)

Reverse the order of a n-D tensor along given axis in dims.

Note

flow.flip makes a copy of input’s data. This is different from NumPy’s np.flip, which returns a view in constant time. Since copying a tensor’s data is more work than viewing that data, flow.flip is expected to be slower than np.flip.

Parameters
  • input (Tensor) – the input tensor

  • dims (a list or tuple) – axis to flip on

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> np_arr = np.arange(0, 8).reshape((2, 2, 2)).astype(np.float32)
>>> input = flow.Tensor(np_arr)
>>> out = flow.flip(input, [0, 1])
>>> out
tensor([[[6., 7.],
         [4., 5.]],

        [[2., 3.],
         [0., 1.]]], dtype=oneflow.float32)
oneflow.floor(input)

Returns a new tensor with the arcsine of the elements of input.

\[\text{out}_{i} = \lfloor \text{input}_{i} \rfloor\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([-0.5,  1.5, 0,  0.8]), dtype=flow.float32)
>>> output = flow.floor(input)
>>> output.shape
oneflow.Size([4])
>>> output.numpy()
array([-1.,  1.,  0.,  0.], dtype=float32)

>>> input1 = flow.tensor(np.array([[0.8, 1.0], [-0.6, 2.5]]), dtype=flow.float32)
>>> output1 = input1.floor()
>>> output1.shape
oneflow.Size([2, 2])
>>> output1.numpy()
array([[ 0.,  1.],
       [-1.,  2.]], dtype=float32)
oneflow.fmod(input, other, *, out=None)Tensor

Computes the element-wise remainder of division.

The dividend and divisor may contain both for integer and floating point numbers. The remainder has the same sign as the dividend input.

Supports broadcasting to a common shape, integer and float inputs.

Parameters
  • input (Tensor) – the dividend

  • other (Tensor or Scalar) – the divisor

Keyword Arguments

out (Tensor, optional) – the output tensor.

Example:

>>> import oneflow as flow
>>> flow.fmod(flow.tensor([-3., -2, -1, 1, 2, 3], dtype=flow.float32), 2.)
tensor([-1., -0., -1.,  1.,  0.,  1.], dtype=oneflow.float32)
>>> flow.fmod(flow.tensor([1, 2, 3, 4, 5.], dtype=flow.float32), 1.5)
tensor([1.0000, 0.5000, 0.0000, 1.0000, 0.5000], dtype=oneflow.float32)
>>> flow.fmod(flow.tensor([1, 2, 3, 4., -5]), flow.tensor([4, 2, 1, 3., 1]))
tensor([1., 0., 0., 1., -0.], dtype=oneflow.float32)
oneflow.full(size: Union[int, Tuple[int, ], oneflow._oneflow_internal.Size], value: Union[float, int], dtype: Optional[oneflow._oneflow_internal.dtype] = None, device: Optional[Union[str, oneflow._oneflow_internal.device]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[oneflow._oneflow_internal.sbp.sbp] = None, requires_grad: bool = False)

Creates a tensor of size size filled with fill_value. The tensor’s dtype is inferred from value.

Parameters
  • size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.

  • fill_value (Scalar) – the value to fill the output tensor with.

  • dtype (flow.dtype, optional) – the desired data type of returned tensor.

  • device (flow.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type

  • placement (flow.placement, optional) – the desired placement of returned consistent tensor. Default: if None, the returned tensor is local one using the argument device.

  • sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional) – the desired sbp descriptor of returned consistent tensor. Default: if None, the returned tensor is local one using the argument device.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow
>>> y = flow.full((5,),5)
>>> y
tensor([5, 5, 5, 5, 5], dtype=oneflow.int64)
>>> y = flow.full((2,3),5.0)
>>> y
tensor([[5., 5., 5.],
        [5., 5., 5.]], dtype=oneflow.float32)
oneflow.gather(input, index, dim=0, sparse_grad=False)

Gathers values along an axis specified by dim.

For a 3-D tensor the output is specified by:

out[i][j][k] = input[index[i][j][k]][j][k]  # if dim == 0
out[i][j][k] = input[i][index[i][j][k]][k]  # if dim == 1
out[i][j][k] = input[i][j][index[i][j][k]]  # if dim == 2

input and index must have the same number of dimensions. It is also required that index.size(d) <= input.size(d) for all dimensions d != dim. out will have the same shape as index. Note that input and index do not broadcast against each other.

Parameters
  • input (Tensor) – the source tensor

  • dim (int) – the axis along which to index

  • index (LongTensor) – the indices of elements to gather

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = np.random.randn(3, 4, 3, 5)
>>> index = np.random.choice(np.arange(3), size=180, replace=True).reshape((3, 4, 3, 5))
>>> output = flow.gather(flow.Tensor(input), flow.tensor(index, dtype=flow.int), dim=1)
>>> output.shape
oneflow.Size([3, 4, 3, 5])
oneflow.gather_nd(input, index)

This operator is a high-dimensional extension of gather, index is a K-dimensional tensor, which is regarded as a index of input Tensor input.

Each element defines a slice of input:

\[output[i_{0},i_{1},...,i_{K-2}] = input[index(i_{0},i_{1},...,i_{K-2})]\]
Parameters
  • input – The input Tensor.

  • index – The slice indices.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.tensor(np.array([[1, 2,3], [4, 5,6],[7,8,9]]), dtype=flow.float)
>>> index_1 = flow.tensor(np.array([[0], [2]]), dtype=flow.int)
>>> out_1 = flow.gather_nd(input,index_1)
>>> print(out_1.shape)
oneflow.Size([2, 3])
>>> out_1
tensor([[1., 2., 3.],
        [7., 8., 9.]], dtype=oneflow.float32)
>>> index_2 = flow.tensor(np.array([[0,2], [2,1]]), dtype=flow.int)
>>> out_2 = flow.gather_nd(input,index_2)
>>> out_2
tensor([3., 8.], dtype=oneflow.float32)
oneflow.gelu(x: Tensor)Tensor

The equation is:

\[out = 0.5 * x * (1 + tanh(\sqrt{\frac{2}{\pi}} * (x + 0.044715x^{3})))\]

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([-0.5, 0, 0.5]).astype(np.float32)
>>> input = flow.tensor(x)

>>> out = flow.gelu(input)
>>> out
tensor([-0.1543,  0.0000,  0.3457], dtype=oneflow.float32)

See GELU for more details.

class oneflow.grad_enable

Context-manager that enabled gradient calculation.

Enables gradient calculation, if it has been disabled via no_grad.

This context manager is thread local; it will not affect computation in other threads.

Also functions as a decorator. (Make sure to instantiate with parenthesis.)

>>> import oneflow as flow
>>> x = flow.ones(2, 3, requires_grad=True)
>>> with flow.no_grad():
...     with flow.grad_enable():
...         y = x * x
>>> y.requires_grad
True
>>> @flow.grad_enable()
... def no_grad_func(x):
...     return x * x
>>> with flow.no_grad():
...     y = no_grad_func(x)
>>> y.requires_grad
True
oneflow.gt()

Returns the truth value of \(input > other\) element-wise.

Parameters
Returns

A Tensor with int8 type.

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input1 = flow.tensor(np.random.randn(2, 6, 5, 3), dtype=flow.float32)
>>> input2 = flow.tensor(np.random.randn(2, 6, 5, 3), dtype=flow.float32)

>>> out = flow.gt(input1, input2).shape
>>> out
oneflow.Size([2, 6, 5, 3])
oneflow.in_top_k(targets, predictions, k)

Says whether the targets are in the top K predictions.

Parameters
  • targets (Tensor) – the target tensor of type int32 or int64.

  • predictions (Tensor) – the predictions tensor of type float32 .

  • k (int) – Number of top elements to look at for computing precision.

Returns

A Tensor of type bool. Computed Precision at k as a bool Tensor.

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> targets1 = flow.tensor(np.array([3, 1]), dtype=flow.int32)
>>> predictions1 = flow.tensor(np.array([[0.0, 1.0, 2.0, 3.0], [3.0, 2.0, 1.0, 0.0],]), dtype=flow.float32)
>>> out1 = flow.in_top_k(targets1, predictions1, k=1)
>>> out1
tensor([1, 0], dtype=oneflow.int8)
>>> out2 = flow.in_top_k(targets1, predictions1, k=2)
>>> out2
tensor([1, 1], dtype=oneflow.int8)
>>> targets2 = flow.tensor(np.array([3, 1]), dtype=flow.int32, device=flow.device('cuda'))
>>> predictions2 = flow.tensor(np.array([[0.0, 1.0, 2.0, 3.0], [3.0, 2.0, 1.0, 0.0],]), dtype=flow.float32, device=flow.device('cuda'))
>>> out3 = flow.in_top_k(targets2, predictions2, k=1)
>>> out3
tensor([1, 0], device='cuda:0', dtype=oneflow.int8)
oneflow.index_select(input, dim, index)

The interface is consistent with PyTorch. The documentation is referenced from: https://pytorch-cn.readthedocs.io/zh/latest/package_references/torch/#torchindex_select

Select values along an axis specified by dim.

index must be an Int32 Tensor with 1-D. dim must be in the range of input Dimensions. value of index must be in the range of the dim-th of input. Note that input and index do not broadcast against each other.

Parameters
  • input (Tensor) – the source tensor

  • dim (int) – the axis along which to index

  • index (Tensor) – the 1-D tensor containing the indices to index

For example:

>>> import oneflow as flow
>>> input = flow.tensor([[1,2,3],[4,5,6]], dtype=flow.int32)
>>> input
tensor([[1, 2, 3],
        [4, 5, 6]], dtype=oneflow.int32)
>>> index = flow.tensor([0,1], dtype=flow.int32)
>>> output = flow.index_select(input, 1, index)
>>> output
tensor([[1, 2],
        [4, 5]], dtype=oneflow.int32)
>>> output = input.index_select(1, index)
>>> output
tensor([[1, 2],
        [4, 5]], dtype=oneflow.int32)
class oneflow.inference_mode(mode=True)

Context-manager that enables or disables inference mode

InferenceMode is a new context manager analogous to no_grad to be used when you arecertain your operations will have no interactions with autograd (e.g., model training). Code run under this mode gets better performance by disabling view tracking and version counter bumps.

This context manager is thread local; it will not affect computation in other threads.

Also functions as a decorator. (Make sure to instantiate with parenthesis.)

Parameters

mode (bool) – Flag whether to enable or disable inference mode. (default: True)

>>> import oneflow as flow
>>> x = flow.ones(2, 3, requires_grad=True)
>>> with flow.inference_mode():
...     y = x * x
>>> y.requires_grad
False
>>> @flow.inference_mode()
... def no_grad_func(x):
...     return x * x
>>> y = no_grad_func(x)
>>> y.requires_grad
False
oneflow.is_floating_point(input)

Returns True if the data type of input is a floating point data type i.e., one of flow.float64, flow.float32, flow.float16.

Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow

>>> input = flow.tensor([1, 2, 3, 4, 5], dtype=flow.int)
>>> output = flow.is_floating_point(input)
>>> output
False
oneflow.is_grad_enabled()

Returns True if grad mode is currently enabled.

oneflow.is_nonzero(input) -> (bool)

Returns True if the input is a single element tensor which is not equal to zero after type conversions. i.e. not equal to flow.tensor([0.]) or flow.tensor([0]).

Throws a RuntimeError if input.shape.numel() != 1

For Example:

>>> import oneflow as flow
>>> flow.is_nonzero(flow.tensor([0.]))
False
>>> flow.is_nonzero(flow.tensor([1.5]))
True
>>> flow.is_nonzero(flow.tensor([3]))
True
oneflow.load(path: str, consistent_src_rank: Optional[int] = None)Dict[str, oneflow._oneflow_internal.Tensor]
oneflow.log()

Returns a new tensor with the natural logarithm of the elements of input.

\[y_{i} = \log_{e} (x_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> arr = np.random.randn(2, 3, 4, 5)
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.log(input)
oneflow.log1p()

Returns a new tensor with the natural logarithm of (1 + input).

\[\text{out}_{i}=\log_e(1+\text{input}_{i})\]

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = flow.tensor(np.array([1.3, 1.5, 2.7]), dtype=flow.float32)
>>> out = flow.log1p(x)
>>> out
tensor([0.8329, 0.9163, 1.3083], dtype=oneflow.float32)
oneflow.logical_and(input, other)

Computes the element-wise logical AND of the given input tensors. Zeros are treated as False and nonzeros are treated as True.

Parameters
Returns

The output Tensor

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input1 = flow.tensor(np.array([1, 0, 1]).astype(np.float32), dtype=flow.float32)
>>> input2 = flow.tensor(np.array([1, 1, 0]).astype(np.float32), dtype=flow.float32)

>>> out = flow.logical_and(input1, input2)
>>> out
tensor([1, 0, 0], dtype=oneflow.int8)
oneflow.logical_or(input, other)

Computes the element-wise logical OR of the given input tensors. Zeros are treated as False and nonzeros are treated as True.

Parameters
Returns

The output Tensor

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input1 = flow.tensor(np.array([1, 0, 1]).astype(np.float32), dtype=flow.float32)
>>> input2 = flow.tensor(np.array([1, 0, 0]).astype(np.float32), dtype=flow.float32)

>>> out = flow.logical_or(input1, input2)
>>> out
tensor([1, 0, 1], dtype=oneflow.int8)
oneflow.logical_xor(input, other)

Computes the element-wise logical XOR of the given input tensors. Zeros are treated as False and nonzeros are treated as True.

Parameters
Returns

The output Tensor

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input1 = flow.tensor(np.array([1, 0, 1]).astype(np.float32), dtype=flow.float32)
>>> input2 = flow.tensor(np.array([1, 0, 0]).astype(np.float32), dtype=flow.float32)
>>> out = flow.logical_xor(input1, input2)
>>> out
tensor([0, 0, 1], dtype=oneflow.int8)
oneflow.lt(input, other)

Returns the truth value of \(input < other\) element-wise.

Parameters
Returns

A Tensor with int8 type.

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input1 = flow.tensor(np.array([1, 2, 3]).astype(np.float32), dtype=flow.float32)
>>> input2 = flow.tensor(np.array([1, 2, 4]).astype(np.float32), dtype=flow.float32)

>>> out = flow.lt(input1, input2)
>>> out
tensor([0, 0, 1], dtype=oneflow.int8)
oneflow.masked_fill(input, mask, value)

Fills elements of self tensor with value where mask is True. The shape of mask must be broadcastable with the shape of the underlying tensor.

Parameters
  • mask (BoolTensor) – the boolean mask

  • value (float) – the value to fill in with

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> in_arr = np.array(
...     [[[-0.13169311,  0.97277078,  1.23305363,  1.56752789],
...     [-1.51954275,  1.87629473, -0.53301206,  0.53006478],
...     [-1.38244183, -2.63448052,  1.30845795, -0.67144869]],
...     [[ 0.41502161,  0.14452418,  0.38968   , -1.76905653],
...     [ 0.34675095, -0.7050969 , -0.7647731 , -0.73233418],
...     [-1.90089858,  0.01262963,  0.74693893,  0.57132389]]]
... )
>>> fill_value = 8.7654321 # random value e.g. -1e9 3.1415
>>> input = flow.tensor(in_arr, dtype=flow.float32)
>>> mask = flow.tensor((in_arr > 0).astype(np.int8), dtype=flow.int)
>>> output = flow.masked_fill(input, mask, fill_value)

# tensor([[[-0.1317,  8.7654,  8.7654,  8.7654],
#  [-1.5195,  8.7654, -0.533 ,  8.7654],
#  [-1.3824, -2.6345,  8.7654, -0.6714]],

# [[ 8.7654,  8.7654,  8.7654, -1.7691],
#  [ 8.7654, -0.7051, -0.7648, -0.7323],
#  [-1.9009,  8.7654,  8.7654,  8.7654]]], dtype=oneflow.float32)
oneflow.masked_select(input, mask)

Returns a new 1-D tensor which indexes the input tensor according to the boolean mask mask which is a BoolTensor(In oneFlow BoolTensor is replaced by Int8Tensor).

The shapes of the mask tensor and the input tensor don’t need to match, but they must be broadcastable.

Parameters
  • input (Tensor) – the input tensor.

  • mask (Tensor) – the tensor containing the binary mask to index with

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> input = flow.tensor(np.array([[-0.4620, 0.3139], [0.3898, -0.7197], [0.0478, -0.1657]]), dtype=flow.float32)
>>> mask = input.gt(0.05)
>>> out = flow.masked_select(input, mask)
>>> out
tensor([0.3139, 0.3898], dtype=oneflow.float32)
oneflow.matmul()

This operator applies matrix multiplication to two Tensor.

Parameters
Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input1 = flow.tensor(np.random.randn(2, 6), dtype=flow.float32)
>>> input2 = flow.tensor(np.random.randn(6, 5), dtype=flow.float32)
>>> of_out = flow.matmul(input1, input2)
>>> of_out.shape
oneflow.Size([2, 5])
oneflow.max(input, dim=None, keepdim=False)

Computes the maximum value of all elements in the input tensor.

For example:

>>> import oneflow as flow
>>> input = flow.Tensor([[4, 1, 5], [2, 6, 3]])
>>> flow.max(input)
tensor(6., dtype=oneflow.float32)
>>> flow.max(input, dim=0)
tensor([4., 6., 5.], dtype=oneflow.float32)
>>> flow.max(input, dim=1)
tensor([5., 6.], dtype=oneflow.float32)
oneflow.mean(input, dim=None, keepdim=False)

Computes the mean of row of elements in a tensor in the given axis, if the axis is None, mean of all elements will be caculated.

For example:

>>> import oneflow as flow
>>> input = flow.Tensor([[1, 2, 3], [4, 5, 6]])
>>> flow.mean(input)
tensor(3.5000, dtype=oneflow.float32)
>>> flow.mean(input, dim=0)
tensor([2.5000, 3.5000, 4.5000], dtype=oneflow.float32)
>>> flow.mean(input, dim=1)
tensor([2., 5.], dtype=oneflow.float32)
oneflow.meshgrid(*tensors)

The interface is consistent with PyTorch. The documentation is referenced from: https://pytorch.org/docs/stable/_modules/torch/functional.html#meshgrid

Take \(N\) tensors, each of which can be either scalar or 1-dimensional vector, and create \(N\) N-dimensional grids, where the \(i\) th grid is defined by expanding the \(i\) th input over dimensions defined by other inputs.

Parameters

tensors (list of Tensor) – list of scalars or 1 dimensional tensors. Scalars will be treated as tensors of size \((1,)\) automatically

Returns

If the input has \(k\) tensors of size \((N_1,), (N_2,), \ldots , (N_k,)\), then the output would also have \(k\) tensors, where all tensors are of size \((N_1, N_2, \ldots , N_k)\).

Return type

seq (sequence of Tensors)

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input1 = flow.tensor(np.array([1, 2, 3]), dtype=flow.float32)
>>> input2 = flow.tensor(np.array([4, 5, 6]), dtype=flow.float32)
>>> of_x, of_y = flow.meshgrid(input1, input2)
>>> of_x
tensor([[1., 1., 1.],
        [2., 2., 2.],
        [3., 3., 3.]], dtype=oneflow.float32)
>>> of_y
tensor([[4., 5., 6.],
        [4., 5., 6.],
        [4., 5., 6.]], dtype=oneflow.float32)
oneflow.min(input, dim=None, keepdim=False)

Computes the minimum value of all elements in the input tensor.

For example:

>>> import oneflow as flow
>>> input = flow.Tensor([[4, 1, 5], [2, 6, 3]])
>>> flow.min(input)
tensor(1., dtype=oneflow.float32)
>>> flow.min(input, dim=0)
tensor([2., 1., 3.], dtype=oneflow.float32)
>>> flow.min(input, dim=1)
tensor([1., 2.], dtype=oneflow.float32)
oneflow.mish(x: Tensor)Tensor

Applies the element-wise function:

\[ext{mish}(x) = x * ext{tanh}( ext{softplus}(x))\]

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([1, 2, 3]).astype(np.float32)
>>> input = flow.tensor(x)

>>> out = flow.mish(input)
>>> out
tensor([0.8651, 1.9440, 2.9865], dtype=oneflow.float32)

See Mish for more details.

oneflow.mul()

Computes the multiplication of input by other for each element, scalar and broadcast promotation are supported.

The formula is:

\[out = input \times other\]

For example:

>>> import numpy as np
>>> import oneflow as flow

# element-wise multiply
>>> input = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.mul(input,other).numpy()
>>> out.shape
(2, 3)

# scalar mutiply
>>> input = 5
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.mul(input,other).numpy()
>>> out.shape
(2, 3)

# broadcast mutiply
>>> input = flow.tensor(np.random.randn(1,1), dtype=flow.float32)
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.mul(input,other).numpy()
>>> out.shape
(2, 3)
oneflow.narrow(x, dim: int, start: int, length: int)

Returns a new tensor that is a narrowed version of x tensor. The dimension dim is input from start to start + length.

Parameters
  • x – the tensor to narrow.

  • dim – the dimension along which to narrow.

  • start – the starting dimension.

  • length – the distance to the ending dimension.

For example:

>>> import oneflow as flow
>>> x = flow.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
>>> flow.narrow(x, 0, 0, 2)
tensor([[1, 2, 3],
        [4, 5, 6]], dtype=oneflow.int64)
>>> flow.narrow(x, 1, 1, 2)
tensor([[2, 3],
        [5, 6],
        [8, 9]], dtype=oneflow.int64)
oneflow.neg(input)

This operator computes the negative value of Tensor.

Parameters

input (oneflow.Tensor) – A Tensor

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input = flow.tensor(
...    np.array([1.0, -1.0, 2.3]).astype(np.float32), dtype=flow.float32
... )
>>> out = flow.negative(input)
>>> out
tensor([-1.0000,  1.0000, -2.3000], dtype=oneflow.float32)
oneflow.negative(input)

This operator computes the negative value of Tensor.

Parameters

input (oneflow.Tensor) – A Tensor

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> input = flow.tensor(
...    np.array([1.0, -1.0, 2.3]).astype(np.float32), dtype=flow.float32
... )
>>> out = flow.negative(input)
>>> out
tensor([-1.0000,  1.0000, -2.3000], dtype=oneflow.float32)
class oneflow.no_grad

Context-manager that disabled gradient calculation.

Disabling gradient calculation is useful for inference, when you are sure that you will not call Tensor.backward(). It will reduce memory consumption for computations that would otherwise have requires_grad=True.

In this mode, the result of every computation will have requires_grad=False, even when the inputs have requires_grad=True.

This context manager is thread local; it will not affect computation in other threads.

Also functions as a decorator. (Make sure to instantiate with parenthesis.)

>>> import oneflow as flow
>>> x = flow.ones(2, 3, requires_grad=True)
>>> with flow.no_grad():
...     y = x * x
>>> y.requires_grad
False
>>> @flow.no_grad()
... def no_grad_func(x):
...     return x * x
>>> y = no_grad_func(x)
>>> y.requires_grad
False
oneflow.nonzero(input, *, out=None, as_tuple=False)Tensor or tuple of Tensors

Note

When as_tuple is False (default): returns a 2-D tensor where each row is the index for a nonzero value.

When as_tuple is True: returns a tuple of 1-D index tensors, allowing for advanced indexing, so x[x.nonzero(as_tuple=True)] gives all nonzero values of tensor x. Of the returned tuple, each index tensor contains nonzero indices for a certain dimension.

See below for more details on the two behaviors.

When as_tuple is False (default):

Returns a tensor containing the indices of all non-zero elements of input. Each row in the result contains the indices of a non-zero element in input. The result is sorted lexicographically, with the last index changing the fastest (C-style).

If input has \(n\) dimensions, then the resulting indices tensor out is of size \((z \times n)\), where \(z\) is the total number of non-zero elements in the input tensor.

When as_tuple is True:

Returns a tuple of 1-D tensors, one for each dimension in input, each containing the indices (in that dimension) of all non-zero elements of input .

If input has \(n\) dimensions, then the resulting tuple contains \(n\) tensors of size \(z\), where \(z\) is the total number of non-zero elements in the input tensor.

As a special case, when input has zero dimensions and a nonzero scalar value, it is treated as a one-dimensional tensor with one element.

Parameters

input (Tensor) – the input tensor.

Keyword Arguments

out (Tensor, optional) – the output tensor containing indices

Returns

If as_tuple is False, the output tensor containing indices. If as_tuple is True, one 1-D tensor for each dimension, containing the indices of each nonzero element along that dimension.

Return type

Tensor or tuple of Tensors

Example:

>>> import oneflow as flow
>>> flow.nonzero(flow.tensor([1, 1, 1, 0, 1]))
tensor([[0],
        [1],
        [2],
        [4]], dtype=oneflow.int32)
>>> flow.nonzero(flow.tensor([[0.6, 0.0, 0.0, 0.0],
...                             [0.0, 0.4, 0.0, 0.0],
...                             [0.0, 0.0, 1.2, 0.0],
...                             [0.0, 0.0, 0.0,-0.4]]))
tensor([[0, 0],
        [1, 1],
        [2, 2],
        [3, 3]], dtype=oneflow.int32)
>>> flow.nonzero(flow.tensor([1, 1, 1, 0, 1]), as_tuple=True)
(tensor([0, 1, 2, 4], dtype=oneflow.int32),)
>>> flow.nonzero(flow.tensor([[0.6, 0.0, 0.0, 0.0],
...                             [0.0, 0.4, 0.0, 0.0],
...                             [0.0, 0.0, 1.2, 0.0],
...                             [0.0, 0.0, 0.0,-0.4]]), as_tuple=True)
(tensor([0, 1, 2, 3], dtype=oneflow.int32), tensor([0, 1, 2, 3], dtype=oneflow.int32))
>>> flow.nonzero(flow.tensor(5), as_tuple=True)
(tensor([0], dtype=oneflow.int32),)
oneflow.ones(*size: Union[int, Tuple[int, ], oneflow._oneflow_internal.Size, List[int]], dtype: Optional[oneflow._oneflow_internal.dtype] = None, device: Optional[Union[str, oneflow._oneflow_internal.device]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[oneflow._oneflow_internal.sbp.sbp] = None, requires_grad: bool = False)

Returns a tensor filled with the scalar value 1, with the shape defined by the variable argument size.

Parameters
  • size (an integer or tuple of integer values) – defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple.

  • dtype (flow.dtype, optional) – the desired data type of returned tensor.

  • device (flow.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type

  • placement (flow.placement, optional) – the desired placement of returned consistent tensor. Default: if None, the returned tensor is local one using the argument device.

  • sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional) – the desired sbp descriptor of returned consistent tensor. Default: if None, the returned tensor is local one using the argument device.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow
>>> y = flow.ones(5)
>>> y
tensor([1., 1., 1., 1., 1.], dtype=oneflow.float32)
>>> y = flow.ones(2,3)
>>> y
tensor([[1., 1., 1.],
        [1., 1., 1.]], dtype=oneflow.float32)
oneflow.ones_like(input) is equivalent to flow.ones(input.shape, dtype=input.dtype)
Parameters

other (Tensor) – The size of input will determine size of the output tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = flow.tensor(np.random.rand(5), dtype=flow.float32)
>>> y = flow.ones_like(x)
>>> y
tensor([1., 1., 1., 1., 1.], dtype=oneflow.float32)
oneflow.pow()
Takes the power of each element in input with exponent and returns a tensor with the result. Exponent can be either a single float number, a single int number, or a tensor with the same shape as input.

When exponent is a scalar value, the operation applied is:

\[\text{out}_i = x_i ^ \text{exponent}\]

When exponent is a tensor, the operation applied is:

\[\text{out}_i = x_i ^ {\text{exponent}_i}\]
Args:
  • input (Tensor): the input tensor.

  • exponent (int, float, Tensor): the exponent.

Returns:

Tensor: The result of variance on the specified axis of input Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> x = flow.tensor(np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0]), dtype=flow.float32)
>>> out = flow.pow(x, 2)
>>> out
tensor([ 1.,  4.,  9., 16., 25., 36.], dtype=oneflow.float32)

>>> x = flow.tensor(np.array([1.0, 2.0, 3.0, 4.0]), dtype=flow.float32)
>>> y = flow.tensor(np.array([1.0, 2.0, 3.0, 4.0]), dtype=flow.float32)
>>> out = flow.pow(x, y)
>>> out
tensor([  1.,   4.,  27., 256.], dtype=oneflow.float32)
oneflow.prod(input, dim=None, keepdim=False)

Computes the product of row of elements in a tensor in the given axis.

note: if the dim is None, it will return a tensor with only one element whose value is the product of all elements of input.

Parameters
  • input (Tensor) – the source tensor

  • dim (int) – the axis along which to prod

For example:

>>> import oneflow as flow
>>> input = flow.Tensor([[1, 2, 3], [4, 5, 6]])
>>> flow.prod(input)
tensor(720., dtype=oneflow.float32)
>>> flow.prod(input, dim=0)
tensor([ 4., 10., 18.], dtype=oneflow.float32)
>>> flow.prod(input, dim=1)
tensor([  6., 120.], dtype=oneflow.float32)
oneflow.rand(*size, out=None, generator=None, dtype: Optional[oneflow._oneflow_internal.dtype] = None, layout=None, device: Optional[Union[str, oneflow._oneflow_internal.device]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[oneflow._oneflow_internal.sbp.sbp] = None, requires_grad: bool = False)

Returns a tensor filled with random numbers from a uniform distribution on the interval [0, 1)

The shape of the tensor is defined by the variable argument size.

Parameters
  • size (int... or oneflow.Size) – Defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple or oneflow.Size.

  • out (optional) – The output tensor.

  • dtype (flow.dtype, optional) – The desired data type of returned tensor. Default: flow.float32.

  • layout (optional) – The desired layout of returned Tensor.

  • generator (flow.Generator, optional) – a pseudorandom number generator for sampling

  • device (flow.device, optional) – The desired device of returned local tensor. If None, uses the current device.

  • placement (flow.placement, optional) – The desired device of returned consistent tensor. If None, will construct local tensor.

  • sbp (flow.sbp, optional) – The desired sbp of returned consistent tensor. It must be equal with the numbers of placement.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow
>>> x = flow.rand(3,3)
>>> x.shape
oneflow.Size([3, 3])
>>> x.is_consistent
False
>>> placement = flow.placement("cpu", {0: [0]})
>>> sbp = flow.sbp.broadcast
>>> x = flow.rand(3, 3, placement=placement, sbp=sbp)
>>> x.is_consistent
True
oneflow.randint(low: oneflow.int64, high: oneflow.int64, size: tuple, out=None, generator=None, dtype: Optional[oneflow._oneflow_internal.dtype] = None, layout=None, device: Optional[Union[str, oneflow._oneflow_internal.device]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[oneflow._oneflow_internal.sbp.sbp] = None, requires_grad: bool = False)

Returns a tensor filled with random integers generated uniformly between low (inclusive) and high (exclusive).

The shape of the tensor is defined by the variable argument size.

Parameters
  • size (int... or oneflow.Size) – Defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple or oneflow.Size.

  • out (optional) – The output tensor.

  • dtype (flow.dtype, optional) – The desired data type of returned tensor. Default: flow.int64.

  • layout (optional) – The desired layout of returned Tensor.

  • generator (flow.Generator, optional) –

  • device (flow.device, optional) – The desired device of returned local tensor. If None, uses the current device.

  • placement (flow.placement, optional) – The desired device of returned consistent tensor. If None, will construct local tensor.

  • sbp (flow.sbp, optional) – The desired sbp of returned consistent tensor. It must be equal with the numbers of placement.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow
>>> generator = flow.Generator()
>>> generator.manual_seed(0)
>>> flow.randint(0, 5, (3,3), generator=generator)
tensor([[2, 2, 3],
        [4, 3, 4],
        [2, 4, 2]], dtype=oneflow.int64)
oneflow.randn(*size, out=None, generator=None, dtype: Optional[oneflow._oneflow_internal.dtype] = None, layout=None, device: Optional[Union[str, oneflow._oneflow_internal.device]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[oneflow._oneflow_internal.sbp.sbp] = None, requires_grad: bool = False)

Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution).

The shape of the tensor is defined by the variable argument size.

Parameters
  • size (int... or oneflow.Size) – Defining the shape of the output tensor. Can be a variable number of arguments or a collection like a list or tuple or oneflow.Size.

  • out (optional) – The output tensor.

  • dtype (flow.dtype, optional) – The desired data type of returned tensor. Default: flow.float32.

  • layout (optional) – The desired layout of returned Tensor.

  • generator (flow.Generator, optional) – a pseudorandom number generator for sampling

  • device (flow.device, optional) – The desired device of returned local tensor. If None, uses the current device.

  • placement (flow.placement, optional) – The desired device of returned consistent tensor. If None, will construct local tensor.

  • sbp (flow.sbp, optional) – The desired sbp of returned consistent tensor. It must be equal with the numbers of placement.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow
>>> x = flow.randn(3,3)
>>> x.shape
oneflow.Size([3, 3])
>>> x.is_consistent
False
>>> placement = flow.placement("cpu", {0:[0]})
>>> sbp = flow.sbp.broadcast
>>> x = flow.randn(3,3,placement=placement,sbp=sbp)
>>> x.is_consistent
True
oneflow.randperm(n: oneflow.int32, generator: Optional[oneflow.framework.generator.create_generator] = None, out=None, dtype: Optional[oneflow._oneflow_internal.dtype] = None, layout=None, device: Optional[Union[str, oneflow._oneflow_internal.device]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[oneflow._oneflow_internal.sbp.sbp] = None, requires_grad: bool = False, pin_memory: bool = False)oneflow._oneflow_internal.Tensor

Returns a random permutation of integers from 0 to n - 1.

Parameters

n (int) – the upper bound (exclusive)

Keyword Arguments
  • generator (oneflow.Generator, optional) – a pseudorandom number generator for sampling

  • out (Tensor, optional) – output Tensor,not supported yet.

  • dtype (oneflow.dtype, optional) – the desired data type of returned tensor. Default: oneflow.int64.

  • layout – layout is not supported yet.

  • device – the desired device of returned tensor. Default: cpu.

  • placement – (flow.placement, optional): The desired device of returned consistent tensor. If None, will construct local tensor.

  • sbp – (flow.sbp, optional): The desired sbp of returned consistent tensor. It must be equal with the numbers of placement.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

  • pin_memory (bool, optional) – pin_memory is not supported yet.

Example:

>>> import oneflow as flow
>>> generator = flow.Generator()
>>> generator.manual_seed(0)
>>> flow.randperm(5, generator=generator)
tensor([2, 4, 3, 0, 1], dtype=oneflow.int32)
oneflow.reciprocal()

Computes the safe reciprocal of x. If x is zero, the reciprocal will be also set to zero.

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = flow.tensor(np.array([[1, 2, 3], [4, 5, 6]]), dtype=flow.float32)
>>> out = flow.reciprocal(x)
>>> out.numpy()
array([[1.        , 0.5       , 0.33333334],
       [0.25      , 0.2       , 0.16666667]], dtype=float32)
oneflow.repeat(input, *sizes)

This operator repeat the input tensor to a larger size along the specified dimensions.

Parameters
  • x (oneflow.Tensor) – The input Tensor.

  • *size (flow.Size or int) – The number of times to repeat this tensor along each dimension

Returns

The result Tensor.

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = np.array([[[[0, 1]],
...               [[2, 3]],
...               [[4, 5]]]]).astype(np.int32)

>>> input = flow.Tensor(x)
>>> out = input.repeat(1, 1, 2, 2)
>>> out.shape
oneflow.Size([1, 3, 2, 4])
oneflow.reshape(input, shape: Optional[Sequence[int]] = None)

This operator reshapes a Tensor.

We can set one dimension in shape as -1, the operator will infer the complete shape.

Parameters
  • x – A Tensor.

  • shape – Shape of the output tensor.

Returns

A Tensor has the same type as x.

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> x = np.array(
...    [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
... ).astype(np.float32)
>>> input = flow.Tensor(x)

>>> y = flow.reshape(input, shape=[2, 2, 2, -1]).shape
>>> y
oneflow.Size([2, 2, 2, 2])
oneflow.round()

This operator rounds the value of Blob to the nearest integer. :param input: A Tensor :type input: oneflow.Tensor

Returns

The result Tensor

Return type

oneflow.Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x1 = flow.tensor(np.array([1.49999, 1.500001, 2.7]).astype(np.float32))
>>> out1 = flow.round(x1)
>>> out1.numpy()
array([1., 2., 3.], dtype=float32)
>>> x2 = flow.tensor(np.array([2.499999, 7.5000001, 5.3, 6.8]).astype(np.float32))
>>> out2 = flow.round(x2)
>>> out2.numpy()
array([2., 8., 5., 7.], dtype=float32)
oneflow.save(var_dict: Dict[str, oneflow._oneflow_internal.Tensor], path: str, consistent_dst_rank: Optional[int] = None)None
oneflow.scatter(input, dim, index, src)

This operator writes the elements specified by index along with the axis dim from the src into the input.

Take a 3-D blob as example, the output is specified by:

input[index[i][j][k]][j][k] = src[i][j][k]  # if dim == 0
input[i][index[i][j][k]][k] = src[i][j][k]  # if dim == 1
input[i][j][index[i][j][k]] = src[i][j][k]  # if dim == 2

input, index and src (if it is a Tensor) should all have the same number of dimensions. It is also required that index.shape(d) <= src.shape(d) for all dimensions d, and that index.shape(d) <= self.shape(d) for all dimensions d != dim. Note that index and src do not broadcast.

Parameters
  • input (Tensor) – The input blob.

  • dim (int) – The axis along which to index

  • index (Tensor) – The index blob of elements to scatter.

  • src (Tensor or float) – The source blob whose elements will be scatterd and updated to output.

Returns

The scatterd Tensor.

Return type

Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> input = flow.ones((3,5))*2
>>> index = flow.tensor(np.array([[0,1,2],[0,1,4]], ), dtype=flow.int32)
>>> src = flow.Tensor(np.array([[0,10,20,30,40],[50,60,70,80,90]]))
>>> out = flow.scatter(input, 1, index, src)
>>> out
tensor([[ 0., 10., 20.,  2.,  2.],
        [50., 60.,  2.,  2., 70.],
        [ 2.,  2.,  2.,  2.,  2.]], dtype=oneflow.float32)
oneflow.scatter_add(input, dim, index, src)

This operator scatter the src with addition operation according to index along dim into the input.

Take a 3-D blob as example, the output is specified by:

input[index[i][j][k]][j][k] += src[i][j][k]  # if dim == 0
input[i][index[i][j][k]][k] += src[i][j][k]  # if dim == 1
input[i][j][index[i][j][k]] += src[i][j][k]  # if dim == 2
Parameters
  • input (Tensor) – The input blob.

  • dim (int) – The axis along which to index

  • index (Tensor) – The index blob of elements to scatter.

  • src (Tensor) – The source blob whose elements will be scatterd and added to output.

Returns

The scatterd Tensor.

Return type

Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.ones((3,5))*2
>>> index = flow.tensor(np.array([[0,1,2],[0,1,4]], ), dtype=flow.int32)
>>> src = flow.Tensor(np.array([[0,10,20,30,40],[50,60,70,80,90]]))
>>> out = flow.scatter_add(input, 1, index, src)
>>> out
tensor([[ 2., 12., 22.,  2.,  2.],
        [52., 62.,  2.,  2., 72.],
        [ 2.,  2.,  2.,  2.,  2.]], dtype=oneflow.float32)
oneflow.scatter_nd(index, update, shape)

This operator inserts the elements in update according to the index and create a new Tensor.

Parameters
  • index – The indices of update. Its type should be flow.int.

  • update – The update Tensor.

  • shape (Sequence[int]) – The constant tensor shape, the constant tensor elements are all zero.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> index = flow.tensor(np.array([[1], [6], [4]]), dtype=flow.int)
>>> update = flow.tensor(np.array([10.2,5.1,12.7]), dtype=flow.float)
>>> out = flow.scatter_nd(index, update, [8])
>>> out
tensor([ 0.0000, 10.2000,  0.0000,  0.0000, 12.7000,  0.0000,  5.1000,  0.0000],
       dtype=oneflow.float32)
oneflow.selu(x: Tensor)Tensor

Applies element-wise function :math:` ext{SELU}(x) = scale * (max(0,x) + min(0, lpha * (exp(x) - 1)))`, with \(lpha=1.6732632423543772848170429916717\) and \(scale=1.0507009873554804934193349852946\).

See SELU for more details.

For examples:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([1, 2, 3]).astype(np.float32)
>>> input = flow.tensor(x)
>>> out = flow.nn.functional.selu(input)
>>> out
tensor([1.0507, 2.1014, 3.1521], dtype=oneflow.float32)
oneflow.sigmoid(input)Tensor

Applies the element-wise function \(\text{Sigmoid}(x) = \frac{1}{1 + \exp(-x)}\)

See Sigmoid for more details.

For examples:

>>> import numpy as np
>>> import oneflow as flow

>>> x = flow.tensor(np.array([0.81733328, 0.43621480, 0.10351428]))
>>> input = flow.tensor(x, dtype=flow.float32)
>>> out = flow.nn.functional.sigmoid(input)
>>> out
tensor([0.6937, 0.6074, 0.5259], dtype=oneflow.float32)
oneflow.sign()

Computes the sign of Tensor.

\[\text{out}_{i} = \text{sgn}(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x1 = flow.tensor(np.array([-2, 0, 2]).astype(np.float32))
>>> out1 = flow.sign(x1)
>>> out1.numpy()
array([-1.,  0.,  1.], dtype=float32)
>>> x2 = flow.tensor(np.array([-3.2, -4.5, 5.8]).astype(np.float32),device=flow.device('cuda'))
>>> out2 = flow.sign(x2)
>>> out2.numpy()
array([-1., -1.,  1.], dtype=float32)
oneflow.silu(x: Tensor)Tensor

The formula is:

\[ext{silu}(x) = x * sigmoid(x)\]

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([1, 2, 3]).astype(np.float32)
>>> input = flow.tensor(x)
>>> out = flow.silu(input)
>>> out
tensor([0.7311, 1.7616, 2.8577], dtype=oneflow.float32)

See SiLU for more details.

oneflow.sin(input)
oneflow.sinh()

Returns a new tensor with the hyperbolic sine of the elements of input.

\[\text{out}_{i} = \sinh(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x1 = flow.tensor(np.array([1, 2, 3]), dtype=flow.float32)
>>> x2 = flow.tensor(np.array([1.53123589,0.54242598,0.15117185]), dtype=flow.float32)
>>> x3 = flow.tensor(np.array([1,0,-1]), dtype=flow.float32)

>>> flow.sinh(x1).numpy()
array([ 1.1752012,  3.6268604, 10.017875 ], dtype=float32)
>>> flow.sinh(x2).numpy()
array([2.20381  , 0.5694193, 0.1517483], dtype=float32)
>>> flow.sinh(x3).numpy()
array([ 1.1752012,  0.       , -1.1752012], dtype=float32)
oneflow.slice(input, slice_tup_list: Sequence[Tuple[int, int, int]])

Extracts a slice from a tensor. The slice_tup_list assigns the slice indices in each dimension, the format is (start, stop, step). The operator will slice the tensor according to the slice_tup_list.

Parameters
  • input – A Tensor.

  • slice_tup_list – A list of slice tuple, indicate each dimension slice (start, stop, step).

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> input = flow.Tensor(np.random.randn(3, 6, 9).astype(np.float32))
>>> tup_list = [[None, None, None], [0, 5, 2], [0, 6, 3]]
>>> y = flow.slice(input, slice_tup_list=tup_list)
>>> y.shape
oneflow.Size([3, 3, 2])
oneflow.slice_update(input, update, slice_tup_list: Sequence[Tuple[int, int, int]])

Update a slice of tensor x. Like x[start:stop:step] = update.

Parameters
  • x – A Tensor, whose slice will be updated.

  • update – A Tensor, indicate the update content.

  • slice_tup_list – A list of slice tuple, indicate each dimension slice (start, stop, step).

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> input = flow.Tensor(np.array([1, 1, 1, 1, 1]).astype(np.float32))
>>> update = flow.Tensor(np.array([2, 3, 4]).astype(np.float32))
>>> y = flow.slice_update(input, update, slice_tup_list=[[1, 4, 1]])
>>> y.numpy()
array([1., 2., 3., 4., 1.], dtype=float32)
oneflow.softmax(tensor, dim=None)

Applies the Softmax function to an n-dimensional input Tensor rescaling them so that the elements of the n-dimensional output Tensor lie in the range [0,1] and sum to 1.

Softmax is defined as:

\[\text{Softmax}(x_{i}) = \frac{\exp(x_i)}{\sum_j \exp(x_j)}\]

When the input Tensor is a sparse tensor then the unspecifed values are treated as -inf.

Shape:
  • Input: \((*)\) where * means, any number of additional dimensions

  • Output: \((*)\), same shape as the input

Returns

a Tensor of the same dimension and shape as the input with values in the range [0, 1]

Parameters

dim (int) – A dimension along which Softmax will be computed (so every slice along dim will sum to 1).

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> m = flow.nn.Softmax(dim = 2)
>>> x = flow.Tensor(
...    np.array(
...        [[[-0.46716809,  0.40112534,  0.61984003],
...        [-1.31244969, -0.42528763,  1.47953856]]]
...    )
... )
>>> out = m(x)
>>> out
tensor([[[0.1575, 0.3754, 0.4671],
         [0.0507, 0.1230, 0.8263]]], dtype=oneflow.float32)
oneflow.softplus(x: Tensor)Tensor

Applies the element-wise function:

\[\text{Softplus}(x) = \frac{1}{\beta} * \log(1 + \exp(\beta * x))\]

See Softplus for more details.

oneflow.softsign(x: Tensor)Tensor

The formula is:

\[softsign(x) = \frac{x}{1 + |x|}\]

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([1, 2, 3]).astype(np.float32)
>>> input = flow.tensor(x)
>>> out = flow._C.softsign(input)
>>> out
tensor([0.5000, 0.6667, 0.7500], dtype=oneflow.float32)

See Softsign for more details.

oneflow.sort(input, dim: int = - 1, descending: bool = False)

Sorts the elements of the input tensor along a given dimension in ascending order by value.

Parameters
  • input (oneflow.Tensor) – The input Tensor.

  • dim (int, optional) – dimension to be sorted. Defaults to the last dim (-1).

  • descending (bool, optional) – controls the sorting order (ascending or descending).

Returns

A tuple of (values, indices), where where the values are the sorted values and the indices are the indices of the elements in the original input tensor.

Return type

Tuple(oneflow.Tensor, oneflow.Tensor(dtype=int32))

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = np.array([[1, 3, 8, 7, 2], [1, 9, 4, 3, 2]], dtype=np.float32)
>>> input = flow.Tensor(x)
>>> (values, indices) = flow.sort(input)
>>> values
tensor([[1., 2., 3., 7., 8.],
        [1., 2., 3., 4., 9.]], dtype=oneflow.float32)
>>> indices
tensor([[0, 4, 1, 3, 2],
        [0, 4, 3, 2, 1]], dtype=oneflow.int32)
>>> (values, indices) = flow.sort(input, descending=True)
>>> values
tensor([[8., 7., 3., 2., 1.],
        [9., 4., 3., 2., 1.]], dtype=oneflow.float32)
>>> indices
tensor([[2, 3, 1, 4, 0],
        [1, 2, 3, 4, 0]], dtype=oneflow.int32)
>>> (values, indices) = flow.sort(input, dim=0)
>>> values
tensor([[1., 3., 4., 3., 2.],
        [1., 9., 8., 7., 2.]], dtype=oneflow.float32)
>>> indices
tensor([[0, 0, 1, 1, 0],
        [1, 1, 0, 0, 1]], dtype=oneflow.int32)
oneflow.split(x, split_size_or_sections: Union[int, List[int]], dim: int = 0)

Splits the tensor into chunks.

If split_size_or_sections is an integer type, then x will be split into equally sized chunks (if possible). Last chunk will be smaller if the tensor size along the given dimension dim is not divisible by split_size.

If split_size_or_sections is a list, then x will be split into len(split_size_or_sections) chunks with sizes in dim according to split_size_or_sections.

Parameters
  • x – tensor to split.

  • split_size_or_sections – size of a single chunk or list of sizes for each chunk.

  • dim – dimension along which to split the tensor.

For example:

>>> import oneflow as flow
>>> a = flow.arange(10).view(5, 2)
>>> flow.split(a, 2)
(tensor([[0, 1],
        [2, 3]], dtype=oneflow.int64), tensor([[4, 5],
        [6, 7]], dtype=oneflow.int64), tensor([[8, 9]], dtype=oneflow.int64))
>>> flow.split(a, [1, 4])
(tensor([[0, 1]], dtype=oneflow.int64), tensor([[2, 3],
        [4, 5],
        [6, 7],
        [8, 9]], dtype=oneflow.int64))
oneflow.sqrt()

Returns a new tensor with the square-root of the elements of input.

\[\text{out}_{i} = \sqrt{\text{input}_{i}}\]
Parameters

input – the input tensor.

>>> import oneflow as flow
>>> import numpy as np

>>> arr = np.array([1.0, 2.0, 3.0])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.sqrt(input).numpy()
>>> output
array([1.       , 1.4142135, 1.7320508], dtype=float32)
oneflow.square()

Returns a new tensor with the square of the elements of input.

\[\text{out}_{i} = \sqrt{\text{input}_{i}}\]
Parameters

input – the input tensor.

>>> import oneflow as flow
>>> import numpy as np

>>> arr = np.array([1.0, 2.0, 3.0])
>>> input = flow.tensor(arr, dtype=flow.float32)
>>> output = flow.square(input).numpy()
>>> output
array([1., 4., 9.], dtype=float32)
oneflow.squeeze(input, dim: Optional[Sequence[int]] = None)

This operator removes the specified dimention which size is 1 of the input Tensor. If the dim is not specified, this operator will remove all the dimention which size is 1 of the input Tensor.

The amount of element in return value is the same as Tensor input.

Parameters
  • input (oneflow.Tensor) – The input Tensor.

  • dim (Optional[Sequence[int]]) – The dim. Defaults to None.

Returns

The result Tensor.

Return type

Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> input = flow.Tensor(np.array([[[[1, 1, 1]]]]).astype(np.int32))
>>> out = flow.squeeze(input, dim=[1, 2]).shape
>>> out
oneflow.Size([1, 3])
oneflow.stack(inputs: oneflow._oneflow_internal.Tensor, dim: int = 0)None

Concatenates a sequence of tensors along a new dimension. The returned tensor shares the same underlying data with input tensors.

A dim value within the range [-input.ndimension() - 1, input.ndimension() + 1] can be used. Negative dim will correspond to stack() applied at dim = dim + input.ndimension() + 1.

Parameters
  • inputs (List[oneflow.Tensor]) – the list of input tensors. Each tensor should have the same shape.

  • dim (int) – the index at which to insert the concatenated dimension.

Returns

A Tensor

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = flow.Tensor(np.random.rand(1, 3, 5))
>>> y = flow.Tensor(np.random.rand(1, 3, 5))
>>> out = flow.stack([x, y], dim = -1)
>>> out.shape
oneflow.Size([1, 3, 5, 2])
oneflow.std(input, dim, unbiased=False, keepdim=False)

Returns the standard-deviation of each row of the input tensor in the dimension dim. If dim is a list of dimensions, reduce over all of them.

If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed, resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).

If unbiased is False, then the standard-deviation will be calculated via the biased estimator. Otherwise, Bessel’s correction will be used.

Parameters
  • input (Tensor) – the input tensor.

  • (int or tuple of python (dim) – ints): the dimension or dimensions to reduce.

  • unbiased (bool) – whether to use the unbiased estimation or not

  • keepdim (bool) – whether the output tensor has dim retained or not.

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> arr = np.array([1.0, 2.0, 3.0])
>>> input = flow.Tensor(arr)
>>> output = flow.std(input, dim=0).numpy()
>>> output
array(0.8164968, dtype=float32)
oneflow.sub()

Computes the subtraction of input by other for each element, scalar and broadcast promotation are supported. The formula is:

\[out = input - other\]

For example:

>>> import numpy as np
>>> import oneflow as flow

# element-wise subtract
>>> input = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.sub(input,other).numpy()
>>> out.shape
(2, 3)

# scalar subtract
>>> input = 5
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.sub(input,other).numpy()
>>> out.shape
(2, 3)

# broadcast subtract
>>> input = flow.tensor(np.random.randn(1,1), dtype=flow.float32)
>>> other = flow.tensor(np.random.randn(2,3), dtype=flow.float32)
>>> out = flow.sub(input,other).numpy()
>>> out.shape
(2, 3)
oneflow.sum(input, dim=None, keepdim=False)

Computes the sum of row of elements in a tensor in the given axis, if the axis is None, sum of all elements will be caculated.

For example:

>>> import oneflow as flow
>>> input = flow.Tensor([[1, 2, 3], [4, 5, 6]])
>>> flow.sum(input)
tensor(21., dtype=oneflow.float32)
>>> flow.sum(input, dim=0)
tensor([5., 7., 9.], dtype=oneflow.float32)
>>> flow.sum(input, dim=1)
tensor([ 6., 15.], dtype=oneflow.float32)
oneflow.tan()

Returns the tan value of the elements of input.

\[\text{out}_{i} = \tan(\text{input}_{i})\]
Parameters

input (Tensor) – the input tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> np_arr = np.array([-1/4*np.pi, 0, 1/4*np.pi]).astype(np.float32)
>>> input = flow.tensor(np_arr, dtype=flow.float32)
>>> output = flow.tan(input)
>>> output
tensor([-1.,  0.,  1.], dtype=oneflow.float32)
oneflow.tanh(x: Tensor)Tensor

The equation is:

\[out = \frac{e^x-e^{-x}}{e^x+e^{-x}}\]

See Tanh for more details.

oneflow.tensor()
Constructs a tensor with data, return a consistent tensor if placement and sbp are in kwargs,

otherwise return a local tensor.

Parameters

data – Initial data for the tensor. Can be a list, tuple, NumPy ndarray, scalar or tensor.

Keyword Arguments
  • dtype (oneflow.dtype, optional) – Default: if None, infers data type from data.

  • device (oneflow.device, optional) – the desired device of returned tensor. If placement and sbp is None, uses the current cpu for the default tensor type.

  • placement (oneflow.placement, optional) – the desired placement of returned tensor.

  • sbp (oneflow.sbp or tuple of oneflow.sbp, optional) – the desired sbp of returned tensor.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False

Noted:

The Keyword Argument device is mutually exclusive with placement and sbp. Consistent tensor only can be constructed from tensor.

For example:

>>> import oneflow as flow

>>> x = flow.tensor([1,2,3])
>>> x
tensor([1, 2, 3], dtype=oneflow.int64)
oneflow.tile(input, reps)

The interface is consistent with PyTorch. The documentation is referenced from: https://pytorch.org/docs/stable/generated/torch.tile.html

Constructs a tensor by repeating the elements of input. The reps argument specifies the number of repetitions in each dimension.

If reps specifies fewer dimensions than input has, then ones are prepended to reps until all dimensions are specified. For example, if input has shape (8, 6, 4, 2) and reps is (2, 2), then reps is treated as (1, 1, 2, 2).

Analogously, if input has fewer dimensions than reps specifies, then input is treated as if it were unsqueezed at dimension zero until it has as many dimensions as reps specifies. For example, if input has shape (4, 2) and reps is (3, 3, 2, 2), then input is treated as if it had the shape (1, 1, 4, 2).

Note

This function is similar to NumPy’s tile function.

Parameters
  • input (oneflow.Tensor) – the tensor whose elements to repeat.

  • reps (tuple) – the number of repetitions per dimension.

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> x = np.array([1, 2]).astype(np.int32)
>>> input = flow.tensor(x, dtype=flow.int32)
>>> out = input.tile(reps=(2,))
>>> out
tensor([1, 2, 1, 2], dtype=oneflow.int32)

>>> x = np.random.randn(5, 2, 1)
>>> input = flow.Tensor(x)
>>> out = input.tile(reps=(3, 4))
>>> out.size()
oneflow.Size([5, 6, 4])
oneflow.to(input, *args, **kwargs)
Performs Tensor dtype and/or device conversion.

A flow.dtype and flow.device are inferred from the arguments of input.to(*args, **kwargs).

Note

If the input Tensor already has the correct flow.dtype and flow.device, then input is returned. Otherwise, the returned tensor is a copy of input with the desired.

Parameters
  • input (oneflow.Tensor) – An input tensor.

  • *args (oneflow.Tensor or oneflow.device or oneflow.dtype) – Positional arguments

  • **kwargs (oneflow.device or oneflow.dtype) – Key-value arguments

Returns

A Tensor.

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> arr = np.random.randint(1, 9, size=(1, 2, 3, 4))
>>> input = flow.Tensor(arr)
>>> output = input.to(dtype=flow.float32)
>>> np.array_equal(arr.astype(np.float32), output.numpy())
True
oneflow.transpose(input, dim0, dim1)

Returns a tensor that is a transposed version of input. The given dimensions dim0 and dim1 are swapped.

The resulting out tensor shares its underlying storage with the input tensor, so changing the content of one would change the content of the other.

Parameters
  • input (oneflow.Tensor) – The input tensor.

  • dim0 (int) – the first dimension to be transposed.

  • dim1 (int) – the second dimension to be transposed.

Returns

A transposed tensor.

Return type

Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> input = flow.tensor(np.random.randn(2, 6, 5, 3), dtype=flow.float32)
>>> out = flow.transpose(input, 0, 1).shape
>>> out
oneflow.Size([6, 2, 5, 3])
oneflow.tril()

Returns the lower triangular part of a matrix (2-D tensor) or batch of matrices input along the specified diagonal, the other elements of the result tensor out are set to 0.

Note

if diagonal = 0, the diagonal of the returned tensor will be the main diagonal, if diagonal > 0, the diagonal of the returned tensor will be above the main diagonal, if diagonal < 0, the diagonal of the returned tensor will be below the main diagonal.

Parameters
  • input (Tensor) – the input tensor.

  • diagonal (int, optional) – the diagonal to specify.

For example:

>>> import oneflow as flow
>>> import numpy as np

>>> x = flow.tensor(np.ones(shape=(3, 3)).astype(np.float32))
>>> flow.tril(x)
tensor([[1., 0., 0.],
        [1., 1., 0.],
        [1., 1., 1.]], dtype=oneflow.float32)
oneflow.unsqueeze(input, dim)

Returns a new tensor with a dimension of size one inserted at the specified position.

The returned tensor shares the same underlying data with this tensor.

A dim value within the range [-input.ndimension() - 1, input.ndimension() + 1) can be used. Negative dim will correspond to unsqueeze() applied at dim = dim + input.ndimension() + 1.

Parameters
  • input (Tensor) – the input tensor.

  • dim (int) – the index at which to insert the singleton dimension

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = flow.Tensor(np.random.rand(2, 3, 4))
>>> y = x.unsqueeze(2)
>>> y.shape
oneflow.Size([2, 3, 1, 4])
oneflow.var(input, dim=None, unbiased=True, keepdim=False)

Returns the variance of each row of the input tensor in the given dimension dim.

If keepdim is True, the output tensor is of the same size as input except in the dimension(s) dim where it is of size 1. Otherwise, dim is squeezed (see flow.squeeze()), resulting in the output tensor having 1 (or len(dim)) fewer dimension(s).

Parameters
  • input (Tensor) – the input tensor.

  • (int or tuple of python (dim) – ints): the dimension or dimensions to reduce. Defaults to None.

  • unbiased (bool, optional) – whether to use Bessel’s correction (\(\delta N = 1\)). Defaults to True.

  • keepdim (bool, optional) – whether the output tensor has dim retained or not. Defaults to False.

Returns

The result of variance on the specified axis of input Tensor

Return type

Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> np_arr = np.random.randn(2,3,4,5)
>>> input = flow.Tensor(np_arr)
>>> output = flow.var(input, 1, True)
oneflow.view(input, *shape)

The interface is consistent with PyTorch. The documentation is referenced from: https://pytorch.org/docs/stable/generated/torch.Tensor.view.html

Returns a new tensor with the same data as the self tensor but of a different shape.

The returned tensor shares the same data and must have the same number of elements, but may have a different size. For a tensor to be viewed, the new view size must be compatible with its original size and stride, i.e., each new view dimension must either be a subspace of an original dimension, or only span across original dimensions \(d, d+1, \dots, d+k\) that satisfy the following contiguity-like condition that \(\forall i = d, \dots, d+k-1\),

\[\text{stride}[i] = \text{stride}[i+1] \times \text{size}[i+1]\]

Otherwise, it will not be possible to view self tensor as shape without copying it (e.g., via contiguous()). When it is unclear whether a view() can be performed, it is advisable to use reshape(), which returns a view if the shapes are compatible, and copies (equivalent to calling contiguous()) otherwise.

Parameters
  • input – A Tensor.

  • *shape – flow.Size or int…

Returns

A Tensor has the same type as input.

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array(
...    [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
... ).astype(np.float32)
>>> input = flow.Tensor(x)

>>> y = input.view(2, 2, 2, -1).numpy().shape
>>> y
(2, 2, 2, 2)
oneflow.where(condition, x=None, y=None)

Return a tensor of elements selected from either x or y, depending on condition. If the element in condition is larger than 0,

it will take the x element, else it will take the y element

Note

If x is None and y is None, flow.where(condition) is identical to flow.nonzero(condition, as_tuple=True).

The tensors condition, x, y must be broadcastable.

Parameters
  • condition (IntTensor) – When 1 (nonzero), yield x, otherwise yield y

  • x (Tensor or Scalar) – value (if :attr:x is a scalar) or values selected at indices where condition is True

  • y (Tensor or Scalar) – value (if :attr:x is a scalar) or values selected at indices where condition is False

Returns

A tensor of shape equal to the broadcasted shape of condition, x, y

Return type

Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> x = flow.tensor(
...    np.array([[-0.4620, 0.3139], [0.3898, -0.7197], [0.0478, -0.1657]]),
...    dtype=flow.float32,
... )
>>> y = flow.tensor(np.ones(shape=(3, 2)), dtype=flow.float32)
>>> condition = flow.tensor(np.array([[0, 1], [1, 0], [1, 0]]), dtype=flow.int32)
>>> out = condition.where(x, y)
>>> out 
tensor([[1.0000, 0.3139],
        ...
        [0.0478, 1.0000]], dtype=oneflow.float32)
oneflow.zeros(*size: Union[int, Tuple[int, ], oneflow._oneflow_internal.Size, List[int]], dtype: Optional[oneflow._oneflow_internal.dtype] = None, device: Optional[Union[str, oneflow._oneflow_internal.device]] = None, placement: Optional[oneflow._oneflow_internal.placement] = None, sbp: Optional[oneflow._oneflow_internal.sbp.sbp] = None, requires_grad: bool = False)

Returns a tensor filled with the scalar value 0, with the shape defined by the variable argument size.

Parameters
  • size (an integer or tuple of integer values) – a variable number of arguments or a collection like a list or tuple.

  • dtype (flow.dtype, optional) – the desired data type of returned tensor.

  • device (flow.device, optional) – the desired device of returned tensor. Default: if None, uses the current device for the default tensor type

  • placement (flow.placement, optional) – the desired placement of returned consistent tensor. Default: if None, the returned tensor is local one using the argument device.

  • sbp (flow.sbp.sbp or tuple of flow.sbp.sbp, optional) – the desired sbp descriptor of returned consistent tensor. Default: if None, the returned tensor is local one using the argument device.

  • requires_grad (bool, optional) – If autograd should record operations on the returned tensor. Default: False.

For example:

>>> import oneflow as flow
>>> y = flow.zeros(5)
>>> y
tensor([0., 0., 0., 0., 0.], dtype=oneflow.float32)
>>> y = flow.zeros(2,3)
>>> y
tensor([[0., 0., 0.],
        [0., 0., 0.]], dtype=oneflow.float32)
oneflow.zeros_like(input) is equivalent to flow.zeros(input.shape, dtype=input.dtype)
Parameters

other (Tensor) – The size of input will determine size of the output tensor.

For example:

>>> import oneflow as flow
>>> import numpy as np
>>> x = flow.tensor(np.random.rand(5), dtype=flow.float32)
>>> y = flow.zeros_like(x)
>>> y
tensor([0., 0., 0., 0., 0.], dtype=oneflow.float32)
oneflow.relu(input, inplace=False)

Applies the rectified linear unit function element-wise. See ReLU for more details.

Parameters

inplace – If set to True, will do this operation in-place. Default: False

For examples:

>>> import oneflow as flow
>>> import numpy as np

>>> ndarr = np.asarray([1, -2, 3])
>>> input = flow.Tensor(ndarr)
>>> output = flow.relu(input)
>>> output
tensor([1., 0., 3.], dtype=oneflow.float32)
oneflow.env.get_rank()

Returns the rank of current process group.

Returns

The rank of the process group.