oneflow¶
The oneflow package contains data structures for multi-dimensional tensors and defines mathematical operations over these tensors. Additionally, it provides many utilities for efficient serializing of Tensors and arbitrary types, and other useful utilities.
It has a CUDA counterpart, that enables you to run your tensor computations on an NVIDIA GPU with compute capability >= 3.0
Tensor¶
Creates a Tensor with the dtype of oneflow.bool and the device on cpu, it has the same parameters as |
|
Creates a Tensor with the dtype of oneflow.uint8 and the device on cpu, it has the same parameters as |
|
Creates a Tensor with the dtype of oneflow.int8 and the device on cpu, it has the same parameters as |
|
Creates a Tensor with the dtype of oneflow.float64 and the device on cpu, it has the same parameters as |
|
Creates a Tensor with the dtype of oneflow.float32 and the device on cpu, it has the same parameters as |
|
Creates a Tensor with the dtype of oneflow.float16 and the device on cpu, it has the same parameters as |
|
Creates a Tensor with the dtype of oneflow.int32 and the device on cpu, it has the same parameters as |
|
Creates a Tensor with the dtype of oneflow.int64 and the device on cpu, it has the same parameters as |
Note that this function is simply doing |
|
Returns True if the data type of input is a floating point data type i.e., one of oneflow.float64 , oneflow.float32 , oneflow.float16, and oneflow.bfloat16. |
|
Returns True if the |
|
Returns the total number of elements in the |
|
Set options for printing. |
|
Returns the default floating point dtype. |
|
Sets the default floating point type for those source operators which create Tensor. |
|
Sets the default floating point type for those source operators which create Tensor. |
Creation Ops¶
Note
Random sampling creation ops are listed under Random sampling and
include:
oneflow.rand()
oneflow.randn()
oneflow.randint()
oneflow.randperm()
Constructs a tensor with data, return a global tensor if placement and sbp are in kwargs, |
|
Converts data into a tensor, sharing data and preserving autograd history if possible. |
|
Create a view of an existing oneflow.Tensor input with specified size, stride and storage_offset. |
|
Creates a |
|
Returns a tensor filled with the scalar value 0, with the shape defined by the variable argument size. |
|
The interface is consistent with PyTorch. |
|
Returns a tensor filled with the scalar value 1, with the shape defined by the variable argument size. |
|
The interface is consistent with PyTorch. |
|
Returns a tensor with the same size as input that is filled with random numbers from a normal distribution with mean 0 and variance 1. |
|
Returns a tensor filled with random integers generated uniformly between low (inclusive) and high (exclusive). |
|
Fills elements of |
|
The interface is consistent with PyTorch. |
|
Returns a 1-D tensor of size \(\left\lfloor \frac{\text{end} - \text{start}}{\text{step}} \right\rfloor + 1\) with values from |
|
Creates a one-dimensional tensor of size |
|
This operator creates a 2-D Tensor with ones on the diagonal and zeros elsewhere. |
|
The interface is consistent with PyTorch. |
|
The interface is consistent with PyTorch. |
|
Creates a tensor of size size filled with fill_value. |
|
Returns a tensor with the same size as |
|
This operation creates a new tensor by applying sparse updates to the input tensor. |
|
This function is equivalent to PyTorch’s logspace function. |
Indexing, Slicing, Joining, Mutating Ops¶
This operator finds the indices of input Tensor input elements that are non-zero. |
|
Returns a 1-dimensional view of each input tensor with zero dimensions. |
|
Returns a 2-dimensional view of each input tensor with zero dimensions. |
|
Returns a 3-dimensional view of each input tensor with zero dimensions. |
|
Concatenate two or more Tensor s at specified dim. |
|
Creates a new tensor by horizontally stacking the tensors in |
|
cat(tensors, dim=0) -> Tensor |
|
Splits a tensor into a specific number of chunks. |
|
Stack tensors in |
|
This operator expand the input tensor to a larger size. |
|
Gathers values along an axis specified by dim. |
|
This operator is a high-dimensional extension of gather, index is a K-dimensional tensor, which is regarded as a index of input Tensor input. |
|
Gather the element in batch dims. |
|
The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.hsplit.html. |
|
Stack tensors in |
|
Splits input, a tensor with two or more dimensions, into multiple tensors vertically according to indices_or_sections. |
|
Stack tensors in |
|
Select values along an axis specified by dim. |
|
See |
|
Returns a new 1-D tensor which indexes the input tensor according to the boolean mask mask which is a BoolTensor(In oneFlow BoolTensor is replaced by Int8Tensor). |
|
Moves the dimension(s) of input at the position(s) in source to the position(s) in destination. |
|
Returns a new tensor that is a narrowed version of input tensor. |
|
Returns a view of the original tensor with its dimensions permuted. |
|
This operator repeat the input tensor to a larger size along the specified dimensions. |
|
This operator reshapes a Tensor. |
|
Alias of |
|
Slices the self tensor along the selected dimension at the given index. |
|
This operator writes the elements specified by index along with the axis dim from the src into the input. |
|
This operator scatter the src with addition operation according to index along dim into the input. |
|
This operator inserts the elements in update according to the index and create a new Tensor. |
|
Extracts a slice from a tensor. |
|
Update a slice of tensor x. |
|
Splits the tensor into chunks. |
|
This operator removes the specified dimention which size is 1 of the input Tensor. |
|
Concatenates a sequence of tensors along a new dimension. |
|
This function is equivalent to NumPy’s swapaxes function. |
|
This function is equivalent to torch’s swapdims function. |
|
oneflow.t(input) → Tensor. |
|
Constructs a tensor by repeating the elements of |
|
Returns a tensor that is a transposed version of input. |
|
Removes a tensor dimension. |
|
Returns a new tensor with a dimension of size one inserted at the specified position. |
|
Return a tensor of elements selected from either |
|
Splits a tensor into multiple sub-tensors, all of which are views of input, along dimension dim according to the indices or number of sections specified by indices_or_sections . |
Random sampling¶
Sets the seed for generating random numbers to a non-deterministic random number. |
|
Sets the seed for generating random numbers. |
|
Returns the initial seed for generating random numbers as a Python long. |
|
Sets the random number generator state. |
|
Returns the random number generator state as a oneflow.ByteTensor. |
|
This operator returns a Tensor with binaray random numbers (0 / 1) from a Bernoulli distribution. |
|
Returns a tensor of random numbers drawn from separate normal distributions whose mean and standard deviation are given. |
|
Returns a tensor filled with random numbers from a uniform distribution on the interval [0, 1) |
|
Returns a tensor filled with random integers generated uniformly between low (inclusive) and high (exclusive). |
|
Returns a tensor filled with random numbers from a normal distribution with mean 0 and variance 1 (also called the standard normal distribution). |
|
Returns a random permutation of integers from |
|
Returns a tensor where each row contains |
In-place random sampling¶
There are a few more in-place random sampling functions defined on Tensors as well. Click through to refer to their documentation:
- oneflow.Tensor.normal_()
- in-place version of oneflow.normal()
- oneflow.Tensor.uniform_()
- numbers sampled from the continuous uniform distribution
Serialization¶
Parallelism¶
Sets the number of threads used for intraop parallelism on CPU. |
Locally disabling gradient computation¶
The context managers oneflow.no_grad()
, oneflow.enable_grad()
, and
oneflow.set_grad_enabled()
are helpful for locally disabling and enabling
gradient computation. These context managers are thread local, so they won’t
work if you send work to another thread using the threading
module, etc.
Examples:
>>> import oneflow
>>> x = oneflow.zeros(1, requires_grad=True)
>>> with oneflow.no_grad():
... y = x * 2
>>> y.requires_grad
False
>>> with oneflow.set_grad_enabled(False):
... y = x * 2
>>> y.requires_grad
False
>>> with oneflow.set_grad_enabled(True):
... y = x * 2
>>> y.requires_grad
True
Context-manager that disabled gradient calculation. |
|
Context-manager that enabled gradient calculation. |
|
Context-manager that enabled gradient calculation. |
|
Returns True if grad mode is currently enabled. |
|
Context-manager that enables or disables inference mode |
Math operations¶
Pointwise Ops¶
Return the absolute value of each element in input tensor:math:y = |x| element-wise. |
|
Returns a new tensor with the inverse cosine of the elements of |
|
Returns a new tensor with the inverse hyperbolic cosine of the elements of |
|
Returns a new tensor with the inverse cosine of the elements of |
|
Returns a new tensor with the inverse hyperbolic cosine of the elements of |
|
Adds other, scaled by alpha, to input. |
|
This function is equivalent to PyTorch’s addcdiv function. |
|
Performs the element-wise multiplication of tensor1 by tensor2, multiply the result by the scalar value and add it to input. |
|
Returns a new tensor with the arcsine of the elements of |
|
Returns a new tensor with the inverse hyperbolic sine of the elements of |
|
Returns a new tensor with the arcsine of the elements of |
|
Returns a new tensor with the inverse hyperbolic sine of the elements of |
|
Returns a new tensor with the arctangent of the elements of |
|
Returns a new tensor with the inverse hyperbolic tangent of the elements of |
|
Returns a new tensor with the arctangent of the elements of |
|
Returns a new tensor with the inverse hyperbolic tangent of the elements of |
|
Element-wise arctangent of input{i}/other{i} with consideration of the quadrant. |
|
Returns a new tensor with the ceil of the elements of |
|
In-place version of |
|
Clamp all elements in |
|
Clamp all elements in |
|
Clamp all elements in |
|
Alias for |
|
Returns a new tensor with the cosine of the elements of |
|
Returns a new tensor with the hyperbolic cosine of the elements of |
|
Computes the division of input by other for each element, scalar and broadcast promotation are supported. |
|
Computes the error function of each element. |
|
Computes the complementary error function of each element of input. |
|
Computes the inverse error function of |
|
This operator computes the exponential of Tensor. |
|
Returns a new tensor with the exponential of the elements minus 1 of |
|
Returns a new tensor with the arcsine of the elements of |
|
In-place version of |
|
frac(input) → Tensor |
|
In-place version of |
|
Computes the element-wise remainder of division. |
|
Applies the Gaussian Error Linear Units function: |
|
Applies GELU approximation that is fast but somewhat inaccurate. |
|
Applies the relu^2 activation introduced in https://arxiv.org/abs/2109.08668v2 |
|
Returns a new tensor with the natural logarithm of the elements of |
|
Returns a new tensor with the natural logarithm of (1 + input). |
|
Returns a new tensor with the natural logarithm to the base 2 of the elements of |
|
Returns a new tensor with the natural logarithm to the base 10 of the elements of |
|
Computes the element-wise logical AND of the given input tensors. |
|
Computes the element-wise logical NOT of the given input tensors. |
|
Computes the element-wise logical OR of the given input tensors. |
|
Computes the element-wise logical XOR of the given input tensors. |
|
Computes the bitwise AND of input and other. |
|
Computes the bitwise OR of input and other. |
|
Computes the bitwise XOR of input and other. |
|
Computes the bitwise NOT of input. |
|
Applies the element-wise function: |
|
Computes the multiplication of input by other for each element, scalar and broadcast promotation are supported. |
|
This operator computes the negative value of Tensor. |
|
This operator computes the negative value of Tensor. |
|
Takes the power of each element in input with exponent and returns a tensor with the result. |
|
Computes the safe reciprocal of x. |
|
This operator rounds the value of Blob to the nearest integer. |
|
In-place version of |
|
Returns a new tensor with the reciprocal of the square-root of each of the elements of |
|
Applies element-wise function |
|
Softmax is defined as: |
|
Applies the element-wise function: |
|
The formula is: |
|
The formula is: |
|
Applies the element-wise function \(\text{Sigmoid}(x) = \frac{1}{1 + \exp(-x)}\) |
|
Computes the sign of Tensor. |
|
Returns a new tensor with the sine of the elements of |
|
Returns a new tensor with the hyperbolic sine of the elements of |
|
In-place version of |
|
Returns a new tensor with the square-root of the elements of |
|
Returns a new tensor with the square of the elements of |
|
Computes the subtraction of input by other for each element, scalar and broadcast promotation are supported. |
|
Returns the tan value of the elements of |
|
The equation is: |
|
The interface is consistent with PyTorch. |
|
The documentation is referenced from: https://pytorch.org/docs/stable/generated/torch.lerp.html. |
|
In-place version of |
|
The documentation is referenced from: https://pytorch.org/docs/stable/generated/torch.quantile.html. |
Reduction Ops¶
The op computes the index with the largest value of a Tensor at specified axis. |
|
The op computes the index with the largest value of a Tensor at specified axis. |
|
Returns the maximum along a dimension. |
|
Returns the minimum value of each slice of the input tensor in the given dimension(s) dim. |
|
For each row of input in the given dimension dim, returns True if any element in the row evaluate to True and False otherwise. |
|
Computes the maximum value of all elements in the input tensor. |
|
Computes the minimum value of all elements in the input tensor. |
|
Computes the mean of row of elements in a tensor in the given dimension. |
|
Returns the median of the values in input. |
|
Returns a namedtuple (values, indices) where values is the mode value of each row of the input tensor in the given dimension dim, i.e. a value which appears most often in that row, and indices is the index location of each mode value found. |
|
Computes the product of row of elements in a tensor in the given dimension. |
|
Returns the sum of each row of the |
|
Returns the standard-deviation of each row of the |
|
Computes the sum of row of elements in a tensor in the given dimension. |
|
Returns the log of summed exponentials of each row of the |
|
Returns the variance of each row of the input tensor in the given dimension dim. |
|
Returns the matrix norm or vector norm of a given tensor. |
|
For each row of input in the given dimension dim, returns True if all element in the row evaluate to True and False otherwise. |
Comparison Ops¶
This operator sorts the input Tensor at specified dim and returns the indices of the sorted Tensor. |
|
Computes element-wise equality. |
|
True if two tensors have the same size and elements, False otherwise. |
|
Returns the truth value of \(input > other\) element-wise. |
|
This function is equivalent to PyTorch’s isinf function. |
|
This function is equivalent to PyTorch’s isnan function. |
|
Returns the truth value of \(input <= other\) element-wise. |
|
Returns the truth value of \(input < other\) element-wise. |
|
Computes element-wise not equality. |
|
Sorts the elements of the input tensor along a given dimension in ascending order by value. |
|
Finds the values and indices of the k largest entries at specified axis. |
|
Returns the truth value of \(input >= other\) element-wise. |
|
Returns the truth value of \(input > other\) element-wise. |
|
Returns the truth value of \(input >= other\) element-wise. |
|
Computes the element-wise maximum of x and y. |
|
Computes the element-wise minimum of x and y. |
|
ne(input, other) -> Tensor |
|
The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.isclose.html |
|
The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.allclose.html |
Spectral Ops¶
This function is equivalent to PyTorch’s hann_window function. |
Other Ops¶
Applies a 1D adaptive average pooling over an input signal composed of several input planes. |
|
Applies a 2D adaptive average pooling over an input signal composed of several input planes. |
|
Applies a 3D adaptive average pooling over an input signal composed of several input planes. |
|
This operator broadcast tensor x to like_tensor according to the broadcast_axes. |
|
The operation takes input tensor x and casts it to the output with dtype |
|
This operator computes the cumulative product of input elements in the given dimension. |
|
This operator computes the cumulative sum of input elements in the given dimension. |
|
If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal. |
|
Returns a partial view of input with the its diagonal elements with respect to dim1 and dim2 appended as a dimension at the end of the shape. |
|
Sums the product of the elements of the input |
|
Flattens a contiguous range of dims into a tensor. |
|
Reverse the order of a n-D tensor along given axis in dims. |
|
Says whether the targets are in the top K predictions. |
|
Take \(N\) tensors, each of which can be either scalar or 1-dimensional vector, and create \(N\) N-dimensional grids, where the \(i\) th grid is defined by expanding the \(i\) th input over dimensions defined by other inputs. |
|
Performs non-maximum suppression (NMS) on the boxes according to their intersection-over-union (IoU). |
|
Roll the tensor along the given dimension(s). |
|
Find the indices from the innermost dimension of sorted_sequence such that, if the corresponding values in values were inserted before the indices, the order of the corresponding innermost dimension within sorted_sequence would be preserved. |
|
Compute tensor dot along given dimensions. |
|
Returns the lower triangular part of a matrix (2-D tensor) or batch of matrices input along the specified diagonal, the other elements of the result tensor out are set to 0. |
|
Repeat elements of a tensor. |
|
Returns the upper triangular part of a matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0. |
|
Returns the cross product of vectors in dimension dim of input and other. |
|
oneflow.bincount(input, weights=None, minlength=0) → Tensor |
|
The interface is consistent with PyTorch. |
|
The interface is consistent with PyTorch. |
|
The interface is consistent with PyTorch. |
|
Returns the unique elements of the input tensor. |
BLAS and LAPACK Operations¶
Performs a matrix multiplication of the matrices |
|
Performs a batch matrix-matrix product of matrices stored in input and mat2. |
|
The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.baddbmm.html. |
|
This operator computes the dot product of tensor input and other. |
|
This operator applies matrix multiplication to two Tensor. |
|
Performs a matrix multiplication of the matrices |
|
Performs a matrix-vector product of the matrix |