oneflow.Tensor¶
A oneflow.Tensor
is a multi-dimensional matrix containing elements of
a single data type.
Data types¶
OneFlow defines 8 Tensor types with CPU and GPU variants which are as follows:
Data type |
dtype |
CPU tensor |
GPU tensor |
---|---|---|---|
Boolean |
|
||
8-bit integer (unsigned) |
|
||
8-bit integer (signed) |
|
||
64-bit floating point |
|
||
32-bit floating point |
|
||
16-bit floating point |
|
||
32-bit integer (signed) |
|
||
64-bit integer (signed) |
|
Initializing and basic operations¶
A tensor can be constructed from a Python list
or sequence using the
oneflow.tensor()
constructor:
>>> import oneflow
>>> import numpy as np
>>> oneflow.tensor([[1., -1.], [1., -1.]])
tensor([[ 1., -1.],
[ 1., -1.]], dtype=oneflow.float32)
>>> oneflow.tensor(np.array([[1, 2, 3], [4, 5, 6]]))
tensor([[ 1, 2, 3],
[ 4, 5, 6]], dtype=oneflow.int64)
Warning
oneflow.tensor()
always copies data
. If you have a Tensor
data
and just want to change its requires_grad
flag, use
requires_grad_()
or
detach()
to avoid a copy.
If you have a numpy array and want to avoid a copy, use
oneflow.as_tensor()
.
>>> import oneflow
>>> oneflow.zeros([2, 4], dtype=oneflow.int32)
tensor([[ 0, 0, 0, 0],
[ 0, 0, 0, 0]], dtype=oneflow.int32)
>>> cuda0 = oneflow.device('cuda:0')
>>> oneflow.ones([2, 4], dtype=oneflow.float64, device=cuda0)
tensor([[ 1., 1., 1., 1.],
[ 1., 1., 1., 1.]], device='cuda:0', dtype=oneflow.float64)
For more information about building tensors, see Creation Ops
The contents of a tensor can be accessed and modified using Python’s indexing and slicing notation:
>>> import oneflow
>>> x = oneflow.tensor([[1, 2, 3], [4, 5, 6]])
>>> print(x[1][2])
tensor(6, dtype=oneflow.int64)
>>> x[0][1] = 8
>>> print(x)
tensor([[1, 8, 3],
[4, 5, 6]], dtype=oneflow.int64)
Use oneflow.Tensor.item()
to get a Python number from a tensor containing a
single value:
>>> import oneflow
>>> x = oneflow.tensor([[1]])
>>> x
tensor([[1]], dtype=oneflow.int64)
>>> x.item()
1
>>> x = oneflow.tensor(2.5)
>>> x
tensor(2.5000, dtype=oneflow.float32)
>>> x.item()
2.5
For more information about indexing, see Indexing, Slicing, Joining, Mutating Ops
A tensor can be created with requires_grad=True
so that
oneflow.autograd
records operations on them for automatic differentiation.
>>> import oneflow
>>> x = oneflow.tensor([[1., -1.], [1., 1.]], requires_grad=True)
>>> out = x.pow(2).sum()
>>> out.backward()
>>> x.grad
tensor([[ 2., -2.],
[ 2., 2.]], dtype=oneflow.float32)
Note
For more information on the oneflow.dtype
, oneflow.device
, and
oneflow.layout
attributes of a oneflow.Tensor
, see
Tensor Attributes.
Note
Methods which mutate a tensor are marked with an underscore suffix.
For example, oneflow.FloatTensor.add_()
computes the absolute value
in-place and returns the modified tensor, while oneflow.FloatTensor.add()
computes the result in a new tensor.
Note
To change an existing tensor’s oneflow.device
and/or oneflow.dtype
, consider using
to()
method of Tensor object.
Warning
Current implementation of oneflow.Tensor
introduces memory overhead,
thus it might lead to unexpectedly high memory usage in the applications with many tiny tensors.
If this is your case, consider using one large structure.
Tensor class reference¶
-
class
oneflow.
Tensor
¶ There are a few main ways to create a tensor, depending on your use case.
To create a tensor with pre-existing data, use
oneflow.tensor()
.To create a tensor with specific size, use
oneflow.*
tensor creation ops (see Creation Ops).To create a tensor with the same size (and similar types) as another tensor, use
oneflow.*_like
tensor creation ops (see Creation Ops).
Returns a Tensor of size |
|
Returns a Tensor of size size filled with 0. |
|
Returns a Tensor of size size filled with fill_value. |
|
Is True if the Tensor is stored on the GPU, False otherwise. |
|
Return whether this Tensor is a global tensor. |
|
Is the |
|
Return the gradient calculated by autograd functions. |
|
See |
|
See |
|
See |
|
See |
|
In-place version of |
|
In-place version of |
|
In-place version of |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
Computes the gradient of current tensor w.r.t. graph leaves. |
|
See |
|
|
|
self.byte() is equivalent to self.to(oneflow.uint8). |
|
See |
|
See |
|
See |
|
See |
|
See |
|
Inplace version of |
|
Alias for |
|
Alias for |
|
See |
|
Copies the elements from src into self tensor and returns self. |
|
See |
|
See |
|
Returns a copy of this object in CPU memory. |
|
Returns a copy of this object in CUDA memory. |
|
See |
|
See |
|
Is the |
|
Is the |
|
See |
|
Tensor.dim() → int |
|
See |
|
In-place version of |
|
|
|
Tensor.element_size() → int |
|
See |
|
See |
|
See |
|
See |
|
See |
|
Inplace version of |
|
See |
|
See |
|
See |
|
Expand this tensor to the same size as |
|
See |
|
Tensor.fill_(value) → Tensor |
|
See |
|
|
|
See |
|
See |
|
See |
|
See |
|
See |
|
For CUDA tensors, this function returns the device ordinal of the GPU on which the tensor resides. |
|
Return the function that created this tensor if it’s |
|
See |
|
In-place version of |
|
self.half() is equivalent to self.to(dtype=oneflow.float16). |
|
The interface is consistent with PyTorch. |
|
|
|
Returns True if self tensor is contiguous in memory. |
|
Return whether this Tensor is a lazy tensor. |
|
All Tensors that have |
|
See |
|
See |
|
Returns the value of this tensor as a standard Python number. |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
|
|
See |
|
In-place version of |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
In-place version of |
|
See |
|
In-place version of |
|
See |
|
See |
|
See |
|
See |
|
Tensor.nelement() → int |
|
See |
|
Fills |
|
See |
|
Tensor.numpy() → numpy.ndarray |
|
Transfer tensor data from GPU memory back to host (CPU) memory. |
|
Load tensor data stored on the host (CPU) back to GPU memory. |
|
Determine whether the tensor has been moved to CPU memory and the CUDA device memory has been released. |
|
See |
|
See |
|
Registers a backward hook. |
|
See |
|
See |
|
Is |
|
Sets this tensor’s requires_grad attribute in-place. |
|
Returns this tensor as the same shape as other. |
|
Enables this Tensor to have their |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
See |
|
Returns the size of the self tensor. |
|
See |
|
See |
|
See |
|
See |
|
In-place version of |
|
See |
|
Returns self tensor’s offset in the underlying storage in terms of number of storage elements (not bytes). |
|
See |
|
See |
|
In-place version of |
|
See |
|
See |
|
See |
|
Performs Tensor dtype and/or device conversion. |
|
Creates a global tensor from a local tensor. |
|
Performs Tensor placement and/or sbp conversion. |
|
Creates a global tensor if this tensor is a local tensor, otherwise performs Tensor placement and/or sbp conversion. |
|
Returns the local component of this global tensor in the current rank. |
|
This interface is no longer available, please use |
|
Returns the tensor as a (nested) list. |
|
See |
|
See |
|
See |
|
See |
|
Returns this tensor cast to the type of the given tensor. |
|
Returns the type if dtype is not provided, else casts this object to the specified type. |
|
See |
|
Is this Tensor with its dimensions reversed. |
|
See |
|
Returns a view of the original tensor which contains all slices of size size from self tensor in the dimension dimension. |
|
Tensor.uniform_(from=0, to=1) → Tensor |
|
In-place version of |
|
In-place version of |
|
See |
|
Returns a new tensor with the same data as the |
|
Expand this tensor to the same size as |
|
See |
|
Fills self tensor with zeros. |
|
See |
|
Copies the tensor to pinned memory, if it’s not already pinned. |
|
Returns true if this tensor resides in pinned memory. |
|
See |
|
Inplace version of |
|
Inplace version of |
|
The inplace version of |
|
See |
|