oneflow.nn.functional

Convolution functions

conv1d

Applies a 1D convolution over an input signal composed of several input planes.

conv2d

Applies a 2D convolution over an input image composed of several input planes.

conv3d

Applies a 3D convolution over an input image composed of several input planes.

conv_transpose1d

Applies a 1D transposed convolution operator over an input signal composed of several input planes, sometimes also called “deconvolution”.

conv_transpose2d

Applies a 2D transposed convolution operator over an input image composed of several input planes, sometimes also called “deconvolution”.

conv_transpose3d

Applies a 3D transposed convolution operator over an input image composed of several input planes, sometimes also called “deconvolution”.

fold

The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.nn.functional.fold.html.

unfold

The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.nn.functional.unfold.html.

BatchNorm functions

batch_norm

Applies Batch Normalization for each channel across a batch of data.

Pooling functions

avg_pool1d

Applies a 1D average pooling over an input signal composed of several input planes.

avg_pool2d

Applies 2D average-pooling operation in \(kH \times kW\) regions by step size \(sH \times sW\) steps.

avg_pool3d

Applies 3D average-pooling operation in \(kT \times kH \times kW\) regions by step size \(sT \times sH \times sW\) steps.

max_pool1d

Applies a 1D max pooling over an input signal composed of several input planes.

max_pool2d

Applies a 2D max pooling over an input signal composed of several input planes.

max_pool3d

Applies a 3D max pooling over an input signal composed of several input planes.

max_unpool1d

Computes a partial inverse of MaxPool1d.

max_unpool2d

Computes a partial inverse of MaxPool2d.

max_unpool3d

Computes a partial inverse of MaxPool3d.

adaptive_avg_pool1d

Applies a 1D adaptive average pooling over an input signal composed of several input planes.

adaptive_avg_pool2d

Applies a 2D adaptive average pooling over an input signal composed of several input planes.

adaptive_avg_pool3d

Applies a 3D adaptive average pooling over an input signal composed of several input planes.

adaptive_max_pool1d

Applies a 1D adaptive max pooling over an input signal composed of several input planes.

adaptive_max_pool2d

Applies a 2D adaptive max pooling over an input signal composed of several input planes.

adaptive_max_pool3d

Applies a 3D adaptive max pooling over an input signal composed of several input planes.

Non-linear activation functions

threshold

Thresholds each element of the input Tensor.

relu

Applies the rectified linear unit function element-wise.

hardtanh

Applies the HardTanh function element-wise.

hardswish

Applies the hardswish function, element-wise, as described in the paper:

relu6

Applies the element-wise function \(\text{ReLU6}(x) = \min(\max(0,x), 6)\).

elu

Applies element-wise,

selu

Applies element-wise function

celu

Applies the element-wise function:

leaky_relu

Applies element-wise, :math:` ext{LeakyReLU}(x) = max(0, x) + ext{negative_slope} * min(0, x)`

square_relu

Applies the relu^2 activation introduced in https://arxiv.org/abs/2109.08668v2

prelu

Applies the element-wise function:

glu

The equation is:

gelu

Applies the Gaussian Error Linear Units function:

quick_gelu

Applies GELU approximation that is fast but somewhat inaccurate.

logsigmoid

Applies the element-wise function:

hardshrink

Applies the hard shrinkage function in an element-wise manner.

softsign

The formula is:

softplus

Applies the element-wise function:

softmax

Applies a softmax function.

softshrink

Applies the soft shrinkage function in an element-wise manner.

log_softmax

LogSoftmax is defined as:

gumbel_softmax

Solve the problem that the output values of argmax do not reflect the probability distribution of the model’s output.

tanh

The equation is:

sigmoid

Applies the element-wise function \(\text{Sigmoid}(x) = \frac{1}{1 + \exp(-x)}\)

hardsigmoid

Applies the element-wise function

silu

The formula is:

mish

Applies the element-wise function:

layer_norm

Applies Layer Normalization for last certain number of dimensions.

normalize

Performs \(L_p\) normalization of inputs over specified dimension

Linear functions

linear

Applies a linear transformation to the incoming data: \(y = xA^T + b\).

Dropout functions

dropout

During training, randomly zeroes some of the elements of the input tensor with probability p using samples from a Bernoulli distribution.

dropout1d

The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.nn.functional.dropout1d.html.

dropout2d

dropout1d(x: Tensor, p: float = 0.5, training: bool = True) -> Tensor

dropout3d

dropout1d(x: Tensor, p: float = 0.5, training: bool = True) -> Tensor

Sparse functions

embedding

A simple lookup table that looks up embeddings in a fixed dictionary and size.

one_hot

This operator generates a onehot Tensor from input Tensor.

Distance functions

cosine_similarity

Returns cosine similarity between x1 and x2, computed along dim.

pairwise_distance

Computes the pairwise distance between vectors \(v_1\), \(v_2\) using the p-norm:

Loss functions

sparse_softmax_cross_entropy

The interface is consistent with TensorFlow.

cross_entropy

See CrossEntropyLoss for details.

ctc_loss

The Connectionist Temporal Classification loss.

l1_loss

This operator computes the L1 loss between each element in input and target.

mse_loss

This operator computes the mean squared error (squared L2 norm) loss between each element in input and target.

smooth_l1_loss

Function that uses a squared term if the absolute element-wise error falls below beta and an L1 term otherwise.

triplet_margin_loss

Creates a criterion that measures the triplet loss given an input tensors \(x1\), \(x2\), \(x3\) and a margin with a value greater than \(0\).

binary_cross_entropy

The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.nn.functional.binary_cross_entropy.html.

binary_cross_entropy_with_logits

The documentation is referenced from: https://pytorch.org/docs/1.10/generated/torch.nn.functional.binary_cross_entropy_with_logits.html.

Vision functions

deform_conv2d

Performs Deformable Convolution v2, described in Deformable ConvNets v2: More Deformable, Better Results if mask is not None and Performs Deformable Convolution, described in Deformable Convolutional Networks if mask is None.

pad

Pads tensor.

interpolate

The interface is consistent with PyTorch.

upsample

alias of oneflow.nn.modules.upsampling.Upsample

grid_sample

The interface is consistent with PyTorch.

affine_grid

The interface is consistent with PyTorch.

Greedy decoder

ctc_greedy_decoder

Performs greedy decoding on the logits given in input (best path).