oneflow.nn.SquareReLU

class oneflow.nn.SquareReLU → Tensor

Applies the relu^2 activation introduced in https://arxiv.org/abs/2109.08668v2

\[:math:`\text{SquareReLU}(x) = \max(0, x) * \max(0, x)`\]

Parameters

input (oneflow.Tensor) – Input Tensor

Returns

A Tensor has same shape as the input.

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([-0.5, 0, 0.5]).astype(np.float32)
>>> input = flow.Tensor(x)
>>> square_relu = flow.nn.SquareReLU()

>>> out = square_relu(input)
>>> out
tensor([0.0000, 0.0000, 0.2500], dtype=oneflow.float32)