oneflow.nn.Hardshrink

class oneflow.nn.Hardshrink(lambd: float = 0.5, inplace: bool = False)

The Hardshrink activation.

The formula is:

\[\begin{split}\text{Hardshrink}(x) = \begin{cases} x, & \text{ if } x > \lambda \\ x, & \text{ if } x < -\lambda \\ 0, & \text{ otherwise } \end{cases}\end{split}\]
Parameters
  • lambd – the \(\lambda\) value for the Hardshrink formulation. Default: 0.5

  • inplace – can optionally do the operation in-place. Default: False

Shape:
  • Input: \((N, *)\) where * means, any number of additional dimensions

  • Output: \((N, *)\), same shape as the input

For example:

>>> import numpy as np
>>> import oneflow as flow
>>> x = np.array([-1.1, 0, 0.2, 0.5]).astype(np.float32)
>>> input = flow.Tensor(x)
>>> hardshrink = flow.nn.Hardshrink(lambd=0.5)
>>> out = hardshrink(input)
>>> out
tensor([-1.1000,  0.0000,  0.0000,  0.0000], dtype=oneflow.float32)