oneflow.nn.GELU

class oneflow.nn.GELU

Gelu activation operator.

The equation is:

\[out = 0.5 * x * (1 + tanh(\sqrt{\frac{2}{\pi}} * (x + 0.044715x^{3})))\]
Parameters

x (oneflow.Tensor) – Input Tensor

Returns

A Tensor.

Return type

oneflow.Tensor

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([-0.5, 0, 0.5]).astype(np.float32)
>>> input = flow.Tensor(x)
>>> gelu = flow.nn.GELU()

>>> out = gelu(input)
>>> out
tensor([-0.1543,  0.0000,  0.3457], dtype=oneflow.float32)