oneflow.nn.functional.gelu

oneflow.nn.functional.gelu(x: Tensor)Tensor

The equation is:

\[out = 0.5 * x * (1 + tanh(\sqrt{\frac{2}{\pi}} * (x + 0.044715x^{3})))\]

For example:

>>> import numpy as np
>>> import oneflow as flow

>>> x = np.array([-0.5, 0, 0.5]).astype(np.float32)
>>> input = flow.tensor(x)

>>> out = flow.gelu(input)
>>> out
tensor([-0.1543,  0.0000,  0.3457], dtype=oneflow.float32)

See GELU for more details.