oneflow.nn.GELU¶
-
class
oneflow.nn.
GELU
¶ Gelu activation operator.
The equation is:
\[out = 0.5 * x * (1 + tanh(\sqrt{\frac{2}{\pi}} * (x + 0.044715x^{3})))\]- Parameters
x (oneflow.Tensor) – Input Tensor
- Returns
A Tensor.
- Return type
For example:
>>> import numpy as np >>> import oneflow as flow >>> x = np.array([-0.5, 0, 0.5]).astype(np.float32) >>> input = flow.Tensor(x) >>> gelu = flow.nn.GELU() >>> out = gelu(input) >>> out tensor([-0.1543, 0.0000, 0.3457], dtype=oneflow.float32)