oneflow.nn.functional.quick_gelu

oneflow.nn.functional.quick_gelu(x: Tensor)Tensor

Applies GELU approximation that is fast but somewhat inaccurate. See: https://github.com/hendrycks/GELUs

\[\begin{split}\\text{QuickGELU}(x) = x * \\sigma(1.702x) = x * \\frac{1}{1 + \\exp(-1.702x)}\end{split}\]
Parameters

input (oneflow.Tensor) – Input Tensor

Returns

A Tensor has same shape as the input.

Return type

oneflow.Tensor

See QuickGELU for more details.