oneflow.quick_gelu¶
-
oneflow.
quick_gelu
(x: Tensor) → Tensor¶ Applies GELU approximation that is fast but somewhat inaccurate. See: https://github.com/hendrycks/GELUs
\[\begin{split}\\text{QuickGELU}(x) = x * \\sigma(1.702x) = x * \\frac{1}{1 + \\exp(-1.702x)}\end{split}\]- Parameters
input (oneflow.Tensor) – Input Tensor
- Returns
A Tensor has same shape as the input.
- Return type
See
QuickGELU
for more details.