torch.nn.functional.gelu

torch.nn.functional.gelu(input) → Tensor

Applies element-wise the function $\text{GELU}(x) = x * \Phi(x)$

where $\Phi(x)$ is the Cumulative Distribution Function for Gaussian Distribution.

See Gaussian Error Linear Units (GELUs).