Lisht

LiSHT: Non-Parameteric Linearly Scaled Hyperbolic Tangent Activation Function.

activation_lisht(x)

Arguments

x	A `Tensor`. Must be one of the following types: `float16`, `float32`, `float64`.

Value

A `Tensor`. Has the same type as `x`.

Details

Computes linearly scaled hyperbolic tangent (LiSHT): `x * tanh(x)` See [LiSHT: Non-Parameteric Linearly Scaled Hyperbolic Tangent Activation Function for Neural Networks](https://arxiv.org/abs/1901.05894).

Examples

if (FALSE) {
library(keras)
library(tfaddons)
model = keras_model_sequential() %>%
layer_conv_2d(filters = 10, kernel_size = c(3,3),input_shape = c(28,28,1),
              activation = activation_lisht)
}