Swish function
In October 2017, Prajit Ramachandran, Barret Zoph and Quoc V. Le from Google Brain proposed the Swish activation function. It’s increase in the networks like Inception-ResNet-v2 by 0.6% and Mobile NASNet-A by 0.9%.
Exey Panteleev | CC BY 2.0