Activation Functions and Their Derivatives 1. Sigmoid Function: when z=0,g'(z)=0.25 2. tanh Function: when x=0,tanh'(x)=1 3. Relu