Deep Learning5: Linear Decoders with Autoencoders

对于

$egin{align} z^{(3)} &= W^{(2)} a^{(2)} + b^{(2)} \ a^{(3)} &= f(z^{(3)}) end{align}$

Linear Decoders设定，a⁽³⁾ = z⁽³⁾则称之为线性编码

sigmoid激活函数要求输入范围在[0,1]之间，某些数据集很难满足，则采用线性编码

此时，误差项更新为

$egin{align} delta_i^{(3)} = frac{partial}{partial z_i} ;; frac{1}{2} left|y - hat{x} ight|^2 = - (y_i - hat{x}_i) cdot f'(z_i^{(3)}) end{align}$

$egin{align} delta_i^{(3)} = - (y_i - hat{x}_i) end{align}$

$egin{align} delta^{(2)} &= left( (W^{(2)})^Tdelta^{(3)} ight) ullet f'(z^{(2)}) end{align}$