性质
- 判别模型
- 分类模型
模型
[P(Y=1|x) = frac{e^{w cdot x}}{1+e^{w cdot x}}\
P(Y=0|x)=frac{1}{1+e^{w cdot x}}
]
损失函数
最大似然估计
[egin{aligned}
L(w) & = prod_{i=1}^N P(y_i=1|x_i)^{y_i}P(y_i=0|x_i)^{(1-y_i)}\
& = prod_{i=1}^N P(y_i=1|x_i)^{y_i}[1-P(y_i=1|x_i)]^{(1-y_i)}\
end{aligned}
]
取负对数
[egin{aligned}
J(w) & = -lnL(w)\
& = - sum_{i=1}^N [y_ilogP(y_i=1|x_i)+(1-y_i)log(1-P(y_i=1|x_i))]\
& = - sum_{i=1}^N [y_ilogfrac{P(y_i=1|x_i)}{1-P(y_i=1|x_i)}+log(1-P(y_i=1|x_i))]\
& = - sum_{i=1}^N [y_i(wx_i)-log(1+e^{wx_i})]
end{aligned}
]
训练算法
梯度下降