NLP
语言模型
最大似然估计
(p(w_{i} | w_{i-1}) = frac{c(w_{i-1}w_{i})}{sum limits_{w_{i}} c(w_{i-1}w_{i})})
句子的概率计算公式
[egin{align}
s & = w_{1}w_{2}...w_{l}, （l 是句子中词的个数） \
p(s) & = p(w_{1}|<BOS>)p(w_{2}|w_{1})p(w_{3} | w_{2}w_{1})...p(w_{l}|w_{1}...w_{l-1}) \
& = prodlimits_{i=1}^{l}p(w_{i}|w_{1}...w_{i-1})
end{align}
]
困惑度
(PP_{T}(T) = 2^{H_{P}(T)})
交叉熵
(H_{p}(T) = -frac{1}{W_{T}}log_{2} p(T))