条件熵
[egin{equation}
H(Y | X) quad=sum_{x in X} p(x) H(Y | X=x)
end{equation}
]
信息增益：熵 - 条件熵
[egin{aligned} i_{t} &=sigmaleft(W_{i i} x_{t}+b_{i i}+W_{h i} h_{(t-1)}+b_{h i}
ight) \ f_{t} &=sigmaleft(W_{i f} x_{t}+b_{i f}+W_{h f} h_{(t-1)}+b_{h f}
ight) \ g_{t} &= anh left(W_{i g} x_{t}+b_{i g}+W_{h g} h_{(t-1)}+b_{h g}
ight) \ o_{t} &=sigmaleft(W_{i o} x_{t}+b_{i o}+W_{h o} h_{(t-1)}+b_{h o}
ight) \ c_{t} &=f_{t} * c_{(t-1)}+i_{t} * g_{t} \ h_{t} &=o_{t} * anh left(c_{t}
ight) end{aligned}
]
到底是拼接起来，还是两个向量分别由一个权重矩阵处理后相加起来？