单向LSTM笔记, LSTM做minist数据集分类

单向LSTM笔记, LSTM做minist数据集分类

先介绍下torch.nn.LSTM()这个API

　　1.input_size: 每一个时步(time_step)输入到lstm单元的维度.(实际输入的数据size为[batch_size, input_size])

       2. hidden_size: 确定了隐含状态hidden_state的维度. 可以简单的看成: 构造了一个权重, 隐含状态 

          

　　3 . num_layers: 叠加的层数。如图所示num_layers为 3

　　4. batch_first: 输入数据的size为[batch_size, time_step, input_size]还是[time_step, batch_size, input_size]

使用单向LSTM对MNIST进行分类,我是在pytorch0.4.1坂本上运行的。

########################## pytorch 用LSTM做minist数据分类 ##################
##########################################################################
import torch
import torch.utils.data as Data
import torchvision
import matplotlib.pyplot as plt
import numpy as np

BATCH_SIZE = 50


class RNN(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.rnn = torch.nn.LSTM(
            input_size=28,
            hidden_size=64,
            num_layers=1,
            batch_first=True
        )
        self.out = torch.nn.Linear(in_features=64, out_features=10)

    def forward(self, x):
        # 一下关于shape的注释只针对单向
        # output: [batch_size, time_step, hidden_size]
        # h_n: [num_layers,batch_size, hidden_size] # 虽然LSTM的batch_first为True,但是h_n/c_n的第一维还是num_layers
        # c_n: 同h_n
        output, (h_n, c_n) = self.rnn(x)
        #print(output.size())
        # output_in_last_timestep=output[:,-1,:] # 也是可以的
        output_in_last_timestep = h_n[-1, :, :]
        # print(output_in_last_timestep.equal(output[:,-1,:])) # ture
        x = self.out(output_in_last_timestep)
        return x


if __name__ == "__main__":
    # 1. 加载数据
    training_dataset = torchvision.datasets.MNIST("./mnist", train=True,
                                                  transform=torchvision.transforms.ToTensor(), download=True)
    dataloader = Data.DataLoader(dataset=training_dataset,
                                 batch_size=BATCH_SIZE, shuffle=True, num_workers=2)
    # showSample(dataloader)
    test_data = torchvision.datasets.MNIST(root="./mnist", train=False,
                                           transform=torchvision.transforms.ToTensor(), download=False)
    test_dataloader = Data.DataLoader(
        dataset=test_data, batch_size=1000, shuffle=False, num_workers=2)
    testdata_iter = iter(test_dataloader)
    test_x, test_y = testdata_iter.next()
    test_x = test_x.view(-1, 28, 28)
    # 2. 网络搭建
    net = RNN()
    # 3. 训练
    # 3. 网络的训练（和之前CNN训练的代码基本一样）
    optimizer = torch.optim.Adam(net.parameters(), lr=0.001)
    loss_F = torch.nn.CrossEntropyLoss()
    for epoch in range(3):  # 数据集只迭代一次
        for step, input_data in enumerate(dataloader):
            x, y = input_data
            pred = net(x.view(-1, 28, 28))
            loss = loss_F(pred,y)  # 计算loss
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            if step % 50 == 49:  # 每50步，计算精度
                with torch.no_grad():
                    test_pred = net(test_x)
                    prob = torch.nn.functional.softmax(test_pred, dim=1)
                    pred_cls = torch.argmax(prob, dim=1)
                    acc = (pred_cls == test_y).sum().numpy() / pred_cls.size()[0]
                    print(f"{epoch}-{step}: accuracy:{acc}")

由上面代码可以看到输出为：output,(h_n,c_n)=self.rnn(x)，解释下代码中的第28行。

• output: 如果num_layer为3，则output只记录最后一层 --------- 第三层的输出。

  • 对应图中向上的h_t
  • 其size根据batch_first而不同。可能是[batch_size, time_step, hidden_size]或[time_step, batch_size, hidden_size]

• h_n: 各个层的最后一个时步的隐含状态h.

  • size为[num_layers,batch_size, hidden_size]
  • 对应图中向右的h_t. 可以看出对于单层单向的LSTM， 其h_n最后一层输出h_n[-1,:,:]，和output最后一个时步的输出output[:,-1,:]相等。在示例代码中print(h_n[-1,:,:].equal(output[:,-1,:]))会打印True

• c_n: 各个层的最后一个时步的隐含状态C

  • c_n可以看成另一个隐含状态，size和h_n相同

我运行了3个epoch效果如下：

0-49: accuracy:0.3
0-99: accuracy:0.596
0-149: accuracy:0.697
0-199: accuracy:0.734
0-249: accuracy:0.769
0-299: accuracy:0.782
0-349: accuracy:0.751
0-399: accuracy:0.843
0-449: accuracy:0.859
0-499: accuracy:0.87
0-549: accuracy:0.857
0-599: accuracy:0.89
0-649: accuracy:0.88
0-699: accuracy:0.883
0-749: accuracy:0.905
0-799: accuracy:0.905
0-849: accuracy:0.902
0-899: accuracy:0.901
0-949: accuracy:0.908
0-999: accuracy:0.921
0-1049: accuracy:0.917
0-1099: accuracy:0.906
0-1149: accuracy:0.941
0-1199: accuracy:0.935
1-49: accuracy:0.935
1-99: accuracy:0.936
1-149: accuracy:0.941
1-199: accuracy:0.923
1-249: accuracy:0.94
1-299: accuracy:0.936
1-349: accuracy:0.941
1-399: accuracy:0.948
1-449: accuracy:0.937
1-499: accuracy:0.939
1-549: accuracy:0.949
1-599: accuracy:0.949
1-649: accuracy:0.953
1-699: accuracy:0.947
1-749: accuracy:0.918
1-799: accuracy:0.944
1-849: accuracy:0.957
1-899: accuracy:0.959
1-949: accuracy:0.947
1-999: accuracy:0.944
1-1049: accuracy:0.961
1-1099: accuracy:0.964
1-1149: accuracy:0.961
1-1199: accuracy:0.952
2-49: accuracy:0.95
2-99: accuracy:0.952
2-149: accuracy:0.957
2-199: accuracy:0.945
2-249: accuracy:0.957
2-299: accuracy:0.953
2-349: accuracy:0.956
2-399: accuracy:0.942
2-449: accuracy:0.946
2-499: accuracy:0.962
2-549: accuracy:0.956
2-599: accuracy:0.957
2-649: accuracy:0.953
2-699: accuracy:0.958
2-749: accuracy:0.963
2-799: accuracy:0.959
2-849: accuracy:0.954
2-899: accuracy:0.961
2-949: accuracy:0.959
2-999: accuracy:0.961
2-1049: accuracy:0.962
2-1099: accuracy:0.958
2-1149: accuracy:0.955
2-1199: accuracy:0.964

主要参考：https://www.jianshu.com/p/043083d114d4