keras使用word2vec pretrained vector注意事项

在使用预训练的embedding层的时候,一定要注意词表的index,在word2vec中,

model.wv.index2word 这个是一个list, index就是词的index,这个是固定的,即便是换到linux平台,这个index也是不变的,所以使用这个。

w2v_for_s2s = Word2Vec.load('model/word2vec_6_3_word.bin')
word2idx = {"UNK": 0}
# vocab_list = [(k, w2v_for_s2s.wv[k]) for k, v in w2v_for_s2s.wv.vocab.items()]
index2word = w2v_for_s2s.wv.index2word
embeddings_matrix = np.zeros((len(index2word) + 1, w2v_for_s2s.vector_size))
print(index2word[:50])
for i in range(len(index2word)):
word = index2word[i]
word2idx[word] = i + 1
embeddings_matrix[i + 1] = w2v_for_s2s.wv[word]
原文地址:https://www.cnblogs.com/yjybupt/p/10997070.html