【464】文本转字符向量bag of words

利用 sklearn.feature_extraction.text 中的 CountVectorizer 来实现

• 首先获取所有的文本信息
• 然后将文本信息转化为从 0 开始的数字
• 获取转换后的字符向量

参见如下代码：

>>> text_01 = "My name is Alex Lee."
>>> text_02 = "I like singing and playing basketball."
>>> text_03 = "I also like swimming during leisure time."
>>> texts = [text_01, text_02, text_03]

>>> texts
['My name is Alex Lee.', 'I like singing and playing basketball.', 'I also like swimming during leisure time.']

>>> import sklearn
>>> from sklearn.feature_extraction.text import CountVectorizer
>>> vect = CountVectorizer().fit(texts)
>>> x = vect.transform(texts)

>>> x
<3x15 sparse matrix of type '<class 'numpy.int64'>'
	with 16 stored elements in Compressed Sparse Row format>

>>> vect.get_feature_names()
['alex', 'also', 'and', 'basketball', 'during', 'is', 'lee', 'leisure', 'like', 'my', 'name', 'playing', 'singing', 'swimming', 'time']
>>> vect.vocabulary_
{'my': 9, 'name': 10, 'is': 5, 'alex': 0, 'lee': 6, 'like': 8, 'singing': 12, 'and': 2, 'playing': 11, 'basketball': 3, 'also': 1, 'swimming': 13, 'during': 4, 'leisure': 7, 'time': 14}

>>> x
<3x15 sparse matrix of type '<class 'numpy.int64'>'
	with 16 stored elements in Compressed Sparse Row format>

>>> print(x)
  (0, 0)	1
  (0, 5)	1
  (0, 6)	1
  (0, 9)	1
  (0, 10)	1
  (1, 2)	1
  (1, 3)	1
  (1, 8)	1
  (1, 11)	1
  (1, 12)	1
  (2, 1)	1
  (2, 4)	1
  (2, 7)	1
  (2, 8)	1
  (2, 13)	1
  (2, 14)	1

>>> x.toarray()
array([[1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0],
       [0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0],
       [0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1]], dtype=int64)