CountVectorizer: transform method returns multidimensional array on a single text line
<p>Firstly, I fit it on the corpus of sms:</p>

<pre><code>from sklearn.feature_extraction.text import CountVectorizer
clf = CountVectorizer()
X_desc = clf.fit_transform(X).toarray()
</code></pre>

<p>Seems to works fine:</p>

<pre><code>X.shape = (5574,)
X_desc.shape = (5574, 8713)
</code></pre>

<p>But then I applied transform method to the textline, as we know, it should have (, 8713) shape as a result, but what we see:</p>

<pre><code>str2 = 'Have you visited the last lecture on physics?'
print len(str2), clf.transform(str2).toarray().shape
</code></pre>

<blockquote>
  <p>52          (52, 8713)</p>
</blockquote>

<p>What is going on here? One more thing - all numbers are zeros</p>

<p>You always need to pass an array or vector to <code>transform</code>; if you just want to transform a single element, you need to pass a singleton array, and then extract its contents:</p>

<pre><code>clf.transform([str1])[0]
</code></pre>

<p>Incidentally the reason that you are getting a 2-dimensional array as output is that the a string is actually stored as a list of characters, and so the vectoriser is treating your string as an array, where each character is being considered as a single document.</p>

python python-2.7 text scikit-learn sklearn-pandas