Clustering text documents and obtaining duplicate top terms
<p>I found the code in <a href="http://stackoverflow.com/questions/27889873/clustering-text-documents-using-scikit-learn-kmeans-in-python?rq=1">this post</a> to be very helpful. (I would add a comment to that post but I need 50 reputation points.)</p>

<p>I used the same code in the post above but added a test document I have been using for debugging my own clustering code. For some reason a word in 1 document appears in both clusters. </p>

<p>The code is:</p>

<p>Update: I added "Unique sentence" to the documents below. </p>

<pre><code>documents = ["I ran yesterday.", 
                "The sun was hot.", 
                "I ran yesterday in the hot and humid sun.", 
                "Yesterday the sun was hot.",
                "Yesterday I ran in the hot sun.",
                "Unique sentence." ]

vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(documents)

#cluster documents    
true_k = 2
model = KMeans(n_clusters=true_k, init='k-means++', max_iter=100, n_init=1)
model.fit(X)

#print top terms per cluster clusters    
print("Top terms per cluster:")
order_centroids = model.cluster_centers_.argsort()[:, ::-1]
terms = vectorizer.get_feature_names()
for i in range(true_k):
    print ("Cluster %d:" % i,)
    for ind in order_centroids[i, :10]:
        print(' %s' % terms[ind])
    print
</code></pre>

<p>The output I receive is:</p>

<p>UPDATE: I updated the output below to reflect the "unique sentence" above. </p>

<p>Cluster 0:
 sun
 hot
 yesterday
 ran
 humid
 unique
 sentence
Cluster 1:
 unique
 sentence
 yesterday
 sun
 ran
 humid
 hot</p>

<p>You'll note that "humid" appears as a top term in both clusters even though it's in just 1 line of the documents above. I would expect a unique word, like "humid" in this case, to be a top term in just 1 of the clusters. </p>

<p>Thanks! </p>

<p>Not necessarily.  The code you are using creates vector space of the bag of words (excluding stop words) of your corpus (I am ignoring the tf-idf weighting.).  Looking at your documents, your vector space is of size 5, with the a word array like (ignoring the order):</p>

<pre><code>word_vec_space = [yesterday, ran, sun, hot, humid]
</code></pre>

<p>Each document is assigned a numeric vector of whether it contains the words in 'word_vec_space'. </p>

<pre><code>"I ran yesterday." -&gt; [1,1,0,0,0]
"The sun was hot." -&gt; [0,0,1,1,0]
...
</code></pre>

<p>When performing k-mean clustering, you pick k starting points in the vector-space and allow the points to move around to optimize the clusters.  You ended up with both cluster centroids containing the a non-zero value for <code>'humid'</code>.  This is due to the one sentence that contains <code>'humid'</code> also had <code>'sun'</code>, <code>'hot'</code>, and <code>'yesterday'</code>.  </p>

python scikit-learn cluster-analysis k-means