Weighted clustering of tags
<p>I have a list of products and each product is tagged and weights associated to it each tag. Now I want to cluster them into similar products. How do I go forward it. I have tried k-means of scikit-learn. But That is not helping much.</p>

<pre><code>Product 1: a=2.5 b=3.5 c=1 d=1
Product 2: a=0.25 c=2
Product 3: e=2 k=5
.
.
.
.
.
.
.
.
Product n: a=3 b=0.75
</code></pre>

<p>Now I want these to be clustered. I also want a product to be in many clusters if necessary. Because 1, 2, 3 can form a cluster and 2, 4, 5 can form other</p>

<p>If the direct and naÃ¯ve application of k-means is <em>not helping much</em>, you may need to dig a bit deeper.</p>

<p>Assuming you have <code>N</code> distinct tags of which <code>0..N</code> can be applied to each product <code>p</code>. Each assignment describes a weighted relationship with a positive weight <code>w</code>. Absence of a tag for a product equals <code>w = 0</code>.</p>

<p>This is your setup that yields an <code>N</code>-dimensional feature space for your products. You should be able to use arbitrary clustering methods; you <em>just</em> have to select the correct measures.</p>

<p><strong>Your distance (or similarity) measure should depend on your data.</strong></p>

<p>Consequently, the first thing to ask yourself is: When are two measures considered <em>similar</em>?</p>

<ul>
<li>If they have as many overlapping tags as possible?</li>
<li>If the sum of differences between non-overlapping tag weights is max?</li>
<li>If the sum of differences between overlapping tags is min?</li>
<li>...</li>
</ul>

<p>Depending on your defined <em>similarity</em>, you should be able to choose or implement a measure that yields the grade of similarity (not just the euclidean distance in <code>N</code> dimensions) when comparing two elements.</p>

<p>Also, you may want to check <a href="http://stats.stackexchange.com/questions/3713/choosing-clustering-method">this post at CrossValidated</a> or (if you want to learn more about clustering) <a href="http://infolab.stanford.edu/~ullman/mmds/ch7.pdf" rel="nofollow">Section 7.3</a> of <em>"Mining of Massive Datasets"</em> (2014, Anand Rajaraman, Jure Leskovec, and Jeffrey D. Ullman) [<a href="http://infolab.stanford.edu/~ullman/mmds/book.pdf" rel="nofollow">Entire book</a>]</p>

python python-3.x machine-learning cluster-analysis data-mining