Weighted clustering of tags
<p>I have a list of products and each product is tagged and weights associated to it each tag. Now I want to cluster them into similar products. How do I go forward it. I have tried k-means of scikit-learn. But That is not helping much.</p>

<pre><code>Product 1: a=2.5 b=3.5 c=1 d=1
Product 2: a=0.25 c=2
Product 3: e=2 k=5
.
.
.
.
.
.
.
.
Product n: a=3 b=0.75
</code></pre>

<p>Now I want these to be clustered. I also want a product to be in many clusters if necessary. Because 1, 2, 3 can form a cluster and 2, 4, 5 can form other</p>

<p>You could use a <a href="https://en.wikipedia.org/wiki/Mixture_model#Gaussian_mixture_model" rel="nofollow">Gaussian Mixture Model</a> which can be seen as a generalisation of k-means which allows soft clusters. You can have K clusters, and each entry belongs to all clusters with a certain amount. This amount is the probability of the entry under that cluster.
Luckily there is <a href="http://scikit-learn.org/stable/modules/mixture.html" rel="nofollow">scikit-learn code</a> for this.</p>

<p>You can treat the set of tags across all products as defining a feature space for the entries. The presence of a tag on a product means that product will have a non-zero entry, equal to the weight, in the position corresponding to that tag. From there, you have a fixed vector to describe entries and GMMs can be applied.</p>

<hr>

<p>However, it is really hard to evaluate unsupervised learning approaches like this. Rather, you should evaluate methods in light of the downstream task they are used for. like suggesting products to people or detecting fraud or detecting duplicates etc. </p>

python python-3.x machine-learning cluster-analysis data-mining