Find the location that occurs most in every cluster in DBSCAN
<p>Playing around with this DBSCAN example:
<a href="http://geoffboeing.com/2014/08/clustering-to-reduce-spatial-data-set-size/" rel="nofollow">http://geoffboeing.com/2014/08/clustering-to-reduce-spatial-data-set-size/</a></p>

<p>The author uses center-most point for each cluster. However, I would like to use the co-ordinates of the location that appears the most number of times in each cluster to represent that cluster. In my dataset, DBSCAN works quite well, but I would like to cluster these clusters together, probably using Hierarchical Clustering on the resulting smaller dataset. Any guidance on how to find the location that occurs most number of time would be great. Any other suggestions to improve clustering are welcome! Thanks!</p>

<p>Data == dataframe similar to locations history in the reference blog</p>

<pre><code>eps_rad = 32 / 6371.0088 #convert to radians
coords = data.as_matrix(columns=['LocLatDegrees', 'LocLongDegrees'])    
db = DBSCAN(eps=eps_rad, min_samples=50, algorithm='ball_tree', metric='haversine').fit(np.radians(coords))
cluster_labels = db.labels_
num_clusters = len(set(cluster_labels))  
n_clusters = len(set(cluster_labels)) - (1 if -1 in cluster_labels else 0)
print('Number of clusters: {:,}'.format(n_clusters))
#remove the noise i.e. cluster label -1
data =data[cluster_labels!=-1]
coords =coords[cluster_labels!=-1]
cluster_labels =cluster_labels[cluster_labels!=-1]
clusters = pd.Series([coords[cluster_labels==n] for n in range(n_clusters)])
</code></pre>

<p>That blog post is not particularly good.</p>

<p>By setting min_samples=1 he is not really using DBSCAN (because this disables <em>density</em>). Instead, he obtained a <em>single-linkage hierarchical clustering</em> result (with the dendrogram 'cut' at height epsilon).</p>

<p>Because DBSCAN allows arbitrary shaped clusters, the center, and centermost point, may actually be a <em>bad</em> choice. And his code assumes earth is flat when determining the center... So also this part of that blog post is not very sound...</p>

<p>If you consider this <a href="https://en.m.wikipedia.org/wiki/DBSCAN#/media/File%3ADBSCAN-density-data.svg" rel="nofollow">image from Wikipedia</a></p>

<p><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/0/05/DBSCAN-density-data.svg/1024px-DBSCAN-density-data.svg.png" alt="enter link description here"></p>

<p>then you can see that the most central point of the red cluster probably is not a good choice.</p>

<p>If you simply want to reduce your data set size, I suggest you use the very simple <strong>Leader</strong> clustering approach.</p>

<blockquote>
  <p>J. A. Hartigan. Clustering Algorithms. John Wiley &amp; Sons, New York, 1975</p>
</blockquote>

<p>This is much closer to the objective of reducing the data set size: essentially you define a threshold d, and you skip points if you already have an object that is closer than d, and keep it otherwise. In contrast to DBSCAN, this will not produce banana-like clusters.</p>

<p>But if you inte d to do hierarchical clustering afterwards, then why first use this approximarion?</p>

<p>As pointed out in another stackoverflow question, beware that <code>scipy.cluster.hierarchy.leaders</code> is NOT the leader-algorithm. There is an R package named <code>leaderCluster</code>, and the ELKI project that I follow for clustering recently <a href="https://github.com/elki-project/elki/commit/dd10823f6c24954c2e5a2810bffe5735257244f4" rel="nofollow">added Leader to Github</a>, too. As the ELKI version can use an index, I expect it to be much faster, but I haven't tried (their DBSCAN and OPTICS are really fast, so I usually use ELKI for large data sets; I like the cover tree index, which doesn't need more parameters than the distance function and just works well - found it to be faster and easier to use than the R*-tree; but these are my personal preferences - I wish jupyter would add some Java support).</p>

python pandas scikit-learn cluster-analysis dbscan