Grouping by almost similar strings
<p>I have a dataset with city names and counts of crimes. The data is dirty such that a name of a city for example 'new york', is written as 'newyork', 'new york us', 'new york city', 'manhattan new york' etc. How can I group all these cities together and sum their crimes?</p>

<p>I tried the 'difflib' package in python that matches strings and gives you a score. It doesn't work well. I also tried the geocode package in python. It has limits on number of times you can access the api, and doesnt work well either. Any suggestions? </p>

<p>Maybe this might help:</p>

<p><a href="http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/" rel="nofollow">http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/</a></p>

<p>Another way: if a string contains 'new' and 'york', then label it 'new york city'. </p>

<p>Another way: Create a dictionary of all the possible fuzzy words that occur and label each of them manually. And use that labelling to replace each of these fuzzy words with the label.</p>

python pandas