Python: classify text into the categories
<p>I have a part of training set</p>

<pre><code>url  category
ebay.com/sch/Mens-Clothing-/1059/i.html?_from=R40&amp;LH_BIN=1&amp;Bottoms%2520Size%2520%2528Men%2527s%2529=33&amp;Size%2520Type=Regular&amp;_nkw=ÐÐ¶Ð¸Ð½ÑÑ&amp;_dcat=11483&amp;Inseam=33&amp;rt=nc&amp;_trksid=p2045573.m1684 ÐÐ½Ð»Ð°Ð¹Ð½-Ð¼Ð°Ð³Ð°Ð·Ð¸Ð½
google.ru/webhp?sourceid=chrome-instant&amp;ion=1&amp;espv=2&amp;ie=UTF-8#q=%D0%BA%D0%BA%D1%83%D0%BF%D0%BE%D0%BD%D1%8B%20aliexpress%202016  Search
google.ru/webhp?sourceid=chrome-instant&amp;ion=1&amp;espv=2&amp;ie=UTF-8#newwindow=1&amp;q=%D0%BA%D1%83%D0%BF%D0%BE%D0%BD%D1%8B+aliexpress+2016    Search
google.ru/search?q=Ð°Ð²Ð¸ÑÐ¾&amp;oq=Ð°Ð²Ð¸ÑÐ¾&amp;aqs=chrome..69i57j0l5.1608j0j7&amp;sourceid=chrome&amp;es_sm=122&amp;ie=UTF-8 Search
irecommend.ru/content/kogda-somnenii-byt-ne-mozhet-tolko-klear-blyu-pomozhet    Ð¤Ð¾ÑÑÐ¼Ñ Ð¸ Ð¾ÑÐ·ÑÐ²Ñ
ebay.com/sch/Mens-Clothing-/1059/i.html?_from=R40&amp;LH_BIN=1&amp;Bottoms%2520Size%2520%2528Men%2527s%2529=33&amp;Size%2520Type=Regular&amp;_dcat=11483&amp;Inseam=33&amp;_nkw=ÐÐ¶Ð¸Ð½ÑÑ&amp;_sop=15  ÐÐ½Ð»Ð°Ð¹Ð½-Ð¼Ð°Ð³Ð°Ð·Ð¸Ð½
ebay.com/sch/Mens-Clothing-/1059/i.html?_from=R40&amp;LH_BIN=1&amp;Bottoms%2520Size%2520%2528Men%2527s%2529=33&amp;Size%2520Type=Regular&amp;_dcat=11483&amp;Inseam=33&amp;_nkw=ÐÐ¶Ð¸Ð½ÑÑ&amp;_sop=15  ÐÐ½Ð»Ð°Ð¹Ð½-Ð¼Ð°Ð³Ð°Ð·Ð¸Ð½
irecommend.ru/content/gramotnyi-razvod-na-dengi-bolshe-ne-kuplyu-vret   Ð¤Ð¾ÑÑÐ¼Ñ Ð¸ Ð¾ÑÐ·ÑÐ²Ñ
google.ru/search?q=ÑÐ½Ð´ÐµÐºÑ&amp;oq=ÑÐ½Ð´ÐµÐºÑ&amp;aqs=chrome..69i57j69i61l3j69i59l2.1383j0j1&amp;sourceid=chrome&amp;es_sm=93&amp;ie=UTF-8    Search
google.ru/search?q=Ð°Ð²Ð¸ÑÐ¾&amp;oq=Ð°Ð²Ð¸ÑÐ¾&amp;aqs=chrome..69i57j69i59j69i60.1095j0j1&amp;sourceid=chrome&amp;es_sm=93&amp;ie=UTF-8  Search
otzovik.com/review_1399716.html#debug   Ð¤Ð¾ÑÑÐ¼Ñ Ð¸ Ð¾ÑÐ·ÑÐ²Ñ
svyaznoy.ru ÐÐ½Ð»Ð°Ð¹Ð½-Ð¼Ð°Ð³Ð°Ð·Ð¸Ð½
mvideo.ru/smartfony-sotovye-telefony/apple-iphone-2927  ÐÐ½Ð»Ð°Ð¹Ð½-Ð¼Ð°Ð³Ð°Ð·Ð¸Ð½
mvideo.ru/promo/rassrochka-0-0-12-mark24197850/f/category=iphone-914?sort=priceLow&amp;_=1453896710474&amp;categoryId=10    ÐÐ½Ð»Ð°Ð¹Ð½-Ð¼Ð°Ð³Ð°Ð·Ð¸Ð½
svyaznoy.ru/catalog/phone/224/tag/windows-phone ÐÐ½Ð»Ð°Ð¹Ð½-Ð¼Ð°Ð³Ð°Ð·Ð¸Ð½
google.it/webhp?sourceid=chrome-instant&amp;ion=1&amp;espv=2&amp;ie=UTF-8#q=%D0%B5%D0%B2%D1%80%D0%BE%D1%81%D0%B5%D1%82%D1%8C    Search
vk.com   Social network
</code></pre>

<p>it's a connection between <code>url</code> and <code>category</code>
And also I have test set and I need to get category to every url.</p>

<pre><code>url    
vk.com/topic-102849764_32295213
stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set
google.ru/search?q=learning+sample&amp;oq=learning+sample&amp;aqs=chrome..69i57.4063j0j1&amp;sourceid=chrome&amp;ie=UTF-8#newwindow=1&amp;q=machine+learning+test+and+learn
facebook.com
locals.ru
tvzvezda.ru/news/vstrane_i_mire/content/201609261038-k6n1.htm
</code></pre>

<p>I don't know, what algorithm should I use to solve this task.
I need the best way to get the most accuracy.
And I think it's a problem, that I have multiple categories.</p>

<p>I try first parse html tag <code>title</code>, because I think, that I can's determine category only with <code>url</code>.</p>

<p>Basically you will classify strings into categories. Therefore you will to use a classifier. But you will not just use one classifier but rather test several and chose the most accurate. </p>

<p>Yet firstly, you will have to think about features of each url. I expect that you will not achieve great accuracy if you are simply feeding the url as a string and as the only feature.</p>

<p>Rather you will preprocess each url to extract features. The choice of relevant/useful features strongly depends on the domain. A feature could be:</p>

<p>simple features</p>

<ul>
<li><p>the first word until the dot such as: facebook for "facebook.com" </p></li>
<li><p>the length of the whole string</p></li>
</ul>

<p>complex features</p>

<p>imagine you define keywords for each cluster such as for "online-shopping"-cluster you will define [promo, buy, shop, sell, price], then you can compute the number of keywords which occur in the string for each cluster as a feature</p>

<p>Therefore, you will have to continue firstly with <strong>feature-engineering</strong> and secondly with a comparing classifier performance. </p>

<p>Additional input: </p>

<p><a href="http://stackoverflow.com/questions/26456904/how-to-classify-urls-what-are-urls-features-how-to-select-and-extract-features">Similiar question on SO (regarding URL features)</a></p>

<p><a href="http://blog.christianperone.com/2011/09/machine-learning-text-feature-extraction-tf-idf-part-i/" rel="nofollow">Text feature extraction</a></p>

<p><a href="https://www.comp.nus.edu.sg/~kanmy/papers/cp689-kan.pdf" rel="nofollow">Fast Webpage Classification Using URL Features</a></p>

<p>EDIT: An example</p>

<pre><code>url = "irecommend.ru/content/kogda-somnenii-byt-ne-mozhet-tolko-klear-blyu-pomozhet"    

f1  = len(url) = 76
f2 = base = str(url).split("/",1)[0] = "irecommend.ru"
f3 = segments = str(a).count("/") = 2
</code></pre>

<p>more solutions from <a href="http://stackoverflow.com/questions/6969268/counting-letters-numbers-and-punctuation-in-a-string">here</a> by <a href="http://stackoverflow.com/users/997301/eiyrio%C3%BC-von-kauyf">EiyrioÃ¼ von Kauyf</a></p>

<pre><code>import string
count = lambda l1,l2: sum([1 for x in l1 if x in l2])

f4 = count_punctuation = count(a,set(string.punctuation))
f5 = count_ascii = count(a,set(string.ascii_letters))
</code></pre>

<p>Yet all these examples are very simple features, which do not cover the semantic content of the URL. Depending on the depth/sophistication of your target variables (clusters), you might need to use features n-gram based features such as in <a href="https://www.google.de/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;ved=0ahUKEwjgrsC2i7TPAhWKaRQKHUfLBD4QFggvMAA&amp;url=http%3A%2F%2Fwww.springer.com%2Fcda%2Fcontent%2Fdocument%2Fcda_downloaddocument%2F9783319258393-c2.pdf%3FSGWID%3D0-0-45-1533089-p177762447&amp;usg=AFQjCNHJeV8TIhj1ylnkG0aMWIZoNJK4bA&amp;cad=rja" rel="nofollow">here</a></p>

python url machine-learning scikit-learn text-classification