Trying to scrape this website using python but unable to get the required data
<p>I am trying to get the company names in this website <a href="https://siftery.com/microsoft-outlook" rel="nofollow">https://siftery.com/microsoft-outlook</a>
Basically it lists some companies that use Microsoft Outlook.
I used BeautifulSoup,requests,urllib and urllib2 but I still am not getting the names of the companies that use Microsoft Outlook not even in the first page of the website. </p>

<p>The code I wrote is below - </p>

<pre><code>r = requests.get('http://siftery.com/microsoft-outlook')

print(str(r.content))
f=open('abc.txt','w')
f.write(r.content)
f.close()
</code></pre>

<p>and part of the output that looks interesting is this -</p>

<p>({"name":"Marketing","handle":"marketing","categories":[{"name":"Marketing Automation","handle":"marketing-automation","external_id":"tgJ_49k7v4J-wV","parent_handle":null,"categories":[{"name":"Marketing Automation Platforms","handle":"marketing-automation-platforms","external_id":"tgJLE9aHoLdneT","parent_handle":"marketing-automation"},</p>

<p>BeautifulSoup also gives me the same output, so do the other libraries.
It seems like "external_id" is where the company name is ? I'm not sure. I also tried to manually find the name of a company for example Acxiom using gedit but couldn't find any occurrence.</p>

<p>The data is available directly as JSON. You can use requests to get it like this:</p>

<pre><code>import requests

r = requests.post('https://siftery.com/product-json/microsoft-outlook')
data = r.json()['content']
companies = data['companies']
for company in companies:
    print(companies[company]['name'])
</code></pre>

<p><strong>Output</strong></p>

<pre>
Public Technologies
Consalta
PagesJaunes.ca
Chumbak
Media Classified
P.I. Works
Saatchi & Saatchi Pro
Tribeck Strategies
Marketecture Solutions, LLC
Trinity Ventures
ARGOS
CFN Services
Last.Backend
Saatchi & Saatchi USA
Netcad
Central Element
NextGear Capital
Masao
Avalon
Motiwe
Bilge Adam
Impakt Athletics
SOZO Design
ThroughTek
Abovo42
Acxiom
ICEPAY
Connexta
Clearview
Mortgage Coach
</pre>

<p>There are other categories of information which you might want to investigate:</p>

<pre><code>&gt;&gt;&gt; data.keys()
[u'product', u'vendor', u'users', u'group_members', u'companies', u'customers', u'other_categories', u'current_user', u'page_info', u'portfolio_products', u'primary_category', u'metadata']
</code></pre>

python web-scraping