web scraping with beautifulsoup
<p>I'm trying to parsing the website only particular part. Here is my code below. Is there anyway to do it more efficient.</p>

<pre><code>from bs4 import BeautifulSoup
import requests
import urllib.request
import json

soup = BeautifulSoup(requests.get("http://www.example.com").content, "html.parser")

for d in soup.select("script[type=text/javascript]"):
    print(d.text[2300:2600])
</code></pre>

<p>Here is the output what i need</p>

<pre><code>&gt; dataLayer = [{
&gt;      'page':'ProductPage',
&gt;       'OAM':'False',
&gt;       'storeNum':'075',
&gt;       'brand':'Seagate',
&gt;       'productPrice':'69.99',
&gt;      'SKU':'106674',
&gt;      'productID':'467336',
&gt;      'mpn':'ST2000DM006',
&gt;      'ean':'763649110218',
&gt;       'category':'Internal Hard Drives',
&gt;      'isMobile':'False' }];
</code></pre>

<p>It can change on other page - (I didn't check it with other pages)</p>

<pre><code>for d in soup.select("script[type=text/javascript]")[27].text.split('\n')[51:62]:
    print(d.strip())
</code></pre>

<p>result   </p>

<pre><code>'page':'ProductPage',
'OAM':'False',
'storeNum':'029',
'brand':'Microsoft',
'productPrice':'129.99',
'SKU':'883785',
'productID':'456088',
'mpn':'QC7-00001',
'ean':'889842010060',
'category':'Tablet Accessories',
'isMobile':'False'
</code></pre>

<hr>

<p><strong>EDIT:</strong> other version:</p>

<pre><code>text = soup.select("head script[type=text/javascript]")[-1].text

start = text.find('dataLayer = [{') + len('dataLayer = [{') 
end = text.rfind('}];')

rows = text[start:end].strip().split('\n')

for d in rows:
    print(d.strip())
</code></pre>

python parsing web-scraping