Retrieve Network Logs using Python
<p>Generally, whenever a page is loaded, it sends several requests, which can be recorded in the <strong>Network</strong> tab in chrome developer tools. </p>

<p>My prime motive is to log all the network requests whenever page is loaded using python script. A sample screen-shot is attached, to illustrate what all request I am trying to collect. </p>

<p><a href="http://i.stack.imgur.com/Bh0sR.png" rel="nofollow">Image for Network Hit Logs</a></p>

<p>I am trying to achieve the same using <em>urllib</em> library in python, however, I am not exactly sure of the usage.</p>

<p>Looking forward for your responses. Thanks in advance. </p>

<p>You can't do this with the <code>urllib</code> family of libraries. To capture AJAX Requests you need to use something that has Javascript support... like a browser.</p>

<p>So your best option in this case is to use <a href="http://docs.seleniumhq.org/" rel="nofollow">Selenium</a> to write a script that uses the <a href="http://docs.seleniumhq.org/projects/webdriver/" rel="nofollow">Selenium Web Driver</a> to drive whatever browser you're using and then capture/log the AJAX requests being pushed out.</p>

python