Python Web Scraping - Table with Dynamic Data
<p>I want to scrape data from a dynamic changing table.</p>

<p>The table is empty when you first open the website, but gets updated every 1-2 seconds with new values.</p>

<p>I tried doing that with the requests and lxml python package (Hitchiker's Guide to Python), but I only get the empty table. </p>

<p>Then I did it with Selenium, but it's a bit too slow to always start up a new browser (I need to get the value every 20-30 seconds).</p>

<p>The table uses a Messaging service called <a href="https://www.lightstreamer.com/" rel="nofollow">Lightstreamer</a>.</p>

<p>Instead of starting up a new browser every time, why not use something similar to <a href="http://phantomjs.org/" rel="nofollow">PhantomJS</a>. It would speed up your code with Selenium. Or try <a href="https://github.com/scrapinghub/splash" rel="nofollow">Splash</a> with Scrappy instead of Selenium. At the end of the day, it's hard to help you without seeing what you've done or tried. Also there are plenty of guides on how to use them on this site or google. </p>

python selenium