Mimicking HTML5 Video support on PhantomJS used through Selenium in Python
<p>I am trying to extract the source link of an HTML5 video found in the video tag . Using Firefox webdrive , I am able to get the desired result ie - </p>

<pre><code>[&lt;video class="video-stream html5-main-video" src='myvideoURL..'&lt;/video&gt;]
</code></pre>

<p>but if I use PhantomJS -</p>

<pre><code> &lt;video class="video-stream html5-main-video" style="width: 854px; height: 480px; left: 0px; top: 0px; -webkit-transform: none;" tabindex="-1"&gt;&lt;/video&gt;
</code></pre>

<p>I suspect this is because of PhantomJS' lack of HTML5 Video support . Is there anyway I can trick the webpage into thinking that HTML5 Video is supported so that it generates the URL ? Or can I do something else ?</p>

<p>tried this</p>

<pre><code>try:

    WebDriverWait(browser,10).until(EC.presence_of_element_located((By.XPATH, "//video")))


finally:


    k = browser.page_source


    browser.quit()


soup = BeautifulSoup(k,'html.parser')


print (soup.find_all('video'))
</code></pre>

<p>The way Firefox and phantomjs webdrivers communicate with Selenium are quite different.</p>

<p>When using Firefox, it signals back that the page has finished loading after it loaded some of the javascript</p>

<p>Differently in phantomjs, it signals Selenium that the page has finished loading as soon as it is able to get the page source meaning it wouldn't have loaded any javascript.</p>

<p>What you need to do is <a href="http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp" rel="nofollow">Wait</a> for the element to be present before extracting it, in this case it would be:</p>

<pre><code>video = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//video")))
</code></pre>

<p>EDIT:</p>

<p>Youtube first checks if the browser supports the video content before deciding whether to provide the source, theres a workaround though described <a href="http://stackoverflow.com/questions/30146353/phantomjs-not-mimicking-browser-behavior-when-looking-at-youtube-videos">here</a></p>

python selenium phantomjs