phantomjs + selenium in python proxy-auth not working
<p>I'm trying to set a proxy for webscraping using selenium + phantomjs. I'm using python. </p>

<p>I've seen in many places that there is a bug in phantomjs such that proxy-auth does not work. </p>

<pre><code>from selenium.webdriver.common.proxy import *
from selenium import webdriver
from selenium.webdriver.common.by import By
service_args = [
'--proxy=http://fr.proxymesh.com:31280',
'--proxy-auth=USER:PWD',
'--proxy-type=http',
]

driver = webdriver.PhantomJS(service_args=service_args)
driver.get("https://www.google.com")
print driver.page_source
</code></pre>

<p>Proxy mesh suggests using the following instead: </p>

<blockquote>
  <p>page.customHeaders={'Proxy-Authorization': 'Basic '+btoa('USERNAME:PASSWORD')};</p>
</blockquote>

<p>but I'm not sure how to translate that into python. </p>

<p>This is what I currently have: </p>

<pre><code>from selenium import webdriver
import base64
from selenium.webdriver.common.proxy import *
from selenium import webdriver
from selenium.webdriver.common.by import By

service_args = [
'--proxy=http://fr.proxymesh.com:31280',
'--proxy-type=http',
]

headers = { 'Proxy-Authorization': 'Basic ' +   base64.b64encode('USERNAME:PASSWORD')}

for key, value in enumerate(headers):
    webdriver.DesiredCapabilities.PHANTOMJS['phantomjs.page.customHeaders.{}'.format(key)] = value

driver = webdriver.PhantomJS(service_args=service_args)
driver.get("https://www.google.com")
print driver.page_source
</code></pre>

<p>but it doesn't work. </p>

<p>Any suggestions for how I could get this to work?</p>

<p>I'm compiling answers from:
<a href="http://stackoverflow.com/questions/37331684/how-to-correctly-pass-basic-auth-every-click-using-selenium-and-phantomjs-webd">How to correctly pass basic auth (every click) using Selenium and phantomjs webdriver</a>
as well as:
<a href="http://stackoverflow.com/questions/16740188/base64-b64encode-error">base64.b64encode error</a></p>

<pre><code>from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import base64

service_args = [
    '--proxy=http://fr.proxymesh.com:31280',
    '--proxy-type=http',
]

authentication_token = "Basic " + base64.b64encode(b'username:password')

capa = DesiredCapabilities.PHANTOMJS
capa['phantomjs.page.customHeaders.Proxy-Authorization'] = authentication_token
driver = webdriver.PhantomJS(desired_capabilities=capa, service_args=service_args)

driver.get("http://...")
</code></pre>

python selenium proxy phantomjs