Compose for Elasticsearch Authentication in Python
<p>I am having issues getting the Python Elasticsearch library working with the Compose for Elasticsearch offering from Bluemix. When I use the following code against an Elasticsearch cluster I created using IBM containers it connects just fine. </p>

<h3>IBM Container Cluster running Elasticsearch with the Shield Plugin for user authentication</h3>

<pre><code>(venv) â¯ python
&gt;&gt;&gt; import requests
&gt;&gt;&gt; import elasticsearch
&gt;&gt;&gt; from elasticsearch import Elasticsearch, RequestsHttpConnection
&gt;&gt;&gt; es = Elasticsearch(
... host='clustertestv5.mybluemix.net',
... port=80,
... http_auth=('es_admin', 'REDACTED'),
... timeout=60)
&gt;&gt;&gt; es.info()
{u'cluster_name': u'elasticsearch', u'tagline': u'You Know, for Search', u'version': {u'lucene_version': u'5.5.0', u'build_hash': u'218bdf10790eef486ff2c41a3df5cfa32dadcfde', u'number': u'2.3.3', u'build_timestamp': u'2016-05-17T15:40:04Z', u'build_snapshot': False}, u'name': u'Forgotten One'}
</code></pre>

<p>When I try to use the same code but authenticate against the Elasticsearch cluster provided by Bluemix it fails with a <code>ConnectionError</code> response. </p>

<pre><code>(venv) â¯ python
&gt;&gt;&gt; import requests
&gt;&gt;&gt; import elasticsearch
&gt;&gt;&gt; from elasticsearch import Elasticsearch, RequestsHttpConnection
&gt;&gt;&gt; es1 = Elasticsearch(
... host='bluemix-sandbox-dal-9-portal.3.dblayer.com',
... port=15206,
... http_auth=('admin', 'REDACTED'),
... timeout=60)
&gt;&gt;&gt; es1.info()
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 220, in info
    return self.transport.perform_request('GET', '/', params=params)
  File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/Users/ryan/Desktop/deltaNiner/venv/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 105, in perform_request
    raise ConnectionError('N/A', str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ProtocolError(('Connection aborted.', BadStatusLine("''",)))
&gt;&gt;&gt; quit()
</code></pre>

<p>A straight curl against the Compose cluster shows that the username and password are correct so I know that is not causing the Connection issue. </p>

<pre><code>(venv) â¯ curl -k 'https://bluemix-sandbox-dal-9-portal.3.dblayer.com:15206/'
&lt;html&gt;&lt;body&gt;&lt;h1&gt;401 Unauthorized&lt;/h1&gt;
You need a valid user and password to access this content.
&lt;/body&gt;&lt;/html&gt;

(venv) â¯ curl -k -u admin:REDACTED 'https://bluemix-sandbox-dal-9-portal.3.dblayer.com:15206/'
{
  "name" : "elastic_search56_bluemix_sandbox_dal_9_data_2_dblayer_com",
  "cluster_name" : "bmix_dal_yp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "version" : {
    "number" : "2.4.0",
    "build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55",
    "build_timestamp" : "2016-08-29T09:14:17Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}
</code></pre>

<p>For reference here is the Elasticsearch python library - <a href="https://elasticsearch-py.readthedocs.io/en/master/" rel="nofollow">https://elasticsearch-py.readthedocs.io/en/master/</a></p>

<p>in your python code that creates the ES connection you're missing an option.</p>

<p>Add: <code>use_ssl=True</code></p>

<p>It is going to complain at that point about missing certificate validation, but will proceed. To round it out you should use the certificate provided in the VCAP connection info, along with the <code>verify_ssl=True</code> option. That option is <code>False</code> by default. Note: default behavior is equivalent to your use of the <code>-k</code> option with curl.</p>

python elasticsearch ibm-bluemix compose