Library `requests` getting different results unpredictably
<p>Why does this code:</p>

<pre><code>import requests


response = requests.post('http://evds.tcmb.gov.tr/cgi-bin/famecgi', data={
    'cgi': '$ozetweb',
    'ARAVERIGRUP': 'bie_yymkpyuk.db',
    'DIL': 'UK',
    'ONDALIK': '5',
    'wfmultiple_selection': 'ZAMANSERILERI',
    'f_begdt': '07-01-2005',
    'f_enddt': '07-10-2016',
    'ZAMANSERILERI': ['TP.PYUK1', 'TP.PYUK2', 'TP.PYUK21', 'TP.PYUK22', 'TP.PYUK3', 'TP.PYUK4', 'TP.PYUK5', 'TP.PYUK6'],
    'YON': '3',
    'SUBMITDEG': 'Report',
    'GRTYPE': '1',
    'EPOSTA': 'xxx',
    'RESIMPOSTA': '***',
})

print(response.text)
</code></pre>

<p>produces different results in Python 2 (<code>2.7.12</code>) and Python 3 (<code>3.5.2</code>)? I'm using <code>requests==2.11.1</code>. Since the <code>requests</code> library supports both Python versions with the same API, I guess the result should be the same.</p>

<p>The expected result is the one obtained from running the code with Python 2. It works every single time. When ran with Python 3, the server sometimes returns an error, and sometimes it works. (This is the intriguing part.)</p>

<p>Since it works with Python 2, I figure the error must happen in the client side. Is there any caveat to how Python 3 handles encoding, or sending the data through the socket, that I should be aware of?</p>

<p><strong>EDIT:</strong> In the comments below, a person was able to reproduce this and confirms this issue exists.</p>

<p>It does seem to come down to different between dicts in python2 vs python3  in relation to <a href="https://docs.python.org/3/whatsnew/3.3.html#porting-python-code" rel="nofollow">Hash randomization is enabled by default</a> since python3.3 and the server needing at least the <em>cgi</em> field to come first, the following can reproduce:</p>

<pre><code>good = requests.post('http://evds.tcmb.gov.tr/cgi-bin/famecgi', data=([
    ('cgi', '$ozetweb'),
    ('ARAVERIGRUP', 'bie_yymkpyuk.db'),
    ('DIL', 'UK'),
    ('ONDALIK', '5'),
    ('wfmultiple_selection', 'ZAMANSERILERI'),
    ('f_begdt', '07-01-2005'),
    ('f_enddt', '07-10-2016'),
    ('ZAMANSERILERI',
     ['TP.PYUK1', 'TP.PYUK2', 'TP.PYUK21', 'TP.PYUK22', 'TP.PYUK3', 'TP.PYUK4', 'TP.PYUK5', 'TP.PYUK6']),
    ('YON', '3'),
    ('SUBMITDEG', 'Report'),
    ('GRTYPE', '1'),
    ('EPOSTA', 'xxx'),
    ('RESIMPOSTA', '***')]))


bad = requests.post('http://evds.tcmb.gov.tr/cgi-bin/famecgi', data=([
    ('ARAVERIGRUP', 'bie_yymkpyuk.db'),
    ('cgi', '$ozetweb'),
    ('DIL', 'UK'),
    ('wfmultiple_selection', 'ZAMANSERILERI'),
    ('ONDALIK', '5'),
    ('f_begdt', '07-01-2005'),
    ('f_enddt', '07-10-2016'),
    ('ZAMANSERILERI',
     ['TP.PYUK1', 'TP.PYUK2', 'TP.PYUK21', 'TP.PYUK22', 'TP.PYUK3', 'TP.PYUK4', 'TP.PYUK5', 'TP.PYUK6']),
    ('YON', '3'),
    ('SUBMITDEG', 'Report'),
    ('GRTYPE', '1'),
    ('EPOSTA', 'xxx'),
    ('RESIMPOSTA', '***')]))
</code></pre>

<p>Running the code above using python2:</p>

<pre><code>In [6]: print(good.request.body)
   ...: print(bad.request.body)
   ...: 
   ...: print(len(good.text), len(bad.text))
   ...: 
cgi=%24ozetweb&amp;ARAVERIGRUP=bie_yymkpyuk.db&amp;DIL=UK&amp;ONDALIK=5&amp;wfmultiple_selection=ZAMANSERILERI&amp;f_begdt=07-01-2005&amp;f_enddt=07-10-2016&amp;ZAMANSERILERI=TP.PYUK1&amp;ZAMANSERILERI=TP.PYUK2&amp;ZAMANSERILERI=TP.PYUK21&amp;ZAMANSERILERI=TP.PYUK22&amp;ZAMANSERILERI=TP.PYUK3&amp;ZAMANSERILERI=TP.PYUK4&amp;ZAMANSERILERI=TP.PYUK5&amp;ZAMANSERILERI=TP.PYUK6&amp;YON=3&amp;SUBMITDEG=Report&amp;GRTYPE=1&amp;EPOSTA=xxx&amp;RESIMPOSTA=%2A%2A%2A
ARAVERIGRUP=bie_yymkpyuk.db&amp;cgi=%24ozetweb&amp;DIL=UK&amp;wfmultiple_selection=ZAMANSERILERI&amp;ONDALIK=5&amp;f_begdt=07-01-2005&amp;f_enddt=07-10-2016&amp;ZAMANSERILERI=TP.PYUK1&amp;ZAMANSERILERI=TP.PYUK2&amp;ZAMANSERILERI=TP.PYUK21&amp;ZAMANSERILERI=TP.PYUK22&amp;ZAMANSERILERI=TP.PYUK3&amp;ZAMANSERILERI=TP.PYUK4&amp;ZAMANSERILERI=TP.PYUK5&amp;ZAMANSERILERI=TP.PYUK6&amp;YON=3&amp;SUBMITDEG=Report&amp;GRTYPE=1&amp;EPOSTA=xxx&amp;RESIMPOSTA=%2A%2A%2A
(71299, 134)
</code></pre>

<p>And python3:</p>

<pre><code>In [4]: print(good.request.body)
   ...: print(bad.request.body)
   ...: 
   ...: print(len(good.text), len(bad.text))
   ...: 
cgi=%24ozetweb&amp;ARAVERIGRUP=bie_yymkpyuk.db&amp;DIL=UK&amp;ONDALIK=5&amp;wfmultiple_selection=ZAMANSERILERI&amp;f_begdt=07-01-2005&amp;f_enddt=07-10-2016&amp;ZAMANSERILERI=TP.PYUK1&amp;ZAMANSERILERI=TP.PYUK2&amp;ZAMANSERILERI=TP.PYUK21&amp;ZAMANSERILERI=TP.PYUK22&amp;ZAMANSERILERI=TP.PYUK3&amp;ZAMANSERILERI=TP.PYUK4&amp;ZAMANSERILERI=TP.PYUK5&amp;ZAMANSERILERI=TP.PYUK6&amp;YON=3&amp;SUBMITDEG=Report&amp;GRTYPE=1&amp;EPOSTA=xxx&amp;RESIMPOSTA=%2A%2A%2A
ARAVERIGRUP=bie_yymkpyuk.db&amp;cgi=%24ozetweb&amp;DIL=UK&amp;wfmultiple_selection=ZAMANSERILERI&amp;ONDALIK=5&amp;f_begdt=07-01-2005&amp;f_enddt=07-10-2016&amp;ZAMANSERILERI=TP.PYUK1&amp;ZAMANSERILERI=TP.PYUK2&amp;ZAMANSERILERI=TP.PYUK21&amp;ZAMANSERILERI=TP.PYUK22&amp;ZAMANSERILERI=TP.PYUK3&amp;ZAMANSERILERI=TP.PYUK4&amp;ZAMANSERILERI=TP.PYUK5&amp;ZAMANSERILERI=TP.PYUK6&amp;YON=3&amp;SUBMITDEG=Report&amp;GRTYPE=1&amp;EPOSTA=xxx&amp;RESIMPOSTA=%2A%2A%2A
71299 134
</code></pre>

<p>Passing your dict as posted in python2:</p>

<pre><code>In [4]: response.request.body
Out[4]: 'cgi=%24ozetweb&amp;DIL=UK&amp;f_enddt=07-10-2016&amp;YON=3&amp;RESIMPOSTA=%2A%2A%2A&amp;wfmultiple_selection=ZAMANSERILERI&amp;ARAVERIGRUP=bie_yymkpyuk.db&amp;GRTYPE=1&amp;SUBMITDEG=Report&amp;f_begdt=07-01-2005&amp;ZAMANSERILERI=TP.PYUK1&amp;ZAMANSERILERI=TP.PYUK2&amp;ZAMANSERILERI=TP.PYUK21&amp;ZAMANSERILERI=TP.PYUK22&amp;ZAMANSERILERI=TP.PYUK3&amp;ZAMANSERILERI=TP.PYUK4&amp;ZAMANSERILERI=TP.PYUK5&amp;ZAMANSERILERI=TP.PYUK6&amp;ONDALIK=5&amp;EPOSTA=xxx'

In [5]: len(response.text)
Out[5]: 71299
</code></pre>

<p>And the same dict in python3:</p>

<pre><code>In [3]: response.request.body
Out[3]: 'EPOSTA=xxx&amp;ARAVERIGRUP=bie_yymkpyuk.db&amp;DIL=UK&amp;SUBMITDEG=Report&amp;cgi=%24ozetweb&amp;GRTYPE=1&amp;f_enddt=07-10-2016&amp;wfmultiple_selection=ZAMANSERILERI&amp;ONDALIK=5&amp;f_begdt=07-01-2005&amp;RESIMPOSTA=%2A%2A%2A&amp;YON=3&amp;ZAMANSERILERI=TP.PYUK1&amp;ZAMANSERILERI=TP.PYUK2&amp;ZAMANSERILERI=TP.PYUK21&amp;ZAMANSERILERI=TP.PYUK22&amp;ZAMANSERILERI=TP.PYUK3&amp;ZAMANSERILERI=TP.PYUK4&amp;ZAMANSERILERI=TP.PYUK5&amp;ZAMANSERILERI=TP.PYUK6'

In [4]: len(response.text)
Out[4]: 134
</code></pre>

<p>And running <code>~$ export PYTHONHASHSEED=1234</code> before starting another ipython2 shell:</p>

<pre><code>In [4]: response.request.body
Out[4]: 'DIL=UK&amp;GRTYPE=1&amp;ARAVERIGRUP=bie_yymkpyuk.db&amp;f_begdt=07-01-2005&amp;RESIMPOSTA=%2A%2A%2A&amp;ONDALIK=5&amp;EPOSTA=xxx&amp;YON=3&amp;SUBMITDEG=Report&amp;wfmultiple_selection=ZAMANSERILERI&amp;cgi=%24ozetweb&amp;ZAMANSERILERI=TP.PYUK1&amp;ZAMANSERILERI=TP.PYUK2&amp;ZAMANSERILERI=TP.PYUK21&amp;ZAMANSERILERI=TP.PYUK22&amp;ZAMANSERILERI=TP.PYUK3&amp;ZAMANSERILERI=TP.PYUK4&amp;ZAMANSERILERI=TP.PYUK5&amp;ZAMANSERILERI=TP.PYUK6&amp;f_enddt=07-10-2016'

In [5]: os.environ["PYTHONHASHSEED"]
Out[5]: '1234'
In [6]: len(response.text)
Out[6]: 134
</code></pre>

<p>You can run the code numerous times to the same end but definitely  <code>('cgi', '$ozetweb')</code> coming first is essential for the code to work, it happened to work using python3 intermittently as the order of the keys sometimes put <em>cgi</em> first. There is a bit more on the <a href="http://stackoverflow.com/a/27522708/2141635">hashing topic</a></p>

python python-requests