How to 'react' to the response of a request using Python
<p>I am using Python requests to get information from the mobile website of the german railways company (<a href="https://mobile.bahn.de/bin/mobil/query.exe/dox" rel="nofollow">https://mobile.bahn.de/bin/mobil/query.exe/dox</a>')</p>

<p>For instance: </p>

<pre><code>import requests
query = {'S':'Stuttgart Hbf', 'Z':'MÃ¼nchen Hbf'}
rsp = requests.get('https://mobile.bahn.de/bin/mobil/query.exe/dox',  params=query)
</code></pre>

<p>which in this case gives the correct page.</p>

<p>However, using the following query:</p>

<pre><code>query = {'S':'Cottbus', 'Z':'MÃ¼nchen Hbf'}
</code></pre>

<p>It gives another response, where the user is required to choose one of the given options (The server is confused about the starting stations, since there are many beginning with 'Cottbus')</p>

<p>Now, my question is: given this response, how can I choose one of the given options, and then repeat the request without getting this error ? </p>

<p>I tried to look at the cookies, to use a session instead of a simple get request. But nothing worked so far.</p>

<p>I hope you can help me.</p>

<p>Thanks.</p>

<p>You can use Beautifulsoup to parse the response and get the options if there is a select on the response: </p>

<pre><code>import requests
from bs4 import BeautifulSoup

query = {'S': u'Cottbus', 'Z': u'MÃ¼nchen Hbf'}
rsp = requests.get('https://mobile.bahn.de/bin/mobil/query.exe/dox',  params=query)
soup = BeautifulSoup(rsp.content, 'lxml')

# check if has choice dropdown
if soup.find('select'):
    # Get list of tuples with text and input values that you will nee do use in the next POST request
    options_value = [(option['value'], option.text) for option in soup.find_all('option')]
</code></pre>

python python-requests