Scraping ASP Page with Python
<p>I'm trying to access an asp page to get data from a website but it always redirects my to its main page. I've tried setting allow_redirect to false but that throws an error, saying "the object has moved and can be found at href=main.htm". The website requires basic auth.</p>

<p>-The url I want the data from is:</p>

<pre><code>example.com.au/blah/webpage.asp
</code></pre>

<p>However, whenever I run the requests.get command, it redirects to example.com.au/blah/main.htm.</p>

<p>my code:</p>

<pre><code>url = "https://example.com.au/blah/webpage.asp"

s = requests.Session()

s.get(url, headers = {"Authorization" : auth})


r = requests.get(url, cookies = s.cookies, headers = {"Authorization" : auth})

print(r.url)
</code></pre>

<p>output:</p>

<pre><code>example.com.au/blah/main.htm
</code></pre>

<p>I was able to access the .asp page by first loading a primary asp page </p>

<pre><code>s.get(example.com.au/blah/menu.asp)
</code></pre>

<p>then loading the asp page I wanted</p>

<pre><code>r = s.get(example.com.au/blah/webpage.asp)
</code></pre>

<p>under the same session.</p>

python asp.net