Python custom 404 response error
<p>I wrote a hiscore checker for a game that I play, basically you enter a list of usernames into the .txt file &amp; it outputs the results in found.txt.</p>

<p>However if the page responds a 404 it throws an error instead of returning output as " 0 " &amp; continuing with the list.</p>

<p>Example of script, </p>

<pre><code>#!/usr/bin/python

import urllib2

def get_total(username):
 try:
  req = urllib2.Request('http://services.runescape.com/m=hiscore/index_lite.ws?player=' + username)
  res = urllib2.urlopen(req).read()
  parts = res.split(',')
  return parts[1]
 except urllib2.HTTPError, e:
  if e.code == 404:
   return "0"
 except:
  return "err"

filename = "check.txt"

accs = []
handler = open(filename)
for entry in handler.read().split('\n'):
 if "No Displayname" not in entry:
  accs.append(entry)
handler.close()

for account in accs:
 display_name = account.split(':')[len(account.split(':')) - 1]
 total = get_total(display_name)
 if "err" not in total:
  rStr = account + ' - ' + total
  handler = open('tried.txt', 'a')
  handler.write(rStr + '\n')
  handler.close()
  if total != "0" and total != "49":
   handler = open('found.txt', 'a')
   handler.write(rStr + '\n')
   handler.close()
  print rStr
 else:
  print "Error searching"
  accs.append(account)

print "Done"
</code></pre>

<p>HTTPERROR exception that doesn't seem to be working,</p>

<pre><code> except urllib2.HTTPError, e:
  if e.code == 404:
   return "0"
 except:
  return "err"
</code></pre>

<p>Error response shown below.</p>

<p><a href="http://i.stack.imgur.com/zPp79.png" rel="nofollow"><img src="http://i.stack.imgur.com/zPp79.png" alt="Python Error"></a></p>

<p>Now I understand the error shown doesn't seem to be related to a response of 404, however this only occurs with users that return a 404 response from the request, any other request works fine. So I can assume the issue is within the 404 response exception.</p>

<p>I believe the issue may lay in the fact that the 404 is a custom page which you get redirected too?</p>

<p>so the original page is " example.com/index.php " but the 404 is " example.com/error.php "?</p>

<p>Not sure how to fix.</p>

<p>For testing purposes, format to use is,</p>

<p>ID:USER:DISPLAY</p>

<p>which is placed into check.txt</p>

<p>It seems that <code>total</code> can end up being <code>None</code>. In that case you can't check that it has <code>'err'</code> in it. To fix the crash, try changing that line to:</p>

<pre><code>if total is not None and "err" not in total:
</code></pre>

<p>To be more specific, <code>get_total</code> is returning <code>None</code>, which means that either</p>

<ul>
<li><code>parts[1]</code> is <code>None</code> <em>or</em> </li>
<li><code>except urllib2.HTTPError, e:</code> is executed <em>but</em> <code>e.code</code> is not 404.</li>
</ul>

<p>In the latter case <code>None</code> is returned as the exception is caught but you're only dealing with the very specific 404 case and ignoring other cases.</p>

python