python urlib in loop
<p>my requirement is to read some page which has so many links available in order
i have to stop at suppose at 4th link
and i have to read and connect to the url at that particular link
save the link contents in a list
again the connected link has so many links and i have to connected to the link at 4th position again
repeat this process for suppose 10 times and finally print the names of the link connected</p>

<p>i am using this code
urlllib is working only once</p>

<pre><code>import urllib
from bs4 import *
url = raw_input('enter url:')
count = raw_input('enter count:')
position = raw_input('enter position:')
count = int(count)
position = int(position)
l = list()
p = 0

for _ in xrange(0,count):
    print 'retrieving:' + url
    html = urllib.urlopen(url).read()
    s = BeautifulSoup(html)
    tags = s.findAll('a')
    for tag in tags:
        w = tag.get('href')
        p = p + 1
        if p == position:
            url = "'" + w + "'"
            l.append(tag.contents[0])

print l
</code></pre>

<p>Without knowing the particular site you're talking about this is just a guess, but could it be that the links in the page you're interested in are relative and not absolute?  If that's the case when you reset url in the for loop then it would be set to an incomplete link like /link.php instead of <a href="http://example.com/link.php" rel="nofollow">http://example.com/link.php</a> and urllib wouldn't know what to do with that.  If you expect all the links you could be interested in to be relative then you'd need to add the base url before appending the new link for it follow.</p>

python urllib