Extracting links and titles only
<p>I am trying to extract links and titles for these links in an anime website, However, I am only able to extract the whole tag, I just want the href and the title.</p>

<p>Here`s the code am using:</p>

<pre><code>import requests
from bs4 import BeautifulSoup

r = requests.get('http://animeonline.vip/info/phi-brain-kami-puzzle-3')
soup = BeautifulSoup(r.content, "html.parser")
for link in soup.find_all('div', class_='list_episode'):
    href = link.get('href')
    print(href)
</code></pre>

<p>And here`s the website html:</p>

<pre><code>&lt;a href="http://animeonline.vip/phi-brain-kami-puzzle-3-episode-25" title="Phi Brain: Kami no Puzzle 3 episode 25"&gt;
                    Phi Brain: Kami no Puzzle 3 episode 25                  &lt;span&gt; 26-03-2014&lt;/span&gt;
        &lt;/a&gt;
</code></pre>

<p>And this is the output:</p>

<pre><code>C:\Python34\python.exe C:/Users/M.Murad/PycharmProjects/untitled/Webcrawler.py
None

Process finished with exit code 0
</code></pre>

<p>All that I want is all links and titles in that class (episodes and their links)</p>

<p>Thanks.</p>

<p>So what is happening is, your link element has all the information in anchor <code>&lt;div&gt;</code> and class = "last_episode" but this has a lot of anchors in it which holds the link in "href" and title in "title".</p>

<p>Just modify the code a little and you will have what you want.</p>

<pre><code>import requests
from bs4 import BeautifulSoup

r = requests.get('http://animeonline.vip/info/phi-brain-kami-puzzle-3')
soup = BeautifulSoup(r.content, "html.parser")
for link in soup.find_all('div', class_='list_episode'):
    href_and_title = [(a.get("href"), a.get("title")) for a in link.find_all("a")]   
    print href_and_title
</code></pre>

<p>output will be in form of [(href,title),(href,title),........(href,title)]</p>

<p>Edit(Explanation):</p>

<p>So what is happening is when you do </p>

<pre><code>soup.find_all('div', class_='list_episode')
</code></pre>

<p>It gives you all details (in html page) with "div" and class "last_episode" but now this anchor holds a huge set of anchors with different "href" and title details, so to get that we use a for loop (there can be multiple anchors (<code>&lt;a&gt;</code>)) and ".get()".</p>

<pre><code> href_and_title = [(a.get("href"), a.get("title")) for a in link.find_all("a")]
</code></pre>

<p>I hope it's clearer this time .</p>

python visual-studio web-scraping python-3.4 ptvs