Using python3, what is the fastest way to extract all <div> blocks from a html str?
<p>there are <code>&lt;div&gt;</code> inner blocks inside a <code>&lt;div&gt;</code> block,
What is the fastest way to extract all <code>&lt;div&gt;</code> blocks from a html str ?
(bs4, lxml or regex ?)</p>

<p><a href="http://lxml.de/" rel="nofollow"><code>lxml</code></a> is generally considered to be the fastest among existing Python parsers, though the parsing speed depends on multiple factors starting with the specific HTML to parse and ending with the computational power you have available. For HTML parsing use the <code>lxml.html</code> subpackage:</p>

<pre><code>from lxml.html import fromstring, tostring

data = """my HTML string"""
root = fromstring(data)

print([tostring(div) for div in root.xpath(".//div")]) 
print([div.text_content() for div in root.xpath(".//div")]) 
</code></pre>

<p>There is also the awesome <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/" rel="nofollow"><code>BeautifulSoup</code></a> parser which, <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/#differences-between-parsers" rel="nofollow">if allowed to use <code>lxml</code> under-the-hood</a>, would be a great combination of convenience, flexibility and speed. It would not be generally faster than pure <code>lxml</code>, but it comes with one of the best APIs I've ever seen allowing you to "view" your XML/HTML from different angles and use a huge variety of techniques:</p>

<pre><code>from bs4 import BeautifulSoup

soup = BeautifulSoup(data, "lxml")
print([str(div) for div in soup.find_all("div")])
print([div.get_text() for div in soup.find_all("div")])
</code></pre>

<p>And, I personally think, <em>there is rarely a case</em> when regex is suitable for HTML parsing:</p>

<ul>
<li><a href="http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags">RegEx match open tags except XHTML self-contained tags</a></li>
</ul>

python html regex parsing