beautifulsoup does not find all p with a certain class
<p>I am using beautifulsoup to find all p in a certain html page I saved locally.
My code is</p>

<pre><code>with open ("./" + str(filename) + ".txt", "r") as myfile:
    data=myfile.read().replace('\n', '')
soup = BeautifulSoup(data)
t11 = soup.findAll("p", {"class": "commentsParagraph"})
</code></pre>

<p>this codes works for part of the page, but some part of the page is loaded with ajax (which I preloaded before I saved the source), and the code is not working on it.</p>

<p>to test this I added to one of the <code>p</code> tags in the ajax portion the class <code>commentsParagraph2</code>, and changed my code to </p>

<pre><code>t11 = soup.findAll("p", {"class": "commentsParagraph2"})
</code></pre>

<p>but t11 is an empty list.</p>

<p>I am attaching the page file as well<a href="https://justpaste.it/z64t" rel="nofollow">here</a></p>

<p>Any ideas?</p>

<p>There is one p tag with the commentsParagraph2 class in your html which bs4 can find without issue using all three parsers:</p>

<pre><code>In [8]: from bs4 import BeautifulSoup
   ...: soup1 = BeautifulSoup(open("/home/padraic
   ...: /t.html").read(),"html5lib")
   ...: soup2 = BeautifulSoup(open("/home/padraic
   ...: /t.html"),"html.parser")
   ...: soup3 = BeautifulSoup(open("/home/padraic
   ...: /t.html"),"lxml")
   ...: print(soup1.select_one("p.commentsParagraph2"))
   ...: print(soup2.select_one("p.commentsParagraph2"))
   ...: print(soup3.select_one("p.commentsParagraph2"))
   ...: 
&lt;p class="commentsParagraph2"&gt;
So much better than Ryder. Only take Econ 11 if she's one of the professors teaching it. Beware her tests though, which are much different from Ryder's.
&lt;/p&gt;
&lt;p class="commentsParagraph2"&gt;
So much better than Ryder. Only take Econ 11 if she's one of the professors teaching it. Beware her tests though, which are much different from Ryder's.
&lt;/p&gt;
&lt;p class="commentsParagraph2"&gt;
So much better than Ryder. Only take Econ 11 if she's one of the professors teaching it. Beware her tests though, which are much different from Ryder's.
&lt;/p&gt;
</code></pre>

<p>So either you are using the broken and no longer maintained <em>BeautifulSoup3</em> or an older version of bs4.</p>

python beautifulsoup