unable to iterate over elements in selenium python
<p>Im a selenium noob and have been struggling to get things done with python.
Im trying to iterate over all user reviews("partial_entry" class) from this page <a href="https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-or560-TAP-Portugal#REVIEWS" rel="nofollow">https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-or560-TAP-Portugal#REVIEWS</a></p>

<pre><code>from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome("C:\Users\shalini\Downloads\chromedriver_win32\chromedriver.exe")
driver.maximize_window()
url="https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-or560-TAP-Portugal#REVIEWS"
driver.get(url)

for i in driver.find_elements_by_xpath("//div[@class='wrap']"):
    print i.find_element(By.XPATH, '//p[@class="partial_entry"]')
        print i.text
    print "=============================================="
    # THIS IF BLOCK IS NECESSARY, I CANT DO AWAY WITH THIS ONE
    if i.find_elements(By.CSS_SELECTOR,"#REVIEWS .googleTranslation&gt;.link"):
        print "======YES TRANSLATION AVAILABLE========"
</code></pre>

<p>Even though Im selecting a different element each time in for loop, but its printing the same element over and over again. (I have to keep the last if block &amp; cant do away with it, so whatever the solution be it has to include that if block also)</p>

<p>======EDIT===================</p>

<p>Even this isnt working (which should actually work, according to <a href="http://selenium-python.readthedocs.io/locating-elements.html" rel="nofollow">http://selenium-python.readthedocs.io/locating-elements.html</a> ) . I dont know whats going on with selenium !!!!!</p>

<pre><code>print i.find_element(By.CSS_SELECTOR, 'p.partial_entry')
</code></pre>

<p>Output:</p>

<pre><code>NoSuchElementException:
</code></pre>

<p><strong>1.</strong> The reason you keep getting the first element repeatedly while iterating over <code>i.find_element(By.XPATH, '//p[@class="partial_entry"]')</code> in the 2nd loop is that the beginning <code>//</code> tries to locate elements from the root/top level, <em>not</em> as a descendant element of <code>i</code>. So that just keeps returning the first <code>p.partial_entry</code> element for each iteration of the outer loop.</p>

<p>To search for descendant elements of <code>i</code> which match <code>p[@class="partial_entry"]</code>, the xpath should start with <code>.//</code>. That's what the dot does.</p>

<p><strong>2.</strong> For the line <code>print i.find_element(By.CSS_SELECTOR, 'p.partial_entry')</code>:<br>
The single <code>find_element</code> either return the first found element or throw an error if none are found. There are some 'div.wrap's which don't have that descendant element, so you will get <code>NoSuchElementException</code>.</p>

<p>The <code>find_elements</code> (note the 's') methods return a list of elements or an empty list if none are found, rather than an error.</p>

<p>So putting all that together:</p>

<pre><code>&gt;&gt;&gt; for i in driver.find_elements_by_xpath("//div[@class='wrap']"):
...     for ent in i.find_elements_by_xpath('.//p[@class="partial_entry"]'):
...         print ent.text
...         if i.find_elements_by_css_selector('#REVIEWS .googleTranslation&gt;.link'):
...             print 'translation available'
...     print  # output clarity
...
</code></pre>

<hr>

<p>Btw, why are you mixing things like <code>find_elements_by_xpath('...')</code> along with <code>find_element(By.XPATH, '...')</code>? Stick to one pattern.</p>

python selenium web-scraping