Python ignore character and print next character from list in for loop
<p>I am using BeautifulSoup4 and requests to scrape information from a website.</p>

<p>I then store the required information in lists, there are two lists for two different types of information I scraped from the page.</p>

<pre><code>    try:
        for i in range(0,1000):
            location = dive_data1[((9*i)-7)].text
            locations.append(location)
            location = dive_data2[((9*i)-7)]
            locations.append(location)
            depth = dive_data1[((9*i)-6)].text
            depths.append(depth)
            depth = dive_data2[((9*i)-6)].text
            depths.append(depth)

    except:
        pass
</code></pre>

<p>After that I try to pass these lists into another for loop to write the contents into a CSV file.</p>

<pre><code>    try:
        writer = csv.writer(dive_log)
        writer.writerow( ("Locations and depths") )
        writer.writerow( ("Sourced from:", str(url_page)) )
        writer.writerow( ("Location", "Depth") )
        for i in range(len(locations)):
            writer.writerow( (locations[i], depths[i]) )
</code></pre>

<p>when i run the script i recieve this error:</p>

<pre><code>writer.writerow( (locations[i], depths[i]) )
UnicodeEncodeError: 'ascii' codec can't encode characters in position 65-66:      ordinal not in range(128)
</code></pre>

<p>I tried this to pass characters it cannot encode:</p>

<pre><code>    writer = csv.writer(dive_log)
    writer.writerow( ("Locations and depths") )
    writer.writerow( ("Sourced from:", str(url_page)) )
    writer.writerow( ("Location", "Depth") )
    for i in range(len(locations)):
        try:
            writer.writerow( (locations[i], depths[i]) )

        except:
            pass
</code></pre>

<p>When I run this, only the lines prior to the for loop are excecuted and it completely passes the repeat of the for loop.</p>

<p>The entirety of the code in my script is copied below in case it is related to something that I had not seen in the rest.</p>

<pre><code>import csv
from bs4 import BeautifulSoup
import requests

dive_log = open("divelog.csv", "wt")
url_page = "https://en.divelogs.de/log/Mark_Gosling"
r = requests.get(url_page)
soup = BeautifulSoup(r.content)

dive_data1 = soup.find_all("tr", {"class": "td2"})
dive_data2 = soup.find_all("td", {"class": "td"})
locations = []
depths = []

try:
    for i in range(0,1000):
        location = dive_data1[((9*i)-7)].text
        locations.append(location)
        location = dive_data2[((9*i)-7)]
        locations.append(location)
        depth = dive_data1[((9*i)-6)].text
        depths.append(depth)
        depth = dive_data2[((9*i)-6)].text
        depths.append(depth)

except:
    pass

try:
    writer = csv.writer(dive_log)
    writer.writerow( ("Locations and depths") )
    writer.writerow( ("Sourced from:", str(url_page)) )
    writer.writerow( ("Location", "Depth") )
    for i in range(len(locations)):
        try:
            writer.writerow( (locations[i], depths[i]) )

        except:
            pass

finally:
    dive_log.close()

print open("divelog.csv", "rt").read()
print "\n\n"
print locations
</code></pre>

<p>Like @yedpodtriztko noted. You can just leave the characters that it cannot decode with following:</p>

<p>instead of doing:</p>

<pre><code>soup = BeautifulSoup(r.content)
</code></pre>

<p>you could use this:</p>

<pre><code>soup = BeautifulSoup(r.content.decode('utf-8', 'ignore'))
</code></pre>

python beautifulsoup export-to-csv