How can I populate a txt with results from a mechanized results?
<p>I am trying to populate a txt file with the response I get from a mechanized form. Here's the form code</p>

<pre><code>import mechanize
from bs4 import BeautifulSoup
br = mechanize.Browser()
br.open ('https://www.cpsbc.ca/physician_search')

first = raw_input('Enter first name: ')
last = raw_input('Enter last name: ')

br.select_form(nr=0)
br.form['filter[first_name]'] = first
br.form['filter[last_name]'] = last
response = br.submit()
content = response.read()
soup = BeautifulSoup(content, "html.parser")

for row in soup.find_all('tbody'):
    print row
</code></pre>

<p>This spits out lines of html code depending on how many privileges the doc has in regards to locations, but the last line has their specialty of training. Please go ahead and test it with any physician from BC, Canada.</p>

<p>I have a txt file that is listed as such:</p>

<pre><code>lastname1, firstname1
lastname2, firstname2
lastname3, firstname3 middlename3
lastname4, firstname4 middlename4
</code></pre>

<p>I hope you get the idea. I would appreciate any help in automatizing the following steps:</p>

<p>go through txt with names one by one and record the output text into a new txt file.</p>

<p>So far, I have this working to spit out the row (which is raw html), which I don't mind, but I can't get it to write into a txt file...</p>

<pre><code>import mechanize
from bs4 import BeautifulSoup

with open('/Users/s/Downloads/hope.txt', 'w') as file_out:
    with open('/Users/s/Downloads/names.txt', 'r') as file_in:
        for line in file_in:
            a = line
            delim = ", "
            i1 = a.find(delim)

            br = mechanize.Browser()
            br.open('https://www.cpsbc.ca/physician_search')

            br.select_form(nr=0)
            br.form['filter[first_name]'] = a[i1+2:]
            br.form['filter[last_name]'] = a[:i1]
            response = br.submit()
            content = response.read()
            soup = BeautifulSoup(content, "html.parser")

            for row in soup.find_all('tbody'):
                print row
</code></pre>

<p>This should not be too complicated. Assuming your file with all the names you want to query upon is called "names.txt" and the output file you want to create is called "output.txt", the code should look something like:</p>

<pre><code>with open('output.txt', 'w') as file_out:
    with open('names.txt', 'r') as file_in:
        for line in file_in:
            &lt;your parsing logic goes here&gt;
            file_out.write(new_record)
</code></pre>

<p>This assumes your parsing logic generates some sort of "record" to be written on file as a string.</p>

<p>If you get more advanced, you can also look into the <code>csv</code> module to import/export data in CSV.</p>

<p>Also have a look at the <a href="https://docs.python.org/2/tutorial/inputoutput.html" rel="nofollow">Input and Output tutorial</a>.</p>

python mechanize