How to web scrape data using Python from an html table and store it in a csv file. I am able to extract some parts but not the others
<p>I am beginner in Web scraping and I have become very much interested in the process. I set for myself a Project that can keep me motivated till I completed the project.</p>

<p><strong>My Project</strong></p>

<p>My Aim is to write a Python Program that goes to my university results page and scrape all the results of a range of students and store each of their marks in each subject in a .csv file or , delimited text file. I have gotten the code working to submit the post request to the .asp page. I would appreciate it if you could guide me on how to store the subject wise details in separate columns like:</p>

<p><strong>Desired Output:</strong></p>

<p>Sl.no,Name,Subject1,Subject2,Subject3,Subject4,Subject5,Subject6,..etc</p>

<p>1,Jason,8,9,8,8,8,9..etc</p>

<p>2,Peter,6,8,9,8,7,7..etc</p>

<p>.</p>

<p>. </p>

<p>.</p>

<p>for a series of exam numbers.</p>

<p><strong>Some Sample Data to try it out</strong></p>

<p><strong>The Results Website</strong>: <a href="http://result.pondiuni.edu.in/candidate.asp" rel="nofollow">http://result.pondiuni.edu.in/candidate.asp</a></p>

<p><strong>Register Number</strong>: 15te1218</p>

<p><strong>Degree</strong>: BTHEE</p>

<p><strong>Exam</strong>: Second</p>

<p>Could anyone give me directions on how I am to accomplish the task?
 Please correct me and would be awesome if you could guide me to solve the problem.</p>

<p>Can this be done in a much more simple way ?</p>

<p>In the code below you can see that I have tried to print out the name of the student but it returns an empty set(doesn't work). and i don't want it to return the data as a set because there is only one occurrence of that detail. </p>

<p>I do not know how to extract the Subject Names and the corresponding mark of that student from the html table in the results page. Some help with this is needed.</p>

<p><strong>Code:</strong> </p>

<pre><code>import requests
from bs4 import BeautifulSoup 
import re
import csv

for x in xrange(44,47):

    EXAMNO ='15te12'+str(x)
    print EXAMNO

    data = {"txtregno": EXAMNO,
        "cmbdegree": r"BTHEE~\BTHEE\result.mdb", # use raw strings
        "cmbexamno": "B",
        "dpath": r"\BTHEE\result.mdb",
        "dname": "BTHEE",
        "txtexamno": "B"}

results_page = requests.post("http://result.pondiuni.edu.in/ResultDisp.asp", data=data).content
soup = BeautifulSoup(results_page, 'html.parser').prettify()
regpa= "&lt;!--Percentage / S.G.P.A : &lt;b&gt;(.+?) &lt;/b&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; --&gt;"
patterngpa =re.compile(regpa)
gpa=re.findall(patterngpa,soup)
print gpa
rename="&lt;font size=3 color=black&gt;(.+?)&lt;/font&gt;"
patternname=re.compile(rename)
name=re.findall(patternname,soup)
    print (name)
</code></pre>

<p><strong>OUTPUT:</strong></p>

<pre><code>15te1244
[u'8.67']
15te1245
[u'8.8']
[]
15te1246
[u'7.8']
[]
</code></pre>

<p>Would be helpful if you could show me how to print it in the desired output format.</p>

<p>Thanks.</p>

<p>Took a lot of time to find a brute force solution.</p>

<pre><code>import requests
from bs4 import BeautifulSoup 
import re
import csv
for x in xrange(44,47):
    EXAMNO ='15te12'+str(x)
    data = {"txtregno": EXAMNO,
    "cmbdegree": r"BTHEE~\BTHEE\result.mdb", # use raw strings
    "cmbexamno": "B",
    "dpath": r"\BTHEE\result.mdb",
    "dname": "BTHEE",
    "txtexamno": "B"}
    results_page = requests.post("http://result.pondiuni.edu.in/ResultDisp.asp", data=data).content
    soup = BeautifulSoup(results_page, 'html.parser').prettify()
    string=str(BeautifulSoup(results_page, 'html.parser'))
    regpa= "&lt;!--Percentage / S.G.P.A : &lt;b&gt;(.+?) &lt;/b&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; --&gt;"
    print (re.search(regpa,string,re.M|re.I )).group(1) 
    regname="&lt;b&gt;Name of the student : &lt;b&gt;&lt;font color=\"black\" size=\"3\"&gt;(.*)&lt;/font&gt;&lt;/b&gt;&lt;/b&gt;"
    print (re.search(regname,string,re.M|re.I )).group(1)
    regsub="66%\"&gt;&lt;font color=\"black\" face=\"arial\" size=\"2\"&gt;(.*)&lt;/font&gt;&lt;/td&gt;"
    matches=(re.findall(regsub,string,re.M|re.I ))

    for i in xrange(len(matches)):
        regsubm="&gt;"+matches[i]+"&lt;/font&gt;&lt;/td&gt;\n&lt;td align=\"center\" bgcolor=\"white\" width=\"2%\"&gt;&lt;font color=\"black\" face=\"arial\" size=\"2\"&gt;..&lt;/font&gt;&lt;/td&gt;\n&lt;td align=\"center\" bgcolor=\"white\" width=\"7%\"&gt;&lt;font color=\"black\" face=\"arial\" size=\"2\"&gt;[\xc2]?[\xa0]?[\xc2]?[\xa0]?-&lt;/font&gt;&lt;/td&gt;\n&lt;td align=\"center\" bgcolor=\"white\" width=\"1%\"&gt;&lt;font color=\"black\" face=\"arial\" size=\"2\"&gt;-&lt;/font&gt;&lt;/td&gt;\n&lt;td align=\"center\" bgcolor=\"white\" width=\"5%\"&gt;&lt;font color=\"black\" face=\"arial\" size=\"2\"&gt;-&lt;/font&gt;&lt;/td&gt;\n&lt;td align=\"center\" bgcolor=\"white\" width=\"5%\"&gt;&lt;font color=\"black\" face=\"arial\" size=\"2\"&gt;(.*)&lt;/font&gt;"
        matchesm=re.findall(regsubm,string,re.M)
        print matches[i],'---&gt;',matchesm[0]
</code></pre>

python html csv asp-classic web-scraping