download specific columns of csv using requests.get
<p>I am using <code>requests.get</code> to download a csv file. I only need two columns from this csv file and the rest of the column are useless for me. Currently I am using </p>

<pre><code>r = requests.get(finalurl, verify=False,stream=True)
shutil.copyfileobj(r.raw, csvfile) 
</code></pre>

<p>to get the complete csv file.</p>

<p>However, I only want to download two column from the csv file. I can always download the entire content and then take what is necessary. </p>

<p>Just checking if there is a way to get specific column using <code>requests.get</code>
Eg: <a href="http://chart.finance.yahoo.com/table.csv?s=AAPL&amp;a=7&amp;b=20&amp;c=2016&amp;d=8&amp;e=20&amp;f=2016&amp;g=d&amp;ignore=.csv" rel="nofollow">http://chart.finance.yahoo.com/table.csv?s=AAPL&amp;a=7&amp;b=20&amp;c=2016&amp;d=8&amp;e=20&amp;f=2016&amp;g=d&amp;ignore=.csv</a> </p>

<p>I need only <code>date</code> and <code>Adj.close</code> from this csv file.</p>

<p>Couldn't find similar questions, please direct me if similar question was asked earlier.</p>

<p>Thanks</p>

<p>You cannot download certain columns only, you can with the regular finance api. You don't have to download all the data in one go either though and then replace after, you can parse as you go:</p>

<pre><code>import csv

final_url = "http://chart.finance.yahoo.com/table.csv?s=AAPL&amp;a=7&amp;b=20&amp;c=2016&amp;d=8&amp;e=20&amp;f=2016&amp;g=d&amp;ignore=.csv"
with open("out.csv", "w") as out:
    writer = csv.writer(out)
    data = requests.get(final_url, verify=False, stream=True).iter_lines()
    headers = fieldnames = next(data).split(",")
    reader = csv.DictReader(data, fieldnames=headers)
    writer.writerow(["Date", "Adj Close"])
    for row in reader:
        writer.writerow([row["Date"], row["Adj Close"]])
</code></pre>

<p>You could just index if the column order is guaranteed to never change but using the <em>DictReader</em> lets you access by key so order is irrelevant. I think it is also safe to presume there will not be any newlines nested in the data.</p>

python csv download python-requests