Open a csv.gz file in Python and print first 100 rows
<p>I'm trying to get only the first 100 rows of a csv.gz file that has over 4 million rows in Python. I also want information on the # of columns and the headers of each. How can I do this? </p>

<p>I looked at <a href="http://stackoverflow.com/questions/10566558/python-read-lines-from-compressed-text-files">python: read lines from compressed text files</a> to figure out how to open the file but I'm struggling to figure out how to actually print the first 100 rows and get some metadata on the information in the columns. </p>

<p>I found this <a href="https://stackoverflow.com/questions/1767513/read-first-n-lines-of-a-file-in-python">Read first N lines of a file in python</a> but not sure how to marry this to opening the csv.gz file and reading it without saving an uncompressed csv file. </p>

<p>I have written this code:</p>

<pre><code>import gzip
import csv
import json
import pandas as pd


df = pd.read_csv('google-us-data.csv.gz', compression='gzip', header=0,    sep=' ', quotechar='"', error_bad_lines=False)
for i in range (100):
print df.next() 
</code></pre>

<p>I'm new to Python and I don't understand the results. I'm sure my code is wrong and I've been trying to debug it but I don't know which documentation to look at. </p>

<p>I get these results (and it keeps going down the console - this is an excerpt): </p>

<pre><code>Skipping line 63: expected 3 fields, saw 7
Skipping line 64: expected 3 fields, saw 7
Skipping line 65: expected 3 fields, saw 7
Skipping line 66: expected 3 fields, saw 7
Skipping line 67: expected 3 fields, saw 7
Skipping line 68: expected 3 fields, saw 7
Skipping line 69: expected 3 fields, saw 7
Skipping line 70: expected 3 fields, saw 7
Skipping line 71: expected 3 fields, saw 7
Skipping line 72: expected 3 fields, saw 7
</code></pre>

<p>The first answer you linked suggests using <a href="https://docs.python.org/3/library/gzip.html#gzip.GzipFile" rel="nofollow"><code>gzip.GzipFile</code></a> - this gives you a file-like object that decompresses for you on the fly.</p>

<p>Now you just need some way to parse csv data out of a file-like object ... like <a href="https://docs.python.org/3/library/csv.html#csv.reader" rel="nofollow">csv.reader</a>.</p>

<p>The <code>csv.reader</code> object will give you a list of fieldnames, so you know the columns, their names, and how many there are.</p>

<p>Then you need to get the first 100 csv row objects, which will work exactly like in the second question you linked, and each of those 100 objects will be a list of fields.</p>

<p>So far this is all covered in your linked questions, apart from knowing about the existence of the csv module, which is listed in the <a href="https://docs.python.org/3/library/index.html" rel="nofollow">library index</a>.</p>

python csv