Number of rows in 40GB tar.gz file without uncompressing it?
<p>I have over 40 gb tar.gz file at <a href="https://ghtstorage.blob.core.windows.net/downloads/mysql-2016-06-16.tar.gz" rel="nofollow">https://ghtstorage.blob.core.windows.net/downloads/mysql-2016-06-16.tar.gz</a>
How can I find the number of rows in the CSV file that is compressed inside this tar.gz file without uncompressing the entire file which might be in 100+ GBs? </p>

<p>If there is only one csv file in that tar.gz, you could do this as a bash one-liner:</p>

<p><code>tar -zxOf mysql-2016-06-16.tar.gz | wc -l</code></p>

<p>It uses tar to extract all the files in the archive to standard output (-O, capital o, not zero), and wc to count the lines.</p>

<p>If there is more files, and only want that one file, you can count the lines in that file like this:</p>

<p><code>tar -zxOf mysql-2016-06-16.tar.gz mysql-2016-06-16/commit_comments.csv| wc -l</code></p>

<p>Here's how to list all files in the archive:</p>

<p><code>tar -zlf mysql-2016-06-16.tar.gz</code></p>

<p>CSV files usually have a header, so remove one line per file and you have the number of rows.</p>

python csv gzip tar