Parquet Files from Spark detected as directory in Linux
<p>I am trying to use Python's parquet module to read in some Parquet files written from a local MapR instance.</p>

<p>The command I used to output these parquet files is: </p>

<pre><code>df.sqlContext.sql("SQL HERE").write.format("parquet").option("mergeSchema", "true").save("/path/to/parquet/test.parquet")
</code></pre>

<p>This is what the file looks like on my Linux host:</p>

<pre><code>drwxr-xr-x  2 mapr mapr 403 Oct  5 13:56 igayfvpwrs.parquet
</code></pre>

<p>Unfortunately, when I use the Python here (<a href="https://pypi.python.org/pypi/parquet" rel="nofollow">https://pypi.python.org/pypi/parquet</a>) - I receive the following exception:</p>

<pre><code>IOError: [Errno 21] Is a directory: '/mnt/mapr/saw/user/mapr/igayfvpwrs.parquet'
</code></pre>

<p>Any idea? These files work great in MapR.</p>

<p>EDIT 2:</p>

<p>I was able to figure it out. Since the original .parquet "file" is a directory, just loop through the directory with glob for all the inner .parquet files - the original code for Python-parquet works in there. </p>

<pre><code>for filename in glob.glob("/mnt/mapr/saw/user/mapr/{0}.parquet/*.parquet".format(tempTableID)):
        with open(filename) as foo:
            for row in parquet.DictReader(foo, columns=["column"]):
                print(json.dumps(row))
</code></pre>

<p>EDIT:
Here is what is inside the parquet file:</p>

<pre><code>-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 _common_metadata
-rwxr-xr-x  1 mapr mapr 2.4K Oct  5 13:58 _metadata
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00000-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00001-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00002-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00003-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00004-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00005-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00006-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00007-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00008-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00009-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00010-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00011-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00012-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00013-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00014-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00015-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00016-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00017-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00018-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00019-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00020-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00021-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00022-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00023-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00024-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00025-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00026-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00027-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00028-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00029-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00030-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00031-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00032-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00033-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00034-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00035-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00036-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00037-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00038-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00039-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00040-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00041-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00042-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00043-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00044-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00045-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00046-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00047-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00048-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00049-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00050-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00051-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00052-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr 1.2K Oct  5 13:58 part-r-00053-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00054-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00055-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00056-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00057-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00058-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00059-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00060-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00061-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00062-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00063-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00064-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00065-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00066-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00067-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00068-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00069-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00070-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00071-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00072-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00073-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00074-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00075-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00076-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00077-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00078-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00079-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00080-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00081-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00082-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00083-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00084-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00085-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00086-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00087-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00088-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00089-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00090-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00091-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00092-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00093-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00094-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00095-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00096-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00097-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00098-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00099-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00100-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00101-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00102-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00103-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00104-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr 1.2K Oct  5 13:58 part-r-00105-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00106-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00107-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00108-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00109-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00110-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00111-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00112-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00113-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00114-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00115-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00116-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00117-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00118-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00119-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00120-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00121-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00122-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00123-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00124-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr 1.2K Oct  5 13:58 part-r-00125-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00126-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00127-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00128-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00129-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00130-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00131-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00132-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00133-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00134-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00135-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00136-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00137-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00138-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00139-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00140-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00141-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00142-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00143-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00144-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00145-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00146-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00147-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00148-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00149-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00150-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00151-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00152-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00153-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00154-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00155-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00156-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00157-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00158-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00159-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00160-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00161-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00162-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00163-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00164-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00165-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00166-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00167-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00168-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00169-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00170-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00171-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00172-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00173-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00174-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00175-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00176-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00177-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00178-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00179-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00180-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00181-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00182-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00183-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00184-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00185-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00186-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00187-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00188-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00189-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00190-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00191-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00192-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00193-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00194-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00195-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00196-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00197-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00198-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00199-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00200-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00201-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00202-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00203-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00204-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00205-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00206-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00207-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00208-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00209-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00210-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00211-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00212-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00213-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00214-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00215-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00216-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00217-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00218-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00219-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00220-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00221-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00222-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00223-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00224-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00225-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00226-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00227-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00228-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00229-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00230-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00231-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00232-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00233-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00234-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00235-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00236-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00237-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00238-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00239-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00240-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00241-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00242-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00243-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00244-02f0b80b-2993-46f8-a191-f111c6db1dad.gz.parquet
-rwxr-xr-x  1 mapr mapr  514 Oct  5 13:58 part-r-00245-02f0b80b-2993-46f8-
[truncated]
-rwxr-xr-x  1 mapr mapr    0 Oct  5 13:58 _SUCCESS
</code></pre>

<p>There is no issue with parquet on spark here. The DataFrameWriter writes to parquet format into a directory and partition the output according the number of partition of the DataFrame it is writing. </p>

<p>What you are getting is absolutely normal. </p>

python hadoop apache-spark parquet