How to copy and convert parquet files to csv
<p>I have access to a hdfs file system and can see parquet files with</p>

<pre><code>hadoop fs -ls /user/foo
</code></pre>

<p>How can I copy those parquet files to my local system and convert them to csv so I can use them?  The files should be simple text files with a number of fields per row.</p>

<p>Try</p>

<pre><code>df = spark.read.parquet("infile.parquet")
df.write.csv("outfile.csv")
</code></pre>

<p>Relevant API documentation:</p>

<ul>
<li><a href="http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrameReader.parquet" rel="nofollow">pyspark.sql.DataFrameReader.parquet</a></li>
<li><a href="http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrameWriter.csv" rel="nofollow">pyspark.sql.DataFrameWriter.csv</a></li>
</ul>

python hadoop apache-spark pyspark parquet