Spark can't read an Orc table (returns empty table)
<p>Do I have to do anything special to be able to read Orc tables with Spark?</p>

<p>I have two table copies in txt and orc. When reading txt table everything is ok. When reading orc table I get no errors but spark returns an empty table. </p>

<p>Here is my code in python:</p>

<pre><code>import pyspark
CONF = (pyspark.SparkConf().setMaster("yarn-client"))
sc = pyspark.SparkContext(conf = CONF)

from pyspark.sql import HiveContext

sq = HiveContext(sc)
df = sq.sql(""" select * from sample_07 """)

print df.show(10)
</code></pre>

<p><strong>UPDATE:</strong></p>

<p>If I specify the path to data stored in sample_07 and register it as temporary table, it works though:</p>

<pre><code>sq = HiveContext(sc)
orcFile = sq.read.format("orc").load("/user/some/sample_07/")
orcFile.registerTempTable("tempTableName");
df = sq.sql("SELECT * FROM tempTableName  LIMIT 10 ")
</code></pre>

<p>I don't think there is anything specific for ORC. can you run query on hive and make sure data is read properly. The empty table might be because hive cannot read the data the way you defined.</p>

python apache-spark orc