Dataframe for HiveContext in spark is not callable
<p>I have the below spark script:</p>

<pre><code>from pyspark import SparkContext, SparkConf
from pyspark.sql import SQLContext, HiveContext
spark_context = SparkContext(conf=SparkConf())
sqlContext = HiveContext(spark_context)
outputPartition=sqlContext.sql("select * from dm_mmx_merge.PLAN_PARTITION ORDER BY PARTITION,ROW_NUM")
outputPartition.printSchema()
outputPartition.filter(outputPartition("partition")==3).show()
</code></pre>

<p>`</p>

<p>I get the output of schema as"</p>

<pre><code>root
 |-- seq: integer (nullable = true)
 |-- cpo_cpo_id: long (nullable = true)
 |-- mo_sesn_yr_cd: string (nullable = true)
 |-- prod_prod_cd: string (nullable = true)
 |-- cmo_ctry_nm: string (nullable = true)
 |-- cmo_cmo_stat_ind: string (nullable = true)
 |-- row_num: integer (nullable = true)
 |-- partition: long (nullable = true)
</code></pre>

<p>But i also get the error:
    <code>Traceback (most recent call last):
  File "hiveSparkTest.py", line 18, in &lt;module&gt;
    outputPartition.filter(outputPartition(partition)==3).show()
TypeError: 'DataFrame' object is not callable</code></p>

<p>I need the get the output for each partition value and do transformation. Any help would be highly appreciable.</p>

<p>In line </p>

<pre><code> outputPartition.filter(outputPartition(partition)==3).show()
</code></pre>

<p>you are trying to use outputPartition as a method.
Use</p>

<pre><code> outputPartition['partition'] 
</code></pre>

<p>instead of</p>

<pre><code> outputPartition(partition)
</code></pre>

python apache-spark pyspark