How could I call a User defined function from spark sql queries in pyspark?
<p>I need to call a function from my spark sql queries. I have tried udf but I don't know how to manipulate it.
Here is the scenario:</p>

<pre><code># my python function example

def sum(effdate, trandate):
  sum=effdate+trandate
  return sum
</code></pre>

<p>and my spark sql query is like:</p>

<pre><code>spark.sql("select sum(cm.effdate, cm.trandate)as totalsum, name from CMLEdG cm ....").show()
</code></pre>

<p>These lines are not my code but I am stating it as an example. How could I  call my sum function inside spark.sql(sql queries) for getting a result?
Could you please kindly suggest me any link or any comment compatible with pyspark?</p>

<p>Any help would be appreciated. </p>

<p>Thanks</p>

<p>Kalyan </p>

<p>Check this</p>

<pre><code>    &gt;&gt;&gt; from pyspark.sql.types import IntegerType
    &gt;&gt;&gt; sqlContext.udf.register("stringLengthInt", lambda x: len(x), IntegerType())
    &gt;&gt;&gt; sqlContext.sql("SELECT stringLengthInt('test')").collect()
    [Row(_c0=4)]
</code></pre>

python apache-spark pyspark apache-spark-sql spark-dataframe