Issue in running hive udf written in python
<p>I have written a simple hive udf in python, but when I run it in hive shell, it throws below error:  </p>

<pre><code>Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
        at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:514)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
        ... 8 more


FAILED: Execution Error, return code 20003 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. An error occurred when trying to close the Operator running your custom script.
</code></pre>

<p>`</p>

<p><strong>Commands used in hive shell:</strong></p>

<p><code>add file /path/to/mycode.py;</code></p>

<p>created the table;</p>

<p>loaded the data;</p>

<p><code>SELECT TRANSFORM (fname,lname) USING 'python mycode.py' AS (fname,LNAME) from table;</code></p>

<p><strong>Hadoop version: 2.7</strong></p>

<p><strong>Hive: 0.13</strong></p>

<p><strong>mycode.py</strong></p>

<pre><code>#!/usr/bin/python
import sys
import string

try:
    for line in sys.stdin:
        lines = string.strip(line,'\n')
        fname,lname = string.split(lines,',')
        #print (fname,lname)
        LNAME = lname.lower()
        #print LNAME
        print([fname,LNAME])
except:
    print sys.exc_info()
</code></pre>

<p><strong>Error:</strong></p>

<p><code>(&lt;type 'exceptions.ValueError'&gt;, ValueError('need more than 1 value to unpack',), &lt;traceback object at 0x7f3441c14050&gt;)    NULL</code></p>

<p>Though, when I try <code>cat pyinp.txt | python mycode.py</code> it gives the desired output.</p>

<p>Can someone help me in resolving this issue?</p>

<p>This solved the problem for others cases like yours: <a href="http://themrmax.github.io/2015/07/16/a-python-nltk-wordnet-udf-in-hive.html" rel="nofollow">here</a> </p>

<p>They are using a try catch block into the function, did you do the same? </p>

python hive