How to get a name of Column or change the name of existing?
<p>I have a task to build a function "removePunctuation" that strips the punctuation and as result pass this test:</p>

<pre><code># TEST Capitalization and punctuation (4b)
testPunctDF = sqlContext.createDataFrame([(" The Elephant's 4 cats. ",)])
testPunctDF.show()
Test.assertEquals(testPunctDF.select(removePunctuation(col('_1'))).first()[0],
                  'the elephants 4 cats',
                  'incorrect definition for removePunctuation function')
</code></pre>

<p>This is what i managed to write. </p>

<pre><code>def removePunctuation(column):
    """Removes punctuation, changes to lower case, and strips leading and trailing spaces.

    Note:
        Only spaces, letters, and numbers should be retained.  Other characters should should be
        eliminated (e.g. it's becomes its).  Leading and trailing spaces should be removed after
        punctuation is removed.

    Args:
        column (Column): A Column containing a sentence.

    Returns:
        Column: A Column named 'sentence' with clean-up operations applied.
    """

    return lower(trim(regexp_replace("column_name", "[\W_]+"," "))).alias("sentence");
</code></pre>

<p>But i still can not make function regexp_replace to use alias "sentence". I am getting this error:</p>

<blockquote>
  <p>AnalysisException: u"cannot resolve 'sentence' given input columns:
  [_1];"</p>
</blockquote>

<p>Surprisingly I was able just to pass column object in <code>regexp_replace()</code> args instead of column name. </p>

python string apache-spark distributed-computing punctuation