How to create new string column extracting integers from a timestamp in Spark?
<p>I have a spark dataframe with a timestamp column, I want a new column which has strings in the format "YYYYMM".</p>

<p>I tried with: </p>

<pre><code>df.withColumn('year_month',year(col("timestamp")).cast("string")+month(col("timestamp")).cast("string"))
</code></pre>

<p>But if my timestamp is 2016-10-12, it returns as 2020 as YYYYMM.</p>

<p>You can use <code>date_format</code>:</p>

<pre><code>from pyspark.sql.functions import date_format
df.withColumn('year_month', date_format('timestamp', 'yyyyMM'))
</code></pre>

python apache-spark pyspark spark-dataframe pyspark-sql