python spark change dataframe column data type to int error
<p>I want to cast the column type to int and get the first 3 rows</p>

<pre><code>    df.withColumn("rn", rowNumber().over(windowSpec).cast('int')).where("rn"&lt;=3).drop("rn").show()
</code></pre>

<p>but I  this error </p>

<pre><code>TypeError: unorderable types: str() &lt;= int()
</code></pre>

<p>The error is here:</p>

<pre><code>.where("rn"&lt;=3)
</code></pre>

<p>And here's how you can figure that out if you ever encounter a similar problem in the future. Following</p>

<pre><code>TypeError: unorderable types: str() &lt;= int()
</code></pre>

<p>is a Python exception and there is no <code>Py4JError</code>. This typically means you can dismiss JVM issues and focus on core Python. The only part of your code where you explicitly compare things is:</p>

<pre><code>"rn" &lt;= 3
</code></pre>

<p>If you want it to be a SQL literal you should pass a string:</p>

<pre><code>.where("rn &lt;= 3")
</code></pre>

<p>If you want <code>rn</code> to be resolved as a column use <code>col</code> function:</p>

<pre><code>from pyspark.sql.functions import col

.where(col("rn") &lt;= 3)
</code></pre>

<p>Also <code>rowNumber</code> function has been removed in the latest release. You should use <code>row_number</code> for a forward compatibility.</p>

python apache-spark types casting