Perhaps
finalDF.repartition(finalDF.rdd.getNumPartitions()).withColumn("status_for_batch
To
finalDF.repartition(finalDF.rdd.getNumPartitions()).withColumn(col("status_for_batch")
On Thu, 9 Jun 2022, 22:32 Sid, <flinkbyhe...@gmail.com> wrote:
> Hi Experts,
>
> I am facing one problem while passing a column to the method. The problem
> is described in detail here:
>
>
> https://stackoverflow.com/questions/72565095/how-to-pass-columns-as-a-json-record-to-the-api-method-using-pyspark
>
> TIA,
> Sid
>
