I need to extract a value from a PySpark structured streaming Dataframe to a string variable to check something.
I tried this code. agentName = kinesisDF.select(kinesisDF.agentName.getItem(0).alias("agentName")).collect()[0][0] This works on a non-streaming Dataframe only. In a streaming Dataframe, collect is not supported. Any workaround for this? Nick