Hi, I am looking for approaches to compare a row with the next one to determine, e.g., differences in event-times/timestamps. I just found a couple of solutions that use Window class, but that does not seem to work on streaming data, such as https://stackoverflow.com/questions/38229659/pyspark-compare-two-rows-in-dataframe https://stackoverflow.com/questions/32148208/how-do-i-compare-multiple-rows-of-a-table-using-spark-sql-data-frames-udf
I got: "org.apache.spark.sql.AnalysisException: Non-time-based windows are not supported on streaming DataFrames/Datasets;;..." It would be great if anybody could give me a hint how to determine row differences in python on streams comming from kafka. Thanks and best regards, on --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
