Hi, I would like to confirm checkpointing behavior, I have observed following scenarios:
*1)* When I set checkpointLocation from streaming query like: val query = rateDF.writeStream.format("console").outputMode("append").trigger(Trigger.ProcessingTime("1 seconds")).*option("checkpointLocation", "/Users/shubham/checkpoint_from_query1")*.queryName("q2").start It generates all the metadata in */Users/shubham/checkpoint_from_query1 *regardless of whether queryName is set or not. *2)* When I set it from conf like: *spark.conf.set("spark.sql.streaming.checkpointLocation", "/Users/shubham/checkpoint_from_conf")* I observed two cases here: *2.1)* When I set the queryName like .queryName("q2"), it generates all metadata under */Users/shubham/checkpoint_from_conf/q2* *2.2)* When queryName is not set, it generates all metadata under */Users/shubham/checkpoint_from_conf/<some-random-uuid>* I have seen query successfully recovers in scenario *1)* and *2.1) *which is fine. It does not recover from *2.2) *which is also fine as it is unable to somehow get the query handle. Can there be any other possibility? Would like to confirm. Thanks, Shubham