Hi, I remember previously that spark.sql(<your query>) wasn’t a final action and you would have needed to run something like show() for the query to actually being performed. Today I noticed that when I do just spark.sql(<my query>) without show() or anything , lots of executors are being fired and reading their logs shows they are actually opening files and reading them. Was there a change in spark 3 and later that changed the behavior? I am using spark 3.1.2. This happens even if I disable AQE. Thanks, S.
