haoyangeng-db commented on PR #57043: URL: https://github.com/apache/spark/pull/57043#issuecomment-5097384606
This change makes me a bit nervous. If I'm reading it correctly, the code does not cover interfaces like `spark.sql()` and others like the DataFrame `checkpoint`, `localCheckpoint`, etc. (And this will be a concern when someone adds a new command.) Thinking more, I wonder if we can do something different: Add a **server-side** SQL conf on whether to skip the refresh; set it to true whenever we create a Spark session from the Spark Connect server (so this covers only the Spark Connect paths), and set refreshPhaseEnabled based on this new config? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
