HeartSaVioR commented on code in PR #42940: URL: https://github.com/apache/spark/pull/42940#discussion_r1329416576
########## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ########## @@ -2180,6 +2180,17 @@ object SQLConf { .booleanConf .createWithDefault(true) + val STREAMING_TRIGGER_AVAILABLE_NOW_WRAPPER_ENABLED = + buildConf("spark.sql.streaming.triggerAvailableNowWrapper.enabled") + .internal() + .doc("Whether to use the wrapper implementation of Trigger.AvailableNow if the source " + + "does not support Trigger.AvailableNow. Enabling this allows the benefits of " + + "Trigger.AvailableNow with sources which don't support it, but some sources " + + "may show unexpected behavior including duplication, data loss, etc. So use with " + Review Comment: This new config is marked as "internal", which is not going to be a part of public documentation. We expect very advanced users and/or operators only know about the existence of the config and use it as their own risk. We don't even expose this config in the warning message - that's the way I avoid users from shooting themselves on the foot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org