HeartSaVioR commented on PR #42940: URL: https://github.com/apache/spark/pull/42940#issuecomment-1726600452
> As we are already in the status that sources not supporting Trigger.AvailableNow can work with the wrapper instead of simply failing since Trigger.Once is deprecated, it sounds reasonable to continue making it work under single batch execution. > > One option is to stop support of it in 4.0 (as it is deprecated since 3.4), but add fallback logic for previous branches which provide the support for now (as it is related to correctness)? Removing API would trigger a long discussion and debate. From my experience of Spark community (you've experienced longer than me :) ), I don't believe the community wants to remove the API unless there is a clear proof that few users use it. I think that's probably Spark 5.0, not 4.0. That said, we couldn't apply more aggressive alternatives. I don't feel like we can persuade community to decide existing query to fail, which users cannot make any change to mitigate (it would require 3rd party data source engineer to work on). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
