GitHub user efimpoberezkin opened a pull request: https://github.com/apache/spark/pull/21392
[SPARK-24063][SS] Control maximum epoch backlog for ContinuousExecution ## What changes were proposed in this pull request? This pull request adds maxEpochBacklog SQL configuration option. EpochCoordinator tracks if the length of the queue of waiting epochs has exceeded it. If so, stream is stopped with an error indicating too many epochs stacked up ## How was this patch tested? Existing unit tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/efimpoberezkin/spark pr/control-epoch-backlog Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/21392.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21392 ---- commit 1d2fc298284f6d553d78035f3095e5d2abe2a8a8 Author: Efim Poberezkin <efim@...> Date: 2018-04-25T13:01:29Z Add max epoch backlog option to SQLConf commit 0919b3f7542aa0a807b0ac56e0da1366f347bb54 Author: Efim Poberezkin <efim@...> Date: 2018-05-07T10:11:21Z Replace logging an error with throwing an exception ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org