Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11440#issuecomment-190568465
Thanks @jerryshao for suggestion!
> Jobs generated in the down time can be used for WAL replay, did you test
when these down jobs are removed, the behavior of WAL replay is still correct?
It seems that the `pendingTimes` is use for WAL replay, i do not skip these
batches
> Also for some windowing operations, I think this removal of down time
jobs may possibly lead to the inconsistent result of windowing aggregation.
Does inconsistent result mean wrong result?
Also, i will running the unit test with the config set to true by default
in my local computer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
