[GitHub] spark pull request #19824: [SPARK][STREAMING] Invoke onBatchCompletion() onl...

2017-12-03 Thread victor-wong
Github user victor-wong closed the pull request at: https://github.com/apache/spark/pull/19824 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19824: [SPARK][STREAMING] Invoke onBatchCompletion() only when ...

2017-12-03 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/19824 @CodingCat Thank you:) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #19824: [SPARK][STREAMING] Invoke onBatchCompletion() only when ...

2017-12-01 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/19824 @CodingCat > One thing to note is that mute an event is a behavior change I agree with that, so we should be careful about changing the current behavior. I will close the PR la

[GitHub] spark issue #19824: [SPARK][STREAMING] Invoke onBatchCompletion() only when ...

2017-12-01 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/19824 @CodingCat please checkout the difference between the two PR. ` if (jobSet.hasCompleted) { - jobSets.remove(jobSet.time) - jobGenerator.onBatchCompletion

[GitHub] spark issue #19824: [SPARK][STREAMING] Invoke onBatchCompletion() only when ...

2017-11-30 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/19824 @CodingCat Yes, this PR wants to solve the same issue in https://github.com/apache/spark/pull/16542, but I think this is a better way to solve it. If a Job failed, I think we should

[GitHub] spark issue #19824: [SPARK][STREAMING] Invoke onBatchCompletion() only when ...

2017-11-30 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/19824 @viirya Sorry for the misleading title, I have changed it now. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #19824: Revert "[SPARK-18905][STREAMING] Fix the issue of...

2017-11-27 Thread victor-wong
GitHub user victor-wong opened a pull request: https://github.com/apache/spark/pull/19824 Revert "[SPARK-18905][STREAMING] Fix the issue of removing a failed jobset from JobScheduler.jobSets" ## What changes were proposed in this pull request? The code changes i

[GitHub] spark issue #17782: Reload credentials file config when app starts with chec...

2017-05-10 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/17782 @srowen Thanks for replying. I tested with master branch and it turned out the issue still existed. I create a new PR against master branch, https://github.com/apache/spark/pull/17937

[GitHub] spark issue #17937: Reload credentials file config when app starts with chec...

2017-05-10 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/17937 Comments on last PR, https://github.com/apache/spark/pull/17782. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #17782: Reload credentials file config when app starts wi...

2017-05-10 Thread victor-wong
Github user victor-wong closed the pull request at: https://github.com/apache/spark/pull/17782 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #17937: Reload credentials file config when app starts wi...

2017-05-10 Thread victor-wong
GitHub user victor-wong opened a pull request: https://github.com/apache/spark/pull/17937 Reload credentials file config when app starts with checkpoint file i… ## What changes were proposed in this pull request? Currently credentials file configuration is recovered from

[GitHub] spark issue #17782: Reload credentials file config when app starts with chec...

2017-05-09 Thread victor-wong
Github user victor-wong commented on the issue: https://github.com/apache/spark/pull/17782 @jerryshao Sorry for replying late. This issue does exist in master branch as I described in https://issues.apache.org/jira/browse/SPARK-19688. What should I do to elaborate more about

[GitHub] spark pull request #17782: Reload credentials file config when app starts wi...

2017-04-27 Thread victor-wong
GitHub user victor-wong opened a pull request: https://github.com/apache/spark/pull/17782 Reload credentials file config when app starts with checkpoint file i… ## What changes were proposed in this pull request? Currently credentials file configuration is recovered