[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #4121 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4121/testReport)** for PR 20650 at commit

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20645 I like the functionality, not necessarily the implementation. Other configs would benefit from this, and having this functionality more easily available without having to change the call sites would

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #4122 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4122/testReport)** for PR 20650 at commit

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20648 **[Test build #87603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87603/testReport)** for PR 20648 at commit

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20648 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/999/

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20648 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20648 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87600/ Test FAILed. ---

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20648 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20648 **[Test build #87600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87600/testReport)** for PR 20648 at commit

[GitHub] spark issue #20442: [SPARK-23265][ML]Update multi-column error handling logi...

2018-02-21 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/20442 Sorry for not working on this earlier. Just came back from China yesterday morning. Not sure if 2.3 RC4 has already get cut. If this still needs to be merged in 2.3, please let me know and I

[GitHub] spark issue #20652: [SPARK-23476][SPARK-SHELL] Generate secret in local mode...

2018-02-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20652 LGTM, but the title should say `[core]` (not spark-shell). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87596/ Test PASSed. ---

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #87596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87596/testReport)** for PR 20650 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #4120 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4120/testReport)** for PR 20650 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #4119 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4119/testReport)** for PR 20650 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #4118 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4118/testReport)** for PR 20650 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #4116 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4116/testReport)** for PR 20650 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #4117 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4117/testReport)** for PR 20650 at commit

[GitHub] spark pull request #20654: [SPARK-23481][WebUI]lastStageAttempt should fail ...

2018-02-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20654 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20654: [SPARK-23481][WebUI]lastStageAttempt should fail when a ...

2018-02-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20654 Thanks! Merging to master and 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20654: [SPARK-23481][WebUI]lastStageAttempt should fail when a ...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20654 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87595/ Test PASSed. ---

[GitHub] spark issue #20654: [SPARK-23481][WebUI]lastStageAttempt should fail when a ...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20654 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20654: [SPARK-23481][WebUI]lastStageAttempt should fail when a ...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20654 **[Test build #87595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87595/testReport)** for PR 20654 at commit

[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...

2018-02-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20656 LGTM pending tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20655: [SPARK-23484][SS] Fix possible race condition in ...

2018-02-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20655 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20655 Merging to master and 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20656 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20656 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/998/

[GitHub] spark issue #20598: [SPARK-23406] [SS] Enable stream-stream self-joins

2018-02-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20598 This is a slightly invasive change in the engine, hence I was not planning to backport. Also, I realized after this PR that self-joins mostly actually work except in few corner cases which I

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20655 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87598/ Test PASSed. ---

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20655 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20655 **[Test build #87598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87598/testReport)** for PR 20655 at commit

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20651 @mgaido91 Spark 2.2 doesn't show skipped stages either. I think this is fine since most of people don't care about them. I posted the root cause and fixed in

[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20656 **[Test build #87602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87602/testReport)** for PR 20656 at commit

[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...

2018-02-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20656 cc @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20656: [SPARK-23475][WebUI]Skipped stages should be evic...

2018-02-21 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/20656 [SPARK-23475][WebUI]Skipped stages should be evicted before completed stages ## What changes were proposed in this pull request? The root cause of missing completed stages is because

[GitHub] spark issue #20598: [SPARK-23406] [SS] Enable stream-stream self-joins

2018-02-21 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/20598 Yeah, this seems risky at RC5. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20653: [SPARK-23459][SQL] Improve the error message when...

2018-02-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20653#discussion_r169801197 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -486,7 +486,8 @@ object PartitioningUtils {

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-21 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r169799465 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -107,17 +106,24 @@ case class

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20655 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87594/ Test PASSed. ---

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20653 **[Test build #87594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87594/testReport)** for PR 20653 at commit

[GitHub] spark issue #20652: [SPARK-23476][SPARK-SHELL] Generate secret in local mode...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20652 **[Test build #87601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87601/testReport)** for PR 20652 at commit

[GitHub] spark pull request #20652: [SPARK-23476][SPARK-SHELL] Generate secret in loc...

2018-02-21 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20652#discussion_r169797492 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -529,7 +529,8 @@ private[spark] class SecurityManager( return

[GitHub] spark issue #20649: [SPARK-23462][SQL] improve missing field error message i...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20649 **[Test build #87599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87599/testReport)** for PR 20649 at commit

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/997/

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20648 **[Test build #87600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87600/testReport)** for PR 20648 at commit

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20648 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20652: [SPARK-23476][SPARK-SHELL] Generate secret in local mode...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20652 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20652: [SPARK-23476][SPARK-SHELL] Generate secret in local mode...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20652 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87593/ Test PASSed. ---

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20655 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20652: [SPARK-23476][SPARK-SHELL] Generate secret in local mode...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20652 **[Test build #87593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87593/testReport)** for PR 20652 at commit

[GitHub] spark issue #20649: [SPARK-23462][SQL] improve missing field error message i...

2018-02-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20649 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20655 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/996/

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20655 **[Test build #87598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87598/testReport)** for PR 20655 at commit

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20655 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20648 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20655: [SPARK-23484][SS] Fix possible race condition in KafkaCo...

2018-02-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20655 @zsxwing thanks for finding this. @jose-torres PTAL. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20655: [SPARK-23484][SS] Fix possible race condition in ...

2018-02-21 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/20655 [SPARK-23484][SS] Fix possible race condition in KafkaContinuousReader ## What changes were proposed in this pull request? var `KafkaContinuousReader.knownPartitions` should be threadsafe as

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #87597 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87597/testReport)** for PR 20650 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/995/

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecutive of...

2018-02-21 Thread justinrmiller
Github user justinrmiller commented on the issue: https://github.com/apache/spark/pull/20572 Otherwise the job crashes, even for gaps of one or two messages on a couple of partitions (and while fail hard is a good philosophy, in this case it's not really practical for us) so it's

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20650 I'm not sure I agree with all the comments on the previous PR, but I agree that this also works. As discussed, the downside to this approach is that people in the future can continue

[GitHub] spark pull request #20652: [SPARK-23476][SPARK-SHELL] Generate secret in loc...

2018-02-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20652#discussion_r169736571 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -529,7 +529,8 @@ private[spark] class SecurityManager( return

[GitHub] spark pull request #20627: [SPARK-23217][ML][PYTHON] Add distanceMeasure par...

2018-02-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20627 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-21 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20539 LGTM merging with branch-2.2 Thanks @zhengruifeng ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20539 **[Test build #4115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4115/testReport)** for PR 20539 at commit

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-02-21 Thread IgorBerman
Github user IgorBerman commented on the issue: https://github.com/apache/spark/pull/20640 @squito, @susanxhuynh no worries, I have my internal patch. As a side note, I'm starting to think that my first proposition of introducing new config that will just override this

[GitHub] spark issue #20622: [SPARK-23441][SS] Remove queryExecutionThread.interrupt(...

2018-02-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20622 Unfortunately, we havent figured out any good to avoid that for MicroBatchExecution till now. The streaming thread is doing a whole lot of different things, and interrupting is the only reliable way

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87592/ Test FAILed. ---

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20650 **[Test build #87596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87596/testReport)** for PR 20650 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/994/

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87591/ Test PASSed. ---

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20057 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #87592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87592/testReport)** for PR 19222 at commit

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20057 **[Test build #87591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87591/testReport)** for PR 20057 at commit

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20650 @zsxwing can you also take a look? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20654: [SPARK-23481][WebUI]lastStageAttempt should fail when a ...

2018-02-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20654 @marmbrus Thanks! Fixed the title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20622: [SPARK-23441][SS] Remove queryExecutionThread.interrupt(...

2018-02-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20622 > Using a thread interrupt to kill the stream appears to be a fundamentally fragile solution, requiring us to maintain a whitelist of exceptions we think Spark execution might surface in response

[GitHub] spark issue #20611: [SPARK-23425][SQL]When wild card is been used in load co...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20611 **[Test build #87590 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87590/testReport)** for PR 20611 at commit

[GitHub] spark pull request #20653: [SPARK-23459][SQL] Improve the error message when...

2018-02-21 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/20653 [SPARK-23459][SQL] Improve the error message when unknown column is specified in partition columns ## What changes were proposed in this pull request? This PR avoids to print schema

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169764458 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +291,34 @@ private[tree] class LearningNode( } } +

[GitHub] spark issue #20611: [SPARK-23425][SQL]When wild card is been used in load co...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20611 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20604: [SPARK-23365][CORE] Do not adjust num executors when kil...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20604 **[Test build #87589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87589/testReport)** for PR 20604 at commit

[GitHub] spark issue #20654: [PARK-23481][WebUI]lastStageAttempt should fail when a s...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20654 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/993/

[GitHub] spark issue #20652: [SPARK-23476][SPARK-SHELL] Generate secret in local mode...

2018-02-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20652 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169737984 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +291,34 @@ private[tree] class LearningNode( } } +

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/992/

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-21 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169736202 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +291,34 @@ private[tree] class LearningNode( } } +

[GitHub] spark issue #20652: [SPARK-23476][SPARK-SHELL] Generate secret in local mode...

2018-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20652 **[Test build #87593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87593/testReport)** for PR 20652 at commit

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169749740 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +291,34 @@ private[tree] class LearningNode( } } +

[GitHub] spark issue #20654: [PARK-23481][WebUI]lastStageAttempt should fail when a s...

2018-02-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20654 cc @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20651#discussion_r169734852 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala --- @@ -172,6 +185,20 @@ private[ui] class AllStagesPage(parent: StagesTab)

[GitHub] spark issue #20622: [SPARK-23441][SS] Remove queryExecutionThread.interrupt(...

2018-02-21 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20622 The alternate solution doesn't solve the problem SPARK-23441 sets out to address. Using a thread interrupt to kill the spark job is a fundamentally fragile solution, requiring us to maintain a

<    1   2   3   4   >