[GitHub] spark issue #20456: [SPARK-22624][PYSPARK] Expose range partitioning shuffle...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20456 **[Test build #86880 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86880/testReport)** for PR 20456 at commit

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20419 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/436/

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20419 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19340: [SPARK-22119][ML] Add cosine distance to KMeans

2018-01-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19340 @zhengruifeng yes I understand why the solutions aren't the same, though it depends on whether you think that's what k-_means_ is supposed to do or not. We're not actually maximizing an expectation

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20419 **[Test build #86882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86882/testReport)** for PR 20419 at commit

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-31 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20164 I might be missing something, but the user sets the raw prediction col name. Now it's going to use a different name than what the user set. Isn't it simply an error to apply a second model,

[GitHub] spark issue #20456: [SPARK-22624][PYSPARK] Expose range partitioning shuffle...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20456 **[Test build #86881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86881/testReport)** for PR 20456 at commit

[GitHub] spark issue #20456: [SPARK-22624][PYSPARK] Expose range partitioning shuffle...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20456 **[Test build #86880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86880/testReport)** for PR 20456 at commit

[GitHub] spark issue #20456: [SPARK-22624][PYSPARK] Expose range partitioning shuffle...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20456 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20456: [SPARK-22624][PYSPARK] Expose range partitioning shuffle...

2018-01-31 Thread xubo245
Github user xubo245 commented on the issue: https://github.com/apache/spark/pull/20456 @gatorsmile please review it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20456: [SPARK-22624][PYSPARK] Expose range partitioning ...

2018-01-31 Thread xubo245
GitHub user xubo245 opened a pull request: https://github.com/apache/spark/pull/20456 [SPARK-22624][PYSPARK] Expose range partitioning shuffle introduced by spark-22614 ## What changes were proposed in this pull request? Expose range partitioning shuffle introduced

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-01-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20455 Once map support is added later, we should also document `getMap`'s behavior. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20455 **[Test build #86878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86878/testReport)** for PR 20455 at commit

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20435 **[Test build #86879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86879/testReport)** for PR 20435 at commit

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/435/

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20455 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/434/

[GitHub] spark issue #20455: [SPARK-23284][SQL] Document the behavior of several Colu...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20455 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20449: [SPARK-23040][CORE]: Returns interruptible iterat...

2018-01-31 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/20449#discussion_r165066153 --- Diff: core/src/main/scala/org/apache/spark/shuffle/BlockStoreShuffleReader.scala --- @@ -104,9 +104,18 @@ private[spark] class

[GitHub] spark issue #11207: [SPARK-12583][Mesos] Mesos shuffle service: Don't delete...

2018-01-31 Thread IgorBerman
Github user IgorBerman commented on the issue: https://github.com/apache/spark/pull/11207 @bbossy thanks. I'm experiencing them too despite the merged PR. I think I have some direction why it fails, here is my setup: 0. v2.2.0 1. Dynamic allocation is on 2. min executors

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20454 **[Test build #86877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86877/testReport)** for PR 20454 at commit

[GitHub] spark pull request #20455: [SPARK-23284][SQL] Document several get API of Co...

2018-01-31 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/20455 [SPARK-23284][SQL] Document several get API of ColumnVector's behavior when accessing null slot ## What changes were proposed in this pull request? For some ColumnVector get APIs such as

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/433/

[GitHub] spark pull request #19984: [SPARK-22789] Map-only continuous processing exec...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19984#discussion_r165055237 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingRelation.scala --- @@ -75,6 +76,52 @@ case class

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20454 **[Test build #86875 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86875/testReport)** for PR 20454 at commit

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86875/ Test FAILed. ---

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/432/

[GitHub] spark issue #20449: [SPARK-23040][CORE]: Returns interruptible iterator for ...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20449 cc @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20449: [SPARK-23040][CORE]: Returns interruptible iterat...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20449#discussion_r165051362 --- Diff: core/src/main/scala/org/apache/spark/shuffle/BlockStoreShuffleReader.scala --- @@ -104,9 +104,18 @@ private[spark] class

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20454 **[Test build #86875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86875/testReport)** for PR 20454 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86876/testReport)** for PR 20448 at commit

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/431/

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20448#discussion_r165049675 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,36 +17,84 @@ package

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20448 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86867/ Test FAILed. ---

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20448 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20435: [SPARK-23268][SQL]Reorganize packages in data sou...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20435#discussion_r165048049 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -403,7 +403,7 @@ class MicroBatchExecution(

[GitHub] spark issue #20448: [SPARK-23203][SQL] make DataSourceV2Relation immutable

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20448 **[Test build #86867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86867/testReport)** for PR 20448 at commit

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20452 **[Test build #86874 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86874/testReport)** for PR 20452 at commit

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/430/

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20452 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/429/

[GitHub] spark pull request #20435: [SPARK-23268][SQL]Reorganize packages in data sou...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20435#discussion_r165047744 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceOffset.scala --- @@ -20,14 +20,15 @@ package

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20435 **[Test build #86873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86873/testReport)** for PR 20435 at commit

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86871/ Test FAILed. ---

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20454 **[Test build #86871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86871/testReport)** for PR 20454 at commit

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20454 @cloud-fan @jose-torres @rdblue --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-31 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20386 @rdblue @cloud-fan @jose-torres thanks for the comments! I was trying to make the API compatible with `onTaskCommit(taskCommit: TaskCommitMessage)` in `FileCommitProtocol` possible.

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20454 **[Test build #86871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86871/testReport)** for PR 20454 at commit

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20450 **[Test build #86872 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86872/testReport)** for PR 20450 at commit

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/428/

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20450 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/427/

[GitHub] spark issue #20454: [SPARK-23202][SQL] Add new API in DataSourceWriter: onDa...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20454 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20454: [SPARK-23202][SQL] Add new DataSourceWriter API: ...

2018-01-31 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/20454 [SPARK-23202][SQL] Add new DataSourceWriter API: onDataWriterCommit ## What changes were proposed in this pull request? Currently, the api

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20452 Overall LGTM. One question is, at #19980, we replace `anyNullsSet` with some predicate of `numNulls` (like `numNulls == 0` or `numNulls` > 0) in many places, shall we use `hasNull` now for those

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20453 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20453 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86870/ Test PASSed. ---

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20453 **[Test build #86870 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86870/testReport)** for PR 20453 at commit

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Thanks all for your review, greatly appreciated. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #11207: [SPARK-12583][Mesos] Mesos shuffle service: Don't delete...

2018-01-31 Thread bbossy
Github user bbossy commented on the issue: https://github.com/apache/spark/pull/11207 @IgorBerman correct, #11272 was a follow up on this and got merged. I still experience issues with the shuffle service on Mesos, but I haven't been able to pin point the cause. ---

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86869/ Test PASSed. ---

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20452 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20452 **[Test build #86869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86869/testReport)** for PR 20452 at commit

[GitHub] spark issue #20449: [SPARK-23040][CORE]: Returns interruptible iterator for ...

2018-01-31 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/20449 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86868/ Test PASSed. ---

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20450 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20450: [SPARK-23280][SQL] add map type support to ColumnVector

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20450 **[Test build #86868 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86868/testReport)** for PR 20450 at commit

[GitHub] spark pull request #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkS...

2018-01-31 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20404 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19340: [SPARK-22119][ML] Add cosine distance to KMeans

2018-01-31 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19340 @zhengruifeng I agree with you, but then we can also normalize the center points

[GitHub] spark issue #11207: [SPARK-12583][Mesos] Mesos shuffle service: Don't delete...

2018-01-31 Thread IgorBerman
Github user IgorBerman commented on the issue: https://github.com/apache/spark/pull/11207 Hi @bbossy just to make sure this PR wasn't merged in favor to https://github.com/apache/spark/pull/11272? --- - To

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20404 I completely agree with @HyukjinKwon. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 How about you @ueshin? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 I am fine with not fixing the case here for now if we go for it only with master branch. Maybe, we can try to fix the pre existing issue too seperately by syncrhoziing everything if anyone

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20452 LGTM, and the name `hasNull` sounds fine to me, too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20450: [SPARK-23280][SQL] add map type support to Column...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20450#discussion_r164997996 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -761,6 +761,43 @@ class ColumnarBatchSuite

[GitHub] spark pull request #20437: [SPARK-23270][Streaming][WEB-UI]FileInputDStream ...

2018-01-31 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20437#discussion_r164997663 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -157,7 +157,7 @@ class FileInputDStream[K, V, F <:

[GitHub] spark pull request #20437: [SPARK-23270][Streaming][WEB-UI]FileInputDStream ...

2018-01-31 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/20437#discussion_r164996836 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -157,7 +157,7 @@ class FileInputDStream[K, V, F

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-31 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Thanks Felix. I would incline to not fix the case mentioned by Felix. What's your opinion @HyukjinKwon @ueshin ? --- - To

[GitHub] spark pull request #20450: [SPARK-23280][SQL] add map type support to Column...

2018-01-31 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20450#discussion_r164996133 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -761,6 +761,43 @@ class ColumnarBatchSuite

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20453 **[Test build #86870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86870/testReport)** for PR 20453 at commit

[GitHub] spark pull request #20450: [SPARK-23280][SQL] add map type support to Column...

2018-01-31 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20450#discussion_r164995018 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -761,6 +761,43 @@ class ColumnarBatchSuite

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20453 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/426/

[GitHub] spark issue #20453: [SPARK-23281][SQL] Query produces results in incorrect o...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20453 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20453: [SPARK-23281][SQL] Query produces results in inco...

2018-01-31 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/20453 [SPARK-23281][SQL] Query produces results in incorrect order when a composite order by clause refers to both original columns and aliases ## What changes were proposed in this pull request?

[GitHub] spark pull request #20450: [SPARK-23280][SQL] add map type support to Column...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20450#discussion_r164993656 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -761,6 +761,43 @@ class ColumnarBatchSuite

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/425/

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20452 **[Test build #86869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86869/testReport)** for PR 20452 at commit

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20452 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-31 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20404 It’s your call - there are some risks but seems like there are pre existing code that has the same problem. ---

[GitHub] spark issue #20452: revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2018-01-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20452 cc @kiszk @ueshin @viirya --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20452: revert [SPARK-22785][SQL] remove ColumnVector.any...

2018-01-31 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20452 revert [SPARK-22785][SQL] remove ColumnVector.anyNullsSet ## What changes were proposed in this pull request? In https://github.com/apache/spark/pull/19980 , we thought `anyNullsSet`

[GitHub] spark pull request #20450: [SPARK-23280][SQL] add map type support to Column...

2018-01-31 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20450#discussion_r164986589 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -761,6 +761,43 @@ class ColumnarBatchSuite

<    1   2   3   4   5   6   7   >