[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90323/ Test PASSed. ---

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #90323 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90323/testReport)** for PR 19602 at commit

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90319/ Test PASSed. ---

[GitHub] spark issue #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol binding to...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21066 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90319/testReport)** for PR 21106 at commit

[GitHub] spark issue #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol binding to...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21066 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3010/

[GitHub] spark issue #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol binding to...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21066 **[Test build #90333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90333/testReport)** for PR 21066 at commit

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting hostPath volumes

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21260 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21095: [SPARK-23529][K8s] Support mounting hostPath volumes

2018-05-07 Thread andrusha
Github user andrusha commented on the issue: https://github.com/apache/spark/pull/21095 @madanadit @liyinan926 @foxish I addressed comments here and added PVC support https://github.com/apache/spark/pull/21260. However I'm unsure if this is the right way to go, please check the PR.

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting hostPath volumes

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21260 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21186 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21186 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3009/

[GitHub] spark pull request #21260: [SPARK-23529][K8s] Support mounting hostPath volu...

2018-05-07 Thread andrusha
GitHub user andrusha opened a pull request: https://github.com/apache/spark/pull/21260 [SPARK-23529][K8s] Support mounting hostPath volumes This PR continues #21095 and intersects with #21238. I've added volume mounts as a separate step and added PersistantVolumeClaim support.

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21186 **[Test build #90332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90332/testReport)** for PR 21186 at commit

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21186 To reduce the review scope, `convertMetastoreTableProperty` goes to #21259 . --- - To unsubscribe, e-mail:

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21259 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21259 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3008/

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21259 **[Test build #90331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90331/testReport)** for PR 21259 at commit

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186484550 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark pull request #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProp...

2018-05-07 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/21259 [SPARK-24112][SQL] Add `convertMetastoreTableProperty` conf ## What changes were proposed in this pull request? In Apache Spark 2.4,

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90311/ Test FAILed. ---

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-05-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21118 > We expect data source to produce `ColumnarBatch` for better performance, and the row interface performance is not that important. I disagree. The vectorized path isn't used for all Parquet

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #90311 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90311/testReport)** for PR 16677 at commit

[GitHub] spark issue #21190: [SPARK-22938][SQL][followup] Assert that SQLConf.get is ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21190 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90317/ Test PASSed. ---

[GitHub] spark issue #21190: [SPARK-22938][SQL][followup] Assert that SQLConf.get is ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21190 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21190: [SPARK-22938][SQL][followup] Assert that SQLConf.get is ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21190 **[Test build #90317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90317/testReport)** for PR 21190 at commit

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186482587 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/package.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186482016 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark issue #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21155 **[Test build #90330 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90330/testReport)** for PR 21155 at commit

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-07 Thread wajda
Github user wajda commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r186479450 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1059,3 +1063,316 @@ case class

[GitHub] spark issue #21186: [SPARK-22279][SPARK-24112] Enable `convertMetastoreOrc` ...

2018-05-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21186 Hmm. I'll split this into two PRs in order to make it easy to review. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21239: [SPARK-24040][SS] Support single partition aggregates in...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21239 **[Test build #90328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90328/testReport)** for PR 21239 at commit

[GitHub] spark issue #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21155 **[Test build #90329 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90329/testReport)** for PR 21155 at commit

[GitHub] spark pull request #21239: [SPARK-24040][SS] Support single partition aggreg...

2018-05-07 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21239#discussion_r186467958 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousAggregationSuite.scala --- @@ -0,0 +1,81 @@ +/* + *

[GitHub] spark pull request #21239: [SPARK-24040][SS] Support single partition aggreg...

2018-05-07 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21239#discussion_r186462620 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/EpochTracker.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed

[GitHub] spark pull request #21239: [SPARK-24040][SS] Support single partition aggreg...

2018-05-07 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21239#discussion_r186476292 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousAggregationSuite.scala --- @@ -0,0 +1,81 @@ +/* + *

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186475370 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-07 Thread aditkumar
Github user aditkumar commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r186475229 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1194,13 +1194,21 @@ case class

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186474730 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186474501 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r186474451 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,69 @@ case class

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186474366 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90313/ Test PASSed. ---

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol binding to...

2018-05-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21066 cc @rxin @JoshRosen @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21236 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21252 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90315/ Test PASSed. ---

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90313 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90313/testReport)** for PR 21106 at commit

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21252 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21236 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90318/ Test FAILed. ---

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21236 **[Test build #90318 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90318/testReport)** for PR 21236 at commit

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21252 **[Test build #90315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90315/testReport)** for PR 21252 at commit

[GitHub] spark issue #21230: [SPARK-24172][SQL] we should not apply operator pushdown...

2018-05-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21230 So it was the use of `transformUp` that caused this rules to match multiple times, right? In that case, would it make more sense to do what @marmbrus suggested in the immutable plan PR and make this

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90325/ Test PASSed. ---

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21250 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21250 **[Test build #90325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90325/testReport)** for PR 21250 at commit

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21070 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3007/

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21070 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186469794 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/BindingParquetOutputCommitter.scala --- @@ -0,0 +1,122 @@ +/*

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21070 **[Test build #90327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90327/testReport)** for PR 21070 at commit

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186469740 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/BindingParquetOutputCommitter.scala --- @@ -0,0 +1,122 @@ +/*

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21070#discussion_r186469029 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -345,7 +345,7 @@ object SQLConf { "snappy, gzip, lzo.")

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21070#discussion_r186468896 --- Diff: dev/deps/spark-deps-hadoop-2.7 --- @@ -163,13 +163,13 @@ orc-mapreduce-1.4.3-nohive.jar oro-2.0.8.jar osgi-resource-locator-1.0.1.jar

[GitHub] spark pull request #21165: [Spark-20087][CORE] Attach accumulators / metrics...

2018-05-07 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/21165#discussion_r186468071 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -212,9 +212,15 @@ case object TaskResultLost extends TaskFailedReason { *

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r186467842 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1194,13 +1194,21 @@ case class

[GitHub] spark pull request #21144: [SPARK-24043][SQL] Interpreted Predicate should i...

2018-05-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21144 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186467415 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90316/ Test FAILed. ---

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186466925 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21258 **[Test build #90316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90316/testReport)** for PR 21258 at commit

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186467007 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90312/ Test FAILed. ---

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186466513 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186466367 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/PathOutputCommitProtocol.scala --- @@ -0,0 +1,260 @@ +/* +

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21258 **[Test build #90312 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90312/testReport)** for PR 21258 at commit

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186466259 --- Diff: hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud/BindingParquetOutputCommitter.scala --- @@ -0,0 +1,122 @@ +/*

[GitHub] spark issue #21144: [SPARK-24043][SQL] Interpreted Predicate should initiali...

2018-05-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/21144 LGTM - merging to master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21070#discussion_r186464674 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java --- @@ -63,115 +59,157 @@ public final

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21070#discussion_r186464557 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java --- @@ -63,115 +59,157 @@ public final

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-07 Thread aditkumar
Github user aditkumar commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r186464442 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1194,13 +1194,21 @@ case class

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r186464404 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,69 @@ case class

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186463919 --- Diff: hadoop-cloud/src/main/scala/org/apache/spark/internal/io/cloud/PathCommitterConstants.scala --- @@ -0,0 +1,87 @@ +/* + * Licensed

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90326/ Test PASSed. ---

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21249 **[Test build #90326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90326/testReport)** for PR 21249 at commit

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186463018 --- Diff: hadoop-cloud/src/main/scala/org/apache/spark/internal/io/cloud/PathCommitterConstants.scala --- @@ -0,0 +1,87 @@ +/* + * Licensed

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90314/ Test FAILed. ---

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21258 **[Test build #90314 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90314/testReport)** for PR 21258 at commit

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-05-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21118 > Actually the `SupportsScanUnsafeRow` is only there to avoid perf regression for migrating file sources. If you think that's not a good public API, we can move it to internal package and only use

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3006/

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21250 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3005/

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20787 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186458781 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import

<    1   2   3   4   5   6   7   >