[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-24 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r197685463 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +270,11 @@ private[parquet]

[GitHub] spark issue #21379: [SPARK-24327][SQL] Verify and normalize a partition colu...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21379 **[Test build #92285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92285/testReport)** for PR 21379 at commit

[GitHub] spark issue #21379: [SPARK-24327][SQL] Verify and normalize a partition colu...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21379 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/452/

[GitHub] spark issue #21379: [SPARK-24327][SQL] Verify and normalize a partition colu...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21379 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21379: [SPARK-24327][SQL] Verify and normalize a partition colu...

2018-06-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21379 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20949: [SPARK-19018][SQL] Add support for custom encoding on cs...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20949 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20949: [SPARK-19018][SQL] Add support for custom encoding on cs...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20949 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92277/ Test PASSed. ---

[GitHub] spark issue #20949: [SPARK-19018][SQL] Add support for custom encoding on cs...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20949 **[Test build #92277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92277/testReport)** for PR 20949 at commit

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r197681414 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -378,6 +378,17 @@ object SQLConf { .booleanConf

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21631 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21631 **[Test build #92284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92284/testReport)** for PR 21631 at commit

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/451/

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92275/ Test PASSed. ---

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #92275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92275/testReport)** for PR 21221 at commit

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92283/ Test FAILed. ---

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21632 **[Test build #92283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92283/testReport)** for PR 21632 at commit

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21632 **[Test build #92283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92283/testReport)** for PR 21632 at commit

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/450/

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 @holdenk @sethah I've updated the PR to latest master (hopefully all of the tests still pass :) ) --- - To unsubscribe,

[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21632: [SPARK-19591][ML][MLlib] Add sample weights to de...

2018-06-24 Thread imatiach-msft
GitHub user imatiach-msft opened a pull request: https://github.com/apache/spark/pull/21632 [SPARK-19591][ML][MLlib] Add sample weights to decision trees This is updated PR https://github.com/apache/spark/pull/16722 What changes were proposed in this pull request?

[GitHub] spark pull request #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPru...

2018-06-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21631#discussion_r197679873 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1602,4 +1602,14 @@ class CSVSuite extends QueryTest

[GitHub] spark issue #21617: [SPARK-24634][SS] Add a new metric regarding number of r...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21617 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21617: [SPARK-24634][SS] Add a new metric regarding number of r...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21617 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92276/ Test PASSed. ---

[GitHub] spark issue #21617: [SPARK-24634][SS] Add a new metric regarding number of r...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21617 **[Test build #92276 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92276/testReport)** for PR 21617 at commit

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21631 @HyukjinKwon sure, I'll do --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21625#discussion_r197679212 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -573,32 +578,6 @@ object

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21625#discussion_r197679077 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -573,32 +578,6 @@ object

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21625#discussion_r197678386 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -573,32 +578,6 @@ object

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21631 **[Test build #92282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92282/testReport)** for PR 21631 at commit

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/449/

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21631 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21631 cc: @HyukjinKwon @MaxGekk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPru...

2018-06-24 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21631 [SPARK-24645][SQL] Skip parsing when csvColumnPruning enabled and partitions scanned only ## What changes were proposed in this pull request? In the master, when `csvColumnPruning` enabled and

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r197674957 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2288,6 +2288,401 @@ case class

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r197676498 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2288,6 +2288,401 @@ case class

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21625#discussion_r197676313 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -573,32 +578,6 @@ object

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21625#discussion_r197676056 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -573,32 +578,6 @@ object

[GitHub] spark pull request #21628: [SPARK-23776][DOC] Update instructions for runnin...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21628#discussion_r197673014 --- Diff: docs/building-spark.md --- @@ -215,19 +215,23 @@ If you are building Spark for use in a Python environment and you wish to pip in

[GitHub] spark pull request #21628: [SPARK-23776][DOC] Update instructions for runnin...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21628#discussion_r197672927 --- Diff: docs/building-spark.md --- @@ -215,19 +215,23 @@ If you are building Spark for use in a Python environment and you wish to pip in

[GitHub] spark pull request #21155: [SPARK-23927][SQL] Add "sequence" expression

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21155#discussion_r197672902 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1887,6 +1889,402 @@ case class

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21495 **[Test build #92281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92281/testReport)** for PR 21495 at commit

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/448/

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21495 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-24 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21495 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21629: Fix minor typo in docs/cloud-integration.md

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21629 It's fine but mind taking another look and see if there are other typos more while we are here? I am pretty sure there are more and it should be nicer if we fix them in batch while we are here.

[GitHub] spark issue #21630: [SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` in funct...

2018-06-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21630 LGTM On Mon, Jun 25, 2018, 11:16 AM Takuya UESHIN wrote: > cc @DylanGuedes @HyukjinKwon > @viirya

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21495 Seems fine to me. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197671215 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-24 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21495 Gently ping @dbtsai , is there any left work? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197670974 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21603 **[Test build #92280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92280/testReport)** for PR 21603 at commit

[GitHub] spark issue #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidation

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21594 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21630: [SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` in funct...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21630 **[Test build #92279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92279/testReport)** for PR 21630 at commit

[GitHub] spark issue #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidation

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21594 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92274/ Test PASSed. ---

[GitHub] spark issue #21630: [SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` in funct...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21630 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21630: [SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` in funct...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21630 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/446/

[GitHub] spark issue #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidation

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21594 **[Test build #92274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92274/testReport)** for PR 21594 at commit

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21603 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/447/

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21603 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21630: [SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` in funct...

2018-06-24 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21630 cc @DylanGuedes @HyukjinKwon @viirya @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21630: [SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` i...

2018-06-24 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/21630 [SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` in function.scala `@scala.annotation.varargs`. ## What changes were proposed in this pull request? This is a follow-up pr of #21045 which

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21073 @ueshin >so I was wondering whether we need the same thing for MapConcat or not. Got it. I will research that, plus I will look at the entire pull request for Concat to see if

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197669221 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197666356 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197665988 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197666062 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197666362 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197666255 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r197665954 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -475,6 +474,231 @@ case class

[GitHub] spark pull request #21628: [SPARK-23776][DOC] Update instructions for runnin...

2018-06-24 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21628#discussion_r197667457 --- Diff: docs/building-spark.md --- @@ -215,19 +215,23 @@ If you are building Spark for use in a Python environment and you wish to pip in

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21628 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92278/ Test PASSed. ---

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21628 **[Test build #92278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92278/testReport)** for PR 21628 at commit

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21628 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21629: Fix minor typo in docs/cloud-integration.md

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21629 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21629: Fix minor typo in docs/cloud-integration.md

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21629 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21629: Fix minor typo in docs/cloud-integration.md

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21629 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21629: Fix minor typo in docs/cloud-integration.md

2018-06-24 Thread jkleckner
GitHub user jkleckner opened a pull request: https://github.com/apache/spark/pull/21629 Fix minor typo in docs/cloud-integration.md ## What changes were proposed in this pull request? Minor typo in docs/cloud-integration.md ## How was this patch tested?

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21628 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21628 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21628 **[Test build #92278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92278/testReport)** for PR 21628 at commit

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92273/ Test PASSed. ---

[GitHub] spark pull request #21628: [SPARK-23776][DOC] Update instructions for runnin...

2018-06-24 Thread bersprockets
GitHub user bersprockets opened a pull request: https://github.com/apache/spark/pull/21628 [SPARK-23776][DOC] Update instructions for running PySpark after building with SBT ## What changes were proposed in this pull request? This update tells the reader how to build Spark

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #92273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92273/testReport)** for PR 21073 at commit

[GitHub] spark pull request #21626: [SPARK-24642][SQL] New function infers schema for...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21626#discussion_r197662799 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/InferSchema.scala --- @@ -0,0 +1,162 @@ +/* + *

[GitHub] spark pull request #21626: [SPARK-24642][SQL] New function infers schema for...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21626#discussion_r197662496 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/InferSchema.scala --- @@ -0,0 +1,162 @@ +/* + *

[GitHub] spark issue #20949: [SPARK-19018][SQL] Add support for custom encoding on cs...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20949 **[Test build #92277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92277/testReport)** for PR 20949 at commit

[GitHub] spark issue #20949: [SPARK-19018][SQL] Add support for custom encoding on cs...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20949 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r197662087 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -513,6 +513,43 @@ class CSVSuite extends

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r197662012 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -512,6 +512,43 @@ class CSVSuite extends

[GitHub] spark issue #21617: [SPARK-24634][SS] Add a new metric regarding number of r...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21617 **[Test build #92276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92276/testReport)** for PR 21617 at commit

[GitHub] spark issue #21617: [SPARK-24634][SS] Add a new metric regarding number of r...

2018-06-24 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21617 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21596: [SPARK-24601] Bump Jackson version

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21596#discussion_r197661126 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonBenchmarks.scala --- @@ -25,8 +25,13 @@ import

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21598 > Based on my understanding, the decision is made case by case. I concur. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21596 Can we target this to Spark 3.0, which should be the next release after Spark 2.4 release? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #92275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92275/testReport)** for PR 21221 at commit

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   >