[GitHub] spark pull request #21602: [SPARK-24613][SQL] Cache with UDF could not be ma...

2018-06-20 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/21602#discussion_r197017717 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala --- @@ -132,4 +132,19 @@ class DatasetCacheSuite extends QueryTest with

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92153/ Test PASSed. ---

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #92153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92153/testReport)** for PR 21073 at commit

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-06-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21599 is this a regression in Spark 2.3? which commit caused it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-20 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r197011649 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +270,11 @@ private[parquet] class

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21603 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/367/

[GitHub] spark issue #18826: [SPARK-14712][ML] LogisticRegressionModel.toString shoul...

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18826 **[Test build #92157 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92157/testReport)** for PR 18826 at commit

[GitHub] spark pull request #21602: [SPARK-24613][SQL] Cache with UDF could not be ma...

2018-06-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21602#discussion_r197011139 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetCacheSuite.scala --- @@ -132,4 +132,19 @@ class DatasetCacheSuite extends QueryTest with

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21603 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18826: [SPARK-14712][ML] LogisticRegressionModel.toString shoul...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18826 I think you cc'ed right ones .. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #18826: [SPARK-14712][ML] LogisticRegressionModel.toString shoul...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18826 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21603 **[Test build #92156 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92156/testReport)** for PR 21603 at commit

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-20 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21603 [SPARK-17091][SQL] Add rule to convert IN predicate to equivalent Parquet filter ## What changes were proposed in this pull request? Add a new optimizer rule to convert an IN predicate to

[GitHub] spark issue #21600: [Spark-24553][WEB-UI] http 302 fixes for href redirect

2018-06-20 Thread SJKallman
Github user SJKallman commented on the issue: https://github.com/apache/spark/pull/21600 Updated title and PR description. Happy to hear thoughts and collaborate on a better solution just let me know. --- - To

[GitHub] spark issue #21600: [Spark-24553][WEB-UI] http 302 fixes for href redirect

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21600 **[Test build #92155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92155/testReport)** for PR 21600 at commit

[GitHub] spark issue #21600: [Spark-24553][WEB-UI] http 302 fixes for href redirect

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21600 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21600: [Spark-24553][WEB-UI]

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21600 I am pretty sure he meant something like `[SPARK-24553][WEB-UI] http 302 fixes for href redirect` ... ok to test ---

[GitHub] spark issue #21600: [Spark-24553][WEB-UI] http 302 fixes for href redirect

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21600 not a big deal but let's remove `Closes #24553` in the PR description BTW. I thought this takes over another PR. --- - To

[GitHub] spark issue #21581: [SPARK-24574][SQL] array_contains, array_position, array...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21581 Oh, I see. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21584: [SPARK-24433][K8S][WIP] Initial R Bindings for SparkR on...

2018-06-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21584 We have a min supported version of R https://github.com/apache/spark/blob/master/R/pkg/DESCRIPTION But beyond that we haven’t seen much problems when new releases of R are coming

[GitHub] spark issue #21581: [SPARK-24574][SQL] array_contains, array_position, array...

2018-06-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21581 But the emails of commits in this pr seem not valid, just for the local computer. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197004678 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197001921 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197005444 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197003678 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197004881 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197005083 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197004822 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197004944 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-20 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197004467 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21560: [SPARK-24386][SS] coalesce(1) aggregates in conti...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21560#discussion_r197004935 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousDataSourceRDD.scala --- @@ -98,6 +98,10 @@ class

[GitHub] spark issue #21581: [SPARK-24574][SQL] array_contains, array_position, array...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21581 oh haha FYI that works after it's merged if @chongguang link the email into his Github profile. I asked the same thing in databricks/spark-xml before :) ---

[GitHub] spark issue #21527: [SPARK-24519] Make the threshold for highly compressed m...

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21527 **[Test build #92154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92154/testReport)** for PR 21527 at commit

[GitHub] spark issue #21600: {Spark-24553}{WEB-UI} http 302 fixes for href redirect

2018-06-20 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21600 Jenkins, this is ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21527: [SPARK-24519] Make the threshold for highly compressed m...

2018-06-20 Thread hthuynh2
Github user hthuynh2 commented on the issue: https://github.com/apache/spark/pull/21527 @vanzin @squito Thank for the comments. I updated the PR, please have a look and let me know if anything need to be changed. Thank you. ---

[GitHub] spark pull request #21050: [SPARK-23912][SQL]add array_distinct

2018-06-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21050 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92151/ Test PASSed. ---

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21590 **[Test build #92151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92151/testReport)** for PR 21590 at commit

[GitHub] spark issue #21600: {Spark-24553}{WEB-UI} http 302 fixes for href redirect

2018-06-20 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21600 LGTM. I think we can do even better, to create some function reformat the URLs in HTML. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21050 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21560: [SPARK-24386][SS] coalesce(1) aggregates in conti...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21560#discussion_r197000483 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala --- @@ -349,6 +349,17 @@ object

[GitHub] spark pull request #21560: [SPARK-24386][SS] coalesce(1) aggregates in conti...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21560#discussion_r196999896 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/ContinuousShuffleReadRDD.scala --- @@ -61,12 +63,14 @@

[GitHub] spark pull request #21560: [SPARK-24386][SS] coalesce(1) aggregates in conti...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21560#discussion_r196999687 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/ContinuousShuffleReadRDD.scala --- @@ -61,12 +63,14 @@

[GitHub] spark pull request #21560: [SPARK-24386][SS] coalesce(1) aggregates in conti...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21560#discussion_r196999745 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousCoalesceRDD.scala --- @@ -0,0 +1,108 @@ +/* +

[GitHub] spark issue #21581: [SPARK-24574][SQL] array_contains, array_position, array...

2018-06-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21581 @chongguang I think this pr is ready to merge, so I tried, but seems like the commits in this pr aren't connected with your GitHub account. If you want to connect the merge commit to your account,

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92149/ Test PASSed. ---

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21602 **[Test build #92149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92149/testReport)** for PR 21602 at commit

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92148/ Test PASSed. ---

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21050 **[Test build #92148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92148/testReport)** for PR 21050 at commit

[GitHub] spark pull request #20640: [SPARK-19755][Mesos] Blacklist is always active f...

2018-06-20 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20640#discussion_r196997885 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -648,14 +645,8 @@

[GitHub] spark issue #21222: [SPARK-24161][SS] Enable debug package feature on struct...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21222 adding cc. to @zsxwing since he has been reviewing PRs for SS so far. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21357: [SPARK-24311][SS] Refactor HDFSBackedStateStoreProvider ...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21357 adding cc. to @zsxwing since he has been reviewing PRs for SS so far. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total estimated size of states ...

2018-06-20 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21469 adding cc. to @zsxwing since he has been reviewing PRs for SS so far. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and...

2018-06-20 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21495#discussion_r196995267 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala --- @@ -21,8 +21,22 @@ import scala.collection.mutable

[GitHub] spark pull request #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21495#discussion_r196993585 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala --- @@ -21,8 +21,22 @@ import scala.collection.mutable

[GitHub] spark pull request #21598: [SPARK-24605][SQL] size(null) returns null instea...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21598#discussion_r196993126 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1314,6 +1314,13 @@ object SQLConf { "Other column

[GitHub] spark pull request #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and...

2018-06-20 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21495#discussion_r196992852 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala --- @@ -21,8 +21,22 @@ import scala.collection.mutable

[GitHub] spark issue #21548: [SPARK-24518][CORE] Using Hadoop credential provider API...

2018-06-20 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21548 > I just have an issue with your summary. It makes it sound like there's no way to securely configure these things today, and that's not true. > > You can have the password in the config

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21542 Unfortunately, looks consistent... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21542 Hmm .. seems roughly the error looks consistent. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #92153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92153/testReport)** for PR 21073 at commit

[GitHub] spark issue #21597: [SPARK-24603] Fix findTightestCommonType reference in co...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21597 @Fokko, thanks for bearing with it. (see also https://spark.apache.org/contributing.html). --- - To unsubscribe, e-mail:

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21596 I tried this in my local too. I didn't mean that we shouldn't but let's fix the test cases and address other concerns. ---

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92152/ Test FAILed. ---

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #92152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92152/testReport)** for PR 21073 at commit

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #92152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92152/testReport)** for PR 21073 at commit

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21073 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92147/ Test FAILed. ---

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #92147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92147/testReport)** for PR 21073 at commit

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21542 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21542 **[Test build #92150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92150/testReport)** for PR 21542 at commit

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21542 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92150/ Test FAILed. ---

[GitHub] spark issue #21560: [SPARK-24386][SS] coalesce(1) aggregates in continuous p...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21560 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92146/ Test FAILed. ---

[GitHub] spark issue #21560: [SPARK-24386][SS] coalesce(1) aggregates in continuous p...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21560 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 > The main thing is that this change is changing test coverage based on the Hadoop version > The Hive 2.1 suite you're disabling is also pretty important to keep working, since it

[GitHub] spark issue #21560: [SPARK-24386][SS] coalesce(1) aggregates in continuous p...

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21560 **[Test build #92146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92146/testReport)** for PR 21560 at commit

[GitHub] spark pull request #21555: [SPARK-24547][K8S] Allow for building spark on k8...

2018-06-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21555 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21555: [SPARK-24547][K8S] Allow for building spark on k8s docke...

2018-06-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/21555 Will follow-up. Thanks all for the comments! Merging to master. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21555: [SPARK-24547][K8S] Allow for building spark on k8s docke...

2018-06-20 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21555 Yup feel free to merge and follow up separately. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-20 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 The main thing is that this change is changing test coverage based on the Hadoop version. So that means that we're effectively changing supported versions of Hive here, and we should do all the

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21598 We will add a configuration for it as a safe safeguard. I think we should incrementally fix things to be the righter behaviour in upper versions .. really. If you meant in the JIRA and PR, this

[GitHub] spark pull request #21548: [SPARK-24518][CORE] Using Hadoop credential provi...

2018-06-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21548#discussion_r196974952 --- Diff: core/src/main/scala/org/apache/spark/SSLOptions.scala --- @@ -179,9 +185,11 @@ private[spark] object SSLOptions extends Logging {

[GitHub] spark pull request #21548: [SPARK-24518][CORE] Using Hadoop credential provi...

2018-06-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21548#discussion_r196976257 --- Diff: docs/security.md --- @@ -446,6 +446,18 @@ replaced with one of the above namespaces. +Spark also supports retrieving

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/366/

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21590 **[Test build #92151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92151/testReport)** for PR 21590 at commit

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21555: [SPARK-24547][K8S] Allow for building spark on k8s docke...

2018-06-20 Thread erikerlandson
Github user erikerlandson commented on the issue: https://github.com/apache/spark/pull/21555 LGMT, I am OK to merge. Most of the automated image build tooling I've seen is custom, but I agree w/ Matt that being able to selectively build is worth supporting, via a followup

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21542 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21542 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/365/

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21542 **[Test build #92150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92150/testReport)** for PR 21542 at commit

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-20 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r196975167 --- Diff: docs/sql-programming-guide.md --- @@ -1302,9 +1302,18 @@ the following case-insensitive options: dbtable -

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 I saw: ``` java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 3.1.0 ``` Other error messages are fixed by the current change. Yup, we

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-06-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21542 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 I saw: ``` Otherwise, we are not even able to start Spark shell. Currently, Hadoop 3 profile in Apache Spark doesn't work at all. It will face an error message such as:

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-20 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 > but the error message was pretty readable to me What is the error message you see? I didn't see any changes in the Spark code that handles that (`IsolatedClientLoader.hiveVersion`), so

  1   2   3   4   5   >