[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21306 @marmbrus, @cloud-fan, @gatorsmile, I've updated this PR to use reflection to instantiate catalogs. This allows implementations to provide named catalogs (and reuse implementations) and configure

[GitHub] spark pull request #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO ...

2018-07-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21847#discussion_r205236872 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroSerializer.scala --- @@ -41,7 +42,7 @@ class AvroSerializer(rootCatalystType: DataType,

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #93565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93565/testReport)** for PR 21847 at commit

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93565/ Test FAILed. ---

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21306 **[Test build #93567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93567/testReport)** for PR 21306 at commit

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-25 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21821 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21867 **[Test build #93566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93566/testReport)** for PR 21867 at commit

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21867 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21867 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21874: [SPARK-23146][K8S][TESTS] Enable client mode inte...

2018-07-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21874 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL][WIP]: Built-in AVRO support...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #93565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93565/testReport)** for PR 21847 at commit

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21867 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93548/ Test FAILed. ---

[GitHub] spark pull request #21875: [SPARK-24288][SQL] Enable preventing predicate pu...

2018-07-25 Thread TomaszGaweda
Github user TomaszGaweda commented on a diff in the pull request: https://github.com/apache/spark/pull/21875#discussion_r205224686 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -183,6 +183,9 @@ class JDBCOptions(

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21867 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21875: [SPARK-24288][SQL] Enable preventing predicate pu...

2018-07-25 Thread TomaszGaweda
Github user TomaszGaweda commented on a diff in the pull request: https://github.com/apache/spark/pull/21875#discussion_r205227335 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala --- @@ -172,7 +172,11 @@ private[sql] case class

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21867 **[Test build #93548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93548/testReport)** for PR 21867 at commit

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21850 **[Test build #93564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93564/testReport)** for PR 21850 at commit

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21758: [SPARK-24795][CORE] Implement barrier execution mode

2018-07-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21758 LGTM @squito Has @jiangxb1987 address your comments? We want to merge this PR ASAP and a few to-be-submitted PRs depend on this PR. This feature is targeting to Spark 2.4

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21850 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21807: [SPARK-24536] Validate that limit clause cannot have a n...

2018-07-25 Thread NiharS
Github user NiharS commented on the issue: https://github.com/apache/spark/pull/21807 Hey @mauropalsgraaf just wanted to check in on this. Have you run into any additional issues or have any questions for this fix? ---

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93554/ Test FAILed. ---

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21850 **[Test build #93554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93554/testReport)** for PR 21850 at commit

[GitHub] spark issue #21875: [SPARK-24288][SQL] Enable preventing predicate pushdown

2018-07-25 Thread TomaszGaweda
Github user TomaszGaweda commented on the issue: https://github.com/apache/spark/pull/21875 Thanks! :) LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93547/ Test PASSed. ---

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21584: [SPARK-24433][K8S] Initial R Bindings for SparkR on K8s

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 **[Test build #93547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93547/testReport)** for PR 21584 at commit

[GitHub] spark issue #21875: [SPARK-24288][SQL] Enable preventing predicate pushdown

2018-07-25 Thread maryannxue
Github user maryannxue commented on the issue: https://github.com/apache/spark/pull/21875 @gatorsmile @TomaszGaweda --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21809 **[Test build #93563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93563/testReport)** for PR 21809 at commit

[GitHub] spark issue #21875: [SPARK-24288][SQL] Enable preventing predicate pushdown

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21875 **[Test build #93562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93562/testReport)** for PR 21875 at commit

[GitHub] spark issue #21875: [SPARK-24288][SQL] Enable preventing predicate pushdown

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21875 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21875: [SPARK-24288][SQL] Enable preventing predicate pushdown

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21875 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21874 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1323/ ---

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21874 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21874 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21874 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21874 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93560/ Test PASSed. ---

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21874 **[Test build #93560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93560/testReport)** for PR 21874 at commit

[GitHub] spark pull request #21875: [SPARK-24288][SQL] Enable preventing predicate pu...

2018-07-25 Thread maryannxue
GitHub user maryannxue opened a pull request: https://github.com/apache/spark/pull/21875 [SPARK-24288][SQL] Enable preventing predicate pushdown ## What changes were proposed in this pull request? Add a JDBC Option "pushDownPredicate" (default `true`) to allow/disallow

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21809 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21809 **[Test build #93561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93561/testReport)** for PR 21809 at commit

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21809 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93561/ Test FAILed. ---

[GitHub] spark issue #21809: [SPARK-24851][UI] Map a Stage ID to it's Associated Job ...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21809 **[Test build #93561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93561/testReport)** for PR 21809 at commit

[GitHub] spark pull request #21809: [SPARK-24851][UI] Map a Stage ID to it's Associat...

2018-07-25 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21809#discussion_r205219399 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -182,6 +184,12 @@ private[ui] class StagePage(parent: StagesTab, store:

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21874 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1323/ ---

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93556/ Test FAILed. ---

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21306 **[Test build #93556 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93556/testReport)** for PR 21306 at commit

[GitHub] spark pull request #21809: [SPARK-24851][UI] Map a Stage ID to it's Associat...

2018-07-25 Thread pgandhi999
Github user pgandhi999 commented on a diff in the pull request: https://github.com/apache/spark/pull/21809#discussion_r205219178 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -115,6 +115,8 @@ private[ui] class StagePage(parent: StagesTab, store:

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93545/ Test PASSed. ---

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21867 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21360: [SPARK-24288] Enable preventing predicate pushdow...

2018-07-25 Thread maryannxue
Github user maryannxue closed the pull request at: https://github.com/apache/spark/pull/21360 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21867 **[Test build #93545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93545/testReport)** for PR 21867 at commit

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21874 **[Test build #93560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93560/testReport)** for PR 21874 at commit

[GitHub] spark issue #21874: [SPARK-23146][K8S][TESTS] Enable client mode integration...

2018-07-25 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/21874 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21874: [SPARK-23146][K8S][TESTS] Enable client mode inte...

2018-07-25 Thread mccheah
GitHub user mccheah opened a pull request: https://github.com/apache/spark/pull/21874 [SPARK-23146][K8S][TESTS] Enable client mode integration test. ## What changes were proposed in this pull request? Enable client mode integration test after merging from master.

[GitHub] spark pull request #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` w...

2018-07-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21850#discussion_r205215698 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -414,6 +414,9 @@ object SimplifyConditionals

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-07-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r205214861 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2320,6 +2320,27 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-07-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r205214601 --- Diff: sql/core/src/test/resources/sql-tests/results/subquery/negative-cases/subq-input-typecheck.sql.out --- @@ -113,15 +105,7 @@ WHERE

[GitHub] spark pull request #21403: [SPARK-24341][SQL] Support only IN subqueries wit...

2018-07-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21403#discussion_r205213736 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -247,6 +249,20 @@ class Analyzer( }

[GitHub] spark issue #20414: [SPARK-23243][SQL] Shuffle+Repartition on an RDD could l...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20414 **[Test build #93558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93558/testReport)** for PR 20414 at commit

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #93557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93557/testReport)** for PR 21221 at commit

[GitHub] spark issue #20405: [SPARK-23229][SQL] Dataset.hint should use planWithBarri...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20405 **[Test build #93559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93559/testReport)** for PR 20405 at commit

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21306 **[Test build #93556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93556/testReport)** for PR 21306 at commit

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21306 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21748: [SPARK-23146][K8S] Support client mode.

2018-07-25 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21748 @mccheah the integration tests did not include the ClientModeTestsSuite. Can you add `with ClientModeTestsSuite` else, the PRB doesn't actually test the client mode support accurately. ---

[GitHub] spark pull request #21748: [SPARK-23146][K8S] Support client mode.

2018-07-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21748 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21803: [SPARK-24849][SPARK-24911][SQL] Converting a valu...

2018-07-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21803 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93546/ Test PASSed. ---

[GitHub] spark issue #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Python UDF...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21650 **[Test build #93546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93546/testReport)** for PR 21650 at commit

[GitHub] spark issue #21803: [SPARK-24849][SPARK-24911][SQL] Converting a value of St...

2018-07-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21803 LGTM Thanks! Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21821 This PR is majorly for Spark 2.3 branch. The code changes will be removed from the master branch when https://github.com/apache/spark/pull/21822 is merged. However, the test cases will

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21821 **[Test build #93555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93555/testReport)** for PR 21821 at commit

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21821 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21821 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21821 @mgaido91 See the comment https://github.com/apache/spark/pull/21821#issuecomment-407096818 --- - To unsubscribe, e-mail:

[GitHub] spark issue #21834: [SPARK-22814][SQL] Support Date/Timestamp in a JDBC part...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21834 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21834: [SPARK-22814][SQL] Support Date/Timestamp in a JDBC part...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21834 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93542/ Test PASSed. ---

[GitHub] spark issue #21834: [SPARK-22814][SQL] Support Date/Timestamp in a JDBC part...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21834 **[Test build #93542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93542/testReport)** for PR 21834 at commit

[GitHub] spark issue #21821: [SPARK-24867] [SQL] Add AnalysisBarrier to DataFrameWrit...

2018-07-25 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21821 Is this still valid since https://github.com/apache/spark/pull/21822 is going on? Shall we have this only on 2.3 main branches? ---

[GitHub] spark pull request #21650: [SPARK-24624][SQL][PYTHON] Support mixture of Pyt...

2018-07-25 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21650#discussion_r205206127 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -94,36 +95,94 @@ object

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93544/ Test FAILed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #93544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93544/testReport)** for PR 21596 at commit

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93543/ Test PASSed. ---

[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21403 **[Test build #93543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93543/testReport)** for PR 21403 at commit

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21102 Just in my opinion, I'd prefer to preserve the element order of the left array. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-25 Thread Fokko
Github user Fokko commented on the issue: https://github.com/apache/spark/pull/21596 @gatorsmile Sure, just checking if it still works against recent master :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21850 **[Test build #93554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93554/testReport)** for PR 21850 at commit

[GitHub] spark pull request #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` w...

2018-07-25 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21850#discussion_r205187664 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -414,6 +414,16 @@ object SimplifyConditionals

<    1   2   3   4   5   6   >