[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-07-07 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/21260 @felixcheung This feature was discussed and this PR was started before https://issues.apache.org/jira/browse/SPARK-24434 was even brought up. Being able to mount commonly used types of volumes

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200828393 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -344,6 +344,36 @@ case class Join(

[GitHub] spark pull request #21664: [SPARK-24678][CORE] NoClassDefFoundError will not...

2018-07-07 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/21664#discussion_r200828384 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1049,6 +1049,13 @@ class DAGScheduler( abortStage(stage,

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-07-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21260 @skonto is it better to generalize the approach to match the one in https://issues.apache.org/jira/browse/SPARK-24435? not sure if @mccheah @foxish @erikerlandson have any last thought

[GitHub] spark issue #21552: [SPARK-24544][SQL] Print actual failure cause when look ...

2018-07-07 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/21552 @maropu May be i will do this check?As @cloud-fan mentioned. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21552: [SPARK-24544][SQL] Print actual failure cause whe...

2018-07-07 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/21552#discussion_r200828333 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -131,6 +132,8 @@ private[sql] class HiveSessionCatalog(

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21608 **[Test build #92715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92715/testReport)** for PR 21608 at commit

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21608 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92713/ Test PASSed. ---

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21608 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21608 **[Test build #92713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92713/testReport)** for PR 21608 at commit

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-07 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/21698 @jiangxb1987 Any closure sensitive to iteration order [1] is effected by this - under the set of circumstances. If we cannot solve it in a principled manner (make shuffle repeatable which I

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21728 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92714/ Test FAILed. ---

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21728 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21728 **[Test build #92714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92714/testReport)** for PR 21728 at commit

[GitHub] spark pull request #21707: Update for spark 2.2.2 release

2018-07-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21707#discussion_r200826322 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala --- @@ -160,7 +160,7 @@ class

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-07-07 Thread edwinalu
Github user edwinalu commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r200826235 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -160,11 +160,29 @@ case class

[GitHub] spark issue #21584: [SPARK-24433][K8S][WIP] Initial R Bindings for SparkR on...

2018-07-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21584 IMO it's fine we have one version supported in the image and stick with that. the tricky thing is having maintainers to keep updating/testing the newer versions in the images (we have

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200825129 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -240,21 +238,27 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200824906 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/WriteSupport.java --- @@ -38,15 +38,16 @@ * If this method fails (by throwing an

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200824639 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2120,6 +2122,99 @@ class Analyzer( } }

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200824599 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2120,6 +2122,99 @@ class Analyzer( } }

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200824602 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2120,6 +2122,99 @@ class Analyzer( } }

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21728 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21728 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/750/

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21728 **[Test build #92714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92714/testReport)** for PR 21728 at commit

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200824532 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -2120,6 +2122,99 @@ class Analyzer( } }

[GitHub] spark pull request #21728: [SPARK-24759] [SQL] No reordering keys for broadc...

2018-07-07 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/21728 [SPARK-24759] [SQL] No reordering keys for broadcast hash join ## What changes were proposed in this pull request? As the implementation of the broadcast hash join is independent of the

[GitHub] spark pull request #21306: [SPARK-24252][SQL] Add DataSourceV2 mix-in for ca...

2018-07-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21306#discussion_r200824504 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/catalog/TableCatalog.java --- @@ -0,0 +1,123 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21608 **[Test build #92713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92713/testReport)** for PR 21608 at commit

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-07-07 Thread Achuth17
Github user Achuth17 commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r200824025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala --- @@ -47,15 +48,26 @@ object CommandUtils extends Logging {

[GitHub] spark pull request #21727: [SPARK-24757][SQL] Improving the error message fo...

2018-07-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21727 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/21727 Merging to master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21727 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92712/ Test PASSed. ---

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21727 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21727 **[Test build #92712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92712/testReport)** for PR 21727 at commit

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21667 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92711/ Test PASSed. ---

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21667 **[Test build #92711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92711/testReport)** for PR 21667 at commit

[GitHub] spark pull request #20933: [SPARK-23817][SQL]Migrate ORC file format read pa...

2018-07-07 Thread tengpeng
Github user tengpeng commented on a diff in the pull request: https://github.com/apache/spark/pull/20933#discussion_r200819285 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -241,39 +240,47 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92710/ Test PASSed. ---

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21556 **[Test build #92710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92710/testReport)** for PR 21556 at commit

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21439 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92708/ Test PASSed. ---

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21439 **[Test build #92708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92708/testReport)** for PR 21439 at commit

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-07 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21698 Thank you for your comments @mridulm ! We focus on resolving the RDD.repartition() correctness issue here in this PR, because it is most commonly used, and that we can still address the

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21727 **[Test build #92712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92712/testReport)** for PR 21727 at commit

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21727 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21727 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21727: [SPARK-24757][SQL] Improving the error message for broad...

2018-07-07 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21727 @hvanhovell Please, have a look at the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21727: [SPARK-24757][SQL] Improving the error message fo...

2018-07-07 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/21727 [SPARK-24757][SQL] Improving the error message for broadcast timeouts ## What changes were proposed in this pull request? In the PR, I propose to provide a tip to user how to resolve the

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21667 **[Test build #92711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92711/testReport)** for PR 21667 at commit

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21667 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/749/

[GitHub] spark issue #21667: [SPARK-24691][SQL]Dispatch the type support check in Fil...

2018-07-07 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21667 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21192: [SPARK-24118][SQL] Flexible format for the lineSep optio...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21192 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92707/ Test PASSed. ---

[GitHub] spark issue #21192: [SPARK-24118][SQL] Flexible format for the lineSep optio...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21192 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21192: [SPARK-24118][SQL] Flexible format for the lineSep optio...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21192 **[Test build #92707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92707/testReport)** for PR 21192 at commit

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92705/ Test PASSed. ---

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 **[Test build #92705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92705/testReport)** for PR 21669 at commit

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92706/ Test PASSed. ---

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #92706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92706/testReport)** for PR 21657 at commit

[GitHub] spark pull request #21556: [SPARK-24549][SQL] Support Decimal type push down...

2018-07-07 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21556#discussion_r200813391 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -82,6 +120,30 @@ private[parquet] class

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21556 **[Test build #92710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92710/testReport)** for PR 21556 at commit

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/748/

[GitHub] spark issue #21556: [SPARK-24549][SQL] Support Decimal type push down to the...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21556 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21690 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92709/ Test PASSed. ---

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21690 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21690 **[Test build #92709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92709/testReport)** for PR 21690 at commit

[GitHub] spark pull request #21146: [SPARK-23654][BUILD] remove jets3t as a dependenc...

2018-07-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21146#discussion_r200812101 --- Diff: dev/deps/spark-deps-hadoop-2.6 --- @@ -21,8 +21,6 @@ automaton-1.11-8.jar avro-1.7.7.jar avro-ipc-1.7.7.jar

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21690 **[Test build #92709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92709/testReport)** for PR 21690 at commit

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21439 **[Test build #92708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92708/testReport)** for PR 21439 at commit

[GitHub] spark issue #21192: [SPARK-24118][SQL] Flexible format for the lineSep optio...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21192 **[Test build #92707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92707/testReport)** for PR 21192 at commit

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/747/

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #92706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92706/testReport)** for PR 21657 at commit

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/746/

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/746/ ---

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/746/ ---

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92704/ Test FAILed. ---

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #92704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92704/testReport)** for PR 21657 at commit

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 **[Test build #92705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92705/testReport)** for PR 21669 at commit

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-07 Thread yuanboliu
Github user yuanboliu commented on the issue: https://github.com/apache/spark/pull/21690 @koeninger Thanks for your reply. Agree with you. there is no need to to use pause repeatedly. This is my test without any pause, and the app master stuck for a long time without any process

[GitHub] spark issue #21260: [SPARK-23529][K8s] Support mounting volumes

2018-07-07 Thread baluchicken
Github user baluchicken commented on the issue: https://github.com/apache/spark/pull/21260 Nice work, only one thing I would consider including a StorageClass name option for the PersistentVolumeClaim volume type which defaults to an empty string. Without that the PVC will always use

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-07-07 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/21698 I did not go over the PR itself in detail, but the proposal sounds very expensive - particularly given the cascading costs involved. Also, I am not sure why we are special case'ing only

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-07-07 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21583 @foxish The problem is the same as described here: https://github.com/moby/moby/issues/32457 which should have been solved in `Docker 17.05`. As such, this is prompted by a deprecated version

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-07-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r200805499 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -160,11 +160,29 @@ case class

[GitHub] spark pull request #21653: [SPARK-13343] speculative tasks that didn't commi...

2018-07-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/21653#discussion_r200805005 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -723,6 +723,13 @@ private[spark] class TaskSetManager( def

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #92704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92704/testReport)** for PR 21657 at commit

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/745/

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21657 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92703/ Test FAILed. ---

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #92703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92703/testReport)** for PR 21657 at commit

  1   2   >