[GitHub] spark pull request #20611: [SPARK-23425][SQL]Support wildcard in HDFS path f...

2018-08-02 Thread sujith71955
Github user sujith71955 commented on a diff in the pull request: https://github.com/apache/spark/pull/20611#discussion_r207320333 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -303,94 +303,44 @@ case class LoadDataCommand(

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21721 The tests keeps failing and looks unrelated. @HyukjinKwon Let me know if you think theres something I should look into. ---

[GitHub] spark issue #21936: [SPARK-24981][Core] ShutdownHook timeout causes job to f...

2018-08-02 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21936 +1 pending jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21953: [SPARK-24992][Core] spark should randomize yarn local di...

2018-08-02 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21953 +1 pending jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21898 **[Test build #94040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94040/testReport)** for PR 21898 at commit

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94040/ Test FAILed. ---

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread ijuma
Github user ijuma commented on the issue: https://github.com/apache/spark/pull/21955 @srowen I think you're pretty close, I added a few comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client ...

2018-08-02 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/spark/pull/21955#discussion_r207318249 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala --- @@ -72,33 +72,39 @@ class KafkaRDDSuite extends

[GitHub] spark pull request #21451: [SPARK-24296][CORE] Replicate large blocks as a s...

2018-08-02 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21451#discussion_r207310135 --- Diff: core/src/main/scala/org/apache/spark/network/netty/NettyBlockRpcServer.scala --- @@ -73,10 +73,32 @@ class NettyBlockRpcServer( }

[GitHub] spark pull request #21451: [SPARK-24296][CORE] Replicate large blocks as a s...

2018-08-02 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21451#discussion_r207310923 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -404,6 +405,47 @@ private[spark] class BlockManager(

[GitHub] spark pull request #21451: [SPARK-24296][CORE] Replicate large blocks as a s...

2018-08-02 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21451#discussion_r207309822 --- Diff: core/src/main/scala/org/apache/spark/network/netty/NettyBlockRpcServer.scala --- @@ -73,10 +73,32 @@ class NettyBlockRpcServer( }

[GitHub] spark pull request #21451: [SPARK-24296][CORE] Replicate large blocks as a s...

2018-08-02 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21451#discussion_r207309551 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -567,4 +567,10 @@ package object config { .intConf

[GitHub] spark pull request #21451: [SPARK-24296][CORE] Replicate large blocks as a s...

2018-08-02 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21451#discussion_r207316984 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -404,6 +405,47 @@ private[spark] class BlockManager(

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21898 **[Test build #94040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94040/testReport)** for PR 21898 at commit

[GitHub] spark pull request #21451: [SPARK-24296][CORE] Replicate large blocks as a s...

2018-08-02 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21451#discussion_r207309499 --- Diff: core/src/main/scala/org/apache/spark/network/netty/NettyBlockRpcServer.scala --- @@ -73,10 +73,32 @@ class NettyBlockRpcServer( }

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client ...

2018-08-02 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/spark/pull/21955#discussion_r207317252 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala --- @@ -72,31 +72,37 @@ class KafkaRDDSuite extends

[GitHub] spark pull request #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client ...

2018-08-02 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/spark/pull/21955#discussion_r207316897 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala --- @@ -72,31 +72,37 @@ class KafkaRDDSuite extends

[GitHub] spark issue #21974: [SPARK-25002][SQL] Avro: revise the output record namesp...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21974 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21974: [SPARK-25002][SQL] Avro: revise the output record namesp...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21974 **[Test build #94035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94035/testReport)** for PR 21974 at commit

[GitHub] spark issue #21974: [SPARK-25002][SQL] Avro: revise the output record namesp...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21974 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94035/ Test PASSed. ---

[GitHub] spark issue #21976: [SPARK-24909] Spark scheduler can hang when fetch failur...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21976 **[Test build #94039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94039/testReport)** for PR 21976 at commit

[GitHub] spark issue #21976: [SPARK-24909] Spark scheduler can hang when fetch failur...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21976 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21976: [SPARK-24909] Spark scheduler can hang when fetch failur...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21976 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21976: [SPARK-24909] Spark scheduler can hang when fetch failur...

2018-08-02 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21976 @squito Note this is just to get people looking at this. I am working on adding some scheduler integration tests but I have to extend those to support multiple executors and allow for tasks

[GitHub] spark pull request #21976: [SPARK-24909] Spark scheduler can hang when fetch...

2018-08-02 Thread tgravescs
GitHub user tgravescs opened a pull request: https://github.com/apache/spark/pull/21976 [SPARK-24909] Spark scheduler can hang when fetch failures, executor …lost, task running on lost executor, and multiple stage attempts ## What changes were proposed in this pull

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19449 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19449 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94014/ Test FAILed. ---

[GitHub] spark issue #21975: [WIP][SPARK-25001][BUILD] Fix miscellaneous build warnin...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21975 **[Test build #94036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94036/testReport)** for PR 21975 at commit

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21721 **[Test build #94038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94038/testReport)** for PR 21721 at commit

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21889 **[Test build #94037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94037/testReport)** for PR 21889 at commit

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19449 **[Test build #94014 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94014/testReport)** for PR 19449 at commit

[GitHub] spark issue #21975: [WIP][SPARK-25001][BUILD] Fix miscellaneous build warnin...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21975 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21975: [WIP][SPARK-25001][BUILD] Fix miscellaneous build warnin...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21975 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21721 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21975: [WIP][SPARK-25001][BUILD] Fix miscellaneous build...

2018-08-02 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/21975 [WIP][SPARK-25001][BUILD] Fix miscellaneous build warnings ## What changes were proposed in this pull request? There are many warnings in the current build (for instance see

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21923 **[Test build #94034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94034/testReport)** for PR 21923 at commit

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94034/ Test FAILed. ---

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21974: [SPARK-25002][SQL] Avro: revise the output record namesp...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21974 **[Test build #94035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94035/testReport)** for PR 21974 at commit

[GitHub] spark issue #21974: [SPARK-25002][SQL] Avro: revise the output record namesp...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21974 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21974: [SPARK-25002][SQL] Avro: revise the output record namesp...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21974 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #21974: [SPARK-25002][SQL] Avro: revise the output record...

2018-08-02 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/21974 [SPARK-25002][SQL] Avro: revise the output record namespace ## What changes were proposed in this pull request? Currently the output namespace is starting with ".", e.g.

[GitHub] spark issue #21973: [BUILD] Fix lint-python.

2018-08-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21973 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21923 **[Test build #94034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94034/testReport)** for PR 21923 at commit

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21955 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21955 **[Test build #94031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94031/testReport)** for PR 21955 at commit

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21955 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94031/ Test FAILed. ---

[GitHub] spark issue #21915: [SPARK-24954][Core] Fail fast on job submit if run a bar...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21915 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94006/ Test FAILed. ---

[GitHub] spark issue #21915: [SPARK-24954][Core] Fail fast on job submit if run a bar...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21915 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21915: [SPARK-24954][Core] Fail fast on job submit if run a bar...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21915 **[Test build #94006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94006/testReport)** for PR 21915 at commit

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94005/ Test FAILed. ---

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94032/ Test FAILed. ---

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21923 **[Test build #94032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94032/testReport)** for PR 21923 at commit

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21964 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94007/ Test PASSed. ---

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21898 **[Test build #94005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94005/testReport)** for PR 21898 at commit

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21964 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21973: [BUILD] Fix lint-python.

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21973 **[Test build #94033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94033/testReport)** for PR 21973 at commit

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21964 **[Test build #94007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94007/testReport)** for PR 21964 at commit

[GitHub] spark issue #21973: [BUILD] Fix lint-python.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21973 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21973: [BUILD] Fix lint-python.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21973 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21955 BTW this only passes, I think, because I snipped out part of the log compaction test. I don't think it's valid to merge as is. Pinging @koeninger in case he's willing and able to weigh in on how to

[GitHub] spark pull request #21973: [BUILD] Fix lint-python.

2018-08-02 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/21973 [BUILD] Fix lint-python. ## What changes were proposed in this pull request? This pr fixes lint-python. ## How was this patch tested? Executed lint-python manually. You

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21923 **[Test build #94032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94032/testReport)** for PR 21923 at commit

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21923 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21955 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21955 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21721 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94008/ Test FAILed. ---

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21721 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21955 **[Test build #94031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94031/testReport)** for PR 21955 at commit

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21721 **[Test build #94008 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94008/testReport)** for PR 21721 at commit

[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...

2018-08-02 Thread jzhuge
Github user jzhuge commented on the issue: https://github.com/apache/spark/pull/21911 Thanks @gatorsmile. Will do. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint ...

2018-08-02 Thread jzhuge
Github user jzhuge commented on a diff in the pull request: https://github.com/apache/spark/pull/21911#discussion_r207303282 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveHintsSuite.scala --- @@ -17,15 +17,25 @@ package

[GitHub] spark pull request #21966: [SPARK-23915][SQL][followup] Add array_except fun...

2018-08-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21966#discussion_r207302241 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -4077,81 +4078,84 @@ case class

[GitHub] spark pull request #21966: [SPARK-23915][SQL][followup] Add array_except fun...

2018-08-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21966#discussion_r207302021 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -4077,81 +4078,84 @@ case class

[GitHub] spark issue #21919: [SPARK-24933][SS] Report numOutputRows in SinkProgress v...

2018-08-02 Thread vackosar
Github user vackosar commented on the issue: https://github.com/apache/spark/pull/21919 @jose-torres I haven't thought about this. Let me investigate bit more. Shall we return to this PR? Do you agree with extending WriterCommitMessage and using in DataWritingSparkTask#run to

[GitHub] spark issue #21952: [SPARK-24993] [SQL] [WIP] Make Avro Fast Again

2018-08-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21952 @dbtsai I didn't use Spark 2.3 when testing databricks-avro. I also used current master. But because a recent change of schema verifying (`FileFormat.supportDataType`) causes incompatibility, I

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #94030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94030/testReport)** for PR 21305 at commit

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21923#discussion_r207300158 --- Diff: core/src/main/java/org/apache/spark/AbstractExecutorPlugin.java --- @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...

2018-08-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21911 cc @maryannxue Please review it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-02 Thread cclauss
Github user cclauss commented on the issue: https://github.com/apache/spark/pull/20838 Several are fixed it this PR. Some others are not fixable without resorting to using __#noqa__. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spar...

2018-08-02 Thread rvesse
Github user rvesse commented on a diff in the pull request: https://github.com/apache/spark/pull/21669#discussion_r207299324 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -107,7 +109,14 @@

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-02 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21889 > I was able to run the first failing test successfully. Can we get a retest, please? @ajacques I just rebased and pushed my branch off of master. Perhaps the easiest thing to do would be

[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...

2018-08-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21911 Generally, it looks good to me. Let us target this to the upcoming 2.4 release We need to improve the test coverage: - Add a test case when users specify multiple REPARTITION,

[GitHub] spark issue #21889: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-02 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21889 > These test failures are in Spark streaming. Is this just an intermittent test failure or actually caused by this PR? I was able to run the first failing test successfully. Can we get a

[GitHub] spark pull request #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint ...

2018-08-02 Thread jzhuge
Github user jzhuge commented on a diff in the pull request: https://github.com/apache/spark/pull/21911#discussion_r207297428 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -102,6 +104,35 @@ object ResolveHints { }

[GitHub] spark pull request #21923: [SPARK-24918][Core] Executor Plugin api

2018-08-02 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/21923#discussion_r207297146 --- Diff: core/src/main/java/org/apache/spark/AbstractExecutorPlugin.java --- @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207296580 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContextImpl.scala --- @@ -39,8 +44,58 @@ private[spark] class BarrierTaskContextImpl(

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-02 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21948 I'm changing my +1 to -1 because read-side changes are mixed in and because copies are the responsibility of data sources if they buffer and hold references to earlier rows. ---

[GitHub] spark issue #21972: [SPARK-24795][CORE][FOLLOWUP] Combine BarrierTaskContext...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21972 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21972: [SPARK-24795][CORE][FOLLOWUP] Combine BarrierTaskContext...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21972 **[Test build #94029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94029/testReport)** for PR 21972 at commit

<    1   2   3   4   5   6   7   8   9   10   >