[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113870538 LGTM, thanks for fixing this! Merging to master and branch-1.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113870521 @andrewor14 They are not the same. #6864 affects dynamic partitioning feature of external data sources, while this one is about dynamic partitions of Hive. --- If

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/6833 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-18 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113351488 i think this pr is for hiveQL and #6864 is for common SQL. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113289197 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113289226 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113289891 [Test build #35171 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35171/consoleFull) for PR 6833 at commit

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113287867 ok to test. Is this issue the same as the one reported in #6864? @liancheng --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113312380 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-113312346 [Test build #35171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35171/console) for PR 6833 at commit

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112914725 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-17 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112914495 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112914749 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112914955 [Test build #35053 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35053/consoleFull) for PR 6833 at commit

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112946390 [Test build #35053 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35053/console) for PR 6833 at commit

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112946466 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-16 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/6833#discussion_r32523129 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveWriterContainers.scala --- @@ -230,7 +230,15 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-16 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/6833#discussion_r32516707 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -197,7 +197,6 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-16 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/6833#discussion_r32589270 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveWriterContainers.scala --- @@ -230,7 +230,15 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-16 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112620496 Seems only affect the dynamic partition in HiveContext, @jeanlyn can you confirm that? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-16 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112627047 also met this issue when dynamic partition in HiveContext --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-16 Thread jeanlyn
Github user jeanlyn commented on a diff in the pull request: https://github.com/apache/spark/pull/6833#discussion_r32592438 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveWriterContainers.scala --- @@ -230,7 +230,15 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-16 Thread jeanlyn
Github user jeanlyn commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112636409 @chenghao-intel ,I think it only affect the dynamic partition.Because `SparkHadoopWriter` get the write by `OutputFormat.getRecordWriter`,most of them use the

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/6833#issuecomment-112259097 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-15 Thread jeanlyn
GitHub user jeanlyn opened a pull request: https://github.com/apache/spark/pull/6833 [SPARK-8379][SQL]avoid speculative tasks write to the same file The issue link [SPARK-8379](https://issues.apache.org/jira/browse/SPARK-8379) Currently,when we insert data to the dynamic

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-15 Thread jeanlyn
Github user jeanlyn commented on a diff in the pull request: https://github.com/apache/spark/pull/6833#discussion_r32492419 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -197,7 +197,6 @@ case class InsertIntoHiveTable(

[GitHub] spark pull request: [SPARK-8379][SQL]avoid speculative tasks write...

2015-06-15 Thread jeanlyn
Github user jeanlyn commented on a diff in the pull request: https://github.com/apache/spark/pull/6833#discussion_r32491951 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -197,7 +197,6 @@ case class InsertIntoHiveTable(