[GitHub] spark pull request #22778: [SPARK-25784][SQL] Infer filters from constraints...

2018-10-29 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22778#discussion_r229181821 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -171,10 +171,13 @@ abstract class

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-10-29 Thread gss2002
Github user gss2002 commented on the issue: https://github.com/apache/spark/pull/22867 @vanzin this seems to work.. Not sure what your thoughts are on this private val tmpDir = broadcastedHadoopConf.value.get("hadoop.tmp.dir", System.getProperty("java.io.tmpdir"))

[GitHub] spark pull request #22778: [SPARK-25784][SQL] Infer filters from constraints...

2018-10-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22778#discussion_r229178084 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -171,10 +171,13 @@ abstract class

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-10-29 Thread gss2002
Github user gss2002 commented on the issue: https://github.com/apache/spark/pull/22867 @vanzin I made the following change and it didn't work. How do you want to proceed? --- a/streaming/src/main/scala/org/apache/spark/streaming/rdd/WriteAheadLogBackedBlockRDD.scala +++

[GitHub] spark issue #22879: [SPARK-25872][SQL][TEST] Add an optimizer tracker for TP...

2018-10-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22879 For generating the golden files, I would not generate the whole plan. Instead, we can keep the node name. That should be good enough. ---

[GitHub] spark issue #22885: [BUILD][MINOR] release script should not interrupt by sv...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22885 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22885: [BUILD][MINOR] release script should not interrupt by sv...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22885 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22857 Please be really careful in null handling. It could easily introduce the correctness bugs like what we recently fixed. ---

[GitHub] spark issue #22885: [BUILD][MINOR] release script should not interrupt by sv...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22885 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98250/ Test FAILed. ---

[GitHub] spark issue #22885: [BUILD][MINOR] release script should not interrupt by sv...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22885 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22885: [BUILD][MINOR] release script should not interrupt by sv...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22885 **[Test build #98250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98250/testReport)** for PR 22885 at commit

[GitHub] spark issue #22885: [BUILD][MINOR] release script should not interrupt by sv...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22885 **[Test build #98250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98250/testReport)** for PR 22885 at commit

[GitHub] spark pull request #22885: [BUILD][MINOR] release script should not interrup...

2018-10-29 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/22885 [BUILD][MINOR] release script should not interrupt by svn ## What changes were proposed in this pull request? When running the release script, you will be interrupted unexpectedly

[GitHub] spark issue #22885: [BUILD][MINOR] release script should not interrupt by sv...

2018-10-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22885 cc @vanzin @srowen @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-10-29 Thread Gauravshah
Github user Gauravshah commented on the issue: https://github.com/apache/spark/pull/21320 backported to 2.3.2 just in case somebody needs it. https://github.com/Gauravshah/spark/tree/branch-2.3_SPARK-4502 Thanks @mallman ---

[GitHub] spark issue #22884: [SPARK-23429][CORE][FOLLOWUP] MetricGetter should rename...

2018-10-29 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22884 good catch, lgtm --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22844: [SPARK-25847][SQL][TEST] Refactor JSONBenchmarks to use ...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22844 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22844: [SPARK-25847][SQL][TEST] Refactor JSONBenchmarks to use ...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22844 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98242/ Test PASSed. ---

[GitHub] spark issue #22881: [SPARK-25855][CORE] Don't use erasure coding for event l...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22881 **[Test build #98249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98249/testReport)** for PR 22881 at commit

[GitHub] spark issue #22844: [SPARK-25847][SQL][TEST] Refactor JSONBenchmarks to use ...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22844 **[Test build #98242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98242/testReport)** for PR 22844 at commit

[GitHub] spark issue #22881: [SPARK-25855][CORE] Don't use erasure coding for event l...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22881 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22881: [SPARK-25855][CORE] Don't use erasure coding for event l...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22881 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22881: [SPARK-25855][CORE] Don't use erasure coding for event l...

2018-10-29 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22881 thanks for the review @vanzin & @kiszk , updated --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22881: [SPARK-25855][CORE] Don't use erasure coding for ...

2018-10-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22881#discussion_r229172448 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -471,4 +473,42 @@ object SparkHadoopUtil {

[GitHub] spark issue #22845: [SPARK-25848][SQL][TEST] Refactor CSVBenchmarks to use m...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22845 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98241/ Test PASSed. ---

[GitHub] spark issue #22845: [SPARK-25848][SQL][TEST] Refactor CSVBenchmarks to use m...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22845 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22845: [SPARK-25848][SQL][TEST] Refactor CSVBenchmarks to use m...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22845 **[Test build #98241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98241/testReport)** for PR 22845 at commit

[GitHub] spark issue #22884: [SPARK-23429][CORE][FOLLOWUP] MetricGetter should rename...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22884 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22884: [SPARK-23429][CORE][FOLLOWUP] MetricGetter should rename...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22884 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22884: [SPARK-23429][CORE][FOLLOWUP] MetricGetter should rename...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22884 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22884: [SPARK-23429][CORE][FOLLOWUP] MetricGetter should...

2018-10-29 Thread LantaoJin
GitHub user LantaoJin opened a pull request: https://github.com/apache/spark/pull/22884 [SPARK-23429][CORE][FOLLOWUP] MetricGetter should rename to ExecutorMetricType in comments ## What changes were proposed in this pull request? MetricGetter should rename to

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22857 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98239/ Test PASSed. ---

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22857 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22857 **[Test build #98239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98239/testReport)** for PR 22857 at commit

[GitHub] spark pull request #22857: [SPARK-25860][SQL] Replace Literal(null, _) with ...

2018-10-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22857#discussion_r229165719 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -736,3 +736,60 @@ object CombineConcats extends

[GitHub] spark pull request #22857: [SPARK-25860][SQL] Replace Literal(null, _) with ...

2018-10-29 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22857#discussion_r229165497 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -736,3 +736,60 @@ object CombineConcats extends

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22309 **[Test build #98248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98248/testReport)** for PR 22309 at commit

[GitHub] spark issue #16429: [SPARK-19019][PYTHON] Fix hijacked `collections.namedtup...

2018-10-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16429 This is fixed from Spark 1.6.4, 2.0.3, 2.1.1 and 2.2.0. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22878 Just quickly and roughly tested. Merge script looks only recognising main author of each commit in a PR. Let's just push a commit into here. ---

[GitHub] spark issue #16429: [SPARK-19019][PYTHON] Fix hijacked `collections.namedtup...

2018-10-29 Thread jamal119
Github user jamal119 commented on the issue: https://github.com/apache/spark/pull/16429 hi,HyukjinKwon,have you solved your problem,my error is exactly the same as yours,looking forward to your reply. --- - To

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22878 I wonder if that can be handled by merge script tho. I think it's okay just to pick up some commits there and rebase them to here even if they are empty commits. That's easier for committers to

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22878 @xuanyuanking , thanks for the work! You can try editing the previous commit message https://help.github.com/articles/creating-a-commit-with-multiple-authors/ , and then `push -f`. ---

[GitHub] spark issue #22861: [SPARK-25663][SPARK-25661][SQL][TEST] Refactor BuiltInDa...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22861 **[Test build #98247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98247/testReport)** for PR 22861 at commit

[GitHub] spark pull request #22861: [SPARK-25663][SPARK-25661][SQL][TEST] Refactor Bu...

2018-10-29 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/22861#discussion_r229162191 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/execution/benchmark/AvroWriteBenchmark.scala --- @@ -19,22 +19,17 @@ package

[GitHub] spark issue #22778: [SPARK-25784][SQL] Infer filters from constraints after ...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22778 **[Test build #98246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98246/testReport)** for PR 22778 at commit

[GitHub] spark issue #22778: [SPARK-25784][SQL] Infer filters from constraints after ...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22778 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22778: [SPARK-25784][SQL] Infer filters from constraints after ...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22778 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22857 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98240/ Test PASSed. ---

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22857 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22857 **[Test build #98240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98240/testReport)** for PR 22857 at commit

[GitHub] spark issue #22861: [SPARK-25663][SPARK-25661][SQL][TEST] Refactor BuiltInDa...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22861 **[Test build #98245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98245/testReport)** for PR 22861 at commit

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22878 also cc @bdrillard, link this to #21348. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22878 @dongjoon-hyun Thanks for your comment, let me see how to achieve this, @bdrillard 's commits based on databricks/spark-avro. ---

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22624 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22624 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98238/ Test PASSed. ---

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22624 **[Test build #98238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98238/testReport)** for PR 22624 at commit

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22878 **[Test build #98244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98244/testReport)** for PR 22878 at commit

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22878 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22878: [SPARK-25789][SQL] Support for Dataset of Avro

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22878 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-29 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r229156708 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -128,6 +128,15 @@ object ScalaReflection extends

[GitHub] spark issue #22666: [SPARK-25672][SQL] schema_of_csv() - schema inference fr...

2018-10-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22666 Thanks, @cloud-fan. The change looks good to me from my side. Let me take another look for this and leave a sign-off (which means a sign-off for @MaxGekk's code changes) ---

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98236/ Test PASSed. ---

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-29 Thread mt40
Github user mt40 commented on the issue: https://github.com/apache/spark/pull/22309 @cloud-fan while what you said is true, I think it's not that bad. Overall, the only major special logic is inside the case class `case` and this is due to the special nature of value class. On

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #98236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98236/testReport)** for PR 21688 at commit

[GitHub] spark pull request #22868: [SPARK-25833][SQL][DOCS] Update migration guide f...

2018-10-29 Thread seancxmao
Github user seancxmao commented on a diff in the pull request: https://github.com/apache/spark/pull/22868#discussion_r229156006 --- Diff: docs/sql-migration-guide-hive-compatibility.md --- @@ -51,6 +51,22 @@ Spark SQL supports the vast majority of Hive features, such as: *

[GitHub] spark pull request #22755: [SPARK-25755][SQL][Test] Supplementation of non-C...

2018-10-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22755#discussion_r229155463 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -669,22 +669,20 @@ class DataFrameAggregateSuite extends

[GitHub] spark pull request #22881: [SPARK-25855][CORE] Don't use erasure coding for ...

2018-10-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22881#discussion_r229155491 --- Diff: docs/configuration.md --- @@ -761,6 +761,17 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #22755: [SPARK-25755][SQL][Test] Supplementation of non-C...

2018-10-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22755#discussion_r229154981 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -459,6 +459,7 @@ private[spark] class Executor(

[GitHub] spark pull request #22868: [SPARK-25833][SQL][DOCS] Update migration guide f...

2018-10-29 Thread seancxmao
Github user seancxmao commented on a diff in the pull request: https://github.com/apache/spark/pull/22868#discussion_r229155030 --- Diff: docs/sql-migration-guide-hive-compatibility.md --- @@ -53,7 +53,20 @@ Spark SQL supports the vast majority of Hive features, such as: *

[GitHub] spark pull request #22881: [SPARK-25855][CORE] Don't use erasure coding for ...

2018-10-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22881#discussion_r229154733 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -471,4 +473,42 @@ object SparkHadoopUtil {

[GitHub] spark pull request #22666: [SPARK-25672][SQL] schema_of_csv() - schema infer...

2018-10-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22666#discussion_r229153592 --- Diff: sql/core/src/test/resources/sql-tests/inputs/csv-functions.sql --- @@ -7,3 +7,11 @@ select from_csv('1', 'a InvalidType'); select

[GitHub] spark issue #22868: [SPARK-25833][SQL][DOCS] Update migration guide for Hive...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22868 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98243/ Test PASSed. ---

[GitHub] spark issue #22868: [SPARK-25833][SQL][DOCS] Update migration guide for Hive...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22868 **[Test build #98243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98243/testReport)** for PR 22868 at commit

[GitHub] spark issue #22868: [SPARK-25833][SQL][DOCS] Update migration guide for Hive...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22868 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22857: [SPARK-25860][SQL] Replace Literal(null, _) with ...

2018-10-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22857#discussion_r229151278 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -736,3 +736,60 @@ object CombineConcats extends

[GitHub] spark issue #22504: [SPARK-25118][Submit] Persist Driver Logs in Client mode...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22504 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22504: [SPARK-25118][Submit] Persist Driver Logs in Client mode...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98234/ Test PASSed. ---

[GitHub] spark pull request #22857: [SPARK-25860][SQL] Replace Literal(null, _) with ...

2018-10-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22857#discussion_r229150341 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -736,3 +736,65 @@ object CombineConcats extends

[GitHub] spark issue #22504: [SPARK-25118][Submit] Persist Driver Logs in Client mode...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22504 **[Test build #98234 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98234/testReport)** for PR 22504 at commit

[GitHub] spark pull request #22868: [SPARK-25833][SQL][DOCS] Update migration guide f...

2018-10-29 Thread seancxmao
Github user seancxmao commented on a diff in the pull request: https://github.com/apache/spark/pull/22868#discussion_r229150147 --- Diff: docs/sql-migration-guide-hive-compatibility.md --- @@ -51,6 +51,22 @@ Spark SQL supports the vast majority of Hive features, such as: *

[GitHub] spark issue #22868: [SPARK-25833][SQL][DOCS] Update migration guide for Hive...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22868 **[Test build #98243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98243/testReport)** for PR 22868 at commit

[GitHub] spark pull request #22857: [SPARK-25860][SQL] Replace Literal(null, _) with ...

2018-10-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22857#discussion_r229150101 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2578,4 +2578,45 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request #22877: [MINOR][SQL] Avoid hardcoded configuration keys i...

2018-10-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22877#discussion_r229148363 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -408,15 +408,16 @@ object SQLConf { val

[GitHub] spark issue #22844: [SPARK-25847][SQL][TEST] Refactor JSONBenchmarks to use ...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22844 **[Test build #98242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98242/testReport)** for PR 22844 at commit

[GitHub] spark issue #22845: [SPARK-25848][SQL][TEST] Refactor CSVBenchmarks to use m...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22845 **[Test build #98241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98241/testReport)** for PR 22845 at commit

[GitHub] spark issue #22882: [SPARK-25871][STREAMING][WIP] Don't use EC for streaming...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22882 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22882: [SPARK-25871][STREAMING][WIP] Don't use EC for streaming...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22882 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98229/ Test PASSed. ---

[GitHub] spark issue #22882: [SPARK-25871][STREAMING][WIP] Don't use EC for streaming...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22882 **[Test build #98229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98229/testReport)** for PR 22882 at commit

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-10-29 Thread gss2002
Github user gss2002 commented on the issue: https://github.com/apache/spark/pull/22867 @vanzin trying this.. I'll advise shortly private val tmpDir = "file:///" + System.getProperty("java.io.tmpdir") ---

[GitHub] spark pull request #22877: [MINOR][SQL] Avoid hardcoded configuration keys i...

2018-10-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22877 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22877: [MINOR][SQL] Avoid hardcoded configuration keys in SQLCo...

2018-10-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22877 Thanks, @kiszk and @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-10-29 Thread erikerlandson
Github user erikerlandson commented on the issue: https://github.com/apache/spark/pull/22146 @mccheah integration testing is passing with the latest container selection policy, good to merge? --- - To unsubscribe,

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-10-29 Thread gss2002
Github user gss2002 commented on the issue: https://github.com/apache/spark/pull/22867 History from JIRA and error WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access to HDFS path due to it using a similar name was $PWD folder from YARN AM Cluster

[GitHub] spark issue #22877: [MINOR][SQL] Avoid hardcoded configuration keys in SQLCo...

2018-10-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22877 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22857: [SPARK-25860][SQL] Replace Literal(null, _) with ...

2018-10-29 Thread aokolnychyi
Github user aokolnychyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22857#discussion_r229133793 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2578,4 +2578,45 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark pull request #22857: [SPARK-25860][SQL] Replace Literal(null, _) with ...

2018-10-29 Thread aokolnychyi
Github user aokolnychyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22857#discussion_r229133550 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -736,3 +736,65 @@ object CombineConcats extends

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22857 **[Test build #98240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98240/testReport)** for PR 22857 at commit

[GitHub] spark issue #22771: [SPARK-25773][Core]Cancel zombie tasks in a result stage...

2018-10-29 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/22771 @markhamstra any further comments? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22857: [SPARK-25860][SQL] Replace Literal(null, _) with FalseLi...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22857 Test PASSed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   >