[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r159889119 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -323,11 +323,13 @@ object SQLConf {

[GitHub] spark pull request #20076: [SPARK-21786][SQL] When acquiring 'compressionCod...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20076#discussion_r159885686 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetCompressionCodecPrecedenceSuite.scala --- @@ -0,0 +1,117 @@

[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...

2018-01-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 Good point @felixcheung though the polling is hard-coded to 100ms, so will be close enough to "immediately". Allowing a value of 0 on both timeouts seems OK, but yeah not clear whether it does have

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20165 **[Test build #85725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85725/testReport)** for PR 20165 at commit

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20165 **[Test build #85724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85724/testReport)** for PR 20165 at commit

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20165 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85724/ Test FAILed. ---

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20165 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20165 **[Test build #85724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85724/testReport)** for PR 20165 at commit

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread xubo245
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159879314 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala --- @@ -62,6 +63,22 @@ class HiveOrcSourceSuite extends OrcSuite

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread xubo245
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159879115 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala --- @@ -62,6 +63,22 @@ class HiveOrcSourceSuite extends OrcSuite

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread xubo245
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159878689 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala --- @@ -62,6 +63,22 @@ class HiveOrcSourceSuite extends OrcSuite

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread xubo245
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159878683 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/HiveSerDe.scala --- @@ -72,7 +72,7 @@ object HiveSerDe { def sourceToSerDe(source:

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159878556 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala --- @@ -62,6 +63,22 @@ class HiveOrcSourceSuite extends

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159878203 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala --- @@ -62,6 +63,22 @@ class HiveOrcSourceSuite extends

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159878257 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcSourceSuite.scala --- @@ -62,6 +63,22 @@ class HiveOrcSourceSuite extends

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20165 **[Test build #85723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85723/testReport)** for PR 20165 at commit

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20165#discussion_r159877971 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/HiveSerDe.scala --- @@ -72,7 +72,7 @@ object HiveSerDe { def

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20165 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20155: [SPARK-22961][REGRESSION] Constant columns should...

2018-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20155 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20165: [SPARK-22972] Couldn't find corresponding Hive SerDe for...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20165 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20155: [SPARK-22961][REGRESSION] Constant columns should genera...

2018-01-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20155 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20165: [SPARK-22972] Couldn't find corresponding Hive Se...

2018-01-05 Thread xubo245
GitHub user xubo245 opened a pull request: https://github.com/apache/spark/pull/20165 [SPARK-22972] Couldn't find corresponding Hive SerDe for data source provider org.apache.spark.sql.hive.orc ## What changes were proposed in this pull request? Fix the warning: Couldn't

[GitHub] spark pull request #20162: [SPARK-22965] [PySpark] [SQL] Add deterministic p...

2018-01-05 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/20162 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20096 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20096 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85720/ Test FAILed. ---

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85720/testReport)** for PR 20096 at commit

[GitHub] spark issue #20163: [SPARK-22966][PySpark] Spark SQL should handle Python UD...

2018-01-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20163 The problem here seems, `returnType` is mismatched to the value. In case of `DateType`, it needs an explicit conversion into integers:

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-05 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20013 @vanzin I haven't run the code. I wonder which changes double the disk usage? The new indices or the cached quantiles? ---

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85721/ Test PASSed. ---

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20164 **[Test build #85721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85721/testReport)** for PR 20164 at commit

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #85722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85722/testReport)** for PR 19893 at commit

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159851476 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala --- @@ -17,4 +17,17 @@ package

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r159835376 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -119,118 +121,115 @@ private class LiveTask( import

[GitHub] spark pull request #20013: [SPARK-20657][core] Speed up rendering of the sta...

2018-01-05 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20013#discussion_r159844819 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -848,7 +853,7 @@ private[spark] class AppStatusListener(

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20164 **[Test build #85721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85721/testReport)** for PR 20164 at commit

[GitHub] spark pull request #20164: [SPARK-22971][ML] OneVsRestModel should use tempo...

2018-01-05 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/20164 [SPARK-22971][ML] OneVsRestModel should use temporary RawPredictionCol ## What changes were proposed in this pull request? use temporary RawPredictionCol in `OneVsRestModel#transform`

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159841944 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -31,15 +31,28 @@ import org.apache.spark.util.AccumulatorContext

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159842007 --- Diff: core/src/test/scala/org/apache/spark/SparkFunSuite.scala --- @@ -31,15 +31,28 @@ import org.apache.spark.util.AccumulatorContext

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159840754 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSessionStateSuite.scala --- @@ -28,14 +28,18 @@ import

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85720/testReport)** for PR 20096 at commit

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159838691 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/DataSourceAnalysisSuite.scala --- @@ -29,16 +29,24 @@ import

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159837804 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SessionStateSuite.scala --- @@ -39,6 +41,7 @@ class SessionStateSuite extends SparkFunSuite

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159834709 --- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159832729 --- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159832598 --- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159831631 --- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-05 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r159831366 --- Diff: core/src/test/scala/org/apache/spark/ThreadAudit.scala --- @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation

<    1   2   3