[GitHub] spark issue #22688: [SPARK-25700][SQL] Creates ReadSupport in only Append Mo...

2018-10-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22688 Another concern is, it doesn't sound to me straightforward that readsupport is created and executed in write path. So, at least, we should just read the schema when it's needed (when

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-10 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22641 Thank you very much @srowen @fjh100456 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread niofire
Github user niofire commented on the issue: https://github.com/apache/spark/pull/22593 From https://spark.apache.org/docs/2.3.2/api/java/org/apache/spark/sql/streaming/DataStreamWriter.html

[GitHub] spark issue #22688: [SPARK-25700][SQL] Creates ReadSupport in only Append Mo...

2018-10-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22688 Other datasources such as Parquet, ORC and JDBC won't be adoptable to the current design since we can't read the schema from the information given the code I provided above. ---

[GitHub] spark pull request #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve te...

2018-10-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22641 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21524: [SPARK-24212][ML][doc] Add the example and user guide fo...

2018-10-10 Thread tengpeng
Github user tengpeng commented on the issue: https://github.com/apache/spark/pull/21524 Let's close it for now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21524: [SPARK-24212][ML][doc] Add the example and user g...

2018-10-10 Thread tengpeng
Github user tengpeng closed the pull request at: https://github.com/apache/spark/pull/21524 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22641: [SPARK-25611][SPARK-25612][SQL][TESTS] Improve test run ...

2018-10-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22641 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22688: [SPARK-25700][SQL] Creates ReadSupport in only Append Mo...

2018-10-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22688 The point is not write only datasource, @cloud-fan. For instance, `spark.read.range(1).format("source").write("non-existent-path")`. There's no way to read the schema. There's no complaints so

[GitHub] spark issue #22671: [SPARK-25615][SQL][TEST] Improve the test runtime of Kaf...

2018-10-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22671 I like the change. This test is down to 5 seconds now. Unfortunately I don't see speedup in other Kafka tests, but I think we should leave this in. ---

[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97197/ Test PASSed. ---

[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #97197 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97197/testReport)** for PR 22583 at commit

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22593 Have a look at `Column.scala` and `Dataset.scala` in `org.apache.spark.sql`. But, on a second look, this is how I see the lists render:

[GitHub] spark pull request #21816: [SPARK-24794][CORE] Driver launched through rest ...

2018-10-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21816#discussion_r224128493 --- Diff: core/src/test/scala/org/apache/spark/deploy/rest/StandaloneRestSubmitSuite.scala --- @@ -83,6 +83,26 @@ class StandaloneRestSubmitSuite extends

[GitHub] spark pull request #22672: [SPARK-25605][TESTS] Alternate take. Run cast str...

2018-10-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22672 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22593 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22593 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97204/ Test FAILed. ---

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22593 **[Test build #97204 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97204/testReport)** for PR 22593 at commit

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread niofire
Github user niofire commented on the issue: https://github.com/apache/spark/pull/22593 @srowen Guessing you're referring to some other files? Could you link? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22593 **[Test build #97204 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97204/testReport)** for PR 22593 at commit

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22237 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97198/ Test FAILed. ---

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22237 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22237 **[Test build #97198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97198/testReport)** for PR 22237 at commit

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-10 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r224113173 --- Diff: docs/configuration.md --- @@ -1583,6 +1583,14 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-10 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r224105069 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -146,21 +146,31 @@ private[scheduler] class BlacklistTracker (

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-10 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r224111068 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -117,6 +117,10 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-10 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r224107221 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +419,61 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22593: [Streaming][DOC] Fix typo & format in DataStreamWriter.s...

2018-10-10 Thread niofire
Github user niofire commented on the issue: https://github.com/apache/spark/pull/22593 Sorry, was out of town, will update --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-10-10 Thread abellina
Github user abellina commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r224108070 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -290,17 +210,11 @@ private[ui] class StagePage(parent: StagesTab, store:

[GitHub] spark issue #22688: [SPARK-25700][SQL] Creates ReadSupport in only Append Mo...

2018-10-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22688 Do we have users complain about it? In the new write API design, a data source must provide a schema. I don't think it's practical that people need a write-only data source which can

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-10-10 Thread abellina
Github user abellina commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r224106218 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/StagesResource.scala --- @@ -102,4 +103,124 @@ private[v1] class StagesResource extends

[GitHub] spark pull request #22668: [SPARK-25675] [Spark Job History] Job UI page doe...

2018-10-10 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22668#discussion_r224101829 --- Diff: core/src/main/scala/org/apache/spark/ui/PagedTable.scala --- @@ -123,10 +123,9 @@ private[ui] trait PagedTable[T] { /** *

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22674 **[Test build #97203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97203/testReport)** for PR 22674 at commit

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22688: [SPARK-25700][SQL] Creates ReadSupport in only Append Mo...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22688 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22688: [SPARK-25700][SQL] Creates ReadSupport in only Append Mo...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97196/ Test PASSed. ---

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22657 **[Test build #97202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97202/testReport)** for PR 22657 at commit

[GitHub] spark issue #22688: [SPARK-25700][SQL] Creates ReadSupport in only Append Mo...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22688 **[Test build #97196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97196/testReport)** for PR 22688 at commit

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97200/ Test FAILed. ---

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22674 **[Test build #97200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97200/testReport)** for PR 22674 at commit

[GitHub] spark pull request #22657: [SPARK-25670][TEST] Reduce number of tested timez...

2018-10-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22657#discussion_r224091433 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTestUtils.scala --- @@ -26,6 +26,16 @@ object DateTimeTestUtils {

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22674 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-10 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22657 > Then we don't need any randomness here, just pick one timezone(like PST?) and test it. I took 8 out of 627. Going to do the same in the PR: https://github.com/apache/spark/pull/22379

[GitHub] spark pull request #22657: [SPARK-25670][TEST] Reduce number of tested timez...

2018-10-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22657#discussion_r224086763 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTestUtils.scala --- @@ -26,6 +26,16 @@ object DateTimeTestUtils {

[GitHub] spark issue #22674: [SPARK-25680][SQL] SQL execution listener shouldn't happ...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22674 **[Test build #97200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97200/testReport)** for PR 22674 at commit

[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22657 **[Test build #97201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97201/testReport)** for PR 22657 at commit

[GitHub] spark issue #22686: [WIP][SPARK-25700][SQL] Partially revert append mode sup...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22686 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97195/ Test PASSed. ---

[GitHub] spark issue #22686: [WIP][SPARK-25700][SQL] Partially revert append mode sup...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22686 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22686: [WIP][SPARK-25700][SQL] Partially revert append mode sup...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22686 **[Test build #97195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97195/testReport)** for PR 22686 at commit

[GitHub] spark pull request #20999: [SPARK-14922][SPARK-17732][SPARK-23866][SQL] Supp...

2018-10-10 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20999#discussion_r224082261 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -521,35 +521,114 @@ case class AlterTableRenamePartitionCommand(

[GitHub] spark issue #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22685 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22685 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97194/ Test PASSed. ---

[GitHub] spark issue #22687: [SPARK-25702][SQL] Push down filters with `Not` operator...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22687 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22687: [SPARK-25702][SQL] Push down filters with `Not` operator...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22687 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97193/ Test PASSed. ---

[GitHub] spark issue #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22685 **[Test build #97194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97194/testReport)** for PR 22685 at commit

[GitHub] spark issue #22687: [SPARK-25702][SQL] Push down filters with `Not` operator...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22687 **[Test build #97193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97193/testReport)** for PR 22687 at commit

[GitHub] spark pull request #22381: [SPARK-25394][CORE] Add an application status met...

2018-10-10 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/22381#discussion_r224071384 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusSource.scala --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #22674: [SPARK-25680][SQL] SQL execution listener shouldn...

2018-10-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22674#discussion_r224071116 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3356,21 +3356,11 @@ class Dataset[T] private[sql]( * user-registered

[GitHub] spark pull request #22381: [SPARK-25394][CORE] Add an application status met...

2018-10-10 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/22381#discussion_r224071170 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -280,6 +284,11 @@ private[spark] class AppStatusListener(

[GitHub] spark issue #22684: [SPARK-25699][SQL] Partially push down conjunctive predi...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22684 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-10 Thread lbialy
Github user lbialy commented on the issue: https://github.com/apache/spark/pull/22309 Hello. I've just found this PR while trying to debug serialization errors for a case class containing value classes. This would be very helpful! ---

[GitHub] spark issue #22684: [SPARK-25699][SQL] Partially push down conjunctive predi...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22684 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22684: [SPARK-25699][SQL] Partially push down conjunctive predi...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22684 **[Test build #97199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97199/testReport)** for PR 22684 at commit

[GitHub] spark issue #22381: [SPARK-25394][CORE] Add an application status metrics so...

2018-10-10 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/22381 Thanx a lot @aditanase ! I will update the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22381: [SPARK-25394][CORE] Add an application status met...

2018-10-10 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/22381#discussion_r224070656 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusSource.scala --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-10-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r224069683 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/FailureSafeParser.scala --- @@ -15,50 +15,57 @@ * limitations under the

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97188/ Test PASSed. ---

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97188/testReport)** for PR 22598 at commit

[GitHub] spark pull request #22682: [SPARK-20946][SPARK-25525][SQL][FOLLOW-UP] Update...

2018-10-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22682 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22682: [SPARK-20946][SPARK-25525][SQL][FOLLOW-UP] Update the mi...

2018-10-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22682 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22684: [SPARK-25699][SQL] Partially push down conjunctive predi...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22684 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97187/ Test PASSed. ---

[GitHub] spark issue #22684: [SPARK-25699][SQL] Partially push down conjunctive predi...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22684 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22684: [SPARK-25699][SQL] Partially push down conjunctive predi...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22684 **[Test build #97187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97187/testReport)** for PR 22684 at commit

[GitHub] spark pull request #22687: [SPARK-25702][SQL] Push down filters with `Not` o...

2018-10-10 Thread gengliangwang
Github user gengliangwang closed the pull request at: https://github.com/apache/spark/pull/22687 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22687: [SPARK-25702][SQL] Push down filters with `Not` operator...

2018-10-10 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22687 It's OK. Close this one. Thanks for reviewing @viirya @HyukjinKwon --- - To unsubscribe, e-mail:

[GitHub] spark issue #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22685 @HyukjinKwon Sorry. Happened to find that some code is not elegant according to my taste. I have not got the fact that massive changes are not good for backporting. As a result, I spent some time

[GitHub] spark issue #22686: [WIP][SPARK-25700][SQL] Partially revert append mode sup...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22686 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22686: [WIP][SPARK-25700][SQL] Partially revert append mode sup...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22686 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97192/ Test FAILed. ---

[GitHub] spark issue #22686: [WIP][SPARK-25700][SQL] Partially revert append mode sup...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22686 **[Test build #97192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97192/testReport)** for PR 22686 at commit

[GitHub] spark pull request #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22685#discussion_r224054521 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala --- @@ -147,7 +147,7 @@ private[sql] object

[GitHub] spark pull request #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22685#discussion_r224054362 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLMetrics.scala --- @@ -95,7 +95,7 @@ object SQLMetrics { def

[GitHub] spark pull request #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22685#discussion_r224053894 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyValueGroupedDataset.scala --- @@ -567,10 +567,10 @@ class KeyValueGroupedDataset[K, V]

[GitHub] spark pull request #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22685#discussion_r224054020 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -188,7 +188,7 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark pull request #22685: [SQL][MINOR][Refactor] Refactor on sql/core

2018-10-10 Thread sadhen
Github user sadhen commented on a diff in the pull request: https://github.com/apache/spark/pull/22685#discussion_r224053653 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala --- @@ -123,7 +123,7 @@ object StaticSQLConf { val

[GitHub] spark issue #22626: [SPARK-25638][SQL] Adding new function - to_csv()

2018-10-10 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22626 @HyukjinKwon Could you look at this PR one more time, please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22666: [SPARK-25672][SQL] schema_of_csv() - schema inference fr...

2018-10-10 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22666 @HyukjinKwon May I ask you to look at this PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22237 **[Test build #97198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97198/testReport)** for PR 22237 at commit

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-10-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r224044435 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/FailureSafeParser.scala --- @@ -15,50 +15,57 @@ * limitations under the

[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22583 **[Test build #97197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97197/testReport)** for PR 22583 at commit

[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22583: [SPARK-10816][SS] SessionWindow support for Structure St...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22583 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-10-10 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/16677 Yes sure , i will create a ticket for this issue and Keep you guys in loop. Thanks --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-10-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r224042803 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,30 @@ case class JsonToStructs(

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-10-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r224042251 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,30 @@ case class JsonToStructs(

[GitHub] spark issue #22687: [SPARK-25702][SQL] Push down filters with `Not` operator...

2018-10-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22687 I prefer not to add code that will not run. Let's see others options too. --- - To unsubscribe, e-mail:

<    1   2   3   4   5   >