[GitHub] spark pull request #22504: [SPARK-25118][Submit] Persist Driver Logs in Clie...

2018-10-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22504#discussion_r226741280 --- Diff: core/src/test/scala/org/apache/spark/util/logging/DriverLoggerSuite.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark ...

2018-10-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22501#discussion_r226740901 --- Diff: sql/core/benchmarks/WideSchemaBenchmark-results.txt --- @@ -1,117 +1,145 @@ -Java HotSpot(TM) 64-Bit Server VM 1.8.0_92-b14 on Mac OS X

[GitHub] spark issue #22501: [SPARK-25492][TEST] Refactor WideSchemaBenchmark to use ...

2018-10-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22501 Hi, @wangyum . I ran the test on EC2 `r3.xlarge`, too. It looks more stable than this. Could you review and merge https://github.com/wangyum/spark/pull/19 ? ---

[GitHub] spark issue #22748: [SPARK-25745][K8S] Improve docker-image-tool.sh script

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22748 **[Test build #97618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97618/testReport)** for PR 22748 at commit

[GitHub] spark issue #22748: [SPARK-25745][K8S] Improve docker-image-tool.sh script

2018-10-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22748 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #15622: [SPARK-18092][ML] Fix column prediction type error

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15622 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22705: [SPARK-25704][CORE] Allocate a bit less than Int....

2018-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22705 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22705: [SPARK-25704][CORE] Allocate a bit less than Int.MaxValu...

2018-10-19 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22705 merged to master / 2.4. Something went wrong w/ the merge script when putting it on 2.4 (I think I accidentally had a dirty working dir at the time), but it was actually a clean cherry

[GitHub] spark pull request #22710: DO NOT MERGE

2018-10-19 Thread squito
Github user squito closed the pull request at: https://github.com/apache/spark/pull/22710 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22756: [SPARK-25758][ML] Deprecate computeCost on BisectingKMea...

2018-10-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22756 I also understand today's situation and agree with @holdenk 's thought about SPARK-25765 as a blocker. Ping @cloud-fan since you are a release manager. How can we proceed SPARK-25765?

[GitHub] spark issue #22504: [SPARK-25118][Submit] Persist Driver Logs in Client mode...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22504 **[Test build #97617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97617/testReport)** for PR 22504 at commit

[GitHub] spark pull request #22504: [SPARK-25118][Submit] Persist Driver Logs in Clie...

2018-10-19 Thread ankuriitg
Github user ankuriitg commented on a diff in the pull request: https://github.com/apache/spark/pull/22504#discussion_r226718233 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -800,14 +817,33 @@ private[history] class

[GitHub] spark issue #22756: [SPARK-25758][ML] Deprecate computeCost on BisectingKMea...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22756 I'm seeing this linked from https://github.com/apache/spark/pull/22764 and I'm wondering if we need to revert this. If the information is not actually available where we tell folks it is I think we

[GitHub] spark pull request #22504: [SPARK-25118][Submit] Persist Driver Logs in Clie...

2018-10-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22504#discussion_r226716880 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -800,14 +817,33 @@ private[history] class

[GitHub] spark issue #12066: [SPARK-7424] [ML] ML ClassificationModel should add meta...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/12066 Hey @yanboliang I think improved metadata on the pipeline would be great, but if this abonded I get it if so do you want to close this PR and switch the JIRA back to open so someone else can take a

[GitHub] spark issue #22729: [SPARK-25737][CORE] Remove JavaSparkContextVarargsWorkar...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22729 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22729: [SPARK-25737][CORE] Remove JavaSparkContextVarargsWorkar...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22729 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21522: [SPARK-24467][ML] VectorAssemblerEstimator

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21522 cc @jkbradley again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22729: [SPARK-25737][CORE] Remove JavaSparkContextVarargsWorkar...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22729 **[Test build #97616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97616/testReport)** for PR 22729 at commit

[GitHub] spark issue #22729: [SPARK-25737][CORE] Remove JavaSparkContextVarargsWorkar...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22729 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22775 The latest Python failure looks relevant to this PR. ```scala AnalysisException: u"cannot resolve 'schemaofjson(`value`)' due to data type mismatch: The input json should be a string

[GitHub] spark pull request #22504: [SPARK-25118][Submit] Persist Driver Logs in Clie...

2018-10-19 Thread ankuriitg
Github user ankuriitg commented on a diff in the pull request: https://github.com/apache/spark/pull/22504#discussion_r226710526 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -800,14 +817,33 @@ private[history] class

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22425 Gentle ping, whats up? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18457: [SPARK-21241][MLlib]- Add setIntercept to StreamingLinea...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18457 Would you be OK closing this PR @SoulGuedria --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22533: [SPARK-18818][PYTHON] Add 'ascending' parameter to Windo...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22533 Jenkins OK to test. @annamolchanova if you want help making the Scala version of this PR first I'd be happy to lend what help I can. ---

[GitHub] spark pull request #22504: [SPARK-25118][Submit] Persist Driver Logs in Clie...

2018-10-19 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22504#discussion_r226706867 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -800,14 +817,33 @@ private[history] class

[GitHub] spark issue #21157: [SPARK-22674][PYTHON] Removed the namedtuple pickling pa...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21157 If removing the hack entirely is going to brake named tuples defined in the repl I'm a -1 on that change. While we certainly are more free to make breaking API changes in a majour version release

[GitHub] spark pull request #22263: [SPARK-25269][SQL] SQL interface support specify ...

2018-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22263 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-19 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22295 I'll leave this for if @HyukjinKwon has any final comments, otherwise I'm happy to merge. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22263: [SPARK-25269][SQL] SQL interface support specify Storage...

2018-10-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22263 Thank you, @wangyum , @cloud-fan , @maropu , @mgaido91 ! --- - To unsubscribe, e-mail:

[GitHub] spark issue #22774: [SPARK-25780][CORE]Scheduling the tasks which have no hi...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97611/ Test PASSed. ---

[GitHub] spark issue #22774: [SPARK-25780][CORE]Scheduling the tasks which have no hi...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22774 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22764: [SPARK-25765][ML] Add training cost to BisectingK...

2018-10-19 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22764#discussion_r226693546 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/BisectingKMeansModel.scala --- @@ -225,13 +227,14 @@ object BisectingKMeansModel extends

[GitHub] spark issue #22774: [SPARK-25780][CORE]Scheduling the tasks which have no hi...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22774 **[Test build #97611 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97611/testReport)** for PR 22774 at commit

[GitHub] spark pull request #22764: [SPARK-25765][ML] Add training cost to BisectingK...

2018-10-19 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22764#discussion_r226688846 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/BisectingKMeansModel.scala --- @@ -225,13 +227,14 @@ object BisectingKMeansModel extends

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22466 **[Test build #97615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97615/testReport)** for PR 22466 at commit

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22466 **[Test build #97614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97614/testReport)** for PR 22466 at commit

[GitHub] spark pull request #22466: [SPARK-25464][SQL] Create Database to the locatio...

2018-10-19 Thread sandeep-katta
Github user sandeep-katta commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r226673513 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -207,6 +207,14 @@ class SessionCatalog(

[GitHub] spark pull request #22764: [SPARK-25765][ML] Add training cost to BisectingK...

2018-10-19 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22764#discussion_r226668414 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/BisectingKMeansModel.scala --- @@ -225,13 +227,14 @@ object BisectingKMeansModel extends

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97613/ Test FAILed. ---

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97613/testReport)** for PR 22598 at commit

[GitHub] spark issue #22771: [SPARK-25773][Core]Cancel zombie tasks in a result stage...

2018-10-19 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22771 Actually as part of the jiras I mentioned above we were looking at killing other task attempts as soon as one task attempt succeeds rather then waiting for the entire job to finish, thoughts on

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22775 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97610/ Test FAILed. ---

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22775 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97613/testReport)** for PR 22598 at commit

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/22598 retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22775 **[Test build #97610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97610/testReport)** for PR 22775 at commit

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-19 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r226659261 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/TokenUtil.scala --- @@ -0,0 +1,111 @@ +/* + * Licensed to the

[GitHub] spark pull request #22466: [SPARK-25464][SQL] Create Database to the locatio...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r226655438 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -207,6 +207,14 @@ class SessionCatalog(

[GitHub] spark issue #22748: [SPARK-25745][K8S] Improve docker-image-tool.sh script

2018-10-19 Thread rvesse
Github user rvesse commented on the issue: https://github.com/apache/spark/pull/22748 Rebased onto master, should be ready for merging --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22764: [SPARK-25765][ML] Add training cost to BisectingK...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22764#discussion_r226652377 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/BisectingKMeansModel.scala --- @@ -225,13 +227,14 @@ object BisectingKMeansModel

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97612/testReport)** for PR 22598 at commit

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97612/ Test FAILed. ---

[GitHub] spark pull request #22771: [SPARK-25773][Core]Cancel zombie tasks in a resul...

2018-10-19 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22771#discussion_r226651308 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1364,6 +1385,16 @@ private[spark] class DAGScheduler(

[GitHub] spark issue #22771: [SPARK-25773][Core]Cancel zombie tasks in a result stage...

2018-10-19 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22771 the change itself seems reasonable we were actually looking at this recently as there are a couple of others bugs around this: https://issues.apache.org/jira/browse/SPARK-25250 and

[GitHub] spark issue #22743: [SPARK-25740][SQL] Refactor DetermineTableStats to inval...

2018-10-19 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22743 Yes. you are right, if datasource table stats is empty, `DetermineTableStats` doesn't set stats for it, so it's only a problem for hive tables. ---

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #97612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97612/testReport)** for PR 22598 at commit

[GitHub] spark pull request #22766: [SPARK-25768][SQL] fix constant argument expectin...

2018-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22766 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22766: [SPARK-25768][SQL] fix constant argument expecting UDAFs

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22766 thanks, merging to master/2.4/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-10-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r226643416 --- Diff: docs/running-on-kubernetes.md --- @@ -799,4 +815,168 @@ specific to Spark on Kubernetes. This sets the major Python version of the docker

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-10-19 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r226642833 --- Diff: docs/running-on-kubernetes.md --- @@ -799,4 +815,168 @@ specific to Spark on Kubernetes. This sets the major Python version of the docker

[GitHub] spark issue #22743: [SPARK-25740][SQL] Refactor DetermineTableStats to inval...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22743 > Datasource table will not cache in tableRelationCache. I don't think so. Spark caches data source table in `FindDataSourceTable` ---

[GitHub] spark pull request #22666: [SPARK-25672][SQL] schema_of_csv() - schema infer...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22666#discussion_r226641023 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3886,6 +3886,31 @@ object functions { withExpr(new

[GitHub] spark pull request #22666: [SPARK-25672][SQL] schema_of_csv() - schema infer...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22666#discussion_r226640860 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CsvExpressionsSuite.scala --- @@ -155,4 +155,15 @@ class

[GitHub] spark pull request #22666: [SPARK-25672][SQL] schema_of_csv() - schema infer...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22666#discussion_r226640362 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -60,7 +63,7 @@ case class CsvToStructs(

[GitHub] spark pull request #22732: [SPARK-25044][FOLLOW-UP] Change ScalaUDF construc...

2018-10-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22732 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22732: [SPARK-25044][FOLLOW-UP] Change ScalaUDF constructor sig...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22732 thanks, merging to master/2.4! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22263: [SPARK-25269][SQL] SQL interface support specify Storage...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22263 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22263: [SPARK-25269][SQL] SQL interface support specify Storage...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22263 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97606/ Test PASSed. ---

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22750 `DataSourceScanExec` does not have `needsUnsafeRowConversion` --- - To unsubscribe, e-mail:

[GitHub] spark issue #22263: [SPARK-25269][SQL] SQL interface support specify Storage...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22263 **[Test build #97606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97606/testReport)** for PR 22263 at commit

[GitHub] spark issue #22482: WIP - [SPARK-10816][SS] Support session window natively

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22482 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22482: WIP - [SPARK-10816][SS] Support session window natively

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22482 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97608/ Test PASSed. ---

[GitHub] spark issue #22482: WIP - [SPARK-10816][SS] Support session window natively

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22482 **[Test build #97608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97608/testReport)** for PR 22482 at commit

[GitHub] spark issue #22776: [SPARK-25779][SQL][TESTS] Remove SQL query tests for fun...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22776 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97609/ Test PASSed. ---

[GitHub] spark issue #22776: [SPARK-25779][SQL][TESTS] Remove SQL query tests for fun...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22776 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22776: [SPARK-25779][SQL][TESTS] Remove SQL query tests for fun...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22776 **[Test build #97609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97609/testReport)** for PR 22776 at commit

[GitHub] spark issue #22666: [SPARK-25672][SQL] schema_of_csv() - schema inference fr...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22666 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22666: [SPARK-25672][SQL] schema_of_csv() - schema inference fr...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22666 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97607/ Test FAILed. ---

[GitHub] spark issue #22666: [SPARK-25672][SQL] schema_of_csv() - schema inference fr...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22666 **[Test build #97607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97607/testReport)** for PR 22666 at commit

[GitHub] spark pull request #22466: [SPARK-25464][SQL] Create Database to the locatio...

2018-10-19 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22466#discussion_r226624218 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -207,6 +207,14 @@ class SessionCatalog(

[GitHub] spark pull request #22576: [SPARK-25560][SQL] Allow FunctionInjection in Spa...

2018-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/22576#discussion_r226623886 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala --- @@ -168,4 +173,21 @@ class SparkSessionExtensions { def

[GitHub] spark pull request #22576: [SPARK-25560][SQL] Allow FunctionInjection in Spa...

2018-10-19 Thread RussellSpitzer
Github user RussellSpitzer commented on a diff in the pull request: https://github.com/apache/spark/pull/22576#discussion_r226622273 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala --- @@ -168,4 +173,21 @@ class SparkSessionExtensions {

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22429 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22429 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97600/ Test PASSed. ---

[GitHub] spark issue #22429: [SPARK-25440][SQL] Dumping query execution info to a fil...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22429 **[Test build #97600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97600/testReport)** for PR 22429 at commit

[GitHub] spark issue #22773: [MINOR][SQL] Add prettyNames for from_json, to_json, fro...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22773 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22773: [MINOR][SQL] Add prettyNames for from_json, to_json, fro...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22773 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97604/ Test PASSed. ---

[GitHub] spark issue #22773: [MINOR][SQL] Add prettyNames for from_json, to_json, fro...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22773 **[Test build #97604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97604/testReport)** for PR 22773 at commit

[GitHub] spark issue #22774: [SPARK-25780][CORE]Scheduling the tasks which have no hi...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22774 **[Test build #97611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97611/testReport)** for PR 22774 at commit

[GitHub] spark issue #22774: [SPARK-25780][CORE]Scheduling the tasks which have no hi...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22774 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22774: [SPARK-25780][CORE]Scheduling the tasks which have no hi...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22774 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22774: [SPARK-25780][CORE]Scheduling the tasks which have no hi...

2018-10-19 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/22774 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22750: [SPARK-25747][SQL] remove ColumnarBatchScan.needsUnsafeR...

2018-10-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22750 I thought about the last line > This PR removes ColumnarBatchScan.needsUnsafeRowConversion, and keep this flag only in FileSourceScanExec ---

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22466 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97602/ Test PASSed. ---

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22466 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22466: [SPARK-25464][SQL] Create Database to the location,only ...

2018-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22466 **[Test build #97602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97602/testReport)** for PR 22466 at commit

[GitHub] spark issue #22732: [SPARK-25044][FOLLOW-UP] Change ScalaUDF constructor sig...

2018-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22732 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   >