[GitHub] spark pull request #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateO...

2017-08-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19029 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19029 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #19066: [SPARK-21255][SQL] simplify encoder for java enum

2017-08-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19066#discussion_r135450475 --- Diff: sql/core/src/test/java/test/org/apache/spark/sql/JavaDatasetSuite.java --- @@ -1302,16 +1302,8 @@ public void setUrl(String url) { }

[GitHub] spark issue #19066: [SPARK-21255][SQL] simplify encoder for java enum

2017-08-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19066 cc @mike0sv @srowen @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #19066: [SPARK-21255][SQL] simplify encoder for java enum

2017-08-27 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/19066 [SPARK-21255][SQL] simplify encoder for java enum ## What changes were proposed in this pull request? This is a follow-up for https://github.com/apache/spark/pull/18488, to simplify the

[GitHub] spark pull request #19065: [SPARK-21729][ML][TEST] Generic test for Probabil...

2017-08-27 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/19065 [SPARK-21729][ML][TEST] Generic test for ProbabilisticClassifier to ensure consistent output columns ## What changes were proposed in this pull request? Add test for prediction using

[GitHub] spark issue #19055: [SPARK-21839][SQL] Support SQL config for ORC compressio...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19055 I am not familiar with ORC. Above is just a quick look about the changes made in this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request #19055: [SPARK-21839][SQL] Support SQL config for ORC com...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19055#discussion_r135449484 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -322,6 +322,15 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #19055: [SPARK-21839][SQL] Support SQL config for ORC com...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19055#discussion_r135449118 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcOptions.scala --- @@ -20,30 +20,35 @@ package org.apache.spark.sql.hive.orc impor

[GitHub] spark pull request #19055: [SPARK-21839][SQL] Support SQL config for ORC com...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19055#discussion_r135449049 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcOptions.scala --- @@ -20,30 +20,35 @@ package org.apache.spark.sql.hive.orc impor

[GitHub] spark issue #19064: [SPARK-21848][SQL] Add trait UDFType to identify user-de...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19064 Yes. This is a potential optimization we should do, but it should be cost based. Let us improve the resolution logics using this common base trait in this PR first. --- If your project is set u

[GitHub] spark pull request #19064: [SPARK-21848][SQL] Add trait UDFType to identify ...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19064#discussion_r135446631 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -22,6 +22,9 @@ import org.apache.spark.sql.cataly

[GitHub] spark pull request #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery ...

2017-08-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r135443202 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -98,6 +99,11 @@ object RewritePredicateSubquery extends

[GitHub] spark pull request #19064: [SPARK-21848][SQL] Add trait UDFType to identify ...

2017-08-27 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/19064 [SPARK-21848][SQL] Add trait UDFType to identify user-defined functions ## What changes were proposed in this pull request? Add trait UDFType to identify user-defined functions. U

[GitHub] spark issue #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery should ...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81173/ Test FAILed. ---

[GitHub] spark issue #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery should ...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery should ...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery should ...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19050 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81172/ Test FAILed. ---

[GitHub] spark issue #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery should ...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81172 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81172/testReport)** for PR 19050 at commit [`121ad5a`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #18787: [SPARK-21583][SQL] Create a ColumnarBatch from Ar...

2017-08-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18787#discussion_r135439310 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -1261,4 +1264,55 @@ class ColumnarBatchSuite ext

[GitHub] spark pull request #18787: [SPARK-21583][SQL] Create a ColumnarBatch from Ar...

2017-08-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18787#discussion_r135439372 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -1261,4 +1264,55 @@ class ColumnarBatchSuite ext

[GitHub] spark pull request #18787: [SPARK-21583][SQL] Create a ColumnarBatch from Ar...

2017-08-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18787#discussion_r135439793 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala --- @@ -1629,6 +1632,39 @@ class ArrowConvertersSuite exte

[GitHub] spark pull request #18787: [SPARK-21583][SQL] Create a ColumnarBatch from Ar...

2017-08-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18787#discussion_r135439857 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala --- @@ -1629,6 +1632,39 @@ class ArrowConvertersSuite exte

[GitHub] spark pull request #18787: [SPARK-21583][SQL] Create a ColumnarBatch from Ar...

2017-08-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18787#discussion_r135438683 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala --- @@ -111,6 +125,66 @@ private[sql] object ArrowConverters {

[GitHub] spark issue #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery should ...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81173/testReport)** for PR 19050 at commit [`bf07e2a`](https://github.com/apache/spark/commit/bf

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81170/ Test PASSed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81170/testReport)** for PR 18581 at commit [`41369cf`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery should ...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19050 **[Test build #81172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81172/testReport)** for PR 19050 at commit [`121ad5a`](https://github.com/apache/spark/commit/12

[GitHub] spark pull request #19050: [SPARK-21835][SQL][WIP] RewritePredicateSubquery ...

2017-08-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19050#discussion_r135434713 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -98,6 +99,11 @@ object RewritePredicateSubquery extends

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19029 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81171/ Test PASSed. ---

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19029 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19029 **[Test build #81171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81171/testReport)** for PR 19029 at commit [`c40eba3`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19062: [SPARK-21845] [SQL] Make codegen fallback of expressions...

2017-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19062 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featur

[GitHub] spark pull request #19062: [SPARK-21845] [SQL] Make codegen fallback of expr...

2017-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19062#discussion_r135430732 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -370,8 +373,7 @@ abstract class SparkPlan extends QueryPlan[SparkPlan

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-27 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19018 ping @felixcheung We can make all R tests for trees deterministic (not only random trees). Leave other problems to separate PR. It would be great to fix it soon, Thanks! --- If your project i

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19029 **[Test build #81171 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81171/testReport)** for PR 19029 at commit [`c40eba3`](https://github.com/apache/spark/commit/c4

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81170/testReport)** for PR 18581 at commit [`41369cf`](https://github.com/apache/spark/commit/41

[GitHub] spark pull request #19049: [WEB-UI]Add the 'master' column to identify the t...

2017-08-27 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/19049#discussion_r135427302 --- Diff: core/src/main/resources/org/apache/spark/ui/static/historypage.js --- @@ -136,6 +136,16 @@ $(document).ready(function() {

[GitHub] spark issue #19061: [SPARK-21568][CORE] ConsoleProgressBar should only be en...

2017-08-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19061 Hi, @vanzin . Could you review this when you have sometime? I'm wondering if this is implemented correctly in a way you expected. Please let me know if there is something to do more. Thank

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19031 ok, I'll close for now. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable m...

2017-08-27 Thread maropu
Github user maropu closed the pull request at: https://github.com/apache/spark/pull/19031 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18945: Add option to convert nullable int columns to float colu...

2017-08-27 Thread logannc
Github user logannc commented on the issue: https://github.com/apache/spark/pull/18945 Sorry for the delay. Things got busy and now there is the storm in Houston. Will update this per these suggestions soon. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark issue #17461: [SPARK-20082][ml] LDA incremental model learning

2017-08-27 Thread mdespriee
Github user mdespriee commented on the issue: https://github.com/apache/spark/pull/17461 I updated the example following your suggestion. It's more consistent with LDAExample this way. --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark issue #18837: [Spark-20812][Mesos] Add secrets support to the dispatch...

2017-08-27 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/18837 Hello @vanzin, thanks for the review. I added `.toSequence` to the new configuration specs, certainly a nice solution to parsing on the fly. Please let me know if there is anything else that needs c

[GitHub] spark pull request #18610: [SPARK-21386] ML LinearRegression supports warm s...

2017-08-27 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/18610#discussion_r135418170 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -226,6 +246,12 @@ class LinearRegression @Since("1.3.0") (@Since

[GitHub] spark pull request #18610: [SPARK-21386] ML LinearRegression supports warm s...

2017-08-27 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/18610#discussion_r135418289 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -72,6 +72,22 @@ private[regression] trait LinearRegressionParams

[GitHub] spark issue #7842: [SPARK-8542][MLlib]PMML export for Decision Trees

2017-08-27 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/7842 @coderxiang what is the plan for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16992 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81169/ Test PASSed. ---

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16992 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16992 **[Test build #81169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81169/testReport)** for PR 16992 at commit [`3d6c80b`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16992 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16992 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81168/ Test FAILed. ---

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16992 **[Test build #81168 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81168/testReport)** for PR 16992 at commit [`77cfb03`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19055: [SPARK-21839][SQL] Support SQL config for ORC compressio...

2017-08-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19055 Hi, @gatorsmile . Could you review this ORC configuration PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request #19063: [SPARK-21846][TEST] Reduce the number of shuffle ...

2017-08-27 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/19063 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #19063: [SPARK-21846][TEST] Reduce the number of shuffle partiti...

2017-08-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19063 I'm closing this PR because the numbers are different than what I expected before. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19060 For Parquet, I can find [TestInputOutputFormat.java](https://github.com/Parquet/parquet-mr/blob/master/parquet-hadoop/src/test/java/parquet/hadoop/example/TestInputOutputFormat.java). Parquet

[GitHub] spark issue #19063: [SPARK-21846][TEST] Reduce the number of shuffle partiti...

2017-08-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19063 I saw that, too. Right. It becomes meaningless. Some is reduced but the other increase. I'm trying to do another approche in this PR and JIRA. I will update more. --- If your project is set

[GitHub] spark issue #18610: [SPARK-21386] ML LinearRegression supports warm start fr...

2017-08-27 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/18610 Just to confirm, so we have agreed that the initialModel should be of type [T <: Model[T]] rather than a String type (path to the saved model)? Sorry I didn't find the related discussion. --- If yo

[GitHub] spark issue #18610: [SPARK-21386] ML LinearRegression supports warm start fr...

2017-08-27 Thread JohnHBrock
Github user JohnHBrock commented on the issue: https://github.com/apache/spark/pull/18610 Is there anymore work to do before this can get merged? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark issue #19031: [SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLines...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19031 This is an internal conf. For the advanced users, we do not encourage them to disable it. If they want to disable it, they can simply set it to a number above 8000. Thus, setting `maxLinesPerFunc

[GitHub] spark pull request #19062: [SPARK-21845] [SQL] Make codegen fallback of expr...

2017-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19062#discussion_r135416594 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -54,6 +54,9 @@ abstract class SparkPlan extends QueryPlan[SparkPlan]

[GitHub] spark issue #18961: [SPARK-21746][SQL]there is an java.lang.IllegalArgumentE...

2017-08-27 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18961 @dongjoon-hyun @cloud-fan Do you have any suggestions? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark issue #19062: [SPARK-21845] [SQL] Make codegen fallback of expressions...

2017-08-27 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/19062 +1,LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread erenavsarogullari
Github user erenavsarogullari commented on the issue: https://github.com/apache/spark/pull/16992 Hi @squito, Thanks for the review this patch. It is ready to re-review / merge. --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark pull request #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler...

2017-08-27 Thread erenavsarogullari
Github user erenavsarogullari commented on a diff in the pull request: https://github.com/apache/spark/pull/16992#discussion_r135415731 --- Diff: docs/job-scheduling.md --- @@ -235,7 +235,7 @@ properties: of the cluster. By default, each pool's `minShare` is 0. The

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16992 **[Test build #81169 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81169/testReport)** for PR 16992 at commit [`3d6c80b`](https://github.com/apache/spark/commit/3d

[GitHub] spark pull request #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions ...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18966#discussion_r135415687 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -769,16 +769,27 @@ class CodegenConte

[GitHub] spark issue #16992: [SPARK-19662][SCHEDULER][TEST] Add Fair Scheduler Unit T...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16992 **[Test build #81168 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81168/testReport)** for PR 16992 at commit [`77cfb03`](https://github.com/apache/spark/commit/77

[GitHub] spark pull request #18581: [SPARK-21289][SQL][ML] Supports custom line separ...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18581#discussion_r135415452 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -32,7 +32,9 @@ import org.apache.hadoop

[GitHub] spark issue #19063: [SPARK-21846][TEST] Reduce the number of shuffle partiti...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19063 - org.apache.spark.sql.hive 10 min - org.apache.spark.sql.hive.client 6 min 20 sec - org.apache.spark.sql.hive.execution 28 min - org.apache.spark.sql.hive.orc 2 min 1 sec

[GitHub] spark issue #19060: [WIP][SQL] Add DataSourceSuite validating data sources l...

2017-08-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19060 I mean, how about Parquet and the others? Do they have the e2e test cases in their projects? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateO...

2017-08-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19029#discussion_r135411403 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala --- @@ -439,8 +439,9 @@ private[ml] object WeightedLeastSquares {

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19029 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19029 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81167/ Test PASSed. ---

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19029 **[Test build #81167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81167/testReport)** for PR 19029 at commit [`21e7ff7`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81166/ Test PASSed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81166 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81166/testReport)** for PR 18581 at commit [`08233e6`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19029: [SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSu...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19029 **[Test build #81167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81167/testReport)** for PR 19029 at commit [`21e7ff7`](https://github.com/apache/spark/commit/21

[GitHub] spark issue #19063: [SPARK-21846][TEST] Reduce the number of shuffle partiti...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19063 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #19063: [SPARK-21846][TEST] Reduce the number of shuffle partiti...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19063 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81163/ Test PASSed. ---

[GitHub] spark issue #19063: [SPARK-21846][TEST] Reduce the number of shuffle partiti...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19063 **[Test build #81163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81163/testReport)** for PR 19063 at commit [`557b0c6`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81165/ Test FAILed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81165/testReport)** for PR 18581 at commit [`cbed415`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81166/testReport)** for PR 18581 at commit [`08233e6`](https://github.com/apache/spark/commit/08

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81164/testReport)** for PR 18581 at commit [`9b1bf10`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81164/ Test FAILed. ---

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18581 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81165/testReport)** for PR 18581 at commit [`cbed415`](https://github.com/apache/spark/commit/cb

[GitHub] spark pull request #19059: [SS] - Avoid using `return` inside `CachedKafkaCo...

2017-08-27 Thread YuvalItzchakov
Github user YuvalItzchakov commented on a diff in the pull request: https://github.com/apache/spark/pull/19059#discussion_r135405801 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaConsumer.scala --- @@ -125,8 +131,11 @@ private[kafka010]

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18581 **[Test build #81164 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81164/testReport)** for PR 18581 at commit [`9b1bf10`](https://github.com/apache/spark/commit/9b

[GitHub] spark issue #18966: [SPARK-21751][SQL] CodeGeneraor.splitExpressions counts ...

2017-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18966 ping @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #18581: [SPARK-21289][SQL][ML] Supports custom line separ...

2017-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18581#discussion_r135405534 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -32,7 +32,9 @@ import org.apache.hadoo

[GitHub] spark issue #18928: [SPARK-21696][SS]Fix a potential issue that may generate...

2017-08-27 Thread YuvalItzchakov
Github user YuvalItzchakov commented on the issue: https://github.com/apache/spark/pull/18928 Right. We've had some problems with reading snapshots after executors dying on OOM, I hope this does the trick :) Thanks. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #19058: SPARK-21843:testNameNote should be "(minNumPostSh...

2017-08-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19058 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #19058: SPARK-21843:testNameNote should be "(minNumPostShufflePa...

2017-08-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19058 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #19063: [SPARK-21846][TEST] Reduce the number of shuffle partiti...

2017-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19063 **[Test build #81163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81163/testReport)** for PR 19063 at commit [`557b0c6`](https://github.com/apache/spark/commit/55

  1   2   >