[GitHub] spark pull request #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16765#discussion_r99066963 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala --- @@ -175,6 +175,7 @@ object Literal { case map

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16775 **[Test build #72276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72276/testReport)** for PR 16775 at commit [`32c90dd`](https://github.com/apache/spark/commit/32

[GitHub] spark issue #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16766 Could you please also add a few test cases? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99065789 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -823,6 +825,17 @@ case class Reparti

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99065474 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sq

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99065162 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sq

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99064944 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sq

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99064595 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -823,6 +825,17 @@ case class Reparti

[GitHub] spark issue #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener callback...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16664 I just quickly went over the code. It looks ok to me, but I will review it again when the comments are resolved. Thanks! --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99064088 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -428,8 +481,14 @@ final class DataFrameWriter[T] private[sql](ds: Dat

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99063701 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -660,12 +660,21 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99063668 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -660,12 +660,21 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-02-01 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r99063259 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/KeyedState.scala --- @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16775 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16775 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72274/ Test PASSed. ---

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16775 **[Test build #72275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72275/testReport)** for PR 16775 at commit [`7a1b300`](https://github.com/apache/spark/commit/7a

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16775 **[Test build #72274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72274/testReport)** for PR 16775 at commit [`5ed5c2a`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-02-01 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r99062793 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/MapGroupsWithStateSuite.scala --- @@ -0,0 +1,240 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99062729 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -428,8 +481,14 @@ final class DataFrameWriter[T] private[sql](ds: Dat

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99062659 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -261,13 +304,19 @@ final class DataFrameWriter[T] private[sql](ds: Da

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99062495 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -218,7 +246,17 @@ final class DataFrameWriter[T] private[sql](ds: Dat

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-02-01 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r99062438 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/MapGroupsWithStateSuite.scala --- @@ -0,0 +1,240 @@ +/* + * Licensed to the Apache S

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99062470 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql](df

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99062185 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -514,6 +576,9 @@ final class DataFrameWriter[T] private[sql](ds: Data

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99062037 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -660,12 +660,21 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99061828 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -190,6 +192,32 @@ final class DataFrameWriter[T] private[sql](ds: Dat

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99061710 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -190,6 +192,32 @@ final class DataFrameWriter[T] private[sql](ds: Dat

[GitHub] spark issue #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener callback...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16664 @marmbrus `DataStreamWriter` has similar issues, right? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99060951 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -190,6 +192,32 @@ final class DataFrameWriter[T] private[sql](ds: Dat

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r99060523 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -660,12 +660,21 @@ object SQLConf { .booleanConf

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/12135 @zhengruifeng Please try to improve the test case coverage in the follow-up PRs. You might find some bugs when you added these test cases. Thanks for your work! --- If your project is set up fo

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99060083 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99059985 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99059884 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql

[GitHub] spark pull request #16758: [SPARK-19413][SS] MapGroupsWithState for arbitrar...

2017-02-01 Thread lw-lin
Github user lw-lin commented on a diff in the pull request: https://github.com/apache/spark/pull/16758#discussion_r99059679 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/KeyedStateImpl.scala --- @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache S

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/12135 @zhengruifeng Actually, I still have a few comments about this PR. I will leave the comments soon. Thanks! --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99059030 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-01 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/12135 @HyukjinKwon @gatorsmile Thanks for pointing out those issues. I will create a followup PR to fix them ASAP. --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark issue #12135: [SPARK-14352][SQL] approxQuantile should support multi c...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/12135 @holdenk When you do the code merge, you need to leave a comment to explain which branch you merged. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99057948 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sql

[GitHub] spark issue #16743: [SPARK-19379][CORE] SparkAppHandle.getState not register...

2017-02-01 Thread thomastechs
Github user thomastechs commented on the issue: https://github.com/apache/spark/pull/16743 One point, as discussed, statusChange gets called for task status change. So, if we can identify the point where the job or that executor(Only one executor for local mode, right) is failed, we c

[GitHub] spark issue #16761: [BackPort-2.1][SPARK-19319][SparkR]:SparkR Kmeans summar...

2017-02-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16761 hmm, I wasn't sure to have the parameter changes in 2.1, what do you think? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99056238 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/RandomForest.scala --- @@ -23,6 +23,7 @@ import scala.util.Try import org.apache.spark.

[GitHub] spark issue #16689: [SPARK-19342][SPARKR] bug fixed in collect method for co...

2017-02-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16689 hmm, that's not a super big issue since vector and list is more or less the same in R. I think it might be better if we are treating the type consistently, although it might be a concerning

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99056047 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala --- @@ -72,6 +72,21 @@ private[ml] trait DecisionTreeParams extends PredictorP

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99055889 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -590,8 +599,8 @@ private[spark] object RandomForest extends Logg

[GitHub] spark pull request #16767: [SPARK-19386][SPARKR][DOC] Bisecting k-means in S...

2017-02-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16767#discussion_r99055560 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -819,6 +821,18 @@ perplexity <- spark.perplexity(model, corpusDF) perplexity ```

[GitHub] spark pull request #16773: [SPARK-19432][Core]Fix an unexpected failure when...

2017-02-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16773 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #16767: [SPARK-19386][SPARKR][DOC] Bisecting k-means in S...

2017-02-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16767#discussion_r99055524 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -494,6 +494,8 @@ SparkR supports the following machine learning models and algorithms. *

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16729 ah thanks. so I were to ``` library(statmod) library(SparkR) ``` could I still access the statmod tweedie function? ie. does statmod::tweedie still work with R base::

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99055341 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/DecisionTreeMetadata.scala --- @@ -42,6 +42,7 @@ import org.apache.spark.rdd.RDD pri

[GitHub] spark issue #16773: [SPARK-19432][Core]Fix an unexpected failure when connec...

2017-02-01 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16773 Thanks. Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16739: [SPARK-19399][SPARKR] Add R coalesce API for DataFrame a...

2017-02-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16739 yap, https://github.com/apache/spark/pull/16739#issuecomment-276739220 - only RDD has `coalesce(.. shuffle)`, in Dataset, it's `coalesce` and `repartition` --- If your project is set up for it

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99054611 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/RandomForestRegressor.scala --- @@ -117,20 +114,20 @@ class RandomForestRegressor @Since

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99054576 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala --- @@ -99,16 +105,31 @@ class DecisionTreeRegressor @Since(

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99054369 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/RandomForestClassifier.scala --- @@ -126,20 +127,20 @@ class RandomForestClassifier

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16729 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72273/ Test PASSed. ---

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99054331 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LabeledPoint.scala --- @@ -35,4 +35,11 @@ case class LabeledPoint(@Since("2.0.0") label: Dou

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16729 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16729 **[Test build #72273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72273/testReport)** for PR 16729 at commit [`a9ac439`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99054115 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -106,14 +122,18 @@ class DecisionTreeClassifier

[GitHub] spark issue #16775: [WIP][ML] Periodic checkout datasets for long ml pipelin...

2017-02-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16775 also cc @MLnick --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99053832 --- Diff: mllib-local/src/test/scala/org/apache/spark/ml/util/TestingUtils.scala --- @@ -48,7 +48,7 @@ object TestingUtils { /** * Priva

[GitHub] spark issue #16775: [WIP][ML] Periodic checkout datasets for long ml pipelin...

2017-02-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16775 cc @mengxr @jkbradley @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #16775: [WIP][ML] Periodic checkout datasets for long ml pipelin...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16775 **[Test build #72274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72274/testReport)** for PR 16775 at commit [`5ed5c2a`](https://github.com/apache/spark/commit/5e

[GitHub] spark pull request #16775: [WIP][ML] Periodic checkout datasets for long ml ...

2017-02-01 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/16775 [WIP][ML] Periodic checkout datasets for long ml pipeline ## What changes were proposed in this pull request? WIP ## How was this patch tested? Jenkins tests. Plea

[GitHub] spark issue #16714: [SPARK-16333][Core] Enable EventLoggingListener to log l...

2017-02-01 Thread drcrallen
Github user drcrallen commented on the issue: https://github.com/apache/spark/pull/16714 @vanzin can you check this out please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16729 **[Test build #72273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72273/testReport)** for PR 16729 at commit [`a9ac439`](https://github.com/apache/spark/commit/a9

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16765 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16765 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72272/ Test PASSed. ---

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16765 **[Test build #72272 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72272/testReport)** for PR 16765 at commit [`af98964`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #16690: [SPARK-19347] ReceiverSupervisorImpl can add block to Re...

2017-02-01 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16690 Thanks a lot for reviewing this PR~ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark issue #16773: [SPARK-19432][Core]Fix an unexpected failure when connec...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16773 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16773: [SPARK-19432][Core]Fix an unexpected failure when connec...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16773 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72266/ Test PASSed. ---

[GitHub] spark issue #16773: [SPARK-19432][Core]Fix an unexpected failure when connec...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16773 **[Test build #72266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72266/testReport)** for PR 16773 at commit [`ee695a8`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #16689: [SPARK-19342][SPARKR] bug fixed in collect method for co...

2017-02-01 Thread titicaca
Github user titicaca commented on the issue: https://github.com/apache/spark/pull/16689 I tried to modify the PRIMITIVE_TYPES for timestamp, but it had a side effect on coltypes method. In test_sparkSQL.R#2262, `expect_equal(coltypes(DF), c("integer", "logical", "POSIXct"))`,

[GitHub] spark issue #16743: [SPARK-19379][CORE] SparkAppHandle.getState not register...

2017-02-01 Thread adamstatdna
Github user adamstatdna commented on the issue: https://github.com/apache/spark/pull/16743 My use case is end-to-end automated testing in local mode using programmatic Launcher. I have tests where the Spark app is expected to be FINISHED and those where it is expected to be FAILED.

[GitHub] spark issue #16763: [SPARK-19422][ML] Cache input data in algorithms

2017-02-01 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16763 @hhbyyh Thanks a lot for pointing this out! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request #12135: [SPARK-14352][SQL] approxQuantile should support ...

2017-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/12135#discussion_r99040876 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -75,13 +76,43 @@ final class DataFrameStatFunctions private[sq

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16772 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72271/ Test PASSed. ---

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16772 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16772 **[Test build #72271 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72271/testReport)** for PR 16772 at commit [`ce59d74`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #12420: [SPARK-14585][ML][WIP] Provide accessor methods for Pipe...

2017-02-01 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/12420 I missed the ClassTag question above. Let me take a look --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16729 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72268/ Test FAILed. ---

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16729 **[Test build #72268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72268/testReport)** for PR 16729 at commit [`b10777e`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16729: [SPARK-19391][SparkR][ML] Tweedie GLM API for SparkR

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16729 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16765 **[Test build #72272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72272/testReport)** for PR 16765 at commit [`af98964`](https://github.com/apache/spark/commit/af

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16765 Simplified the code change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #16723: [SPARK-19389][ML][PYTHON][DOC] Minor doc fixes for ML Py...

2017-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16723 (I just rebased it based on this PR and built the javadoc8 for sure. I believe it should emit an error if this PR introduce the break but it seems not. So, LGTM for doc changes.) --- If your p

[GitHub] spark issue #16737: [SPARK-19397] [SQL] Make option names of LIBSVM and TEXT...

2017-02-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16737 Please hold on this PR. Found a serious bug to fix in case insensitive option support. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark issue #16774: [SPARK-19357][ML][WIP] Adding parallel model evaluation ...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16774 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72267/ Test PASSed. ---

[GitHub] spark issue #16774: [SPARK-19357][ML][WIP] Adding parallel model evaluation ...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16774 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16774: [SPARK-19357][ML][WIP] Adding parallel model evaluation ...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16774 **[Test build #72267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72267/testReport)** for PR 16774 at commit [`5650e98`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16772 **[Test build #72271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72271/testReport)** for PR 16772 at commit [`ce59d74`](https://github.com/apache/spark/commit/ce

[GitHub] spark issue #16771: [SPARK-19429][PYTHON][SQL] Support slice arguments in Co...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16771 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72269/ Test PASSed. ---

[GitHub] spark issue #16771: [SPARK-19429][PYTHON][SQL] Support slice arguments in Co...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16771 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16771: [SPARK-19429][PYTHON][SQL] Support slice arguments in Co...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16771 **[Test build #72269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72269/testReport)** for PR 16771 at commit [`c1f5110`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16772 **[Test build #72270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72270/testReport)** for PR 16772 at commit [`c25c127`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16772 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72270/ Test FAILed. ---

[GitHub] spark issue #16772: [SPARK-14772][PYTHON][ML] Fixed Params.copy method to ma...

2017-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16772 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

  1   2   3   4   >