[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isInCollection in DataFrame AP...

2018-05-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 @cloud-fan Let me know if the new API looks good to you. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-25 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r190966442 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -786,6 +787,24 @@ class Column(val expr: Expression) extends Logging

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-25 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-25 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-25 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r190799418 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -149,7 +149,7 @@ object RewritePredicateSubquery

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r190754229 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -220,6 +219,7 @@ object OptimizeIn extends Rule

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 test this again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame API for S...

2018-05-24 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21416 +@cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r190472138 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -219,7 +219,11 @@ object ReorderAssociativeOperator

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21416#discussion_r190470525 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala --- @@ -397,6 +399,68 @@ class ColumnExpressionSuite extends QueryTest

[GitHub] spark pull request #21416: [SPARK-24371] [SQL] Added isinSet in DataFrame AP...

2018-05-23 Thread dbtsai
GitHub user dbtsai opened a pull request: https://github.com/apache/spark/pull/21416 [SPARK-24371] [SQL] Added isinSet in DataFrame API for Scala and Java. ## What changes were proposed in this pull request? Implemented **`isinSet`** in DataFrame API for both Scala and Java

[GitHub] spark pull request #21322: [SPARK-24225][CORE] Support closing AutoClosable ...

2018-05-18 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21322#discussion_r189187759 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -404,6 +428,7 @@ private[spark] class MemoryStore( def

[GitHub] spark pull request #21322: [SPARK-24225][CORE] Support closing AutoClosable ...

2018-05-18 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21322#discussion_r189187178 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -384,14 +385,37 @@ private[spark] class MemoryStore

[GitHub] spark pull request #21322: [SPARK-24225][CORE] Support closing AutoClosable ...

2018-05-14 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21322#discussion_r188033806 --- Diff: core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala --- @@ -526,4 +526,84 @@ class MemoryStoreSuite

[GitHub] spark pull request #21322: [SPARK-24225][CORE] Support closing AutoClosable ...

2018-05-14 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21322#discussion_r188034118 --- Diff: core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala --- @@ -526,4 +526,84 @@ class MemoryStoreSuite

[GitHub] spark pull request #21322: [SPARK-24225][CORE] Support closing AutoClosable ...

2018-05-14 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21322#discussion_r188035443 --- Diff: core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala --- @@ -526,4 +526,84 @@ class MemoryStoreSuite

[GitHub] spark pull request #21322: [SPARK-24225][CORE] Support closing AutoClosable ...

2018-05-14 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21322#discussion_r188032854 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -384,15 +385,36 @@ private[spark] class MemoryStore

[GitHub] spark pull request #21322: [SPARK-24225][CORE] Support closing AutoClosable ...

2018-05-14 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21322#discussion_r188032698 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -384,15 +385,36 @@ private[spark] class MemoryStore

[GitHub] spark issue #21322: [SPARK-24225] Support closing AutoClosable objects in Me...

2018-05-14 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21322 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21322: [SPARK-24225] Support closing AutoClosable objects in Me...

2018-05-14 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21322 Jenkins, add to whitelist. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21322: [SPARK-24225] Support closing AutoClosable objects in Me...

2018-05-14 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21322 Could you change the title to `[SPARK-24225][CORE] Support closing AutoClosable objects...`? Thanks. --- - To unsubscribe, e

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-09 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21235 Merged into master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-09 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21235 test it again --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21235 I'll merge into master once the test passes. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21235: [SPARK-24181][SQL] Better error message for writi...

2018-05-07 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21235#discussion_r186532502 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -339,9 +339,16 @@ final class DataFrameWriter[T] private[sql](ds

[GitHub] spark pull request #21235: [SPARK-24181][SQL] Better error message for writi...

2018-05-04 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21235#discussion_r186240261 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -339,9 +339,16 @@ final class DataFrameWriter[T] private[sql](ds

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-04 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21235 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21235: [SPARK-24181][SQL] Better error message for writi...

2018-05-04 Thread dbtsai
GitHub user dbtsai opened a pull request: https://github.com/apache/spark/pull/21235 [SPARK-24181][SQL] Better error message for writing sorted data ## What changes were proposed in this pull request? The exception message should clearly distinguish sorting and bucketing

[GitHub] spark issue #20907: [SPARK-11237][ML] Add pmml export for k-means in Spark M...

2018-04-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/20907 LGTM too! +1 on the documentation which can be a followup PR. Merged into master, and thanks. DB Tsai | Siri Open Source Technologies |  Apple, Inc

[GitHub] spark issue #21125: [Spark-24024][ML] Fix poisson deviance calculations in G...

2018-04-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21125 LGTM, merged into master. Thanks. DB Tsai | Siri Open Source Technologies |  Apple, Inc --- - To unsubscribe, e-mail

[GitHub] spark pull request #21125: [Spark-24024][ML] Fix poisson deviance calculatio...

2018-04-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21125#discussion_r183475337 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -782,8 +782,12 @@ object GeneralizedLinearRegression

[GitHub] spark issue #21120: [SPARK-22448][ML] Added sum function to Summerizer and M...

2018-04-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21120 Why do you want to add this? Once we have mean, it's easy to compute sum. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21120: [SPARK-22448][ML] Added sum function to Summerizer and M...

2018-04-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21120 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21125: [Spark-24024] Fix poisson deviance calculations i...

2018-04-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21125#discussion_r183282075 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -782,8 +782,12 @@ object GeneralizedLinearRegression

[GitHub] spark pull request #21125: [Spark-24024] Fix poisson deviance calculations i...

2018-04-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21125#discussion_r183283116 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -495,8 +495,8 @@ class

[GitHub] spark pull request #21125: [Spark-24024] Fix poisson deviance calculations i...

2018-04-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21125#discussion_r183282476 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -495,8 +495,8 @@ class

[GitHub] spark pull request #21125: [Spark-24024] Fix poisson deviance calculations i...

2018-04-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21125#discussion_r183283144 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -507,7 +507,8 @@ class

[GitHub] spark issue #21125: [Spark-24024] Fix poisson deviance calculations in GLM t...

2018-04-22 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21125 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21125: [Spark-24024] Fix poisson deviance calculations in GLM t...

2018-04-22 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21125 Jenkins, please test this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183257701 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -217,33 +295,32 @@ class StringIndexerModel ( @Since("

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183255152 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -130,21 +161,53 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183253488 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -79,26 +80,56 @@ private[feature] trait StringIndexerBase extends

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r18325 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -217,33 +295,32 @@ class StringIndexerModel ( @Since("

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183253932 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -79,26 +80,56 @@ private[feature] trait StringIndexerBase extends

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183257676 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -217,33 +295,32 @@ class StringIndexerModel ( @Since("

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183254078 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -130,21 +161,53 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183258353 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -217,33 +295,32 @@ class StringIndexerModel ( @Since("

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183257799 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -217,33 +295,32 @@ class StringIndexerModel ( @Since("

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-04-22 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r183253904 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -79,26 +80,56 @@ private[feature] trait StringIndexerBase extends

[GitHub] spark issue #19381: [SPARK-10884][ML] Support prediction on single instance ...

2018-04-17 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/19381 @WeichenXu123 we were discussing this when we moved out common math code into `mllib-local`, but there is no umbrella ticket around it. I talked to many companies, and that is one of the pain point

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-04-16 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/20633 Can you address the conflicts? Gonna start to review it soon. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #19381: [SPARK-10884][ML] Support prediction on single instance ...

2018-04-16 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/19381 Too late to the party! Great PR. We should also think about refactor the predictors to `mllib-local` eventually so people can use them in prod without depending on Spark

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-20 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/18346 @cloud-fan +1 on this. This can clear up the testability issue we have when implementing `Expression` since it's challenging to test properly for both `eval` and `codegen` path. If we can

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-20 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/18346 Does it mean all the expressions **have to** implement codegen, and no `eval` any more? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-19 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/18346 Thanks, @viirya for this PR. We hit this issue, and @viirya was kindly helping us to find the root cause. This approach LGTM. One alternative approach we took in the end to unblock our use

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-05-17 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17746 @superbobry As you can see in this PR, one of them is ```scala -override def link(mu: Double): Double = dist.Gaussian(0.0, 1.0).icdf(mu) +override def link(mu: Double

[GitHub] spark pull request #15628: [SPARK-17471][ML] Add compressed method to ML mat...

2017-05-15 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/15628#discussion_r116603167 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala --- @@ -148,7 +154,8 @@ sealed trait Matrix extends Serializable

[GitHub] spark issue #17887: [SPARK-20399][SQL] Add a config to fallback string liter...

2017-05-10 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17887 Thanks @viirya We'll backport it and test it out soon. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17894: [SPARK-17134][ML] Use level 2 BLAS operations in ...

2017-05-09 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17894#discussion_r115423381 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -23,6 +23,7 @@ import scala.collection.mutable

[GitHub] spark pull request #17894: [SPARK-17134][ML] Use level 2 BLAS operations in ...

2017-05-09 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17894#discussion_r115423264 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1722,25 +1723,22 @@ private class LogisticAggregator

[GitHub] spark pull request #17894: [SPARK-17134][ML] Use level 2 BLAS operations in ...

2017-05-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17894#discussion_r115313579 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -23,6 +23,7 @@ import scala.collection.mutable

[GitHub] spark pull request #17894: [SPARK-17134][ML] Use level 2 BLAS operations in ...

2017-05-08 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17894#discussion_r115313235 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1722,25 +1723,22 @@ private class LogisticAggregator

[GitHub] spark issue #17829: [SPARK-20047][FOLLOWUP][ML] Constrained Logistic Regress...

2017-05-02 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17829 LGTM. Wait for more reviews. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-05-01 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17746 The motivation to have this one merged in Spark 2.2 is not only just for #17715 but also because Breeze 0.13.x fixes many bugs in upstream. Since Spark was tightened to 0.12, many users (including

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 @mgummelt We tested this in our production env, and it solves our issue. Since it seems to be a trivial change, I made my judgement. Gonna wait for more feedback. Thanks. --- If your project

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 @mgummelt We tested this in our production env, and it solves our issue. Since it seems to be a trivial change, I made my judgement. Gonna wait for more feedback. --- If your project is set up

[GitHub] spark pull request #17786: [SPARK-20483] Mesos Coarse mode may starve other ...

2017-04-27 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17786#discussion_r113809015 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -60,8 +60,16

[GitHub] spark issue #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17715 LGTM. Merged into master and branch-2.2 Thanks @yanboliang for delivering this big feature which is very useful for many practical use-cases in the industry. Thanks @WeichenXu123

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 Tests are added in a followup PR. https://github.com/apache/spark/pull/17788 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17788: [SPARK-20483][MINOR] Test for Mesos Coarse mode may star...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17788 LGTM. Merged into master and branch 2.1. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 LGTM. Merged into master and branch 2.2. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 Jenkins, ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17786: [SPARK-20483] Mesos Coarse mode may starve other Mesos f...

2017-04-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17786 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17757: [Minor][ML] Fix some PySpark & SparkR flaky tests

2017-04-25 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17757 LGTM. Please merge the current master to resolve the conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-04-25 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17746 @srowen Couple APIs changes in Breeze 0.13 are not source code compatible with 0.12. We should tell users about that in the release note, and they need to do their migration if they are using Breeze

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-04-25 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17746 Many thanks for @WeichenXu123 helping to fix this bug in breeze! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13....

2017-04-25 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17746#discussion_r113256277 --- Diff: R/pkg/inst/tests/testthat/test_mllib_classification.R --- @@ -288,18 +288,18 @@ test_that("spark.mlp", { c(0, 0, 0, 0, 0, 5, 5,

[GitHub] spark issue #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13.1

2017-04-25 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17746 LGTM. Merged into master and branch 2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-25 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17715 Many use-cases are setting the bounds as a constant instead of setting each dimensional individually. Maybe we can add the following APIs. ```scala def setLowerBoundsOnIntercepts(bound

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113052881 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -434,8 +587,45 @@ class LogisticRegression @Since("

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113042111 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -178,6 +179,68 @@ private[classification] trait

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113066007 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -434,8 +587,45 @@ class LogisticRegression @Since("

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113073040 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -815,6 +904,40 @@ class LogisticRegressionSuite

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113073212 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -1149,6 +1300,49 @@ class LogisticRegressionSuite

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113072397 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -1149,6 +1300,49 @@ class LogisticRegressionSuite

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113065566 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -434,8 +587,45 @@ class LogisticRegression @Since("

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113071333 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -650,6 +711,34 @@ class LogisticRegressionSuite

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113067669 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -546,6 +736,21 @@ class LogisticRegression @Since("

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113047003 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -324,6 +460,18 @@ class LogisticRegression @Since("

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113070161 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -610,6 +639,38 @@ class LogisticRegressionSuite

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113042705 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -178,6 +179,68 @@ private[classification] trait

[GitHub] spark pull request #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17715#discussion_r113048193 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -434,8 +587,45 @@ class LogisticRegression @Since("

[GitHub] spark pull request #17746: [SPARK-20449][ML] Upgrade breeze version to 0.13....

2017-04-24 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17746#discussion_r112999875 --- Diff: R/pkg/inst/tests/testthat/test_mllib_classification.R --- @@ -288,18 +288,18 @@ test_that("spark.mlp", { c(0, 0, 0, 0, 0, 5, 5,

[GitHub] spark issue #17736: [SPARK-20399][SQL][WIP] Can't use same regex pattern bet...

2017-04-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17736 LGTM. Thanks. @cloud-fan @rxin this fixes our production jobs when we port our applications from 1.6 to 2.0. I think it's a important bug fix. Thanks. --- If your project is set up for it, you can

[GitHub] spark pull request #17706: [SPARK-20423][ML] fix MLOR coeffs centering when ...

2017-04-21 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17706#discussion_r112747202 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -1204,6 +1207,9 @@ class LogisticRegressionSuite

[GitHub] spark pull request #17706: [SPARK-20423][ML] fix MLOR coeffs centering when ...

2017-04-21 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17706#discussion_r112745777 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -1204,6 +1207,9 @@ class LogisticRegressionSuite

[GitHub] spark issue #17706: [SPARK-20423][ML] fix MLOR coeffs centering when reg == ...

2017-04-21 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17706 Merged into master and 2.2 branch. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17706: [SPARK-20423][ML] fix MLOR coeffs centering when reg == ...

2017-04-21 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17706 LGTM. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17715: [SPARK-20047][ML] Constrained Logistic Regression

2017-04-21 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17715 High level questions, what happen to LBFGSB if the initial condition is out of bound? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

<    1   2   3   4   5   6   7   8   9   10   >