[jira] [Resolved] (SPARK-16107) Group GLM-related methods in generated doc

2016-06-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16107. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13820

[jira] [Resolved] (SPARK-16118) getDropLast is missing in OneHotEncoder

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16118. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13821

[jira] [Created] (SPARK-16118) getDropLast is missing in OneHotEncoder

2016-06-21 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16118: - Summary: getDropLast is missing in OneHotEncoder Key: SPARK-16118 URL: https://issues.apache.org/jira/browse/SPARK-16118 Project: Spark Issue Type: New

[jira] [Created] (SPARK-16117) Hide LibSVMFileFormat in public API docs

2016-06-21 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16117: - Summary: Hide LibSVMFileFormat in public API docs Key: SPARK-16117 URL: https://issues.apache.org/jira/browse/SPARK-16117 Project: Spark Issue Type:

[jira] [Closed] (SPARK-16113) Deprecate (or remove) multiclass APIs in ml.LogisticRegression

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-16113. - Resolution: Not A Problem > Deprecate (or remove) multiclass APIs in ml.LogisticRegression >

[jira] [Commented] (SPARK-16113) Deprecate (or remove) multiclass APIs in ml.LogisticRegression

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342642#comment-15342642 ] Xiangrui Meng commented on SPARK-16113: --- Just realized that `thresholds` was inherited from

[jira] [Created] (SPARK-16113) Deprecate (or remove) multiclass APIs in ml.LogisticRegression

2016-06-21 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16113: - Summary: Deprecate (or remove) multiclass APIs in ml.LogisticRegression Key: SPARK-16113 URL: https://issues.apache.org/jira/browse/SPARK-16113 Project: Spark

[jira] [Comment Edited] (SPARK-16111) Hide SparkOrcNewRecordReader in API docs

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342498#comment-15342498 ] Xiangrui Meng edited comment on SPARK-16111 at 6/21/16 7:26 PM: Ping

[jira] [Comment Edited] (SPARK-16111) Hide SparkOrcNewRecordReader in API docs

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342498#comment-15342498 ] Xiangrui Meng edited comment on SPARK-16111 at 6/21/16 7:26 PM: Ping

[jira] [Commented] (SPARK-16111) Hide SparkOrcNewRecordReader in API docs

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342498#comment-15342498 ] Xiangrui Meng commented on SPARK-16111: --- Ping [~ rbalamohan] > Hide SparkOrcNewRecordReader in API

[jira] [Created] (SPARK-16111) Hide SparkOrcNewRecordReader in API docs

2016-06-21 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16111: - Summary: Hide SparkOrcNewRecordReader in API docs Key: SPARK-16111 URL: https://issues.apache.org/jira/browse/SPARK-16111 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16086: -- Fix Version/s: 1.6.2 1.5.3 > Python UDF failed when there is no arguments >

[jira] [Updated] (SPARK-15741) PySpark Cleanup of _setDefault with seed=None

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15741: -- Assignee: Bryan Cutler > PySpark Cleanup of _setDefault with seed=None >

[jira] [Updated] (SPARK-15741) PySpark Cleanup of _setDefault with seed=None

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15741: -- Target Version/s: 2.0.0 > PySpark Cleanup of _setDefault with seed=None >

[jira] [Resolved] (SPARK-15741) PySpark Cleanup of _setDefault with seed=None

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15741. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13672

[jira] [Updated] (SPARK-16107) Group GLM-related methods in generated doc

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16107: -- Assignee: Junyang Qian > Group GLM-related methods in generated doc >

[jira] [Updated] (SPARK-16107) Group GLM-related methods in generated doc

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16107: -- Labels: starter (was: ) > Group GLM-related methods in generated doc >

[jira] [Updated] (SPARK-16107) Group GLM-related methods in generated doc

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16107: -- Description: Group API docs of spark.glm, glm, predict(GLM), summary(GLM), read/write.ml(GLM)

[jira] [Updated] (SPARK-16107) Group GLM-related methods in generated doc

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16107: -- Description: spark.glm: spark.glm, glm, predict(GLM), summary(GLM), read/write.ml(GLM) >

[jira] [Commented] (SPARK-16107) Group GLM-related methods in generated doc

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342178#comment-15342178 ] Xiangrui Meng commented on SPARK-16107: --- ping [~junyangq] > Group GLM-related methods in generated

[jira] [Created] (SPARK-16107) Group GLM-related methods in generated doc

2016-06-21 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16107: - Summary: Group GLM-related methods in generated doc Key: SPARK-16107 URL: https://issues.apache.org/jira/browse/SPARK-16107 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16090) Improve method grouping in SparkR generated docs

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342173#comment-15342173 ] Xiangrui Meng commented on SPARK-16090: --- I changed the issue type to umbrella since there could be

[jira] [Updated] (SPARK-16090) Improve method grouping in SparkR generated docs

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16090: -- Issue Type: Umbrella (was: Improvement) > Improve method grouping in SparkR generated docs >

[jira] [Updated] (SPARK-15177) SparkR 2.0 QA: make SparkR model params and default values consistent with MLlib

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15177: -- Summary: SparkR 2.0 QA: make SparkR model params and default values consistent with MLlib

[jira] [Updated] (SPARK-15177) SparkR 2.0 QA: New R APIs and API docs for mllib.R

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15177: -- Description: Audit new public R APIs in mllib.R (was: Audit new public R APIs in mllib.R.) >

[jira] [Updated] (SPARK-15177) SparkR 2.0 QA: make SparkR model params and default values consistent with MLlib

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15177: -- Description: Make SparkR model params and default values consistent with MLlib (was: Audit

[jira] [Updated] (SPARK-15177) SparkR 2.0 QA: New R APIs and API docs for mllib.R

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15177: -- Shepherd: Xiangrui Meng > SparkR 2.0 QA: New R APIs and API docs for mllib.R >

[jira] [Resolved] (SPARK-15177) SparkR 2.0 QA: New R APIs and API docs for mllib.R

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15177. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 I marked

[jira] [Updated] (SPARK-15177) SparkR 2.0 QA: New R APIs and API docs for mllib.R

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15177: -- Assignee: Yanbo Liang > SparkR 2.0 QA: New R APIs and API docs for mllib.R >

[jira] [Commented] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341330#comment-15341330 ] Xiangrui Meng commented on SPARK-16071: --- [~ding] This JIRA is not to solve this particular issue

[jira] [Resolved] (SPARK-16045) Spark 2.0 ML.feature: doc update for stopwords and binarizer

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16045. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13375

[jira] [Updated] (SPARK-16045) Spark 2.0 ML.feature: doc update for stopwords and binarizer

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16045: -- Assignee: yuhao yang > Spark 2.0 ML.feature: doc update for stopwords and binarizer >

[jira] [Updated] (SPARK-16045) Spark 2.0 ML.feature: doc update for stopwords and binarizer

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16045: -- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > Spark 2.0 ML.feature: doc update for

[jira] [Resolved] (SPARK-7751) Add @Since annotation to stable and experimental methods in MLlib

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7751. -- Resolution: Fixed Fix Version/s: 2.0.0 Mark this umbrella as resolved since all

[jira] [Updated] (SPARK-10258) Add @Since annotation to ml.feature

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10258: -- Shepherd: Nick Pentreath > Add @Since annotation to ml.feature >

[jira] [Resolved] (SPARK-10258) Add @Since annotation to ml.feature

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10258. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13641

[jira] [Commented] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341297#comment-15341297 ] Xiangrui Meng commented on SPARK-16086: --- Reverted the changes in master and branch-2.0 since they

[jira] [Reopened] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reopened SPARK-16086: --- > Python UDF failed when there is no arguments > >

[jira] [Updated] (SPARK-16086) Python UDF failed when there is no arguments

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16086: -- Fix Version/s: (was: 2.0.0) > Python UDF failed when there is no arguments >

[jira] [Commented] (SPARK-16090) Improve method grouping in SparkR generated docs

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341258#comment-15341258 ] Xiangrui Meng commented on SPARK-16090: --- For ML methods, I'd like to propose the following

[jira] [Updated] (SPARK-16090) Improve method grouping in SparkR generated docs

2016-06-21 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16090: -- Description: This JIRA follows the discussion on https://github.com/apache/spark/pull/13109 to

[jira] [Created] (SPARK-16090) Improve method grouping in SparkR generated docs

2016-06-21 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16090: - Summary: Improve method grouping in SparkR generated docs Key: SPARK-16090 URL: https://issues.apache.org/jira/browse/SPARK-16090 Project: Spark Issue

[jira] [Resolved] (SPARK-16079) PySpark ML classification missing import of DecisionTreeRegressionModel for GBTClassificationModel

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16079. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13787

[jira] [Updated] (SPARK-16079) PySpark ML classification missing import of DecisionTreeRegressionModel for GBTClassificationModel

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16079: -- Assignee: Bryan Cutler > PySpark ML classification missing import of

[jira] [Updated] (SPARK-16079) PySpark ML classification missing import of DecisionTreeRegressionModel for GBTClassificationModel

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16079: -- Affects Version/s: 2.0.0 > PySpark ML classification missing import of

[jira] [Commented] (SPARK-16074) Expose VectorUDT/MatrixUDT in a public API

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340642#comment-15340642 ] Xiangrui Meng commented on SPARK-16074: --- Picked option 2) because we don't have any Java source

[jira] [Assigned] (SPARK-16074) Expose VectorUDT/MatrixUDT in a public API

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-16074: - Assignee: Xiangrui Meng > Expose VectorUDT/MatrixUDT in a public API >

[jira] [Commented] (SPARK-16075) Make VectorUDT/MatrixUDT singleton under spark.ml package

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340419#comment-15340419 ] Xiangrui Meng commented on SPARK-16075: --- I'm not sure whether we should make this change in 2.0. It

[jira] [Updated] (SPARK-16075) Make VectorUDT/MatrixUDT singleton under spark.ml package

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16075: -- Description: Both VectorUDT and MatrixUDT are implemented as normal classes and their could

[jira] [Updated] (SPARK-16074) Expose VectorUDT/MatrixUDT in a public API

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16074: -- Description: Both VectorUDT and MatrixUDT are private APIs, because UserDefinedType itself is

[jira] [Created] (SPARK-16075) Make VectorUDT/MatrixUDT singleton under spark.ml package

2016-06-20 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16075: - Summary: Make VectorUDT/MatrixUDT singleton under spark.ml package Key: SPARK-16075 URL: https://issues.apache.org/jira/browse/SPARK-16075 Project: Spark

[jira] [Created] (SPARK-16074) Expose VectorUDT/MatrixUDT in a public API

2016-06-20 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16074: - Summary: Expose VectorUDT/MatrixUDT in a public API Key: SPARK-16074 URL: https://issues.apache.org/jira/browse/SPARK-16074 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-16073) Performance of Parquet encodings on saving primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16073: -- Description: Spark supports both uncompressed and compressed (snappy, gzip, lzo) Parquet

[jira] [Updated] (SPARK-16073) Performance of Parquet encodings on saving primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16073: -- Description: Spark supports both uncompressed and compressed (snappy, gzip, lzo) Parquet

[jira] [Created] (SPARK-16073) Performance of Parquet encodings on saving primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16073: - Summary: Performance of Parquet encodings on saving primitive arrays Key: SPARK-16073 URL: https://issues.apache.org/jira/browse/SPARK-16073 Project: Spark

[jira] [Updated] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16070: -- Description: I created this umbrella JIRA to track DataFrame/Parquet issues with primitive

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16070: -- Description: I created this umbrella JIRA to track DataFrame/Parquet issues with primitive

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Assignee: Yin Huai > Not sufficient array size checks to avoid integer overflows in Tungsten >

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Description: Several bugs have been found caused by integer overflows in Tungsten. This JIRA

[jira] [Updated] (SPARK-16071) Not sufficient array size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16071: -- Summary: Not sufficient array size checks to avoid integer overflows in Tungsten (was: Not

[jira] [Created] (SPARK-16071) Not sufficient size checks to avoid integer overflows in Tungsten

2016-06-20 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16071: - Summary: Not sufficient size checks to avoid integer overflows in Tungsten Key: SPARK-16071 URL: https://issues.apache.org/jira/browse/SPARK-16071 Project: Spark

[jira] [Updated] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16070: -- Description: I created this umbrella JIRA to track DataFrame/Parquet issues with primitive

[jira] [Updated] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16070: -- Description: I created this umbrella JIRA to track DataFrame/Parquet issues with primitive

[jira] [Updated] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16070: -- Description: I created this umbrella JIRA to track DataFrame/Parquet issues with primitive

[jira] [Updated] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16070: -- Description: I created this umbrella JIRA to track DataFrame/Parquet issues with primitive

[jira] [Created] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-20 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16070: - Summary: DataFrame/Parquet issues with primitive arrays Key: SPARK-16070 URL: https://issues.apache.org/jira/browse/SPARK-16070 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16035: -- Assignee: Andrea Pasqua > The SparseVector parser fails checking for valid end parenthesis >

[jira] [Resolved] (SPARK-16035) The SparseVector parser fails checking for valid end parenthesis

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16035. --- Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by

[jira] [Resolved] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15129. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13285

[jira] [Updated] (SPARK-15892) Incorrectly merged AFTAggregator with zero total count

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15892: -- Fix Version/s: 2.0.0 > Incorrectly merged AFTAggregator with zero total count >

[jira] [Resolved] (SPARK-15892) Incorrectly merged AFTAggregator with zero total count

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15892. --- Resolution: Fixed Fix Version/s: (was: 2.0.0) 1.6.2 Issue

[jira] [Resolved] (SPARK-15603) Replace SQLContext with SparkSession in ML/MLLib

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15603. --- Resolution: Fixed Fix Version/s: 2.0.0 > Replace SQLContext with SparkSession in

[jira] [Resolved] (SPARK-16008) ML Logistic Regression aggregator serializes unnecessary data

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-16008. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13729

[jira] [Updated] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16000: -- Assignee: yuhao yang > Make model loading backward compatible with saved models using old

[jira] [Commented] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

2016-06-17 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336446#comment-15336446 ] Xiangrui Meng commented on SPARK-16000: --- That's great! Please let me know if you want to split the

[jira] [Closed] (SPARK-15947) Make pipeline components backward compatible with old vector columns in Scala/Java

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-15947. - > Make pipeline components backward compatible with old vector columns in > Scala/Java >

[jira] [Assigned] (SPARK-15946) Wrap the conversion utils in Python

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-15946: - Assignee: Xiangrui Meng > Wrap the conversion utils in Python >

[jira] [Updated] (SPARK-15947) Make pipeline components backward compatible with old vector columns in Scala/Java

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15947: -- Summary: Make pipeline components backward compatible with old vector columns in Scala/Java

[jira] [Closed] (SPARK-15948) Make pipeline components backward compatible with old vector columns in Python

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-15948. - Resolution: Won't Fix > Make pipeline components backward compatible with old vector columns in

[jira] [Resolved] (SPARK-15947) Make pipeline components backward compatible with old vector columns in Scala/Java

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-15947. --- Resolution: Won't Fix > Make pipeline components backward compatible with old vector columns

[jira] [Commented] (SPARK-15948) Make pipeline components backward compatible with old vector columns in Python

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334732#comment-15334732 ] Xiangrui Meng commented on SPARK-15948: --- Marked this as "Won't Do". See SPARK-15947 for reasons. >

[jira] [Updated] (SPARK-15643) ML 2.0 QA: migration guide update

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15643: -- Assignee: Yanbo Liang > ML 2.0 QA: migration guide update > -

[jira] [Commented] (SPARK-15643) ML 2.0 QA: migration guide update

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334729#comment-15334729 ] Xiangrui Meng commented on SPARK-15643: --- [~yanboliang] Please include a paragraph to help users

[jira] [Comment Edited] (SPARK-15947) Make pipeline components backward compatible with old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334725#comment-15334725 ] Xiangrui Meng edited comment on SPARK-15947 at 6/16/16 9:30 PM: Had an

[jira] [Commented] (SPARK-15947) Make pipeline components backward compatible with old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334725#comment-15334725 ] Xiangrui Meng commented on SPARK-15947: --- Had an offline discussion with [~josephkb]. There would be

[jira] [Updated] (SPARK-15947) Make pipeline components backward compatible with old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15947: -- Description: After SPARK-15945, we should make ALL pipeline components accept old vector

[jira] [Updated] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16000: -- Description: To help users migrate from Spark 1.6. to 2.0, we should make model loading

[jira] [Updated] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16000: -- Summary: Make model loading backward compatible with saved models using old vector columns

[jira] [Updated] (SPARK-15947) Make pipeline components backward compatible with old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15947: -- Description: After SPARK-15945, we should make ALL pipeline components accept old vector

[jira] [Updated] (SPARK-15947) Make pipeline components backward compatible with old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-15947: -- Summary: Make pipeline components backward compatible with old vector columns (was: Make

[jira] [Created] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns

2016-06-16 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-16000: - Summary: Make model loading backward compatible with saved models using old vector columns Key: SPARK-16000 URL: https://issues.apache.org/jira/browse/SPARK-16000

[jira] [Updated] (SPARK-16000) Make model loading backward compatible with saved models using old vector columns in Scala/Java

2016-06-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-16000: -- Summary: Make model loading backward compatible with saved models using old vector columns in

[jira] [Assigned] (SPARK-15947) Make pipeline components backward compatible with old vector columns in Scala/Java

2016-06-15 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-15947: - Assignee: Xiangrui Meng > Make pipeline components backward compatible with old vector

<    3   4   5   6   7   8   9   10   11   12   >