[jira] [Resolved] (SPARK-23758) MLlib 2.4 Roadmap

2019-07-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-23758. --- Resolution: Done > MLlib 2.4 Roadmap > - > > Key:

[jira] [Updated] (SPARK-23758) MLlib 2.4 Roadmap

2019-07-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23758: -- Affects Version/s: (was: 3.0.0) 2.4.0 > MLlib 2.4 Roadmap

[jira] [Commented] (SPARK-23758) MLlib 2.4 Roadmap

2019-07-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889240#comment-16889240 ] Joseph K. Bradley commented on SPARK-23758: --- Ah sorry, we stopped using this. I'll close it.

[jira] [Updated] (SPARK-26960) Reduce flakiness of Spark ML Listener test suite by waiting for listener bus to clear

2019-02-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-26960: -- Description: [SPARK-23674] added SparkListeners for some spark.ml events, as well as

[jira] [Created] (SPARK-26960) Reduce flakiness of Spark ML Listener test suite by waiting for listener bus to clear

2019-02-21 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-26960: - Summary: Reduce flakiness of Spark ML Listener test suite by waiting for listener bus to clear Key: SPARK-26960 URL: https://issues.apache.org/jira/browse/SPARK-26960

[jira] [Updated] (SPARK-25994) SPIP: Property Graphs, Cypher Queries, and Algorithms

2018-11-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25994: -- Labels: SPIP (was: ) > SPIP: Property Graphs, Cypher Queries, and Algorithms >

[jira] [Updated] (SPARK-25324) ML 2.4 QA: API: Java compatibility, docs

2018-09-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25324: -- Fix Version/s: 2.4.0 > ML 2.4 QA: API: Java compatibility, docs >

[jira] [Updated] (SPARK-25320) ML, Graph 2.4 QA: API: Binary incompatible changes

2018-09-28 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25320: -- Fix Version/s: 2.4.0 > ML, Graph 2.4 QA: API: Binary incompatible changes >

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618118#comment-16618118 ] Joseph K. Bradley commented on SPARK-25321: --- [~WeichenXu123] Have you been able to look into

[jira] [Commented] (SPARK-25321) ML, Graph 2.4 QA: API: New Scala APIs, docs

2018-09-12 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16612436#comment-16612436 ] Joseph K. Bradley commented on SPARK-25321: --- You're right; these are breaking changes. If

[jira] [Commented] (SPARK-25397) SparkSession.conf fails when given default value with Python 3

2018-09-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16609593#comment-16609593 ] Joseph K. Bradley commented on SPARK-25397: --- CC [~smilegator], [~cloud_fan] for visibility >

[jira] [Updated] (SPARK-25397) SparkSession.conf fails when given default value with Python 3

2018-09-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25397: -- Priority: Minor (was: Major) > SparkSession.conf fails when given default value with

[jira] [Created] (SPARK-25397) SparkSession.conf fails when given default value with Python 3

2018-09-10 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-25397: - Summary: SparkSession.conf fails when given default value with Python 3 Key: SPARK-25397 URL: https://issues.apache.org/jira/browse/SPARK-25397 Project:

[jira] [Resolved] (SPARK-25268) runParallelPersonalizedPageRank throws serialization Exception

2018-09-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-25268. --- Resolution: Fixed Fix Version/s: 2.4.0 3.0.0 Issue

[jira] [Assigned] (SPARK-25268) runParallelPersonalizedPageRank throws serialization Exception

2018-09-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-25268: - Assignee: shahid > runParallelPersonalizedPageRank throws serialization

[jira] [Updated] (SPARK-25268) runParallelPersonalizedPageRank throws serialization Exception

2018-09-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25268: -- Shepherd: Joseph K. Bradley > runParallelPersonalizedPageRank throws serialization

[jira] [Resolved] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-25124. --- Resolution: Fixed Fix Version/s: 2.3.2 Issue resolved by pull request 8

[jira] [Updated] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25124: -- Target Version/s: 2.3.2 > VectorSizeHint.size is buggy, breaking streaming pipeline >

[jira] [Commented] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590919#comment-16590919 ] Joseph K. Bradley commented on SPARK-25124: --- I merged

[jira] [Updated] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25124: -- Target Version/s: 2.3.2, 2.4.0 (was: 2.3.2) > VectorSizeHint.size is buggy, breaking

[jira] [Updated] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25124: -- Fix Version/s: 2.4.0 > VectorSizeHint.size is buggy, breaking streaming pipeline >

[jira] [Updated] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25124: -- Shepherd: Joseph K. Bradley > VectorSizeHint.size is buggy, breaking streaming

[jira] [Assigned] (SPARK-25124) VectorSizeHint.size is buggy, breaking streaming pipeline

2018-08-23 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-25124: - Assignee: Huaxin Gao > VectorSizeHint.size is buggy, breaking streaming

[jira] [Resolved] (SPARK-25149) Personalized PageRank raises an error if vertexIDs are > MaxInt

2018-08-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-25149. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22139

[jira] [Assigned] (SPARK-25149) Personalized PageRank raises an error if vertexIDs are > MaxInt

2018-08-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-25149: - Assignee: Bago Amirbekian > Personalized PageRank raises an error if vertexIDs

[jira] [Updated] (SPARK-25149) Personalized PageRank raises an error if vertexIDs are > MaxInt

2018-08-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25149: -- Summary: Personalized PageRank raises an error if vertexIDs are > MaxInt (was:

[jira] [Updated] (SPARK-25149) Personalized Page Rank raises an error if vertexIDs are > MaxInt

2018-08-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-25149: -- Shepherd: Joseph K. Bradley > Personalized Page Rank raises an error if vertexIDs are

[jira] [Assigned] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-24632: - Assignee: (was: Joseph K. Bradley) > Allow 3rd-party libraries to use

[jira] [Commented] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565769#comment-16565769 ] Joseph K. Bradley commented on SPARK-24632: --- I'm unassigning myself since I don't have time to

[jira] [Commented] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-08-01 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565767#comment-16565767 ] Joseph K. Bradley commented on SPARK-24632: --- That's a good point. Let's do it your way. : )

[jira] [Resolved] (SPARK-24852) Have spark.ml training use updated `Instrumentation` APIs.

2018-07-20 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-24852. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21799

[jira] [Updated] (SPARK-24852) Have spark.ml training use updated `Instrumentation` APIs.

2018-07-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24852: -- Shepherd: Joseph K. Bradley > Have spark.ml training use updated `Instrumentation`

[jira] [Assigned] (SPARK-24852) Have spark.ml training use updated `Instrumentation` APIs.

2018-07-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-24852: - Assignee: Bago Amirbekian > Have spark.ml training use updated

[jira] [Resolved] (SPARK-24747) Make spark.ml.util.Instrumentation class more flexible

2018-07-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-24747. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21719

[jira] [Commented] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-06-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522557#comment-16522557 ] Joseph K. Bradley commented on SPARK-24632: --- CC [~yanboliang], [~holden.ka...@gmail.com]: You

[jira] [Updated] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-06-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24632: -- Description: This is a follow-up for [SPARK-17025], which allowed users to implement

[jira] [Assigned] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-24632: - Assignee: Joseph K. Bradley > Allow 3rd-party libraries to use pyspark.ml

[jira] [Updated] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24632: -- Description: This is a follow-up for [SPARK-17025], which allowed users to implement

[jira] [Resolved] (SPARK-21926) Compatibility between ML Transformers and Structured Streaming

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-21926. --- Resolution: Fixed Fix Version/s: 2.3.0 Marking fix version as 2.3.0 since

[jira] [Updated] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24465: -- Description: Locality Sensitive Hashing (LSH) Models

[jira] [Comment Edited] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520677#comment-16520677 ] Joseph K. Bradley edited comment on SPARK-24465 at 6/22/18 6:39 PM:

[jira] [Commented] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520677#comment-16520677 ] Joseph K. Bradley commented on SPARK-24465: --- Oh actually I think I made this by mistake? I

[jira] [Resolved] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-24465. --- Resolution: Fixed Assignee: Joseph K. Bradley Fix Version/s: 2.3.1

[jira] [Updated] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24465: -- Description: Locality Sensitive Hashing (LSH) Models

[jira] [Commented] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520671#comment-16520671 ] Joseph K. Bradley commented on SPARK-24465: --- You're right; I did not read [SPARK-12878]

[jira] [Updated] (SPARK-12878) Dataframe fails with nested User Defined Types

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-12878: -- Description: Spark 1.6.0 crashes when using nested User Defined Types in a Dataframe.

[jira] [Commented] (SPARK-19498) Discussion: Making MLlib APIs extensible for 3rd party libraries

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520666#comment-16520666 ] Joseph K. Bradley commented on SPARK-19498: --- Sure, comments are welcome! Or links to JIRAs,

[jira] [Issue Comment Deleted] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-17025: -- Comment: was deleted (was: Thank you for your e-mail. I am on businees travel until

[jira] [Assigned] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-17025: - Assignee: Ajay Saini > Cannot persist PySpark ML Pipeline model that includes

[jira] [Resolved] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-17025. --- Resolution: Fixed Fix Version/s: 2.3.0 Fixed by linked JIRAs > Cannot

[jira] [Created] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

2018-06-22 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-24632: - Summary: Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence Key: SPARK-24632 URL:

[jira] [Commented] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520646#comment-16520646 ] Joseph K. Bradley commented on SPARK-17025: --- We've tested it with Python-only implementations,

[jira] [Commented] (SPARK-4591) Algorithm/model parity for spark.ml (Scala)

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520635#comment-16520635 ] Joseph K. Bradley commented on SPARK-4591: -- There are still a few contained tasks which are

[jira] [Commented] (SPARK-11107) spark.ml should support more input column types: umbrella

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16520634#comment-16520634 ] Joseph K. Bradley commented on SPARK-11107: --- There are still lots of Transformers and

[jira] [Resolved] (SPARK-11107) spark.ml should support more input column types: umbrella

2018-06-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-11107. --- Resolution: Done > spark.ml should support more input column types: umbrella >

[jira] [Commented] (SPARK-24467) VectorAssemblerEstimator

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511852#comment-16511852 ] Joseph K. Bradley commented on SPARK-24467: --- True, we would have to make the VectorAssembler

[jira] [Commented] (SPARK-15882) Discuss distributed linear algebra in spark.ml package

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511845#comment-16511845 ] Joseph K. Bradley commented on SPARK-15882: --- I'm afraid I don't have time to prioritize this

[jira] [Updated] (SPARK-3723) DecisionTree, RandomForest: Add more instrumentation

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3723: - Shepherd: (was: Joseph K. Bradley) > DecisionTree, RandomForest: Add more

[jira] [Updated] (SPARK-3727) Trees and ensembles: More prediction functionality

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3727: - Shepherd: (was: Joseph K. Bradley) > Trees and ensembles: More prediction

[jira] [Updated] (SPARK-5362) Gradient and Optimizer to support generic output (instead of label) and data batches

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5362: - Shepherd: (was: Joseph K. Bradley) > Gradient and Optimizer to support generic output

[jira] [Updated] (SPARK-5556) Latent Dirichlet Allocation (LDA) using Gibbs sampler

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5556: - Shepherd: (was: Joseph K. Bradley) > Latent Dirichlet Allocation (LDA) using Gibbs

[jira] [Updated] (SPARK-4591) Algorithm/model parity for spark.ml (Scala)

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4591: - Shepherd: (was: Joseph K. Bradley) > Algorithm/model parity for spark.ml (Scala) >

[jira] [Updated] (SPARK-8799) OneVsRestModel should extend ClassificationModel

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8799: - Shepherd: (was: Joseph K. Bradley) > OneVsRestModel should extend ClassificationModel

[jira] [Updated] (SPARK-9120) Add multivariate regression (or prediction) interface

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9120: - Shepherd: (was: Joseph K. Bradley) > Add multivariate regression (or prediction)

[jira] [Updated] (SPARK-8767) Abstractions for InputColParam, OutputColParam

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8767: - Shepherd: (was: Joseph K. Bradley) > Abstractions for InputColParam, OutputColParam >

[jira] [Updated] (SPARK-8799) OneVsRestModel should extend ClassificationModel

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8799: - Target Version/s: (was: 3.0.0) > OneVsRestModel should extend ClassificationModel >

[jira] [Updated] (SPARK-7424) spark.ml classification, regression abstractions should add metadata to output column

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7424: - Shepherd: (was: Joseph K. Bradley) > spark.ml classification, regression abstractions

[jira] [Updated] (SPARK-21166) Automated ML persistence

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21166: -- Shepherd: (was: Joseph K. Bradley) > Automated ML persistence >

[jira] [Updated] (SPARK-14585) Provide accessor methods for Pipeline stages

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14585: -- Shepherd: (was: Joseph K. Bradley) > Provide accessor methods for Pipeline stages >

[jira] [Updated] (SPARK-19591) Add sample weights to decision trees

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-19591: -- Shepherd: (was: Joseph K. Bradley) > Add sample weights to decision trees >

[jira] [Updated] (SPARK-9140) Replace TimeTracker by Stopwatch

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9140: - Shepherd: (was: Joseph K. Bradley) > Replace TimeTracker by Stopwatch >

[jira] [Updated] (SPARK-15573) Backwards-compatible persistence for spark.ml

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15573: -- Shepherd: (was: Joseph K. Bradley) > Backwards-compatible persistence for spark.ml

[jira] [Updated] (SPARK-19498) Discussion: Making MLlib APIs extensible for 3rd party libraries

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-19498: -- Shepherd: (was: Joseph K. Bradley) > Discussion: Making MLlib APIs extensible for

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24359: -- Shepherd: Xiangrui Meng (was: Joseph K. Bradley) > SPIP: ML Pipelines in R >

[jira] [Updated] (SPARK-24097) Instruments improvements - RandomForest and GradientBoostedTree

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24097: -- Shepherd: (was: Joseph K. Bradley) > Instruments improvements - RandomForest and

[jira] [Updated] (SPARK-21926) Compatibility between ML Transformers and Structured Streaming

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21926: -- Shepherd: (was: Joseph K. Bradley) > Compatibility between ML Transformers and

[jira] [Updated] (SPARK-24212) PrefixSpan in spark.ml: user guide section

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24212: -- Shepherd: (was: Joseph K. Bradley) > PrefixSpan in spark.ml: user guide section >

[jira] [Resolved] (SPARK-14376) spark.ml parity for trees

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14376. --- Resolution: Fixed Fix Version/s: 2.4.0 > spark.ml parity for trees >

[jira] [Assigned] (SPARK-10817) ML abstraction umbrella

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-10817: - Assignee: (was: Joseph K. Bradley) > ML abstraction umbrella >

[jira] [Assigned] (SPARK-5572) LDA improvement listing

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-5572: Assignee: (was: Joseph K. Bradley) > LDA improvement listing >

[jira] [Assigned] (SPARK-4285) Transpose RDD[Vector] to column store for ML

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-4285: Assignee: (was: Joseph K. Bradley) > Transpose RDD[Vector] to column store

[jira] [Updated] (SPARK-5572) LDA improvement listing

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5572: - Shepherd: (was: Joseph K. Bradley) > LDA improvement listing > ---

[jira] [Assigned] (SPARK-7206) Gaussian Mixture Model (GMM) improvements

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-7206: Assignee: (was: Joseph K. Bradley) > Gaussian Mixture Model (GMM)

[jira] [Commented] (SPARK-14376) spark.ml parity for trees

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511751#comment-16511751 ] Joseph K. Bradley commented on SPARK-14376: --- Thanks! I'll close it. > spark.ml parity for

[jira] [Assigned] (SPARK-14604) Modify design of ML model summaries

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-14604: - Assignee: (was: Joseph K. Bradley) > Modify design of ML model summaries >

[jira] [Commented] (SPARK-22666) Spark datasource for image format

2018-06-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511745#comment-16511745 ] Joseph K. Bradley commented on SPARK-22666: --- Side note: The Java library we use for reading

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

2018-06-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505049#comment-16505049 ] Joseph K. Bradley commented on SPARK-24359: --- It sounds like everyone is agreed on the fact

[jira] [Created] (SPARK-24467) VectorAssemblerEstimator

2018-06-05 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-24467: - Summary: VectorAssemblerEstimator Key: SPARK-24467 URL: https://issues.apache.org/jira/browse/SPARK-24467 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-04 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24465: -- Description: Locality Sensitive Hashing (LSH) Models

[jira] [Created] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-04 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-24465: - Summary: LSHModel should support Structured Streaming for transform Key: SPARK-24465 URL: https://issues.apache.org/jira/browse/SPARK-24465 Project: Spark

[jira] [Updated] (SPARK-24465) LSHModel should support Structured Streaming for transform

2018-06-04 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24465: -- Environment: (was: Locality Sensitive Hashing (LSH) Models

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-31 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497203#comment-16497203 ] Joseph K. Bradley commented on SPARK-24359: --- Clarification question: [~falaki] did you mean to

[jira] [Updated] (SPARK-22666) Spark datasource for image format

2018-05-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22666: -- Summary: Spark datasource for image format (was: Spark reader source for image

[jira] [Commented] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491412#comment-16491412 ] Joseph K. Bradley commented on SPARK-24359: --- Regarding separating repos: What's the conclusion?

[jira] [Updated] (SPARK-24359) SPIP: ML Pipelines in R

2018-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24359: -- Description: h1. Background and motivation SparkR supports calling MLlib

[jira] [Commented] (SPARK-23455) Default Params in ML should be saved separately

2018-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491354#comment-16491354 ] Joseph K. Bradley commented on SPARK-23455: --- Yep, thanks [~viirya] for answering! It will

[jira] [Updated] (SPARK-24300) generateLDAData in ml.cluster.LDASuite didn't set seed correctly

2018-05-25 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24300: -- Shepherd: Joseph K. Bradley > generateLDAData in ml.cluster.LDASuite didn't set seed

[jira] [Created] (SPARK-24333) Add fit with validation set to spark.ml GBT: Python API

2018-05-21 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-24333: - Summary: Add fit with validation set to spark.ml GBT: Python API Key: SPARK-24333 URL: https://issues.apache.org/jira/browse/SPARK-24333 Project: Spark

[jira] [Resolved] (SPARK-7132) Add fit with validation set to spark.ml GBT

2018-05-21 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7132. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21129

[jira] [Assigned] (SPARK-22884) ML test for StructuredStreaming: spark.ml.clustering

2018-05-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-22884: - Assignee: Sandor Murakozi > ML test for StructuredStreaming:

  1   2   3   4   5   6   7   8   9   10   >