[jira] [Closed] (SPARK-14640) @DeveloperApi on PythonUserDefinedType

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-14640. --- Resolution: Not A Problem I'm closing this. PythonUserDefinedType is actually an internal class. >

[jira] [Commented] (SPARK-13801) DataFrame.col should return unresolved attribute

2016-04-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242436#comment-15242436 ] Takeshi Yamamuro commented on SPARK-13801: -- I got this cause; wrong nullability is set in

[jira] [Created] (SPARK-14653) Remove NumericParser and jackson dependency from mllib-local

2016-04-14 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14653: - Summary: Remove NumericParser and jackson dependency from mllib-local Key: SPARK-14653 URL: https://issues.apache.org/jira/browse/SPARK-14653 Project: Spark

[jira] [Resolved] (SPARK-14374) PySpark ml GBTClassifier, Regressor support export/import

2016-04-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14374. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12383

[jira] [Created] (SPARK-14652) pyspark streaming driver unable to cleanup metadata for cached RDDs leading to driver OOM

2016-04-14 Thread Wei Deng (JIRA)
Wei Deng created SPARK-14652: Summary: pyspark streaming driver unable to cleanup metadata for cached RDDs leading to driver OOM Key: SPARK-14652 URL: https://issues.apache.org/jira/browse/SPARK-14652

[jira] [Resolved] (SPARK-14275) Reimplement TypedAggregateExpression to DeclarativeAggregate

2016-04-14 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-14275. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12067

[jira] [Updated] (SPARK-14447) Speed up TungstenAggregate w/ keys using AggregateHashMap

2016-04-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14447: - Assignee: Sameer Agarwal > Speed up TungstenAggregate w/ keys using AggregateHashMap >

[jira] [Resolved] (SPARK-14447) Speed up TungstenAggregate w/ keys using AggregateHashMap

2016-04-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-14447. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12345

[jira] [Closed] (SPARK-14432) Add API to calculate the approximate quantiles for multiple columns

2016-04-14 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-14432. --- Resolution: Duplicate > Add API to calculate the approximate quantiles for multiple columns

[jira] [Commented] (SPARK-14649) DagScheduler runs duplicate tasks on fetch failure

2016-04-14 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242376#comment-15242376 ] Sital Kedia commented on SPARK-14649: - [~kayousterhout] - Any idea how to handle this? >

[jira] [Created] (SPARK-14651) CREATE TEMPORARY TABLE is not supported yet

2016-04-14 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-14651: --- Summary: CREATE TEMPORARY TABLE is not supported yet Key: SPARK-14651 URL: https://issues.apache.org/jira/browse/SPARK-14651 Project: Spark Issue

[jira] [Created] (SPARK-14650) Compile Spark REPL for Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-14650: -- Summary: Compile Spark REPL for Scala 2.12 Key: SPARK-14650 URL: https://issues.apache.org/jira/browse/SPARK-14650 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242366#comment-15242366 ] Apache Spark commented on SPARK-14622: -- User 'yang0228' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14622: Assignee: (was: Apache Spark) > Retain lost executors status >

[jira] [Assigned] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14622: Assignee: Apache Spark > Retain lost executors status > > >

[jira] [Updated] (SPARK-14104) All Python param setters should use the `_set` method.

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14104: -- Target Version/s: 2.0.0 > All Python param setters should use the `_set` method. >

[jira] [Updated] (SPARK-14104) All Python param setters should use the `_set` method.

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14104: -- Shepherd: Joseph K. Bradley Assignee: Seth Hendrickson > All Python param setters

[jira] [Commented] (SPARK-14460) DataFrameWriter JDBC doesn't Quote/Escape column names

2016-04-14 Thread Bo Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242330#comment-15242330 ] Bo Meng commented on SPARK-14460: - I have added the test case that is using "order" as column name.

[jira] [Updated] (SPARK-14622) Retain lost executors status

2016-04-14 Thread Qingyang Hong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qingyang Hong updated SPARK-14622: -- Target Version/s: 2.0.0 (was: 1.6.0) Fix Version/s: (was: 1.6.0)

[jira] [Created] (SPARK-14649) DagScheduler runs duplicate tasks on fetch failure

2016-04-14 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-14649: --- Summary: DagScheduler runs duplicate tasks on fetch failure Key: SPARK-14649 URL: https://issues.apache.org/jira/browse/SPARK-14649 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14649) DagScheduler runs duplicate tasks on fetch failure

2016-04-14 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-14649: Description: When running a job we found out that there are many duplicate tasks running after

[jira] [Resolved] (SPARK-14601) Minor doc/usage changes related to removal of Spark assembly

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14601. - Resolution: Fixed Assignee: Mark Grover Fix Version/s: 2.0.0 > Minor doc/usage

[jira] [Commented] (SPARK-14346) SHOW CREATE TABLE command (Native)

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242281#comment-15242281 ] Apache Spark commented on SPARK-14346: -- User 'xwu0226' has created a pull request for this issue:

[jira] [Created] (SPARK-14648) Spark EC2 script creates cluster but spark is not installed properly.

2016-04-14 Thread Nikhil (JIRA)
Nikhil created SPARK-14648: -- Summary: Spark EC2 script creates cluster but spark is not installed properly. Key: SPARK-14648 URL: https://issues.apache.org/jira/browse/SPARK-14648 Project: Spark

[jira] [Commented] (SPARK-14489) RegressionEvaluator returns NaN for ALS in Spark ml

2016-04-14 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242251#comment-15242251 ] Seth Hendrickson commented on SPARK-14489: -- [~mlnick] I am skeptical that

[jira] [Commented] (SPARK-14609) LOAD DATA

2016-04-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242252#comment-15242252 ] Xiao Li commented on SPARK-14609: - BTW, in the implementation, we need to read the following documents:

[jira] [Assigned] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14647: Assignee: Andrew Or (was: Apache Spark) > Group SQLContext/HiveContext state into

[jira] [Assigned] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14647: Assignee: Apache Spark (was: Andrew Or) > Group SQLContext/HiveContext state into

[jira] [Commented] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242250#comment-15242250 ] Apache Spark commented on SPARK-14647: -- User 'andrewor14' has created a pull request for this issue:

[jira] [Created] (SPARK-14647) Group SQLContext/HiveContext state into PersistentState

2016-04-14 Thread Andrew Or (JIRA)
Andrew Or created SPARK-14647: - Summary: Group SQLContext/HiveContext state into PersistentState Key: SPARK-14647 URL: https://issues.apache.org/jira/browse/SPARK-14647 Project: Spark Issue

[jira] [Commented] (SPARK-14609) LOAD DATA

2016-04-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242238#comment-15242238 ] Xiao Li commented on SPARK-14609: - https://issues.apache.org/jira/browse/SPARK-14603 is time consuming.

[jira] [Commented] (SPARK-14460) DataFrameWriter JDBC doesn't Quote/Escape column names

2016-04-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242231#comment-15242231 ] Andrew Ash commented on SPARK-14460: I also had a problem with a column named "order" so we should

[jira] [Created] (SPARK-14646) k-means save/load should put one cluster per row

2016-04-14 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-14646: - Summary: k-means save/load should put one cluster per row Key: SPARK-14646 URL: https://issues.apache.org/jira/browse/SPARK-14646 Project: Spark Issue

[jira] [Resolved] (SPARK-12869) Optimize conversion from BlockMatrix to IndexedRowMatrix

2016-04-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12869. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10839

[jira] [Resolved] (SPARK-14565) RandomForest should use parseInt and parseDouble for feature subset size instead of regexes

2016-04-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-14565. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12360

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting

[jira] [Assigned] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14644: Assignee: (was: Apache Spark) > Binary param can be a shared param with rewording >

[jira] [Assigned] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14644: Assignee: Apache Spark > Binary param can be a shared param with rewording >

[jira] [Commented] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242180#comment-15242180 ] Apache Spark commented on SPARK-14644: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14645: Assignee: (was: Apache Spark) > non local Python resource doesn't work with Mesos

[jira] [Assigned] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14645: Assignee: Apache Spark > non local Python resource doesn't work with Mesos cluster mode >

[jira] [Commented] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242172#comment-15242172 ] Apache Spark commented on SPARK-14645: -- User 'tnachen' has created a pull request for this issue:

[jira] [Created] (SPARK-14645) non local Python resource doesn't work with Mesos cluster mode

2016-04-14 Thread Timothy Chen (JIRA)
Timothy Chen created SPARK-14645: Summary: non local Python resource doesn't work with Mesos cluster mode Key: SPARK-14645 URL: https://issues.apache.org/jira/browse/SPARK-14645 Project: Spark

[jira] [Commented] (SPARK-14540) Support Scala 2.12 closures and Java 8 lambdas in ClosureCleaner

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242160#comment-15242160 ] Josh Rosen commented on SPARK-14540: There's now a Scala PR to fix this:

[jira] [Updated] (SPARK-14583) SparkSQL doesn't apply TBLPROPERTIES('serialization.null.format'='') when Hive Table has partitions

2016-04-14 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephane Maarek updated SPARK-14583: Summary: SparkSQL doesn't apply TBLPROPERTIES('serialization.null.format'='') when Hive

[jira] [Updated] (SPARK-14583) SparkSQL doesn't read TBLPROPERTIES('serialization.null.format'='') when Hive Table has partitions

2016-04-14 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephane Maarek updated SPARK-14583: Summary: SparkSQL doesn't read TBLPROPERTIES('serialization.null.format'='') when Hive

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Since the PipelineMLWriter/PipelineMLReader/PipelineModelMLWriter/PipelineModelMLReader

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Since the PipelineMLWriter/PipelineMLReader/PipelineModelMLWriter/PipelineModelMLReader

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Since the PipelineMLWriter/PipelineMLReader/PipelineModelMLWriter/PipelineModelMLReader

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting

[jira] [Updated] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-14280: --- Description: The following instructions will be kept quasi-up-to-date and are the best starting

[jira] [Updated] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-14440: -- Description: Remove * PipelineMLWriter * PipelineMLReader * PipelineModelMLWriter *

[jira] [Commented] (SPARK-14440) Remove PySpark ml.pipeline's specific Reader and Writer

2016-04-14 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242147#comment-15242147 ] Xusen Yin commented on SPARK-14440: --- Sorry for the late response, I'll update it soon. > Remove

[jira] [Updated] (SPARK-14130) [Table related commands] Alter column

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-14130: -- Assignee: Yin Huai (was: Andrew Or) > [Table related commands] Alter column >

[jira] [Commented] (SPARK-12741) DataFrame count method return wrong size.

2016-04-14 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242132#comment-15242132 ] Stephane Maarek commented on SPARK-12741: - Hi Sean, What do you mean by the behavior on master?

[jira] [Commented] (SPARK-14519) Cross-publish Kafka for Scala 2.12.0-M4

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242130#comment-15242130 ] Josh Rosen commented on SPARK-14519: I emailed the Kafka dev list:

[jira] [Created] (SPARK-14644) Binary param can be a shared param with rewording

2016-04-14 Thread holdenk (JIRA)
holdenk created SPARK-14644: --- Summary: Binary param can be a shared param with rewording Key: SPARK-14644 URL: https://issues.apache.org/jira/browse/SPARK-14644 Project: Spark Issue Type:

[jira] [Created] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-14643: -- Summary: Remove overloaded methods which become ambiguous in Scala 2.12 Key: SPARK-14643 URL: https://issues.apache.org/jira/browse/SPARK-14643 Project: Spark

[jira] [Assigned] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

2016-04-14 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14643: -- Assignee: Josh Rosen > Remove overloaded methods which become ambiguous in Scala 2.12 >

[jira] [Commented] (SPARK-14434) User guide doc and examples for GaussianMixture in spark.ml

2016-04-14 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242080#comment-15242080 ] Miao Wang commented on SPARK-14434: --- Sent pull request to SPARK-14433. Start working on this one. >

[jira] [Assigned] (SPARK-14433) PySpark ml GaussianMixture

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14433: Assignee: (was: Apache Spark) > PySpark ml GaussianMixture >

[jira] [Commented] (SPARK-14433) PySpark ml GaussianMixture

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241981#comment-15241981 ] Apache Spark commented on SPARK-14433: -- User 'wangmiao1981' has created a pull request for this

[jira] [Assigned] (SPARK-14433) PySpark ml GaussianMixture

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14433: Assignee: Apache Spark > PySpark ml GaussianMixture > -- > >

[jira] [Updated] (SPARK-14642) import org.apache.spark.sql.expressions._ breaks udf under functions

2016-04-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-14642: - Description: The following code works {code} scala> import org.apache.spark.sql.functions._ import

[jira] [Created] (SPARK-14642) import org.apache.spark.sql.expressions._ breaks udf under functions

2016-04-14 Thread Yin Huai (JIRA)
Yin Huai created SPARK-14642: Summary: import org.apache.spark.sql.expressions._ breaks udf under functions Key: SPARK-14642 URL: https://issues.apache.org/jira/browse/SPARK-14642 Project: Spark

[jira] [Commented] (SPARK-10179) LogisticRegressionWithSGD does not multiclass

2016-04-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241933#comment-15241933 ] DB Tsai commented on SPARK-10179: - Since we're moving to ML api, we will not add new features into mllib

[jira] [Closed] (SPARK-10179) LogisticRegressionWithSGD does not multiclass

2016-04-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai closed SPARK-10179. --- Resolution: Won't Fix > LogisticRegressionWithSGD does not multiclass >

[jira] [Created] (SPARK-14641) Specify worker log dir separately from scratch space dir

2016-04-14 Thread Philipp Hanslovsky (JIRA)
Philipp Hanslovsky created SPARK-14641: -- Summary: Specify worker log dir separately from scratch space dir Key: SPARK-14641 URL: https://issues.apache.org/jira/browse/SPARK-14641 Project: Spark

[jira] [Closed] (SPARK-14499) Add tests to make sure drop partitions of an external table will not delete data

2016-04-14 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-14499. --- Resolution: Resolved > Add tests to make sure drop partitions of an external table will not delete > data >

[jira] [Updated] (SPARK-7159) Support multiclass logistic regression in spark.ml

2016-04-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-7159: --- Issue Type: New Feature (was: Improvement) > Support multiclass logistic regression in spark.ml >

[jira] [Commented] (SPARK-14306) PySpark ml.classification OneVsRest support export/import

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241899#comment-15241899 ] Joseph K. Bradley commented on SPARK-14306: --- Right! Thanks > PySpark ml.classification

[jira] [Commented] (SPARK-14638) Threads of Spark Streaming (with Kafka) looses sight of the executor classpath

2016-04-14 Thread Younos Aboulnaga (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241893#comment-15241893 ] Younos Aboulnaga commented on SPARK-14638: -- But I have shown that the worker directory contains

[jira] [Commented] (SPARK-14638) Threads of Spark Streaming (with Kafka) looses sight of the executor classpath

2016-04-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241885#comment-15241885 ] Sean Owen commented on SPARK-14638: --- The stack trace you show demonstrates it's an initialization

[jira] [Updated] (SPARK-14374) PySpark ml GBTClassifier, Regressor support export/import

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14374: -- Shepherd: Joseph K. Bradley Assignee: Yanbo Liang > PySpark ml GBTClassifier,

[jira] [Closed] (SPARK-14616) TreeNodeException running Q44 and 58 on Parquet tables

2016-04-14 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JESSE CHEN closed SPARK-14616. -- Resolution: Not A Problem > TreeNodeException running Q44 and 58 on Parquet tables >

[jira] [Resolved] (SPARK-14545) Improve `LikeSimplification` by adding `a%b` rule

2016-04-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-14545. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 > Improve

[jira] [Commented] (SPARK-14616) TreeNodeException running Q44 and 58 on Parquet tables

2016-04-14 Thread JESSE CHEN (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241860#comment-15241860 ] JESSE CHEN commented on SPARK-14616: Build from yesterday did not have this problem. Closing. >

[jira] [Resolved] (SPARK-14238) Add binary toggle Param to PySpark HashingTF in ML & MLlib

2016-04-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-14238. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12079

[jira] [Resolved] (SPARK-14618) RegressionEvaluator doc out of date

2016-04-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14618. --- Resolution: Fixed Fix Version/s: 1.6.2 1.5.3

[jira] [Updated] (SPARK-14639) Add `bround` function in Python/R.

2016-04-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-14639: -- Description: This issue aims to expose Scala `bround` function in Python/R API. `bround`

[jira] [Updated] (SPARK-14639) Add `bround` function in Python/R.

2016-04-14 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-14639: -- Description: This issue aims to expose Scala `bround` function in Python/R API. `bround`

[jira] [Assigned] (SPARK-14640) @DeveloperApi on PythonUserDefinedType

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14640: Assignee: (was: Apache Spark) > @DeveloperApi on PythonUserDefinedType >

[jira] [Assigned] (SPARK-14640) @DeveloperApi on PythonUserDefinedType

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14640: Assignee: Apache Spark > @DeveloperApi on PythonUserDefinedType >

[jira] [Commented] (SPARK-14640) @DeveloperApi on PythonUserDefinedType

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241759#comment-15241759 ] Apache Spark commented on SPARK-14640: -- User 'joan38' has created a pull request for this issue:

[jira] [Created] (SPARK-14640) @DeveloperApi on PythonUserDefinedType

2016-04-14 Thread Joan Goyeau (JIRA)
Joan Goyeau created SPARK-14640: --- Summary: @DeveloperApi on PythonUserDefinedType Key: SPARK-14640 URL: https://issues.apache.org/jira/browse/SPARK-14640 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-14639) Add `bround` function in Python/R.

2016-04-14 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-14639: - Summary: Add `bround` function in Python/R. Key: SPARK-14639 URL: https://issues.apache.org/jira/browse/SPARK-14639 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-14352) approxQuantile should support multi columns

2016-04-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241729#comment-15241729 ] Nick Pentreath edited comment on SPARK-14352 at 4/14/16 7:02 PM: - This

[jira] [Commented] (SPARK-14352) approxQuantile should support multi columns

2016-04-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241729#comment-15241729 ] Nick Pentreath commented on SPARK-14352: This duplicates SPARK-14432 - which did come a little

[jira] [Resolved] (SPARK-13967) Add binary toggle Param to PySpark CountVectorizer

2016-04-14 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-13967. Resolution: Fixed Fix Version/s: 2.0.0 > Add binary toggle Param to PySpark

[jira] [Commented] (SPARK-14638) Threads of Spark Streaming (with Kafka) looses sight of the executor classpath

2016-04-14 Thread Younos Aboulnaga (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241714#comment-15241714 ] Younos Aboulnaga commented on SPARK-14638: -- But the same code works when used in a Spark batch

[jira] [Commented] (SPARK-11834) Ignore thresholds in LogisticRegression and update documentation

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241701#comment-15241701 ] Apache Spark commented on SPARK-11834: -- User 'JeremyNixon' has created a pull request for this

[jira] [Assigned] (SPARK-11834) Ignore thresholds in LogisticRegression and update documentation

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11834: Assignee: Xiangrui Meng (was: Apache Spark) > Ignore thresholds in LogisticRegression

[jira] [Assigned] (SPARK-11834) Ignore thresholds in LogisticRegression and update documentation

2016-04-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11834: Assignee: Apache Spark (was: Xiangrui Meng) > Ignore thresholds in LogisticRegression

[jira] [Resolved] (SPARK-14592) Create table like

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14592. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.0.0 Target

[jira] [Resolved] (SPARK-14558) In ClosureCleaner, clean the outer pointer if it's a REPL line object

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14558. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > In ClosureCleaner,

[jira] [Resolved] (SPARK-14617) Remove deprecated APIs in TaskMetrics

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14617. --- Resolution: Fixed > Remove deprecated APIs in TaskMetrics > - >

[jira] [Resolved] (SPARK-14619) Track internal accumulators (metrics) by stage attempt rather than stage

2016-04-14 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-14619. --- Resolution: Fixed > Track internal accumulators (metrics) by stage attempt rather than stage >

[jira] [Resolved] (SPARK-14612) Consolidate the version of dependencies in mllib and mllib-local into one place

2016-04-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-14612. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12390

  1   2   3   >