[jira] [Commented] (SPARK-9447) Python RandomForestClassifier probabilityCol, rawPredictionCol

[ https://issues.apache.org/jira/browse/SPARK-9447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652504#comment-14652504 ] Apache Spark commented on SPARK-9447: - User 'jkbradley' has created a pull request for

[jira] [Updated] (SPARK-9570) Consistent recommendation for spark-submit --master

[ https://issues.apache.org/jira/browse/SPARK-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neelesh Srinivas Salian updated SPARK-9570: --- Summary: Consistent recommendation for spark-submit --master (was:

[jira] [Resolved] (SPARK-9544) RFormula in Python

[ https://issues.apache.org/jira/browse/SPARK-9544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9544. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7879

[jira] [Updated] (SPARK-9570) Consistent recommendation for submitting spark apps to YARN, -master yarn --deploy-mode x vs -master yarn-x'.

[ https://issues.apache.org/jira/browse/SPARK-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neelesh Srinivas Salian updated SPARK-9570: --- Summary: Consistent recommendation for submitting spark apps to YARN, -master

[jira] [Resolved] (SPARK-9191) Add ml.PCA user guide and code examples

[ https://issues.apache.org/jira/browse/SPARK-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-9191. -- Resolution: Fixed Fix Version/s: 1.5.0 Add ml.PCA user guide and code examples

[jira] [Commented] (SPARK-7165) Sort Merge Join for outer joins

[ https://issues.apache.org/jira/browse/SPARK-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652527#comment-14652527 ] Apache Spark commented on SPARK-7165: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-8333) Spark failed to delete temp directory created by HiveContext

[ https://issues.apache.org/jira/browse/SPARK-8333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652531#comment-14652531 ] Sudhakar Thota commented on SPARK-8333: --- Hi Sheng, I have verified in spark

[jira] [Updated] (SPARK-9512) RemoveEvaluationFromSort reorders sort order

[ https://issues.apache.org/jira/browse/SPARK-9512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9512: --- Shepherd: Michael Armbrust RemoveEvaluationFromSort reorders sort order

[jira] [Updated] (SPARK-7550) Support setting the right schema serde when writing to Hive metastore

[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7550: --- Shepherd: liancheng (was: Yin Huai) Support setting the right schema serde when writing to Hive

[jira] [Updated] (SPARK-7550) Support setting the right schema serde when writing to Hive metastore

[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7550: --- Shepherd: Cheng Lian (was: liancheng) Support setting the right schema serde when writing to Hive

[jira] [Resolved] (SPARK-9287) Speedup unit test of Date expressions

[ https://issues.apache.org/jira/browse/SPARK-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9287. --- Resolution: Fixed Assignee: Davies Liu Fix Version/s: 1.5.0 Target

[jira] [Commented] (SPARK-9568) Spark MLlib 1.5.0 testing umbrella

[ https://issues.apache.org/jira/browse/SPARK-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652354#comment-14652354 ] Reynold Xin commented on SPARK-9568: [~srowen] I think QA is a definitely blocker for

[jira] [Updated] (SPARK-9257) Fix the false negative of Aggregate2Sort and FinalAndCompleteAggregate2Sort's missingInput

[ https://issues.apache.org/jira/browse/SPARK-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9257: Priority: Minor (was: Major) Fix the false negative of Aggregate2Sort and

[jira] [Updated] (SPARK-9205) org.apache.spark.sql.hive.HiveSparkSubmitSuite failing for Scala 2.11

[ https://issues.apache.org/jira/browse/SPARK-9205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9205: Summary: org.apache.spark.sql.hive.HiveSparkSubmitSuite failing for Scala 2.11 (was: Test

[jira] [Updated] (SPARK-5027) add SVMWithLBFGS interface in MLLIB

[ https://issues.apache.org/jira/browse/SPARK-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5027: - Shepherd: (was: zhengbing li) add SVMWithLBFGS interface in MLLIB

[jira] [Updated] (SPARK-5564) Support sparse LDA solutions

[ https://issues.apache.org/jira/browse/SPARK-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5564: - Target Version/s: 1.6.0 (was: 1.5.0) Support sparse LDA solutions

[jira] [Updated] (SPARK-5027) add SVMWithLBFGS interface in MLLIB

[ https://issues.apache.org/jira/browse/SPARK-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5027: - Target Version/s: 1.6.0 (was: 1.5.0) add SVMWithLBFGS interface in MLLIB

[jira] [Updated] (SPARK-8884) 1-sample Anderson-Darling Goodness-of-Fit test

[ https://issues.apache.org/jira/browse/SPARK-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8884: - Target Version/s: 1.6.0 (was: 1.5.0) 1-sample Anderson-Darling Goodness-of-Fit test

[jira] [Updated] (SPARK-7653) ML Pipeline and meta-algs should take random seed param

[ https://issues.apache.org/jira/browse/SPARK-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7653: - Target Version/s: 1.6.0 (was: 1.5.0) ML Pipeline and meta-algs should take random seed

[jira] [Commented] (SPARK-9568) Spark MLlib 1.5.0 testing umbrella

[ https://issues.apache.org/jira/browse/SPARK-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652387#comment-14652387 ] Sean Owen commented on SPARK-9568: -- Cool, if it helps more work get done on the important

[jira] [Updated] (SPARK-4980) Add decay factors to streaming linear methods

[ https://issues.apache.org/jira/browse/SPARK-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4980: - Target Version/s: 1.6.0 (was: 1.5.0) Add decay factors to streaming linear methods

[jira] [Updated] (SPARK-7653) ML Pipeline and meta-algs should take random seed param

[ https://issues.apache.org/jira/browse/SPARK-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7653: - Priority: Major (was: Minor) ML Pipeline and meta-algs should take random seed param

[jira] [Updated] (SPARK-8486) SIFT Feature Transformer

[ https://issues.apache.org/jira/browse/SPARK-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8486: - Target Version/s: (was: 1.5.0) SIFT Feature Transformer

[jira] [Commented] (SPARK-9357) Remove JoinedRow

[ https://issues.apache.org/jira/browse/SPARK-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652390#comment-14652390 ] Herman van Hovell commented on SPARK-9357: -- +1 for removing this. The

[jira] [Updated] (SPARK-6312) ChiSqTest should check for too few counts

[ https://issues.apache.org/jira/browse/SPARK-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6312: - Target Version/s: (was: 1.5.0) ChiSqTest should check for too few counts

[jira] [Commented] (SPARK-9483) UTF8String.getPrefix only works in little-endian order

[ https://issues.apache.org/jira/browse/SPARK-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652412#comment-14652412 ] Apache Spark commented on SPARK-9483: - User 'mtbrandy' has created a pull request for

[jira] [Assigned] (SPARK-7736) Exception not failing Python applications (in yarn cluster mode)

[ https://issues.apache.org/jira/browse/SPARK-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7736: --- Assignee: Apache Spark Exception not failing Python applications (in yarn cluster mode)

[jira] [Commented] (SPARK-7736) Exception not failing Python applications (in yarn cluster mode)

[ https://issues.apache.org/jira/browse/SPARK-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652429#comment-14652429 ] Apache Spark commented on SPARK-7736: - User 'vanzin' has created a pull request for

[jira] [Commented] (SPARK-5439) Expose yarn app id for yarn mode

[ https://issues.apache.org/jira/browse/SPARK-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652456#comment-14652456 ] Marcelo Vanzin commented on SPARK-5439: --- There are 3 ways to get the app id: -

[jira] [Resolved] (SPARK-5439) Expose yarn app id for yarn mode

[ https://issues.apache.org/jira/browse/SPARK-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-5439. --- Resolution: Implemented Expose yarn app id for yarn mode

[jira] [Updated] (SPARK-5890) Add FeatureDiscretizer

[ https://issues.apache.org/jira/browse/SPARK-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5890: - Target Version/s: 1.6.0 (was: 1.5.0) Add FeatureDiscretizer --

[jira] [Updated] (SPARK-5890) Add FeatureDiscretizer

[ https://issues.apache.org/jira/browse/SPARK-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5890: - Shepherd: Joseph K. Bradley Assignee: Xusen Yin Add FeatureDiscretizer

[jira] [Updated] (SPARK-8515) Improve ML attribute API

[ https://issues.apache.org/jira/browse/SPARK-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8515: - Target Version/s: 1.6.0 (was: 1.5.0) Improve ML attribute API

[jira] [Updated] (SPARK-8515) Improve ML attribute API

[ https://issues.apache.org/jira/browse/SPARK-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8515: - Description: In 1.4.0, we introduced ML attribute API to embed feature/label attribute

[jira] [Updated] (SPARK-8516) ML attribute API in PySpark

[ https://issues.apache.org/jira/browse/SPARK-8516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8516: - Target Version/s: 1.6.0 (was: 1.5.0) ML attribute API in PySpark

[jira] [Updated] (SPARK-6530) ChiSqSelector transformer

[ https://issues.apache.org/jira/browse/SPARK-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6530: - Shepherd: Joseph K. Bradley Assignee: Xusen Yin Target Version/s:

[jira] [Resolved] (SPARK-8131) Improve Database support

[ https://issues.apache.org/jira/browse/SPARK-8131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8131. - Resolution: Fixed Fix Version/s: 1.5.0 Improve Database support

[jira] [Updated] (SPARK-7492) Convert LocalDataFrame to LocalMatrix

[ https://issues.apache.org/jira/browse/SPARK-7492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-7492: Target Version/s: 1.6.0 (was: 1.5.0) Convert LocalDataFrame to LocalMatrix

[jira] [Commented] (SPARK-9568) Spark MLlib 1.5.0 testing umbrella

[ https://issues.apache.org/jira/browse/SPARK-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652365#comment-14652365 ] Joseph K. Bradley commented on SPARK-9568: -- [~srowen] I think these

[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-1359: - Target Version/s: (was: 1.5.0) SGD implementation is not efficient

[jira] [Resolved] (SPARK-9558) Update docs to follow the increase of memory defaults.

[ https://issues.apache.org/jira/browse/SPARK-9558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9558. Resolution: Fixed Assignee: Kousuke Saruta Fix Version/s: 1.5.0 Update docs to

[jira] [Updated] (SPARK-9548) BytesToBytesMap could have a destructive iterator

[ https://issues.apache.org/jira/browse/SPARK-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9548: Issue Type: Improvement (was: Bug) BytesToBytesMap could have a destructive iterator

[jira] [Commented] (SPARK-8459) Add import/export to spark.mllib Hierarchical Clustering

[ https://issues.apache.org/jira/browse/SPARK-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652375#comment-14652375 ] Joseph K. Bradley commented on SPARK-8459: -- We can set the target version once

[jira] [Updated] (SPARK-8459) Add import/export to spark.mllib Hierarchical Clustering

[ https://issues.apache.org/jira/browse/SPARK-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8459: - Target Version/s: (was: 1.5.0) Add import/export to spark.mllib Hierarchical

[jira] [Updated] (SPARK-9486) Add aliasing to data sources to allow external packages to register themselves with Spark

[ https://issues.apache.org/jira/browse/SPARK-9486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9486: Target Version/s: 1.5.0 Add aliasing to data sources to allow external packages to

[jira] [Commented] (SPARK-8884) 1-sample Anderson-Darling Goodness-of-Fit test

[ https://issues.apache.org/jira/browse/SPARK-8884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652383#comment-14652383 ] Joseph K. Bradley commented on SPARK-8884: -- Modifying target to 1.6; please say

[jira] [Updated] (SPARK-8546) PMML export for Naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-8546: - Target Version/s: 1.6.0 (was: 1.5.0) PMML export for Naive Bayes

[jira] [Updated] (SPARK-6160) ChiSqSelector should keep test statistic info

[ https://issues.apache.org/jira/browse/SPARK-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6160: - Target Version/s: 1.6.0 (was: 1.5.0) ChiSqSelector should keep test statistic info

[jira] [Updated] (SPARK-6634) Allow replacing columns in Transformers

[ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6634: - Target Version/s: 1.6.0 (was: 1.5.0) Allow replacing columns in Transformers

[jira] [Updated] (SPARK-7316) Add step capability to RDD sliding window

[ https://issues.apache.org/jira/browse/SPARK-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7316: - Assignee: Alexander Ulanov Add step capability to RDD sliding window

[jira] [Updated] (SPARK-7316) Add step capability to RDD sliding window

[ https://issues.apache.org/jira/browse/SPARK-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7316: - Target Version/s: 1.6.0 (was: 1.5.0) Add step capability to RDD sliding window

[jira] [Updated] (SPARK-5114) Should Evaluator be a PipelineStage

[ https://issues.apache.org/jira/browse/SPARK-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5114: - Target Version/s: 1.6.0 (was: 1.5.0) Should Evaluator be a PipelineStage

[jira] [Updated] (SPARK-7130) spark.ml RandomForest* should always do bootstrapping

[ https://issues.apache.org/jira/browse/SPARK-7130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7130: - Assignee: (was: Joseph K. Bradley) spark.ml RandomForest* should always do

[jira] [Updated] (SPARK-7130) spark.ml RandomForest* should always do bootstrapping

[ https://issues.apache.org/jira/browse/SPARK-7130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7130: - Target Version/s: 1.6.0 (was: 1.5.0) spark.ml RandomForest* should always do

[jira] [Reopened] (SPARK-8160) Tungsten style external aggregation

[ https://issues.apache.org/jira/browse/SPARK-8160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reopened SPARK-8160: --- Actually, going to re-open since I think we're waiting on a code-review followup patch. Tungsten style

[jira] [Assigned] (SPARK-7736) Exception not failing Python applications (in yarn cluster mode)

[ https://issues.apache.org/jira/browse/SPARK-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7736: --- Assignee: (was: Apache Spark) Exception not failing Python applications (in yarn

[jira] [Commented] (SPARK-6689) MiniYarnCLuster still test failed with hadoop-2.2

[ https://issues.apache.org/jira/browse/SPARK-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652437#comment-14652437 ] Marcelo Vanzin commented on SPARK-6689: --- The two errors are completely unrelated.

[jira] [Commented] (SPARK-6508) error handling issue running python in yarn cluster mode

[ https://issues.apache.org/jira/browse/SPARK-6508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652443#comment-14652443 ] Marcelo Vanzin commented on SPARK-6508: --- I believe both errors here will be fixed by

[jira] [Updated] (SPARK-7290) Add StringVectorizer

[ https://issues.apache.org/jira/browse/SPARK-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7290: - Target Version/s: 1.6.0 (was: 1.5.0) Add StringVectorizer

[jira] [Commented] (SPARK-7290) Add StringVectorizer

[ https://issues.apache.org/jira/browse/SPARK-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652465#comment-14652465 ] Joseph K. Bradley commented on SPARK-7290: -- We'll need to retarget this to 1.6

[jira] [Resolved] (SPARK-4696) Yarn: spark.driver.extra* variables not applied consistently to yarn client mode AM

[ https://issues.apache.org/jira/browse/SPARK-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-4696. --- Resolution: Implemented We have {{spark.yarn.am.extraJavaOptions}} and

[jira] [Commented] (SPARK-9357) Remove JoinedRow

[ https://issues.apache.org/jira/browse/SPARK-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652467#comment-14652467 ] Reynold Xin commented on SPARK-9357: [~hvanhovell] would be great to have a pr!

[jira] [Updated] (SPARK-7736) Exception not failing Python applications (in yarn cluster mode)

[ https://issues.apache.org/jira/browse/SPARK-7736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-7736: -- Target Version/s: 1.5.0 Exception not failing Python applications (in yarn cluster mode)

<    1   2   3   4   5