[jira] [Assigned] (SPARK-9747) Avoid starving an unsafe operator in an aggregate

2015-08-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9747: --- Assignee: Apache Spark (was: Andrew Or) > Avoid starving an unsafe operator in an aggregate

[jira] [Commented] (SPARK-9747) Avoid starving an unsafe operator in an aggregate

2015-08-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662413#comment-14662413 ] Apache Spark commented on SPARK-9747: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662410#comment-14662410 ] Feynman Liang commented on SPARK-9660: -- [Word2VecModel aux constructor|https://githu

[jira] [Resolved] (SPARK-9733) Improve explain message for data source scan node

2015-08-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9733. Resolution: Fixed Fix Version/s: 1.5.0 > Improve explain message for data source scan node >

[jira] [Comment Edited] (SPARK-9720) spark.ml Identifiable types should have UID in toString methods

2015-08-07 Thread Bertrand Dechoux (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662333#comment-14662333 ] Bertrand Dechoux edited comment on SPARK-9720 at 8/7/15 8:41 PM: ---

[jira] [Commented] (SPARK-9738) remove FromUnsafe and add its codegen version to GenerateSafe

2015-08-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662403#comment-14662403 ] Josh Rosen commented on SPARK-9738: --- [~davies], [~cloud_fan], [~rxin], should this JIRA

[jira] [Commented] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662398#comment-14662398 ] Sean Owen commented on SPARK-9746: -- No, that's wrong. The *input* are arbitrary types K a

[jira] [Commented] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Andreas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662390#comment-14662390 ] Andreas commented on SPARK-9746: Btw the value cannot be a "ham sandwich": according to me

[jira] [Commented] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662383#comment-14662383 ] Sean Owen commented on SPARK-9746: -- Yes. That is what it means to count the # of occurren

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662382#comment-14662382 ] Feynman Liang commented on SPARK-9660: -- {{GaussianMixtureModel.predict(Vector)}} has

[jira] [Commented] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Andreas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662381#comment-14662381 ] Andreas commented on SPARK-9746: Sorry, but it doesn't count anything, the values of the r

[jira] [Comment Edited] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662382#comment-14662382 ] Feynman Liang edited comment on SPARK-9660 at 8/7/15 8:30 PM: --

[jira] [Comment Edited] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662374#comment-14662374 ] Feynman Liang edited comment on SPARK-9660 at 8/7/15 8:26 PM: --

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662378#comment-14662378 ] Feynman Liang commented on SPARK-9660: -- [~josephkb] Don't users have to provide {{thr

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662374#comment-14662374 ] Feynman Liang commented on SPARK-9660: -- {{SVMModel}} missing short descriptions for {

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662367#comment-14662367 ] Feynman Liang commented on SPARK-9660: -- {{LogisticRegressionModel.toString()}} missin

[jira] [Comment Edited] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662364#comment-14662364 ] Feynman Liang edited comment on SPARK-9660 at 8/7/15 8:16 PM: --

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662364#comment-14662364 ] Feynman Liang commented on SPARK-9660: -- {{LogisticRegressionModel$.load}} missing sho

[jira] [Commented] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Andreas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662360#comment-14662360 ] Andreas commented on SPARK-9746: May be I'm to dumb: but the count for each key is always

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662356#comment-14662356 ] Joseph K. Bradley commented on SPARK-9660: -- Sure, sounds good. (same as for DTCl

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662358#comment-14662358 ] Joseph K. Bradley commented on SPARK-9660: -- I want to add it as public for all Pr

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662354#comment-14662354 ] Feynman Liang commented on SPARK-9660: -- {{CrossValidatorModel}}'s {{avgMetrics}} and

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662355#comment-14662355 ] Joseph K. Bradley commented on SPARK-9660: -- I believe we're adding the "threshold

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662352#comment-14662352 ] Joseph K. Bradley commented on SPARK-9660: -- I do want it public at some point, bu

[jira] [Closed] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-9746. > PairRDDFunctions.countByKey: values/counts always 1 > --- > >

[jira] [Resolved] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9746. -- Resolution: Not A Problem Do not reopen an issue unless it's clear that the original reason it was reso

[jira] [Commented] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662348#comment-14662348 ] Sean Owen commented on SPARK-9746: -- It counts the number of entries whose key is K, for a

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662347#comment-14662347 ] Feynman Liang commented on SPARK-9660: -- In Decision Tree models, {{numFeatures}} is o

[jira] [Comment Edited] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Andreas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662341#comment-14662341 ] Andreas edited comment on SPARK-9746 at 8/7/15 8:04 PM: Sorry, but

[jira] [Comment Edited] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662317#comment-14662317 ] Feynman Liang edited comment on SPARK-9660 at 8/7/15 8:04 PM: --

[jira] [Comment Edited] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662317#comment-14662317 ] Feynman Liang edited comment on SPARK-9660 at 8/7/15 8:04 PM: --

[jira] [Reopened] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Andreas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas reopened SPARK-9746: Sorry, but I don't agree. cntxt..parallelize(List (("a", 1), ("a", 2))).groupBy(_._1).countByKey() returns 'Ma

[jira] [Comment Edited] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662317#comment-14662317 ] Feynman Liang edited comment on SPARK-9660 at 8/7/15 7:58 PM: --

[jira] [Comment Edited] (SPARK-9720) spark.ml Identifiable types should have UID in toString methods

2015-08-07 Thread Bertrand Dechoux (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662333#comment-14662333 ] Bertrand Dechoux edited comment on SPARK-9720 at 8/7/15 7:59 PM: ---

[jira] [Commented] (SPARK-9720) spark.ml Identifiable types should have UID in toString methods

2015-08-07 Thread Bertrand Dechoux (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662333#comment-14662333 ] Bertrand Dechoux commented on SPARK-9720: - I could take care of it. Here is the l

[jira] [Commented] (SPARK-9678) HTTP request to BlockManager port yields exception

2015-08-07 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662321#comment-14662321 ] Aseem Bansal commented on SPARK-9678: - I understand. Just thought to mention that. >

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662317#comment-14662317 ] Feynman Liang commented on SPARK-9660: -- Should [RandomForestClassificationModel's aux

[jira] [Comment Edited] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662303#comment-14662303 ] Feynman Liang edited comment on SPARK-9660 at 8/7/15 7:23 PM: --

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662303#comment-14662303 ] Feynman Liang commented on SPARK-9660: -- Logistic regression [only supports binary cl

[jira] [Comment Edited] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662296#comment-14662296 ] Jerry Z edited comment on SPARK-9744 at 8/7/15 7:20 PM: Fixed! Sor

[jira] [Comment Edited] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662296#comment-14662296 ] Jerry Z edited comment on SPARK-9744 at 8/7/15 7:19 PM: Fixed! Sor

[jira] [Issue Comment Deleted] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Z updated SPARK-9744: --- Comment: was deleted (was: Fixed! Sorry didn't know you were referring to the title. So I think for performan

[jira] [Commented] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662297#comment-14662297 ] Jerry Z commented on SPARK-9744: Fixed! Sorry didn't know you were referring to the title.

[jira] [Issue Comment Deleted] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Z updated SPARK-9744: --- Comment: was deleted (was: Fixed! Sorry didn't know you were referring to the title. So I think for performan

[jira] [Commented] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662296#comment-14662296 ] Jerry Z commented on SPARK-9744: Fixed! Sorry didn't know you were referring to the title.

[jira] [Commented] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662299#comment-14662299 ] Jerry Z commented on SPARK-9744: Fixed! Sorry didn't know you were referring to the title.

[jira] [Issue Comment Deleted] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Z updated SPARK-9744: --- Comment: was deleted (was: Fixed! Sorry didn't know you were referring to the title. So I think for performan

[jira] [Issue Comment Deleted] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Z updated SPARK-9744: --- Comment: was deleted (was: Fixed! Sorry didn't know you were referring to the title. So I think for performan

[jira] [Commented] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662301#comment-14662301 ] Jerry Z commented on SPARK-9744: Fixed! Sorry didn't know you were referring to the title.

[jira] [Commented] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662300#comment-14662300 ] Jerry Z commented on SPARK-9744: Fixed! Sorry didn't know you were referring to the title.

[jira] [Commented] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662295#comment-14662295 ] Jerry Z commented on SPARK-9744: Fixed! Sorry didn't know you were referring to the title.

[jira] [Issue Comment Deleted] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Z updated SPARK-9744: --- Comment: was deleted (was: Fixed! Sorry didn't know you were referring to the title. So I think for performan

[jira] [Issue Comment Deleted] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Z updated SPARK-9744: --- Comment: was deleted (was: Fixed! Sorry didn't know you were referring to the title. So I think for performan

[jira] [Commented] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662298#comment-14662298 ] Jerry Z commented on SPARK-9744: Fixed! Sorry didn't know you were referring to the title.

[jira] [Updated] (SPARK-9744) Add RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Z updated SPARK-9744: --- Summary: Add RDD method to map with lag and lead (was: Add Java RDD method to map with lag and lead) > Add R

[jira] [Commented] (SPARK-9744) Add Java RDD method to map with lag and lead

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662275#comment-14662275 ] Sean Owen commented on SPARK-9744: -- The title of the JIRA is "Add Java RDD method..." but

[jira] [Resolved] (SPARK-5111) HiveContext and Thriftserver cannot work in secure cluster beyond hadoop2.5

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-5111. --- Resolution: Done Fix Version/s: 1.5.0 > HiveContext and Thriftserver cannot work in sec

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662270#comment-14662270 ] Feynman Liang commented on SPARK-9660: -- {{DecisionTreeClassificationModel.rawPredicti

[jira] [Assigned] (SPARK-9748) Centriod typo in KMeansModel

2015-08-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9748: --- Assignee: Apache Spark > Centriod typo in KMeansModel > > >

[jira] [Issue Comment Deleted] (SPARK-9748) Centriod typo in KMeansModel

2015-08-07 Thread Bertrand Dechoux (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertrand Dechoux updated SPARK-9748: Comment: was deleted (was: Pull request done : https://github.com/apache/spark/pull/8037) >

[jira] [Assigned] (SPARK-9748) Centriod typo in KMeansModel

2015-08-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9748: --- Assignee: (was: Apache Spark) > Centriod typo in KMeansModel > --

[jira] [Commented] (SPARK-9744) Add Java RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662267#comment-14662267 ] Jerry Z commented on SPARK-9744: Not sure what you mean by "Why just the Java RDD?". Also,

[jira] [Commented] (SPARK-9748) Centriod typo in KMeansModel

2015-08-07 Thread Bertrand Dechoux (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662269#comment-14662269 ] Bertrand Dechoux commented on SPARK-9748: - Pull request done : https://github.com/

[jira] [Commented] (SPARK-9748) Centriod typo in KMeansModel

2015-08-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662268#comment-14662268 ] Apache Spark commented on SPARK-9748: - User 'BertrandDechoux' has created a pull reque

[jira] [Created] (SPARK-9748) Centriod typo in KMeansModel

2015-08-07 Thread Bertrand Dechoux (JIRA)
Bertrand Dechoux created SPARK-9748: --- Summary: Centriod typo in KMeansModel Key: SPARK-9748 URL: https://issues.apache.org/jira/browse/SPARK-9748 Project: Spark Issue Type: Task C

[jira] [Commented] (SPARK-9742) NullPointerException when using --packages

2015-08-07 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662260#comment-14662260 ] Burak Yavuz commented on SPARK-9742: Did the behavior of Option's change for some reas

[jira] [Commented] (SPARK-9660) ML 1.5 QA: API: New Scala APIs, docs

2015-08-07 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662257#comment-14662257 ] Feynman Liang commented on SPARK-9660: -- [~josephkb] Should [DecisionTreeClassificatio

[jira] [Updated] (SPARK-9720) spark.ml Identifiable types should have UID in toString methods

2015-08-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9720: - Description: It would be nice to print the UID (instance name) in toString methods. That'

[jira] [Commented] (SPARK-9720) spark.ml Identifiable types should have UID in toString methods

2015-08-07 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662253#comment-14662253 ] Joseph K. Bradley commented on SPARK-9720: -- Good point about the default toString

[jira] [Commented] (SPARK-9720) spark.ml Identifiable types should have UID in toString methods

2015-08-07 Thread Bertrand Dechoux (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662248#comment-14662248 ] Bertrand Dechoux commented on SPARK-9720: - I might not understand but isn't it alr

[jira] [Commented] (SPARK-9744) Add Java RDD method to map with lag and lead

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662233#comment-14662233 ] Sean Owen commented on SPARK-9744: -- Yeah that's the idea. It works though is potentially

[jira] [Created] (SPARK-9747) Avoid starving an unsafe operator in an aggregate

2015-08-07 Thread Andrew Or (JIRA)
Andrew Or created SPARK-9747: Summary: Avoid starving an unsafe operator in an aggregate Key: SPARK-9747 URL: https://issues.apache.org/jira/browse/SPARK-9747 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-9693) Reserve a page in all unsafe operators to avoid starving an operator

2015-08-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9693: - Priority: Blocker (was: Critical) > Reserve a page in all unsafe operators to avoid starving an operator

[jira] [Updated] (SPARK-9709) Avoid starving an unsafe operator in a sort

2015-08-07 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9709: - Priority: Blocker (was: Critical) > Avoid starving an unsafe operator in a sort > ---

[jira] [Resolved] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-9746. -- Resolution: Not A Problem This should start as a question on user@. https://cwiki.apache.org/confluence

[jira] [Commented] (SPARK-5111) HiveContext and Thriftserver cannot work in secure cluster beyond hadoop2.5

2015-08-07 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662227#comment-14662227 ] Zhan Zhang commented on SPARK-5111: --- Since hive upgrade is done. This jira is not valid

[jira] [Resolved] (SPARK-8382) Improve Analysis Unit test framework

2015-08-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8382. - Resolution: Fixed Fix Version/s: 1.5.0 > Improve Analysis Unit test framework > ---

[jira] [Updated] (SPARK-8382) Improve Analysis Unit test framework

2015-08-07 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8382: Assignee: Wenchen Fan > Improve Analysis Unit test framework > -

[jira] [Resolved] (SPARK-9736) JoinedRow.anyNull should delegate to the underlying rows

2015-08-07 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-9736. - Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8027 [https://github.com/apac

[jira] [Commented] (SPARK-7789) sql on security hbase:Token generation only allowed for Kerberos authenticated clients

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662221#comment-14662221 ] Steve Loughran commented on SPARK-7789: --- With SPARK-8064, Hive 1.2.1 is the hive lib

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-9745: -- Priority: Critical (was: Major) > Applications hangs when the last executor fails with dynamic allocati

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-9745: -- Assignee: Andrew Or > Applications hangs when the last executor fails with dynamic allocation >

[jira] [Updated] (SPARK-7252) Add support for creating new Hive and HBase delegation tokens

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-7252: -- Affects Version/s: (was: 1.3.1) 1.5.0 > Add support for creating new

[jira] [Commented] (SPARK-9019) spark-submit fails on yarn with kerberos enabled

2015-08-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662211#comment-14662211 ] Steve Loughran commented on SPARK-9019: --- # If this problem exists (I don't have test

[jira] [Created] (SPARK-9746) PairRDDFunctions.countByKey: values/counts always 1

2015-08-07 Thread Andreas (JIRA)
Andreas created SPARK-9746: -- Summary: PairRDDFunctions.countByKey: values/counts always 1 Key: SPARK-9746 URL: https://issues.apache.org/jira/browse/SPARK-9746 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-9667) Remove SparkSqlSerializer2 in favor of Unsafe exchange

2015-08-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662193#comment-14662193 ] Apache Spark commented on SPARK-9667: - User 'rxin' has created a pull request for this

[jira] [Comment Edited] (SPARK-9744) Add Java RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662188#comment-14662188 ] Jerry Z edited comment on SPARK-9744 at 8/7/15 6:14 PM: Let me kno

[jira] [Commented] (SPARK-9744) Add Java RDD method to map with lag and lead

2015-08-07 Thread Jerry Z (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662188#comment-14662188 ] Jerry Z commented on SPARK-9744: Let me know if I'm missing/doing more work than necessary

[jira] [Commented] (SPARK-6392) [SQL]class not found exception thows when `add jar` use spark cli

2015-08-07 Thread Jonathan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662189#comment-14662189 ] Jonathan Kelly commented on SPARK-6392: --- Are you sure this is fixed by https://githu

[jira] [Resolved] (SPARK-9667) Remove SparkSqlSerializer2 in favor of Unsafe exchange

2015-08-07 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9667. Resolution: Fixed Fix Version/s: 1.5.0 > Remove SparkSqlSerializer2 in favor of Unsafe exchan

[jira] [Updated] (SPARK-9744) Add Java RDD method to map with lag and lead

2015-08-07 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9744: - Priority: Minor (was: Major) This sounds a bit like a windowing function. This is something you can do wi

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Alex Angelini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Angelini updated SPARK-9745: - Description: When a job has only a single executor remaining and that executor dies (due to somet

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Alex Angelini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Angelini updated SPARK-9745: - Description: When a job has only a single executor remaining and that executor dies (due to somet

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Alex Angelini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Angelini updated SPARK-9745: - Description: When a job has only a single executor remaining and that executor dies (due to somet

[jira] [Updated] (SPARK-9600) DataFrameWriter.saveAsTable always writes data to "/user/hive/warehouse"

2015-08-07 Thread Sudhakar Thota (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudhakar Thota updated SPARK-9600: -- Attachment: SPARK-9600-fl1.txt my screen content > DataFrameWriter.saveAsTable always writes da

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Alex Angelini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Angelini updated SPARK-9745: - Description: When a job has only a single executor remaining and that executor dies (due to somet

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Alex Angelini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Angelini updated SPARK-9745: - Description: When a job has only a single executor remaining and that executor dies (due to somet

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Alex Angelini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Angelini updated SPARK-9745: - Description: When a job has only a single executor remaining and that executor dies (due to somet

[jira] [Updated] (SPARK-9745) Applications hangs when the last executor fails with dynamic allocation

2015-08-07 Thread Alex Angelini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Angelini updated SPARK-9745: - Attachment: tasks_hung_job.png logs_hung_job.png executors_hung_job

[jira] [Commented] (SPARK-9600) DataFrameWriter.saveAsTable always writes data to "/user/hive/warehouse"

2015-08-07 Thread Sudhakar Thota (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662125#comment-14662125 ] Sudhakar Thota commented on SPARK-9600: --- Hi, I tested this on 1.4.1 and 1.5.0 both s

<    1   2   3   >