[jira] [Updated] (SPARK-7141) saveAsTextFile() on S3 first creates empty prefix

2015-04-24 Thread Eric O. LEBIGOT (EOL) (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric O. LEBIGOT (EOL) updated SPARK-7141: - Description: Using {{saveAsTextFile("s3://bucket/prefix"}} actually adds an empty

[jira] [Updated] (SPARK-7141) saveAsTextFile() on S3 first creates empty prefix

2015-04-24 Thread Eric O. LEBIGOT (EOL) (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric O. LEBIGOT (EOL) updated SPARK-7141: - Description: Using {{saveAsTextFile("s3://bucket/prefix"}} actually adds an empty

[jira] [Updated] (SPARK-7141) saveAsTextFile() on S3 first creates empty prefix

2015-04-24 Thread Eric O. LEBIGOT (EOL) (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric O. LEBIGOT (EOL) updated SPARK-7141: - Description: Using {{saveAsTextFile("s3://bucket/prefix"}} actually adds an empty

[jira] [Created] (SPARK-7141) saveAsTextFile() on S3 first creates empty prefix

2015-04-24 Thread Eric O. LEBIGOT (EOL) (JIRA)
Eric O. LEBIGOT (EOL) created SPARK-7141: Summary: saveAsTextFile() on S3 first creates empty prefix Key: SPARK-7141 URL: https://issues.apache.org/jira/browse/SPARK-7141 Project: Spark

[jira] [Comment Edited] (SPARK-7108) Setting spark.local.dir in driver no longer overrides the standalone worker's local directory setting

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512280#comment-14512280 ] Patrick Wendell edited comment on SPARK-7108 at 4/25/15 6:02 AM: ---

[jira] [Comment Edited] (SPARK-7108) Setting spark.local.dir in driver no longer overrides the standalone worker's local directory setting

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512280#comment-14512280 ] Patrick Wendell edited comment on SPARK-7108 at 4/25/15 6:01 AM: ---

[jira] [Commented] (SPARK-7108) Setting spark.local.dir in driver no longer overrides the standalone worker's local directory setting

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512280#comment-14512280 ] Patrick Wendell commented on SPARK-7108: Hey I think [~joshrosen] actually miswrot

[jira] [Updated] (SPARK-7123) support table.star in sqlcontext

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7123: --- Issue Type: New Feature (was: Bug) > support table.star in sqlcontext > -

[jira] [Commented] (SPARK-6961) Cannot save data to parquet files when executing from Windows from a Maven Project

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512272#comment-14512272 ] Patrick Wendell commented on SPARK-6961: Hey [~bogdannb] - so Spark actually uses

[jira] [Assigned] (SPARK-7140) Do not scan all values in Vector.hashCode

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7140: --- Assignee: Apache Spark (was: Xiangrui Meng) > Do not scan all values in Vector.hashCode > --

[jira] [Assigned] (SPARK-7140) Do not scan all values in Vector.hashCode

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7140: --- Assignee: Xiangrui Meng (was: Apache Spark) > Do not scan all values in Vector.hashCode > --

[jira] [Commented] (SPARK-7140) Do not scan all values in Vector.hashCode

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512269#comment-14512269 ] Apache Spark commented on SPARK-7140: - User 'mengxr' has created a pull request for th

[jira] [Updated] (SPARK-7140) Do not scan all values in Vector.hashCode

2015-04-24 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7140: - Target Version/s: 1.3.2, 1.4.0 > Do not scan all values in Vector.hashCode > -

[jira] [Created] (SPARK-7140) Do not scan all values in Vector.hashCode

2015-04-24 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-7140: Summary: Do not scan all values in Vector.hashCode Key: SPARK-7140 URL: https://issues.apache.org/jira/browse/SPARK-7140 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-5529) BlockManager heartbeat expiration does not kill executor

2015-04-24 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512244#comment-14512244 ] Hong Shen commented on SPARK-5529: -- 1.4.0 version would be release in june. > BlockManag

[jira] [Commented] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-24 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512238#comment-14512238 ] Guoqiang Li commented on SPARK-7008: In practice, relative to the {{LBFGS}} ,{{SGD +Ad

[jira] [Comment Edited] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-24 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14504435#comment-14504435 ] Guoqiang Li edited comment on SPARK-7008 at 4/25/15 4:16 AM: -

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with "Launch More Like This"

2015-04-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512233#comment-14512233 ] Nicholas Chammas commented on SPARK-3213: - Thanks for the background, Joseph and X

[jira] [Commented] (SPARK-5895) Add VectorSlicer

2015-04-24 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512229#comment-14512229 ] Xusen Yin commented on SPARK-5895: -- Sure, thanks. > Add VectorSlicer >

[jira] [Closed] (SPARK-7095) Pass DataType to source.Filter classes

2015-04-24 Thread Alex Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu closed SPARK-7095. --- Resolution: Not A Problem Spark SQL only pushes filter to datasource if the value is the same data type as th

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with "Launch More Like This"

2015-04-24 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512215#comment-14512215 ] Xiangrui Meng commented on SPARK-3213: -- We reverted this change in (https://github.co

[jira] [Resolved] (SPARK-7136) Spark SQL and DataFrame Guide - missing file paths and non-existent example file

2015-04-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7136. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Deborah Siegel > Spark SQL and Data

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512209#comment-14512209 ] Bryan Cutler commented on SPARK-6980: - Thanks for the clarification [~imranr], that ma

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with "Launch More Like This"

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512208#comment-14512208 ] Joseph K. Bradley commented on SPARK-3213: -- When I made this issue, it was becaus

[jira] [Commented] (SPARK-3213) spark_ec2.py cannot find slave instances launched with "Launch More Like This"

2015-04-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512188#comment-14512188 ] Nicholas Chammas commented on SPARK-3213: - Hey people, is the main motivation for

[jira] [Updated] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

2015-04-24 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-4066: -- Description: Here is the thread Koert started: http://search-hadoop.com/m/JW1q5j8z422/scalastyle+annoys+me+a+li

[jira] [Updated] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-24 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-7008: Description: An implementation of Factorization Machines based on Scala and Spark MLlib. FM is a kin

[jira] [Comment Edited] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512148#comment-14512148 ] Imran Rashid edited comment on SPARK-6980 at 4/25/15 1:20 AM: --

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512148#comment-14512148 ] Imran Rashid commented on SPARK-6980: - Hi [~bryanc] [~harshg], sorry I didn't notice

[jira] [Comment Edited] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-24 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512110#comment-14512110 ] zhengruifeng edited comment on SPARK-7008 at 4/25/15 12:46 AM: -

[jira] [Comment Edited] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-24 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512110#comment-14512110 ] zhengruifeng edited comment on SPARK-7008 at 4/25/15 12:44 AM: -

[jira] [Commented] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-24 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512110#comment-14512110 ] zhengruifeng commented on SPARK-7008: - The convergence curves of Binary Classification

[jira] [Updated] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2015-04-24 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-7008: Attachment: FM_CR.xlsx > An implementation of Factorization Machine (LibFM) > --

[jira] [Issue Comment Deleted] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-6980: Comment: was deleted (was: Modified ActorWordCount example to produce akka timeout) > Akka timeout

[jira] [Updated] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-6980: Attachment: Spark-6980-Test.scala Modified ActorWordCount example to produce akka timeout > Akka ti

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512069#comment-14512069 ] Bryan Cutler commented on SPARK-6980: - I'm working out of trunk. Changing the ActorWo

[jira] [Assigned] (SPARK-3090) Avoid not stopping SparkContext with YARN Client mode

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3090: --- Assignee: (was: Apache Spark) > Avoid not stopping SparkContext with YARN Client mode >

[jira] [Commented] (SPARK-3090) Avoid not stopping SparkContext with YARN Client mode

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512048#comment-14512048 ] Apache Spark commented on SPARK-3090: - User 'vanzin' has created a pull request for th

[jira] [Assigned] (SPARK-3090) Avoid not stopping SparkContext with YARN Client mode

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3090: --- Assignee: Apache Spark > Avoid not stopping SparkContext with YARN Client mode > ---

[jira] [Closed] (SPARK-7134) Add regParam and featureScaling options to Logistic regression 'train' methods

2015-04-24 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng closed SPARK-7134. Resolution: Won't Fix I'm closing in favor of SPARK-6682. > Add regParam and featureScaling options

[jira] [Resolved] (SPARK-1457) Change APIs for training algorithms to take optimizer as parameter

2015-04-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1457. -- Resolution: Won't Fix I think this is not a bad idea but think this might have timed out, and will be s

[jira] [Commented] (SPARK-7138) Add method to BlockGenerator to add multiple records to BlockGenerator with single callback

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511902#comment-14511902 ] Apache Spark commented on SPARK-7138: - User 'tdas' has created a pull request for this

[jira] [Updated] (SPARK-7138) Add method to BlockGenerator to add multiple records to BlockGenerator with single callback

2015-04-24 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7138: - Component/s: Streaming > Add method to BlockGenerator to add multiple records to BlockGenerator wi

[jira] [Updated] (SPARK-5895) Add VectorSlicer

2015-04-24 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5895: - Assignee: Xusen Yin > Add VectorSlicer > > > Key: SPARK-5895 >

[jira] [Created] (SPARK-7139) Allow received block metadata to be saved to WAL and recovered on driver failure

2015-04-24 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-7139: Summary: Allow received block metadata to be saved to WAL and recovered on driver failure Key: SPARK-7139 URL: https://issues.apache.org/jira/browse/SPARK-7139 Projec

[jira] [Assigned] (SPARK-6214) Allow configuration options to use a simple expression language

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6214: --- Assignee: (was: Apache Spark) > Allow configuration options to use a simple expression la

[jira] [Assigned] (SPARK-6214) Allow configuration options to use a simple expression language

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6214: --- Assignee: Apache Spark > Allow configuration options to use a simple expression language > --

[jira] [Updated] (SPARK-6599) Improve reliability and usability of Kinesis-based Spark Streaming

2015-04-24 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6599: - Description: Currently, the KinesisReceiver can loose some data in the case of certain failures (

[jira] [Updated] (SPARK-6599) Improve reliability and usability of Kinesis-based Spark Streaming

2015-04-24 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-6599: - Description: Currently, the KinesisReceiver can loose some data in the case of certain failures (

[jira] [Resolved] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-04-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6122. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5354 [https://github.com/ap

[jira] [Updated] (SPARK-7138) Add method to BlockGenerator to add multiple records to BlockGenerator with single callback

2015-04-24 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7138: - Issue Type: Improvement (was: Bug) > Add method to BlockGenerator to add multiple records to Bloc

[jira] [Created] (SPARK-7138) Add method to BlockGenerator to add multiple records to BlockGenerator with single callback

2015-04-24 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-7138: Summary: Add method to BlockGenerator to add multiple records to BlockGenerator with single callback Key: SPARK-7138 URL: https://issues.apache.org/jira/browse/SPARK-7138

[jira] [Updated] (SPARK-7138) Add method to BlockGenerator to add multiple records to BlockGenerator with single callback

2015-04-24 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7138: - Description: This is to ensure that receivers that receive data in small batches (like Kinesis) a

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511844#comment-14511844 ] Bryan Cutler commented on SPARK-7127: - Sounds good, thank you :D > Broadcast spark.ml

[jira] [Commented] (SPARK-5895) Add VectorSlicer

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511820#comment-14511820 ] Joseph K. Bradley commented on SPARK-5895: -- Based on the updated [https://cwiki.

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511815#comment-14511815 ] Joseph K. Bradley commented on SPARK-7127: -- [~bryanc] Thanks for your interest.

[jira] [Assigned] (SPARK-7017) Refactor dev/run-tests into Python

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7017: --- Assignee: Brennon York (was: Apache Spark) > Refactor dev/run-tests into Python > --

[jira] [Commented] (SPARK-7017) Refactor dev/run-tests into Python

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511811#comment-14511811 ] Apache Spark commented on SPARK-7017: - User 'brennonyork' has created a pull request f

[jira] [Assigned] (SPARK-7017) Refactor dev/run-tests into Python

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7017: --- Assignee: Apache Spark (was: Brennon York) > Refactor dev/run-tests into Python > --

[jira] [Resolved] (SPARK-6290) spark.ml.param.Params.checkInputColumn bug upon error

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-6290. -- Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Xiangrui Meng

[jira] [Created] (SPARK-7137) Add checkInputColumn back to Params and print more info

2015-04-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7137: Summary: Add checkInputColumn back to Params and print more info Key: SPARK-7137 URL: https://issues.apache.org/jira/browse/SPARK-7137 Project: Spark

[jira] [Commented] (SPARK-6290) spark.ml.param.Params.checkInputColumn bug upon error

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511786#comment-14511786 ] Joseph K. Bradley commented on SPARK-6290: -- Thanks! I should have realized that.

[jira] [Commented] (SPARK-7136) Spark SQL and DataFrame Guide - missing file paths and non-existent example file

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511781#comment-14511781 ] Apache Spark commented on SPARK-7136: - User 'd3borah' has created a pull request for t

[jira] [Assigned] (SPARK-7136) Spark SQL and DataFrame Guide - missing file paths and non-existent example file

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7136: --- Assignee: Apache Spark > Spark SQL and DataFrame Guide - missing file paths and non-existent

[jira] [Assigned] (SPARK-7136) Spark SQL and DataFrame Guide - missing file paths and non-existent example file

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7136: --- Assignee: (was: Apache Spark) > Spark SQL and DataFrame Guide - missing file paths and no

[jira] [Created] (SPARK-7136) Spark SQL and DataFrame Guide - missing file paths and non-existent example file

2015-04-24 Thread Deborah Siegel (JIRA)
Deborah Siegel created SPARK-7136: - Summary: Spark SQL and DataFrame Guide - missing file paths and non-existent example file Key: SPARK-7136 URL: https://issues.apache.org/jira/browse/SPARK-7136 Proj

[jira] [Updated] (SPARK-7107) Add parameter for zookeeper.znode.parent to hbase_inputformat.py

2015-04-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7107: --- Assignee: Ted Yu > Add parameter for zookeeper.znode.parent to hbase_inputformat.py >

[jira] [Created] (SPARK-7135) Add a Column expression that can generates unique IDs for each row

2015-04-24 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7135: -- Summary: Add a Column expression that can generates unique IDs for each row Key: SPARK-7135 URL: https://issues.apache.org/jira/browse/SPARK-7135 Project: Spark

[jira] [Assigned] (SPARK-7134) Add regParam and featureScaling options to Logistic regression 'train' methods

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7134: --- Assignee: (was: Apache Spark) > Add regParam and featureScaling options to Logistic regre

[jira] [Commented] (SPARK-7134) Add regParam and featureScaling options to Logistic regression 'train' methods

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511717#comment-14511717 ] Apache Spark commented on SPARK-7134: - User 'rakeshchalasani' has created a pull reque

[jira] [Assigned] (SPARK-7134) Add regParam and featureScaling options to Logistic regression 'train' methods

2015-04-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7134: --- Assignee: Apache Spark > Add regParam and featureScaling options to Logistic regression 'trai

[jira] [Commented] (SPARK-7108) Setting spark.local.dir in driver no longer overrides the standalone worker's local directory setting

2015-04-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511698#comment-14511698 ] Marcelo Vanzin commented on SPARK-7108: --- I did that because the documentation explic

[jira] [Commented] (SPARK-7103) SparkContext.union crashed when some RDDs have no partitioner

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511689#comment-14511689 ] Patrick Wendell commented on SPARK-7103: Escalated the priority since IMO this is

[jira] [Commented] (SPARK-7108) Setting spark.local.dir in driver no longer overrides the standalone worker's local directory setting

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511687#comment-14511687 ] Patrick Wendell commented on SPARK-7108: Ping [~vanzin] who authored SPARK-4834 >

[jira] [Updated] (SPARK-7120) ClosureCleaner lacks documentation

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7120: --- Issue Type: Improvement (was: Bug) > ClosureCleaner lacks documentation > ---

[jira] [Updated] (SPARK-7103) SparkContext.union crashed when some RDDs have no partitioner

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7103: --- Priority: Critical (was: Minor) > SparkContext.union crashed when some RDDs have no partition

[jira] [Updated] (SPARK-7103) SparkContext.union crashed when some RDDs have no partitioner

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7103: --- Target Version/s: 1.3.2, 1.4.0 > SparkContext.union crashed when some RDDs have no partitioner

[jira] [Created] (SPARK-7134) Add regParam and featureScaling options to Logistic regression 'train' methods

2015-04-24 Thread Rakesh Chalasani (JIRA)
Rakesh Chalasani created SPARK-7134: --- Summary: Add regParam and featureScaling options to Logistic regression 'train' methods Key: SPARK-7134 URL: https://issues.apache.org/jira/browse/SPARK-7134 Pr

[jira] [Updated] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-04-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7133: --- Labels: starter (was: ) > Implement struct, array, and map field accessor using apply in Scala and >

[jira] [Updated] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-04-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7133: --- Description: Typing {code} df.col[1] {code} and {code} df.col['field'] {code} is so much eaiser than

[jira] [Updated] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-04-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7133: --- Description: Typing {code} df.col[1] and df.col['field'] {code} is so much eaiser than {code} df.

[jira] [Commented] (SPARK-7035) Drop __getattr__ on pyspark.sql.DataFrame

2015-04-24 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511641#comment-14511641 ] Reynold Xin commented on SPARK-7035: [~kalle] while we debate about whether we want to

[jira] [Created] (SPARK-7133) Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python

2015-04-24 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7133: -- Summary: Implement struct, array, and map field accessor using apply in Scala and __getitem__ in Python Key: SPARK-7133 URL: https://issues.apache.org/jira/browse/SPARK-7133

[jira] [Updated] (SPARK-6852) Accept numeric as numPartitions in SparkR

2015-04-24 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-6852: - Assignee: Sun Rui > Accept numeric as numPartitions in SparkR > --

[jira] [Resolved] (SPARK-6852) Accept numeric as numPartitions in SparkR

2015-04-24 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-6852. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 561

[jira] [Resolved] (SPARK-4590) Early investigation of parameter server

2015-04-24 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Zadeh resolved SPARK-4590. --- Resolution: Fixed > Early investigation of parameter server > ---

[jira] [Commented] (SPARK-4590) Early investigation of parameter server

2015-04-24 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511621#comment-14511621 ] Reza Zadeh commented on SPARK-4590: --- I am resolving this ticket as it has served its pur

[jira] [Comment Edited] (SPARK-4514) SparkContext localProperties does not inherit property updates across thread reuse

2015-04-24 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511537#comment-14511537 ] Ilya Ganelin edited comment on SPARK-4514 at 4/24/15 7:37 PM: --

[jira] [Created] (SPARK-7132) Add fit with validation set to spark.ml GBT

2015-04-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7132: Summary: Add fit with validation set to spark.ml GBT Key: SPARK-7132 URL: https://issues.apache.org/jira/browse/SPARK-7132 Project: Spark Issue Type:

[jira] [Created] (SPARK-7131) Move tree,forest implementation from spark.mllib to spark.ml

2015-04-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7131: Summary: Move tree,forest implementation from spark.mllib to spark.ml Key: SPARK-7131 URL: https://issues.apache.org/jira/browse/SPARK-7131 Project: Spark

[jira] [Created] (SPARK-7130) spark.ml RandomForest* should always do bootstrapping

2015-04-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7130: Summary: spark.ml RandomForest* should always do bootstrapping Key: SPARK-7130 URL: https://issues.apache.org/jira/browse/SPARK-7130 Project: Spark I

[jira] [Created] (SPARK-7129) Add generic boosting algorithm to spark.ml

2015-04-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7129: Summary: Add generic boosting algorithm to spark.ml Key: SPARK-7129 URL: https://issues.apache.org/jira/browse/SPARK-7129 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6290) spark.ml.param.Params.checkInputColumn bug upon error

2015-04-24 Thread Glenn Weidner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511540#comment-14511540 ] Glenn Weidner commented on SPARK-6290: -- After synchronizing with latest from master,

[jira] [Commented] (SPARK-4514) SparkContext localProperties does not inherit property updates across thread reuse

2015-04-24 Thread Ilya Ganelin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511537#comment-14511537 ] Ilya Ganelin commented on SPARK-4514: - [~joshrosen] - given your work on SPARK-6629 is

[jira] [Commented] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-24 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511526#comment-14511526 ] Bryan Cutler commented on SPARK-7127: - Hey [~josephkb], I'd love to work on this to st

[jira] [Comment Edited] (SPARK-2516) Bootstrapping

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511493#comment-14511493 ] Joseph K. Bradley edited comment on SPARK-2516 at 4/24/15 6:37 PM: -

[jira] [Commented] (SPARK-2516) Bootstrapping

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14511493#comment-14511493 ] Joseph K. Bradley commented on SPARK-2516: -- [~mengxr] Just to confirm, am I corr

[jira] [Created] (SPARK-7128) Add generic bagging algorithm to spark.ml

2015-04-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7128: Summary: Add generic bagging algorithm to spark.ml Key: SPARK-7128 URL: https://issues.apache.org/jira/browse/SPARK-7128 Project: Spark Issue Type: N

[jira] [Updated] (SPARK-6479) Create external block store API

2015-04-24 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6479: --- Target Version/s: 1.4.0 > Create external block store API > --- >

[jira] [Updated] (SPARK-7127) Broadcast spark.ml tree ensemble models for predict

2015-04-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7127: - Labels: starter (was: ) > Broadcast spark.ml tree ensemble models for predict > -

  1   2   >