[jira] [Commented] (SPARK-3714) Spark workflow scheduler

2014-09-29 Thread Egor Pakhomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151387#comment-14151387 ] Egor Pakhomov commented on SPARK-3714: -- Yes, I tried, please see [Design doc |

[jira] [Created] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2014-09-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-3717: Summary: DecisionTree, RandomForest: Partition by feature Key: SPARK-3717 URL: https://issues.apache.org/jira/browse/SPARK-3717 Project: Spark Issue

[jira] [Updated] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2014-09-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3717: - Description: h1. Summary Currently, data are partitioned by row/instance for

[jira] [Created] (SPARK-3718) FsHistoryProvider should consider spark.eventLog.dir not only spark.history.fs.logDirectory

2014-09-29 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3718: - Summary: FsHistoryProvider should consider spark.eventLog.dir not only spark.history.fs.logDirectory Key: SPARK-3718 URL: https://issues.apache.org/jira/browse/SPARK-3718

[jira] [Closed] (SPARK-3567) appId field in SparkDeploySchedulerBackend should be volatile

2014-09-29 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta closed SPARK-3567. - Resolution: Fixed appId field in SparkDeploySchedulerBackend should be volatile

[jira] [Commented] (SPARK-2516) Bootstrapping

2014-09-29 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151416#comment-14151416 ] Yu Ishikawa commented on SPARK-2516: Thank you for assigning me with this issue and

[jira] [Commented] (SPARK-3718) FsHistoryProvider should consider spark.eventLog.dir not only spark.history.fs.logDirectory

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151418#comment-14151418 ] Apache Spark commented on SPARK-3718: - User 'sarutak' has created a pull request for

[jira] [Commented] (SPARK-3714) Spark workflow scheduler

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151422#comment-14151422 ] Sean Owen commented on SPARK-3714: -- Another meta-question for everyone: at what point

[jira] [Commented] (SPARK-3274) Spark Streaming Java API reports java.lang.ClassCastException when calling collectAsMap on JavaPairDStream

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151432#comment-14151432 ] Sean Owen commented on SPARK-3274: -- I don't think that's the same thing. It is just

[jira] [Commented] (SPARK-3274) Spark Streaming Java API reports java.lang.ClassCastException when calling collectAsMap on JavaPairDStream

2014-09-29 Thread Pulkit Bhuwalka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151447#comment-14151447 ] Pulkit Bhuwalka commented on SPARK-3274: [~sowen] - you are right. I was making

[jira] [Comment Edited] (SPARK-3274) Spark Streaming Java API reports java.lang.ClassCastException when calling collectAsMap on JavaPairDStream

2014-09-29 Thread Pulkit Bhuwalka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151447#comment-14151447 ] Pulkit Bhuwalka edited comment on SPARK-3274 at 9/29/14 7:27 AM:

[jira] [Resolved] (SPARK-2159) Spark shell exit() does not stop SparkContext

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2159. -- Resolution: Won't Fix Fix Version/s: (was: 1.2.0) The discussion in the PR suggests this is

[jira] [Commented] (SPARK-3714) Spark workflow scheduler

2014-09-29 Thread Egor Pakhomov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151458#comment-14151458 ] Egor Pakhomov commented on SPARK-3714: -- I agree with your concerns - it should be

[jira] [Resolved] (SPARK-2643) Stages web ui has ERROR when pool name is None

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2643. -- Resolution: Fixed Discussion suggests this was fixed by a related change:

[jira] [Resolved] (SPARK-1208) after some hours of working the :4040 monitoring UI stops working.

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1208. -- Resolution: Fixed This appears to be a similar, if not the same issue, as in SPARK-2643. The

[jira] [Updated] (SPARK-3203) ClassNotFoundException in spark-shell with Cassandra

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3203: - Summary: ClassNotFoundException in spark-shell with Cassandra (was: ClassNotFound Exception)

[jira] [Updated] (SPARK-1381) Spark to Shark direct streaming

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-1381: - Priority: Major (was: Blocker) It sounds like this is WontFix at this point, if there was a problem to

[jira] [Resolved] (SPARK-1381) Spark to Shark direct streaming

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1381. -- Resolution: Won't Fix Spark to Shark direct streaming ---

[jira] [Commented] (SPARK-3714) Spark workflow scheduler

2014-09-29 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151486#comment-14151486 ] Mridul Muralidharan commented on SPARK-3714: Most of the drawbacks mentioned

[jira] [Created] (SPARK-3719) Spark UI: complete/failed stages is better to show the total number of stages

2014-09-29 Thread uncleGen (JIRA)
uncleGen created SPARK-3719: --- Summary: Spark UI: complete/failed stages is better to show the total number of stages Key: SPARK-3719 URL: https://issues.apache.org/jira/browse/SPARK-3719 Project: Spark

[jira] [Commented] (SPARK-3719) Spark UI: complete/failed stages is better to show the total number of stages

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151490#comment-14151490 ] Apache Spark commented on SPARK-3719: - User 'uncleGen' has created a pull request for

[jira] [Commented] (SPARK-3582) Spark SQL having issue with existing Hive UDFs which take Map as a parameter

2014-09-29 Thread Saurabh Santhosh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151511#comment-14151511 ] Saurabh Santhosh commented on SPARK-3582: - Issue resolved by Pull request :

[jira] [Updated] (SPARK-3601) Kryo NPE for output operations on Avro complex Objects even after registering.

2014-09-29 Thread mohan gaddam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mohan gaddam updated SPARK-3601: Shepherd: Reynold Xin Kryo NPE for output operations on Avro complex Objects even after

[jira] [Commented] (SPARK-2626) Stop SparkContext in all examples

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151575#comment-14151575 ] Apache Spark commented on SPARK-2626: - User 'srowen' has created a pull request for

[jira] [Commented] (SPARK-2805) update akka to version 2.3

2014-09-29 Thread Quinton Anderson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151646#comment-14151646 ] Quinton Anderson commented on SPARK-2805: - Any progress on this front? update

[jira] [Updated] (SPARK-1313) Shark- JDBC driver

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-1313: - Priority: Minor (was: Blocker) Issue Type: Question (was: Task) Shark- JDBC driver

[jira] [Resolved] (SPARK-1313) Shark- JDBC driver

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1313. -- Resolution: Not a Problem This looks like it was a question more than anything, and was answered.

[jira] [Resolved] (SPARK-1884) Shark failed to start

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-1884. -- Resolution: Won't Fix This appears to be a protobuf version mismatch, which suggests Shark is being

[jira] [Commented] (SPARK-3687) Spark hang while processing more than 100 sequence files

2014-09-29 Thread Yi Tian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151683#comment-14151683 ] Yi Tian commented on SPARK-3687: The stack you print is the worker process. I think you

[jira] [Commented] (SPARK-3685) Spark's local dir should accept only local paths

2014-09-29 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151691#comment-14151691 ] Matthew Farrellee commented on SPARK-3685: -- [~andrewor] thanks for the info.

[jira] [Created] (SPARK-3720) support ORC in spark sql

2014-09-29 Thread wangfei (JIRA)
wangfei created SPARK-3720: -- Summary: support ORC in spark sql Key: SPARK-3720 URL: https://issues.apache.org/jira/browse/SPARK-3720 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Updated] (SPARK-3720) support ORC in spark sql

2014-09-29 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-3720: --- Description: The Optimized Row Columnar (ORC) file format provides a highly efficient way to store data on

[jira] [Commented] (SPARK-3720) support ORC in spark sql

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151727#comment-14151727 ] Apache Spark commented on SPARK-3720: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-3522) Make spark-ec2 verbosity configurable

2014-09-29 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151787#comment-14151787 ] Nicholas Chammas commented on SPARK-3522: - Always logging to a file sounds like a

[jira] [Commented] (SPARK-3627) spark on yarn reports success even though job fails

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151784#comment-14151784 ] Apache Spark commented on SPARK-3627: - User 'tgravescs' has created a pull request for

[jira] [Created] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
Brad Miller created SPARK-3721: -- Summary: Broadcast Variables above 2GB break in PySpark Key: SPARK-3721 URL: https://issues.apache.org/jira/browse/SPARK-3721 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: The bug displays 3 unique failure modes in PySpark, all of which seem to be related to

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Created] (SPARK-3722) Spark on yarn docs work

2014-09-29 Thread WangTaoTheTonic (JIRA)
WangTaoTheTonic created SPARK-3722: -- Summary: Spark on yarn docs work Key: SPARK-3722 URL: https://issues.apache.org/jira/browse/SPARK-3722 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3722) Spark on yarn docs work

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151869#comment-14151869 ] Apache Spark commented on SPARK-3722: - User 'WangTaoTheTonic' has created a pull

[jira] [Updated] (SPARK-3721) Broadcast Variables above 2GB break in PySpark

2014-09-29 Thread Brad Miller (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brad Miller updated SPARK-3721: --- Description: Attempting to reproduce the bug in isolation in iPython notebook I've observed the

[jira] [Updated] (SPARK-3366) Compute best splits distributively in decision tree

2014-09-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3366: - Assignee: Qiping Li Compute best splits distributively in decision tree

[jira] [Updated] (SPARK-3677) Scalastyle is never applyed to the sources under yarn/common

2014-09-29 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3677: -- Description: When we run sbt -Pyarn scalastyle or mvn package, scalastyle is not applied to

[jira] [Updated] (SPARK-2626) Stop SparkContext in all examples

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2626: --- Labels: starter (was: ) Stop SparkContext in all examples

[jira] [Commented] (SPARK-2805) update akka to version 2.3

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151976#comment-14151976 ] Patrick Wendell commented on SPARK-2805: I'm working on publishing akka today.

[jira] [Commented] (SPARK-3479) Have Jenkins show which tests failed in his GitHub messages

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151980#comment-14151980 ] Patrick Wendell commented on SPARK-3479: I think this is more than minor - it

[jira] [Updated] (SPARK-3479) Have Jenkins show which tests failed in his GitHub messages

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3479: --- Priority: Major (was: Minor) Have Jenkins show which tests failed in his GitHub messages

[jira] [Updated] (SPARK-3479) Have Jenkins show which tests failed in his GitHub messages

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3479: --- Issue Type: Improvement (was: Sub-task) Parent: (was: SPARK-2230) Have Jenkins

[jira] [Resolved] (SPARK-2885) All-pairs similarity via DIMSUM

2014-09-29 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-2885. -- Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 1778

[jira] [Resolved] (SPARK-2230) Improvements to Jenkins QA Harness

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2230. Resolution: Fixed This was tracking an earlier initiative to clean up this harness. Since

[jira] [Resolved] (SPARK-3032) Potential bug when running sort-based shuffle with sorting using TimSort

2014-09-29 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3032. -- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Potential bug when

[jira] [Commented] (SPARK-3032) Potential bug when running sort-based shuffle with sorting using TimSort

2014-09-29 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152014#comment-14152014 ] Matei Zaharia commented on SPARK-3032: -- Yup, this will appear in 1.1.1. I've merged

[jira] [Commented] (SPARK-2331) SparkContext.emptyRDD has wrong return type

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152018#comment-14152018 ] Patrick Wendell commented on SPARK-2331: Yeah we could have made this a wider type

[jira] [Resolved] (SPARK-2331) SparkContext.emptyRDD has wrong return type

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2331. Resolution: Won't Fix SparkContext.emptyRDD has wrong return type

[jira] [Comment Edited] (SPARK-2331) SparkContext.emptyRDD has wrong return type

2014-09-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152018#comment-14152018 ] Patrick Wendell edited comment on SPARK-2331 at 9/29/14 6:34 PM:

[jira] [Created] (SPARK-3725) Link to building spark returns a 404

2014-09-29 Thread Anant Daksh Asthana (JIRA)
Anant Daksh Asthana created SPARK-3725: -- Summary: Link to building spark returns a 404 Key: SPARK-3725 URL: https://issues.apache.org/jira/browse/SPARK-3725 Project: Spark Issue Type:

[jira] [Created] (SPARK-3726) RandomForest: Support for bootstrap options

2014-09-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-3726: Summary: RandomForest: Support for bootstrap options Key: SPARK-3726 URL: https://issues.apache.org/jira/browse/SPARK-3726 Project: Spark Issue

[jira] [Created] (SPARK-3727) DecisionTree, RandomForest: More prediction functionality

2014-09-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-3727: Summary: DecisionTree, RandomForest: More prediction functionality Key: SPARK-3727 URL: https://issues.apache.org/jira/browse/SPARK-3727 Project: Spark

[jira] [Commented] (SPARK-3725) Link to building spark returns a 404

2014-09-29 Thread Anant Daksh Asthana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152047#comment-14152047 ] Anant Daksh Asthana commented on SPARK-3725: Would it make sense to add a

[jira] [Commented] (SPARK-3725) Link to building spark returns a 404

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152050#comment-14152050 ] Sean Owen commented on SPARK-3725: -- Yes of course, it's already in the repo and has been

[jira] [Comment Edited] (SPARK-922) Update Spark AMI to Python 2.7

2014-09-29 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150695#comment-14150695 ] Andrew Davidson edited comment on SPARK-922 at 9/29/14 7:05 PM:

[jira] [Created] (SPARK-3729) Null-pointer when constructing a HiveContext when settings are present

2014-09-29 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3729: --- Summary: Null-pointer when constructing a HiveContext when settings are present Key: SPARK-3729 URL: https://issues.apache.org/jira/browse/SPARK-3729 Project:

[jira] [Assigned] (SPARK-3729) Null-pointer when constructing a HiveContext when settings are present

2014-09-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-3729: --- Assignee: Michael Armbrust Null-pointer when constructing a HiveContext when

[jira] [Commented] (SPARK-1547) Add gradient boosting algorithm to MLlib

2014-09-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152101#comment-14152101 ] Joseph K. Bradley commented on SPARK-1547: -- This will be great to have! The WIP

[jira] [Commented] (SPARK-3729) Null-pointer when constructing a HiveContext when settings are present

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152113#comment-14152113 ] Apache Spark commented on SPARK-3729: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-3725) Link to building spark returns a 404

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152138#comment-14152138 ] Sean Owen commented on SPARK-3725: -- No, that links to the raw markdown. Truly, the fix is

[jira] [Commented] (SPARK-3730) Any one else having building spark recently

2014-09-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152140#comment-14152140 ] Sean Owen commented on SPARK-3730: -- (The profile is hadoop-2.3 but that's not the issue.)

[jira] [Commented] (SPARK-3685) Spark's local dir should accept only local paths

2014-09-29 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152152#comment-14152152 ] Matthew Farrellee commented on SPARK-3685: -- the root of the resource problem is

[jira] [Updated] (SPARK-2693) Support for UDAF Hive Aggregates like PERCENTILE

2014-09-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2693: Priority: Critical (was: Major) Support for UDAF Hive Aggregates like PERCENTILE

[jira] [Updated] (SPARK-2693) Support for UDAF Hive Aggregates like PERCENTILE

2014-09-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2693: Assignee: Ravindra Pesala Support for UDAF Hive Aggregates like PERCENTILE

[jira] [Created] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-09-29 Thread Milan Straka (JIRA)
Milan Straka created SPARK-3731: --- Summary: RDD caching stops working in pyspark after some time Key: SPARK-3731 URL: https://issues.apache.org/jira/browse/SPARK-3731 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-09-29 Thread Milan Straka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milan Straka updated SPARK-3731: Attachment: worker.log Sample worker.log showing the problem. For example, consider rdd_1_1. It has

[jira] [Commented] (SPARK-3685) Spark's local dir should accept only local paths

2014-09-29 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152228#comment-14152228 ] Andrew Or commented on SPARK-3685: -- Not sure if I fully understand what you mean. If I'm

[jira] [Commented] (SPARK-3709) BroadcastSuite.Unpersisting rg.apache.spark.broadcast.BroadcastSuite.Unpersisting TorrentBroadcast is flaky

2014-09-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152238#comment-14152238 ] Reynold Xin commented on SPARK-3709: Adding stack trace {code} [info] - Unpersisting

[jira] [Created] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Sotos Matzanas (JIRA)
Sotos Matzanas created SPARK-3732: - Summary: Yarn Client: Add option to NOT System.exit() at end of main() Key: SPARK-3732 URL: https://issues.apache.org/jira/browse/SPARK-3732 Project: Spark

[jira] [Commented] (SPARK-1547) Add gradient boosting algorithm to MLlib

2014-09-29 Thread Manish Amde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152251#comment-14152251 ] Manish Amde commented on SPARK-1547: Sure. I like your naming suggestion. I will

[jira] [Commented] (SPARK-3730) Any one else having building spark recently

2014-09-29 Thread Anant Daksh Asthana (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152269#comment-14152269 ] Anant Daksh Asthana commented on SPARK-3730: Definately not a spark issue.

[jira] [Updated] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2014-09-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3717: - Description: h1. Summary Currently, data are partitioned by row/instance for

[jira] [Commented] (SPARK-3434) Distributed block matrix

2014-09-29 Thread Reza Zadeh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152275#comment-14152275 ] Reza Zadeh commented on SPARK-3434: --- It looks like Shivaram Venkataraman from the AMPlab

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152291#comment-14152291 ] Apache Spark commented on SPARK-3732: - User 'smatzana' has created a pull request for

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152294#comment-14152294 ] Marcelo Vanzin commented on SPARK-3732: --- I think that explicit System.exit() could

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Sotos Matzanas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152302#comment-14152302 ] Sotos Matzanas commented on SPARK-3732: --- we added the option as insurance against

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152308#comment-14152308 ] Thomas Graves commented on SPARK-3732: -- I think you should just change the name of

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152309#comment-14152309 ] Marcelo Vanzin commented on SPARK-3732: --- Removing the call should work regardless;

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152318#comment-14152318 ] Marcelo Vanzin commented on SPARK-3732: --- BTW, if the call is removed, it should be

[jira] [Commented] (SPARK-3708) Backticks aren't handled correctly is aliases

2014-09-29 Thread Ravindra Pesala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152321#comment-14152321 ] Ravindra Pesala commented on SPARK-3708: I guess here you mentioned about

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Sotos Matzanas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152323#comment-14152323 ] Sotos Matzanas commented on SPARK-3732: --- [~tgraves] this jira is the first step for

[jira] [Created] (SPARK-3733) Support for programmatically submitting Spark jobs

2014-09-29 Thread Sotos Matzanas (JIRA)
Sotos Matzanas created SPARK-3733: - Summary: Support for programmatically submitting Spark jobs Key: SPARK-3733 URL: https://issues.apache.org/jira/browse/SPARK-3733 Project: Spark Issue

[jira] [Commented] (SPARK-3717) DecisionTree, RandomForest: Partition by feature

2014-09-29 Thread Sung Chung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152347#comment-14152347 ] Sung Chung commented on SPARK-3717: --- I think that this would be great as an alternative

[jira] [Commented] (SPARK-3630) Identify cause of Kryo+Snappy PARSING_ERROR

2014-09-29 Thread Arun Ahuja (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152351#comment-14152351 ] Arun Ahuja commented on SPARK-3630: --- We have seen this issue as well: {code}

[jira] [Updated] (SPARK-3709) BroadcastSuite.Unpersisting rg.apache.spark.broadcast.BroadcastSuite.Unpersisting TorrentBroadcast is flaky

2014-09-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3709: --- Assignee: Reynold Xin (was: Cheng Lian) BroadcastSuite.Unpersisting

[jira] [Commented] (SPARK-3709) BroadcastSuite.Unpersisting rg.apache.spark.broadcast.BroadcastSuite.Unpersisting TorrentBroadcast is flaky

2014-09-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152377#comment-14152377 ] Apache Spark commented on SPARK-3709: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-09-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152409#comment-14152409 ] Thomas Graves commented on SPARK-3732: -- I understand your usecase and need for it,

  1   2   >