[jira] [Commented] (SPARK-5444) 'spark.blockManager.port' conflict in netty service

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294840#comment-14294840 ] Apache Spark commented on SPARK-5444: - User 'SaintBacchus' has created a pull request

[jira] [Commented] (SPARK-5395) Large number of Python workers causing resource depletion

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294820#comment-14294820 ] Apache Spark commented on SPARK-5395: - User 'davies' has created a pull request for th

[jira] [Updated] (SPARK-5395) Large number of Python workers causing resource depletion

2015-01-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-5395: -- Affects Version/s: 1.3.0 > Large number of Python workers causing resource depletion > -

[jira] [Updated] (SPARK-5426) SchemaRDD is java incompatible

2015-01-27 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuldeep updated SPARK-5426: --- Priority: Minor (was: Major) > SchemaRDD is java incompatible > -- > >

[jira] [Created] (SPARK-5444) 'spark.blockManager.port' conflict in netty service

2015-01-27 Thread SaintBacchus (JIRA)
SaintBacchus created SPARK-5444: --- Summary: 'spark.blockManager.port' conflict in netty service Key: SPARK-5444 URL: https://issues.apache.org/jira/browse/SPARK-5444 Project: Spark Issue Type: B

[jira] [Updated] (SPARK-5426) SchemaRDD is java incompatible

2015-01-27 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuldeep updated SPARK-5426: --- Priority: Major (was: Minor) > SchemaRDD is java incompatible > -- > >

[jira] [Commented] (SPARK-5426) SchemaRDD is java incompatible

2015-01-27 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294760#comment-14294760 ] Kuldeep commented on SPARK-5426: `toJavaRDD` seems to be the trick. But this does seems a

[jira] [Updated] (SPARK-5382) Scripts do not use SPARK_CONF_DIR where they should

2015-01-27 Thread Jacek Lewandowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated SPARK-5382: - Fix Version/s: 1.2.1 1.3.0 > Scripts do not use SPARK_CONF_DIR where th

[jira] [Updated] (SPARK-5426) SchemaRDD is java incompatible

2015-01-27 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuldeep updated SPARK-5426: --- Priority: Minor (was: Blocker) > SchemaRDD is java incompatible > -- > >

[jira] [Issue Comment Deleted] (SPARK-5377) Dynamically add jar into Spark Driver's classpath.

2015-01-27 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-5377: Comment: was deleted (was: It's really needed. When we use HiveContext and run sql, {{\-\-driver-cl

[jira] [Created] (SPARK-5443) jsonRDD with schema should ignore sub-objects that are omitted in schema

2015-01-27 Thread Derrick Burns (JIRA)
Derrick Burns created SPARK-5443: Summary: jsonRDD with schema should ignore sub-objects that are omitted in schema Key: SPARK-5443 URL: https://issues.apache.org/jira/browse/SPARK-5443 Project: Spark

[jira] [Updated] (SPARK-5198) Change executorId more unique on mesos fine-grained mode

2015-01-27 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-5198: Fix Version/s: (was: 1.2.1) (was: 1.3.0) > Change executorId more unique

[jira] [Commented] (SPARK-5376) [Mesos] MesosExecutor should have correct resources

2015-01-27 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294674#comment-14294674 ] Jongyoul Lee commented on SPARK-5376: - Please change {{Unresolved}} to {{Not a problem

[jira] [Commented] (SPARK-5198) Change executorId more unique on mesos fine-grained mode

2015-01-27 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294672#comment-14294672 ] Jongyoul Lee commented on SPARK-5198: - Please change {{Unresolved}} to {{Not a problem

[jira] [Comment Edited] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

2015-01-27 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294648#comment-14294648 ] Kuldeep edited comment on SPARK-5308 at 1/28/15 3:14 AM: - Thanks [

[jira] [Commented] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

2015-01-27 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294648#comment-14294648 ] Kuldeep commented on SPARK-5308: Thanks! > MD5 / SHA1 hash format doesn't match standard

[jira] [Commented] (SPARK-5324) Results of describe can't be queried

2015-01-27 Thread shengli (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294627#comment-14294627 ] shengli commented on SPARK-5324: hi, [~yanboliang] Thanks for working on this. But by the

[jira] [Updated] (SPARK-5361) python tuple not supported while converting PythonRDD back to JavaRDD

2015-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5361: -- Affects Version/s: 1.2.0 > python tuple not supported while converting PythonRDD back to JavaRDD > -

[jira] [Updated] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5441: --- Target Version/s: 1.3.0 > SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs >

[jira] [Updated] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5441: --- Assignee: Michael Nazario > SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

[jira] [Commented] (SPARK-5395) Large number of Python workers causing resource depletion

2015-01-27 Thread Sven Krasser (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294570#comment-14294570 ] Sven Krasser commented on SPARK-5395: - Some new findings: I can trigger the problem no

[jira] [Commented] (SPARK-5162) Python yarn-cluster mode

2015-01-27 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294567#comment-14294567 ] Lianhui Wang commented on SPARK-5162: - [~vgrigor] i think that a quick way to resolve

[jira] [Closed] (SPARK-5442) Docs claim users must explicitly depend on a hadoop client, but it is not actually required.

2015-01-27 Thread Juliet Hougland (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliet Hougland closed SPARK-5442. -- Resolution: Not a Problem > Docs claim users must explicitly depend on a hadoop client, but it i

[jira] [Commented] (SPARK-5436) Validate GradientBoostedTrees during training

2015-01-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294523#comment-14294523 ] Joseph K. Bradley commented on SPARK-5436: -- That sound good. I think the main "c

[jira] [Commented] (SPARK-5440) Add toLocalIterator to pyspark rdd

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294469#comment-14294469 ] Apache Spark commented on SPARK-5440: - User 'mnazario' has created a pull request for

[jira] [Commented] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294465#comment-14294465 ] Apache Spark commented on SPARK-5441: - User 'mnazario' has created a pull request for

[jira] [Created] (SPARK-5442) Docs claim users must explicitly depend on a hadoop client, but it is not actually required.

2015-01-27 Thread Juliet Hougland (JIRA)
Juliet Hougland created SPARK-5442: -- Summary: Docs claim users must explicitly depend on a hadoop client, but it is not actually required. Key: SPARK-5442 URL: https://issues.apache.org/jira/browse/SPARK-5442

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294454#comment-14294454 ] Apache Spark commented on SPARK-5097: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

2015-01-27 Thread Michael Nazario (JIRA)
Michael Nazario created SPARK-5441: -- Summary: SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs Key: SPARK-5441 URL: https://issues.apache.org/jira/browse/SPARK-5441 Project: Spark

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

2015-01-27 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294445#comment-14294445 ] Tomer Kaftan commented on SPARK-5097: - What are the current plans for how UDFs will fi

[jira] [Created] (SPARK-5440) Add toLocalIterator to pyspark rdd

2015-01-27 Thread Michael Nazario (JIRA)
Michael Nazario created SPARK-5440: -- Summary: Add toLocalIterator to pyspark rdd Key: SPARK-5440 URL: https://issues.apache.org/jira/browse/SPARK-5440 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-5199) Input metrics should show up for InputFormats that return CombineFileSplits

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5199. Resolution: Fixed Fix Version/s: 1.3.0 > Input metrics should show up for InputFormat

[jira] [Commented] (SPARK-5439) Expose yarn app id for yarn mode

2015-01-27 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294406#comment-14294406 ] Xuefu Zhang commented on SPARK-5439: Yeah. This is certainly something desirable in Hi

[jira] [Commented] (SPARK-5436) Validate GradientBoostedTrees during training

2015-01-27 Thread Chris T (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294403#comment-14294403 ] Chris T commented on SPARK-5436: I think, then, the only addition needed is to retain the

[jira] [Commented] (SPARK-5191) Pyspark: scheduler hangs when importing a standalone pyspark app

2015-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294400#comment-14294400 ] Josh Rosen commented on SPARK-5191: --- I think this is actually caused by an interaction b

[jira] [Commented] (SPARK-3290) No unpersist callls in SVDPlusPlus

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294393#comment-14294393 ] Apache Spark commented on SPARK-3290: - User 'srowen' has created a pull request for th

[jira] [Commented] (SPARK-5439) Expose yarn app id for yarn mode

2015-01-27 Thread bc Wong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294378#comment-14294378 ] bc Wong commented on SPARK-5439: Parsing the stderr for YARN cluster mode isn't a good sol

[jira] [Commented] (SPARK-5439) Expose yarn app id for yarn mode

2015-01-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294362#comment-14294362 ] Marcelo Vanzin commented on SPARK-5439: --- Hi bc, can you provide more context here? T

[jira] [Commented] (SPARK-5436) Validate GradientBoostedTrees during training

2015-01-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294360#comment-14294360 ] Joseph K. Bradley commented on SPARK-5436: -- Yes, it would be reasonable to take t

[jira] [Created] (SPARK-5439) Expose yarn app id for yarn mode

2015-01-27 Thread bc Wong (JIRA)
bc Wong created SPARK-5439: -- Summary: Expose yarn app id for yarn mode Key: SPARK-5439 URL: https://issues.apache.org/jira/browse/SPARK-5439 Project: Spark Issue Type: Improvement Componen

[jira] [Resolved] (SPARK-5418) Output directory for shuffle should consider left space of each directory set in conf

2015-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5418. -- Resolution: Duplicate > Output directory for shuffle should consider left space of each directory set >

[jira] [Resolved] (SPARK-2751) Geeting error to run graphx.LiveJournalPageRank

2015-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2751. -- Resolution: Not a Problem The correct usage is --numEPart=n, which works and is what's documented. You

[jira] [Commented] (SPARK-3454) Expose JSON representation of data shown in WebUI

2015-01-27 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294348#comment-14294348 ] Ryan Williams commented on SPARK-3454: -- +1 to making the web UI a consumer of a JSON

[jira] [Commented] (SPARK-5191) Pyspark: scheduler hangs when importing a standalone pyspark app

2015-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294351#comment-14294351 ] Josh Rosen commented on SPARK-5191: --- I'm investigating this now and it seems to affect e

[jira] [Commented] (SPARK-5206) Accumulators are not re-registered during recovering from checkpoint

2015-01-27 Thread vincent ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294339#comment-14294339 ] vincent ye commented on SPARK-5206: --- I suggest to add life-cycle events into DStream, li

[jira] [Commented] (SPARK-5436) Validate GradientBoostedTrees during training

2015-01-27 Thread Chris T (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294335#comment-14294335 ] Chris T commented on SPARK-5436: The usual way that GBT models are evaluated is by calcula

[jira] [Commented] (SPARK-5438) Graph aggregateMessages should support zero/seqOp/combOp

2015-01-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294319#comment-14294319 ] Joseph K. Bradley commented on SPARK-5438: -- Note: My use case for this is in LDA:

[jira] [Created] (SPARK-5438) Graph aggregateMessages should support zero/seqOp/combOp

2015-01-27 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5438: Summary: Graph aggregateMessages should support zero/seqOp/combOp Key: SPARK-5438 URL: https://issues.apache.org/jira/browse/SPARK-5438 Project: Spark

[jira] [Commented] (SPARK-4587) Model export/import

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294312#comment-14294312 ] Apache Spark commented on SPARK-4587: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-2802) Improve the Cassandra sample and Add a new sample for Streaming to Cassandra

2015-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294314#comment-14294314 ] Sean Owen commented on SPARK-2802: -- Did this materialize? I imagine at this point the rec

[jira] [Commented] (SPARK-3454) Expose JSON representation of data shown in WebUI

2015-01-27 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294311#comment-14294311 ] Imran Rashid commented on SPARK-3454: - There are a few ways we could move forward on t

[jira] [Commented] (SPARK-2559) Add A Link to Download the Application Events Log for Offline Analysis

2015-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294306#comment-14294306 ] Sean Owen commented on SPARK-2559: -- I think this duplicates SPARK-2450 > Add A Link to D

[jira] [Closed] (SPARK-5432) SparkSubmitSuite should sc.stop()

2015-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-5432. Resolution: Won't Fix It may be intended behavior for the test suites to ensure that the driver does termin

[jira] [Updated] (SPARK-5191) Pyspark: scheduler hangs when importing a standalone pyspark app

2015-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5191: -- Affects Version/s: 1.2.1 1.3.0 1.0.2

[jira] [Updated] (SPARK-5437) DriverSuite and SparkSubmitSuite incorrect timeout behavior

2015-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5437: - Summary: DriverSuite and SparkSubmitSuite incorrect timeout behavior (was: DriverSuite and SparkSubmitSui

[jira] [Updated] (SPARK-5191) Pyspark: scheduler hangs when importing a standalone pyspark app

2015-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5191: -- Description: In a.py: {code} from pyspark import SparkContext sc = SparkContext("local", "test spark")

[jira] [Updated] (SPARK-5191) Pyspark: scheduler hangs when importing a standalone pyspark app

2015-01-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5191: -- Component/s: PySpark > Pyspark: scheduler hangs when importing a standalone pyspark app > --

[jira] [Commented] (SPARK-5437) DriverSuite and SparkSubmitSuite leak JVMs on timeout

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294293#comment-14294293 ] Apache Spark commented on SPARK-5437: - User 'andrewor14' has created a pull request fo

[jira] [Created] (SPARK-5437) DriverSuite and SparkSubmitSuite leak JVMs on timeout

2015-01-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-5437: Summary: DriverSuite and SparkSubmitSuite leak JVMs on timeout Key: SPARK-5437 URL: https://issues.apache.org/jira/browse/SPARK-5437 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5432) SparkSubmitSuite should sc.stop()

2015-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5432: - Priority: Minor (was: Major) > SparkSubmitSuite should sc.stop() > - > >

[jira] [Updated] (SPARK-5432) SparkSubmitSuite should sc.stop()

2015-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5432: - Description: The reason why we need all the hacks around the UI ports for these two suites is because we

[jira] [Updated] (SPARK-5432) SparkSubmitSuite should sc.stop()

2015-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5432: - Summary: SparkSubmitSuite should sc.stop() (was: DriverSuite and SparkSubmitSuite should sc.stop()) > Sp

[jira] [Commented] (SPARK-5338) Support cluster mode with Mesos

2015-01-27 Thread Timothy Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294264#comment-14294264 ] Timothy Chen commented on SPARK-5338: - Started a doc to begin the design of this, will

[jira] [Created] (SPARK-5436) Validate GradientBoostedTrees during training

2015-01-27 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-5436: Summary: Validate GradientBoostedTrees during training Key: SPARK-5436 URL: https://issues.apache.org/jira/browse/SPARK-5436 Project: Spark Issue Typ

[jira] [Updated] (SPARK-5338) Support cluster mode with Mesos

2015-01-27 Thread Timothy Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated SPARK-5338: Component/s: Mesos > Support cluster mode with Mesos > --- > >

[jira] [Commented] (SPARK-5428) Declare the 'assembly' module at the bottom of the element in the parent POM

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294246#comment-14294246 ] Apache Spark commented on SPARK-5428: - User 'tzolov' has created a pull request for th

[jira] [Created] (SPARK-5435) saveAsNewAPIHadoopDataset is not setting up the local configuration

2015-01-27 Thread Joe Mudd (JIRA)
Joe Mudd created SPARK-5435: --- Summary: saveAsNewAPIHadoopDataset is not setting up the local configuration Key: SPARK-5435 URL: https://issues.apache.org/jira/browse/SPARK-5435 Project: Spark Issu

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

2015-01-27 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294193#comment-14294193 ] Sandy Ryza commented on SPARK-5097: --- Ah, yeah, I hadn't considered that aspect. I defin

[jira] [Commented] (SPARK-3381) DecisionTree: eliminate bins for unordered features

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294161#comment-14294161 ] Apache Spark commented on SPARK-3381: - User 'MechCoder' has created a pull request for

[jira] [Commented] (SPARK-5434) Preserve spaces in path to spark-ec2

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294139#comment-14294139 ] Apache Spark commented on SPARK-5434: - User 'nchammas' has created a pull request for

[jira] [Created] (SPARK-5434) Preserve spaces in path to spark-ec2

2015-01-27 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-5434: --- Summary: Preserve spaces in path to spark-ec2 Key: SPARK-5434 URL: https://issues.apache.org/jira/browse/SPARK-5434 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5433) Spark EC2 doesn't mount local disks for all instance types

2015-01-27 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-5433: Description: Launching a cluster using spark-ec2 will currently mount all local disks for the r3 in

[jira] [Updated] (SPARK-5428) Declare the 'assembly' module at the bottom of the element in the parent POM

2015-01-27 Thread Christian Tzolov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Tzolov updated SPARK-5428: Description: For multiple-modules projects, Maven follows those execution order rules: http:

[jira] [Commented] (SPARK-5432) DriverSuite and SparkSubmitSuite should sc.stop()

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294080#comment-14294080 ] Apache Spark commented on SPARK-5432: - User 'andrewor14' has created a pull request fo

[jira] [Updated] (SPARK-5433) Spark EC2 doesn't mount local disks for all instance types

2015-01-27 Thread Tomer Kaftan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomer Kaftan updated SPARK-5433: Summary: Spark EC2 doesn't mount local disks for all instance types (was: Spark EC2 doesn't mount l

[jira] [Created] (SPARK-5433) Spark EC2 doesn't mount local disks for some instance types

2015-01-27 Thread Tomer Kaftan (JIRA)
Tomer Kaftan created SPARK-5433: --- Summary: Spark EC2 doesn't mount local disks for some instance types Key: SPARK-5433 URL: https://issues.apache.org/jira/browse/SPARK-5433 Project: Spark Issu

[jira] [Updated] (SPARK-5428) Declare the 'assembly' module at the bottom of the element in the parent POM

2015-01-27 Thread Christian Tzolov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Tzolov updated SPARK-5428: Description: For multiple-modules projects, Maven follows those execution order rules: http:

[jira] [Created] (SPARK-5432) DriverSuite and SparkSubmitSuite should sc.stop()

2015-01-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-5432: Summary: DriverSuite and SparkSubmitSuite should sc.stop() Key: SPARK-5432 URL: https://issues.apache.org/jira/browse/SPARK-5432 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5431) SparkSubmitSuite and DriverSuite hang indefinitely if Master fails

2015-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5431: - Affects Version/s: 1.0.0 > SparkSubmitSuite and DriverSuite hang indefinitely if Master fails > --

[jira] [Updated] (SPARK-5431) SparkSubmitSuite and DriverSuite hang indefinitely if Master fails

2015-01-27 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5431: - Priority: Major (was: Critical) > SparkSubmitSuite and DriverSuite hang indefinitely if Master fails > --

[jira] [Commented] (SPARK-5119) java.lang.ArrayIndexOutOfBoundsException on trying to train decision tree model

2015-01-27 Thread Nicolas Garneau (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294046#comment-14294046 ] Nicolas Garneau commented on SPARK-5119: Hey guys, I am wondering what you think a

[jira] [Commented] (SPARK-4587) Model export/import

2015-01-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294035#comment-14294035 ] Joseph K. Bradley commented on SPARK-4587: -- Thanks for reading the doc! The Parq

[jira] [Commented] (SPARK-5155) Python API for MQTT streaming

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294006#comment-14294006 ] Apache Spark commented on SPARK-5155: - User 'prabeesh' has created a pull request for

[jira] [Commented] (SPARK-5430) Move treeReduce and treeAggregate to core

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294000#comment-14294000 ] Apache Spark commented on SPARK-5430: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-5135) Add support for describe [extended] table to DDL in SQLContext

2015-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293994#comment-14293994 ] Apache Spark commented on SPARK-5135: - User 'OopsOutOfMemory' has created a pull reque

[jira] [Resolved] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5299. Resolution: Fixed Okay I've now added every key ever used to publish a Spark release (I thin

[jira] [Reopened] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-5299: Actually I need to deal with past releases as well, so re-opening. > Is http://www.apache.org/d

[jira] [Resolved] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5299. Resolution: Fixed Thanks I've fixed this. > Is http://www.apache.org/dist/spark/KEYS out of

[jira] [Created] (SPARK-5431) SparkSubmitSuite and DriverSuite hang indefinitely if Master fails

2015-01-27 Thread Andrew Or (JIRA)
Andrew Or created SPARK-5431: Summary: SparkSubmitSuite and DriverSuite hang indefinitely if Master fails Key: SPARK-5431 URL: https://issues.apache.org/jira/browse/SPARK-5431 Project: Spark Iss

[jira] [Created] (SPARK-5430) Move treeReduce and treeAggregate to core

2015-01-27 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-5430: Summary: Move treeReduce and treeAggregate to core Key: SPARK-5430 URL: https://issues.apache.org/jira/browse/SPARK-5430 Project: Spark Issue Type: Improveme

[jira] [Commented] (SPARK-1835) sbt gen-idea includes both mesos and mesos with shaded-protobuf into dependencies

2015-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293929#comment-14293929 ] Sean Owen commented on SPARK-1835: -- FWIW I can confirm this still occurs: {code} $ grep

[jira] [Resolved] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5308. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Assignee: Sea

[jira] [Commented] (SPARK-5051) python: module pyspark.daemon not found

2015-01-27 Thread Sven Krasser (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293916#comment-14293916 ] Sven Krasser commented on SPARK-5051: - I assume this is related to this thread: http:

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

2015-01-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293905#comment-14293905 ] Reynold Xin commented on SPARK-5097: I've debating that myself for a while. The main q

[jira] [Commented] (SPARK-4964) Exactly-once + WAL-free Kafka Support in Spark Streaming

2015-01-27 Thread vincent ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293899#comment-14293899 ] vincent ye commented on SPARK-4964: --- I have pretty much the same idea as mentioned in Ta

[jira] [Commented] (SPARK-4587) Model export/import

2015-01-27 Thread Peter Prettenhofer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293892#comment-14293892 ] Peter Prettenhofer commented on SPARK-4587: --- I read the design document for mode

[jira] [Resolved] (SPARK-1283) Create spark-contrib repo for 1.0

2015-01-27 Thread Evan Chan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evan Chan resolved SPARK-1283. -- Resolution: Won't Fix Yeah this is not needed anymore, with spark-packages.org > Create spark-contrib r

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

2015-01-27 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293849#comment-14293849 ] Sandy Ryza commented on SPARK-5097: --- Would it be possible to keep the Python versions of

[jira] [Commented] (SPARK-1283) Create spark-contrib repo for 1.0

2015-01-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293848#comment-14293848 ] Sean Owen commented on SPARK-1283: -- Is this obsolete given the existence of spark-package

[jira] [Commented] (SPARK-5395) Large number of Python workers causing resource depletion

2015-01-27 Thread Sven Krasser (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293843#comment-14293843 ] Sven Krasser commented on SPARK-5395: - I've definitely seen this behavior when adding

  1   2   >