[jira] [Updated] (SPARK-16762) spark hanging when action method print

2016-07-27 Thread Anh Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anh Nguyen updated SPARK-16762: --- Attachment: Screen Shot 2016-07-28 at 12.34.22 PM.png > spark hanging when action method print >

[jira] [Updated] (SPARK-16762) spark hanging when action method print

2016-07-27 Thread Anh Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anh Nguyen updated SPARK-16762: --- Attachment: Screen Shot 2016-07-28 at 12.33.35 PM.png > spark hanging when action method print >

[jira] [Updated] (SPARK-16762) spark hanging when action method print

2016-07-27 Thread Anh Nguyen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anh Nguyen updated SPARK-16762: --- Description: I write code spark Streaming (consumer) intergate kafaka and deploy on mesos FW: import

[jira] [Created] (SPARK-16762) spark hanging when action method print

2016-07-27 Thread Anh Nguyen (JIRA)
Anh Nguyen created SPARK-16762: -- Summary: spark hanging when action method print Key: SPARK-16762 URL: https://issues.apache.org/jira/browse/SPARK-16762 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-16761) Fix doc link in docs/ml-guide.md

2016-07-27 Thread Dapeng Sun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dapeng Sun updated SPARK-16761: --- Description: Fix the invalid link at http://spark.apache.org/docs/latest/ml-guide.html > Fix doc

[jira] [Assigned] (SPARK-16761) Fix doc link in docs/ml-guide.md

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16761: Assignee: (was: Apache Spark) > Fix doc link in docs/ml-guide.md >

[jira] [Assigned] (SPARK-16761) Fix doc link in docs/ml-guide.md

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16761: Assignee: Apache Spark > Fix doc link in docs/ml-guide.md >

[jira] [Commented] (SPARK-16761) Fix doc link in docs/ml-guide.md

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396865#comment-15396865 ] Apache Spark commented on SPARK-16761: -- User 'sundapeng' has created a pull request for this issue:

[jira] [Created] (SPARK-16761) Fix doc link in docs/ml-guide.md

2016-07-27 Thread Dapeng Sun (JIRA)
Dapeng Sun created SPARK-16761: -- Summary: Fix doc link in docs/ml-guide.md Key: SPARK-16761 URL: https://issues.apache.org/jira/browse/SPARK-16761 Project: Spark Issue Type: Documentation

[jira] [Assigned] (SPARK-16312) Docs for Kafka 0.10 consumer integration

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16312: Assignee: (was: Apache Spark) > Docs for Kafka 0.10 consumer integration >

[jira] [Commented] (SPARK-16312) Docs for Kafka 0.10 consumer integration

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396833#comment-15396833 ] Apache Spark commented on SPARK-16312: -- User 'koeninger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16312) Docs for Kafka 0.10 consumer integration

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16312: Assignee: Apache Spark > Docs for Kafka 0.10 consumer integration >

[jira] [Commented] (SPARK-16085) Spark stand-alone ui redirects to RM application master UI for yarn-client mode

2016-07-27 Thread Lve Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396791#comment-15396791 ] Lve Chen commented on SPARK-16085: -- Thanks for the info~ I'm right now getting the driver node by "yarn

[jira] [Commented] (SPARK-15857) Add Caller Context in Spark

2016-07-27 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396677#comment-15396677 ] Weiqing Yang commented on SPARK-15857: -- That single PR is for subtasks 1, 2 and 4. Based on the

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396651#comment-15396651 ] Sean Owen commented on SPARK-15541: --- What's your fix -- avoiding directly calling this method? >

[jira] [Resolved] (SPARK-16450) Build failes for Mesos 0.28.x

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16450. --- Resolution: Duplicate This is subsumed by the containerizer change, which updates for Mesos 1.x >

[jira] [Updated] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16751: -- Priority: Minor (was: Critical) We can do this, sure, but why is it Critical? tests have zero surface

[jira] [Commented] (SPARK-15857) Add Caller Context in Spark

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396638#comment-15396638 ] Sean Owen commented on SPARK-15857: --- How about elaborating with a design doc before making all these

[jira] [Commented] (SPARK-15857) Add Caller Context in Spark

2016-07-27 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396583#comment-15396583 ] Weiqing Yang commented on SPARK-15857: -- To make review easier, subtasks have been created. > Add

[jira] [Assigned] (SPARK-16443) ALS wrapper in SparkR

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16443: Assignee: Junyang Qian (was: Apache Spark) > ALS wrapper in SparkR >

[jira] [Commented] (SPARK-16443) ALS wrapper in SparkR

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396580#comment-15396580 ] Apache Spark commented on SPARK-16443: -- User 'junyangq' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16443) ALS wrapper in SparkR

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16443: Assignee: Apache Spark (was: Junyang Qian) > ALS wrapper in SparkR >

[jira] [Created] (SPARK-16760) Pass 'jobId' to Task

2016-07-27 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-16760: Summary: Pass 'jobId' to Task Key: SPARK-16760 URL: https://issues.apache.org/jira/browse/SPARK-16760 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16759) Spark expose an API to pass in Caller Context into it

2016-07-27 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-16759: Summary: Spark expose an API to pass in Caller Context into it Key: SPARK-16759 URL: https://issues.apache.org/jira/browse/SPARK-16759 Project: Spark Issue

[jira] [Created] (SPARK-16758) Set up caller context to YARN

2016-07-27 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-16758: Summary: Set up caller context to YARN Key: SPARK-16758 URL: https://issues.apache.org/jira/browse/SPARK-16758 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-16757) Set up caller context to HDFS

2016-07-27 Thread Weiqing Yang (JIRA)
Weiqing Yang created SPARK-16757: Summary: Set up caller context to HDFS Key: SPARK-16757 URL: https://issues.apache.org/jira/browse/SPARK-16757 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-16365) Ideas for moving "mllib-local" forward

2016-07-27 Thread Jay White Bear (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396480#comment-15396480 ] Jay White Bear commented on SPARK-16365: The discussion is about adding a full blown linear

[jira] [Updated] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Roberts updated SPARK-16751: - Description: This JIRA is to upgrade the derby version from 10.11.1.1 to 10.12.1.1 Sean and I

[jira] [Resolved] (SPARK-16110) Can't set Python via spark-submit for YARN cluster mode when PYSPARK_PYTHON & PYSPARK_DRIVER_PYTHON are set

2016-07-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16110. Resolution: Fixed Assignee: Kevin Grealish Fix Version/s: 2.1.0 > Can't

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396341#comment-15396341 ] Reynold Xin commented on SPARK-16756: - That makes sense. > Add `sql` function to LogicalPlan and

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396340#comment-15396340 ] Dongjoon Hyun commented on SPARK-16756: --- If there is anything you have in mind, please give me

[jira] [Comment Edited] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396337#comment-15396337 ] Dongjoon Hyun edited comment on SPARK-16756 at 7/27/16 8:45 PM: Until

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396337#comment-15396337 ] Dongjoon Hyun commented on SPARK-16756: --- Until now, there are no flags. I'm trying to not use any

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396331#comment-15396331 ] Reynold Xin commented on SPARK-16756: - This makes sense. Are there flags we might want to pass into

[jira] [Commented] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396321#comment-15396321 ] Dongjoon Hyun commented on SPARK-16756: --- Hi, [~rxin]. So far, I've been working in this direction.

[jira] [Updated] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16756: -- Description: Like `Expression`, this issue adds `sql` function for `LogicalPlan` and

[jira] [Created] (SPARK-16756) Add `sql` function to LogicalPlan and `NonSQLPlan` trait

2016-07-27 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-16756: - Summary: Add `sql` function to LogicalPlan and `NonSQLPlan` trait Key: SPARK-16756 URL: https://issues.apache.org/jira/browse/SPARK-16756 Project: Spark

[jira] [Assigned] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15232: Assignee: Apache Spark > Add subquery SQL building tests to LogicalPlanToSQLSuite >

[jira] [Assigned] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15232: Assignee: (was: Apache Spark) > Add subquery SQL building tests to

[jira] [Commented] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396283#comment-15396283 ] Apache Spark commented on SPARK-15232: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-16450) Build failes for Mesos 0.28.x

2016-07-27 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396236#comment-15396236 ] Michael Gummelt commented on SPARK-16450: - I'll update soon. Though this pending PR updates to

[jira] [Resolved] (SPARK-16755) Enable external shuffle service with Mesos Docker Containers

2016-07-27 Thread Niels Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niels Becker resolved SPARK-16755. -- Resolution: Not A Problem > Enable external shuffle service with Mesos Docker Containers >

[jira] [Created] (SPARK-16755) Enable external shuffle service with Mesos Docker Containers

2016-07-27 Thread Niels Becker (JIRA)
Niels Becker created SPARK-16755: Summary: Enable external shuffle service with Mesos Docker Containers Key: SPARK-16755 URL: https://issues.apache.org/jira/browse/SPARK-16755 Project: Spark

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396209#comment-15396209 ] Jurriaan Pruis commented on SPARK-16753: It's not only more memory, they also take up a lot more

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396192#comment-15396192 ] Reynold Xin commented on SPARK-16753: - Yea I understand some tasks are using more memory due to data

[jira] [Updated] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-16754: - Priority: Minor (was: Major) > NPE when defining case class and searching Encoder in the same

[jira] [Updated] (SPARK-15675) Implicit resolution doesn't work in multiple Statements in Spark Repl

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-15675: - Priority: Major (was: Minor) > Implicit resolution doesn't work in multiple Statements in Spark

[jira] [Updated] (SPARK-15675) Implicit resolution doesn't work in multiple Statements in Spark Repl

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-15675: - Priority: Minor (was: Major) > Implicit resolution doesn't work in multiple Statements in Spark

[jira] [Assigned] (SPARK-16673) New Executor Page displays columns that used to be conditionally hidden

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16673: Assignee: (was: Apache Spark) > New Executor Page displays columns that used to be

[jira] [Commented] (SPARK-16673) New Executor Page displays columns that used to be conditionally hidden

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396099#comment-15396099 ] Apache Spark commented on SPARK-16673: -- User 'ajbozarth' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16673) New Executor Page displays columns that used to be conditionally hidden

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16673: Assignee: Apache Spark > New Executor Page displays columns that used to be conditionally

[jira] [Commented] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396085#comment-15396085 ] Shixiong Zhu commented on SPARK-16754: -- Perhaps we should inject

[jira] [Commented] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396082#comment-15396082 ] Shixiong Zhu commented on SPARK-16754: -- The issue is because OuterScopes.getOuterScope assumes

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396078#comment-15396078 ] Jurriaan Pruis commented on SPARK-16753: This is looks like a skew problem to me since the tasks

[jira] [Created] (SPARK-16754) NPE when defining case class and searching Encoder in the same line

2016-07-27 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-16754: Summary: NPE when defining case class and searching Encoder in the same line Key: SPARK-16754 URL: https://issues.apache.org/jira/browse/SPARK-16754 Project: Spark

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396067#comment-15396067 ] Reynold Xin commented on SPARK-16753: - Is this a problem with skew, or a problem with GC? Have you

[jira] [Commented] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396053#comment-15396053 ] Jurriaan Pruis commented on SPARK-16753: [~rxin] Do you know something about this? I've seen a

[jira] [Created] (SPARK-16753) Spark SQL doesn't handle skewed dataset joins properly

2016-07-27 Thread Jurriaan Pruis (JIRA)
Jurriaan Pruis created SPARK-16753: -- Summary: Spark SQL doesn't handle skewed dataset joins properly Key: SPARK-16753 URL: https://issues.apache.org/jira/browse/SPARK-16753 Project: Spark

[jira] [Commented] (SPARK-16450) Build failes for Mesos 0.28.x

2016-07-27 Thread Niels Becker (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396032#comment-15396032 ] Niels Becker commented on SPARK-16450: -- Its out > Build failes for Mesos 0.28.x >

[jira] [Resolved] (SPARK-5847) Allow for configuring MetricsSystem's use of app ID to namespace all metrics

2016-07-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-5847. --- Resolution: Fixed Assignee: Mark Grover Fix Version/s: 2.1.0 > Allow for

[jira] [Issue Comment Deleted] (SPARK-15872) Dataset of Array of Custom case class throws MissingRequirementError

2016-07-27 Thread Matthew Gwynne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Gwynne updated SPARK-15872: --- Comment: was deleted (was: I have had what appears to be exactly the same issue in 1.6.2.

[jira] [Commented] (SPARK-15872) Dataset of Array of Custom case class throws MissingRequirementError

2016-07-27 Thread Matthew Gwynne (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395938#comment-15395938 ] Matthew Gwynne commented on SPARK-15872: I have had what appears to be exactly the same issue in

[jira] [Resolved] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-16752. Resolution: Invalid This is not the place to report bugs about the SJS, which is unrelated

[jira] [Comment Edited] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395907#comment-15395907 ] Ash Pran edited comment on SPARK-16752 at 7/27/16 4:11 PM: --- Please see the

[jira] [Updated] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Pran updated SPARK-16752: - Description: We are having a strange issue with Spark Job Server (SJS) We are using SJS 0.6.1 and Spark

[jira] [Updated] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Pran updated SPARK-16752: - Attachment: SJS_JOBS_RUNNING SJS_JOB_LOG_CONSOLE SJS_JOB_COMP_YARN

[jira] [Created] (SPARK-16752) Spark Job Server not releasing jobs from "running list" even after yarn completes the job

2016-07-27 Thread Ash Pran (JIRA)
Ash Pran created SPARK-16752: Summary: Spark Job Server not releasing jobs from "running list" even after yarn completes the job Key: SPARK-16752 URL: https://issues.apache.org/jira/browse/SPARK-16752

[jira] [Updated] (SPARK-16729) Spark should throw analysis exception for invalid casts to date type

2016-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16729: Fix Version/s: 2.0.1 > Spark should throw analysis exception for invalid casts to date type >

[jira] [Commented] (SPARK-15541) SparkContext.stop throws error

2016-07-27 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395815#comment-15395815 ] Maciej BryƄski commented on SPARK-15541: [~srowen] I have patch that resolve this issue. Do you

[jira] [Commented] (SPARK-16085) Spark stand-alone ui redirects to RM application master UI for yarn-client mode

2016-07-27 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395705#comment-15395705 ] Saisai Shao commented on SPARK-16085: - Unfortunately, there's no such configuration for Spark to

[jira] [Updated] (SPARK-16726) Improve `Union/Intersect/Except` error messages on incompatible types

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16726: -- Description: Currently, `UNION/INTERSECT/EXCEPT` query on incompatible types shows a

[jira] [Updated] (SPARK-15254) Improve ML pipeline Cross Validation Scaladoc & PyDoc

2016-07-27 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-15254: --- Assignee: Krishna Kalyan > Improve ML pipeline Cross Validation Scaladoc & PyDoc >

[jira] [Resolved] (SPARK-15254) Improve ML pipeline Cross Validation Scaladoc & PyDoc

2016-07-27 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-15254. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13894

[jira] [Closed] (SPARK-13106) KafkaUtils.createDirectStream method with messageHandler and topics

2016-07-27 Thread Luis Alves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luis Alves closed SPARK-13106. -- Resolution: Duplicate > KafkaUtils.createDirectStream method with messageHandler and topics >

[jira] [Commented] (SPARK-16485) Additional fixes to Mllib 2.0 documentation

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395675#comment-15395675 ] Apache Spark commented on SPARK-16485: -- User 'lins05' has created a pull request for this issue:

[jira] [Commented] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395617#comment-15395617 ] Apache Spark commented on SPARK-16751: -- User 'a-roberts' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16751: Assignee: (was: Apache Spark) > Upgrade derby to 10.12.1.1 from 10.11.1.1 >

[jira] [Assigned] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16751: Assignee: Apache Spark > Upgrade derby to 10.12.1.1 from 10.11.1.1 >

[jira] [Created] (SPARK-16751) Upgrade derby to 10.12.1.1 from 10.11.1.1

2016-07-27 Thread Adam Roberts (JIRA)
Adam Roberts created SPARK-16751: Summary: Upgrade derby to 10.12.1.1 from 10.11.1.1 Key: SPARK-16751 URL: https://issues.apache.org/jira/browse/SPARK-16751 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16744) spark.yarn.appMasterEnv handling assumes values should be appended

2016-07-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395611#comment-15395611 ] Thomas Graves commented on SPARK-16744: --- Along with this we may consider thinking about an admin

[jira] [Commented] (SPARK-16745) Spark job completed however have to wait for 13 mins (data size is small)

2016-07-27 Thread Joe Chong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395567#comment-15395567 ] Joe Chong commented on SPARK-16745: --- Thanks for the reply. This is a standalone Spark instantiation.

[jira] [Commented] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395520#comment-15395520 ] Dongjoon Hyun commented on SPARK-15232: --- Thank you! > Add subquery SQL building tests to

[jira] [Commented] (SPARK-15581) MLlib 2.1 Roadmap

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395489#comment-15395489 ] Alexander Ulanov commented on SPARK-15581: -- [~bordaw] sounds great! Just in case, I have

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Commented] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395434#comment-15395434 ] Alexander Ulanov commented on SPARK-9120: - Thanks for the comment, RegressionModel does not extend

[jira] [Updated] (SPARK-9120) Add multivariate regression (or prediction) interface

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-9120: Description: org.apache.spark.ml.regression.RegressionModel supports prediction only for a

[jira] [Commented] (SPARK-16606) Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect."

2016-07-27 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395420#comment-15395420 ] Jacek Laskowski commented on SPARK-16606: - Haven't really started to be honest. I'd like to work

[jira] [Commented] (SPARK-10627) Regularization for artificial neural networks

2016-07-27 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395379#comment-15395379 ] Alexander Ulanov commented on SPARK-10627: -- [~RubenJanssen] These are major features. Could you

[jira] [Commented] (SPARK-16606) Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect."

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395378#comment-15395378 ] Sean Owen commented on SPARK-16606: --- [~ja...@japila.pl] are you working on this or should I zap it? >

[jira] [Resolved] (SPARK-15817) Spark client picking hive 1.2.1 by default which failed to alter a table name

2016-07-27 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15817. --- Resolution: Not A Problem > Spark client picking hive 1.2.1 by default which failed to alter a table

[jira] [Comment Edited] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-27 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395354#comment-15395354 ] Hongyao Zhao edited comment on SPARK-16746 at 7/27/16 10:06 AM: I think

[jira] [Commented] (SPARK-16746) Spark streaming lost data when ReceiverTracker writes Blockinfo to hdfs timeout

2016-07-27 Thread Hongyao Zhao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395354#comment-15395354 ] Hongyao Zhao commented on SPARK-16746: -- I think if spark.streaming.receiver.writeAheadLog.enable is

[jira] [Assigned] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16750: Assignee: Yanbo Liang (was: Apache Spark) > ML GaussianMixture training failed due to

[jira] [Assigned] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

2016-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16750: Assignee: Apache Spark (was: Yanbo Liang) > ML GaussianMixture training failed due to

  1   2   >