[jira] [Updated] (SPARK-3461) Support external groupByKey using repartitionAndSortWithinPartitions

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3461: --- Description: Given that we have SPARK-2978, it seems like we could support an external group

[jira] [Updated] (SPARK-3461) Support external groupBy and groupByKey using repartitionAndSortWithinPartitions

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3461: --- Description: Given that we have SPARK-2978, it seems like we could support an external group

[jira] [Updated] (SPARK-3461) Support external groupByKey using repartitionAndSortWithinPartitions

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3461: --- Summary: Support external groupByKey using repartitionAndSortWithinPartitions (was: Support e

[jira] [Commented] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160037#comment-14160037 ] Apache Spark commented on SPARK-3731: - User 'davies' has created a pull request for th

[jira] [Updated] (SPARK-3461) Support external groupBy and groupByKey using repartitionAndSortWithinPartitions

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3461: --- Summary: Support external groupBy and groupByKey using repartitionAndSortWithinPartitions (wa

[jira] [Updated] (SPARK-3461) Support external groupBy

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3461: --- Assignee: Davies Liu (was: Patrick Wendell) > Support external groupBy >

[jira] [Issue Comment Deleted] (SPARK-3034) [HIve] java.sql.Date cannot be cast to java.sql.Timestamp

2014-10-05 Thread Venkata Ramana G (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Ramana G updated SPARK-3034: Comment: was deleted (was: Requires adding another data type DateType Modifications required

[jira] [Commented] (SPARK-3568) Add metrics for ranking algorithms

2014-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160022#comment-14160022 ] Apache Spark commented on SPARK-3568: - User 'coderxiang' has created a pull request fo

[jira] [Updated] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2331: --- Component/s: Spark Core > SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] > ---

[jira] [Updated] (SPARK-3694) Allow printing object graph of tasks/RDD's with a debug flag

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3694: --- Issue Type: Improvement (was: Bug) > Allow printing object graph of tasks/RDD's with a debug

[jira] [Updated] (SPARK-3782) Direct use of log4j in AkkaUtils interferes with certain logging configurations

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3782: --- Component/s: Spark Core > Direct use of log4j in AkkaUtils interferes with certain logging >

[jira] [Updated] (SPARK-3694) Allow printing object graph of tasks/RDD's with a debug flag

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3694: --- Component/s: Spark Core > Allow printing object graph of tasks/RDD's with a debug flag > -

[jira] [Updated] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2331: --- Summary: SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] (was: SparkContext.emptyR

[jira] [Updated] (SPARK-2885) All-pairs similarity via DIMSUM

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2885: --- Component/s: MLlib > All-pairs similarity via DIMSUM > --- > >

[jira] [Updated] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3732: --- Component/s: YARN > Yarn Client: Add option to NOT System.exit() at end of main() > --

[jira] [Updated] (SPARK-3742) SparkUI page unable to open

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3742: --- Component/s: (was: Spark Core) YARN > SparkUI page unable to open > -

[jira] [Updated] (SPARK-3742) Link to Spark UI sometimes fails when using H/A RM's

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3742: --- Summary: Link to Spark UI sometimes fails when using H/A RM's (was: SparkUI page unable to op

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3731: --- Assignee: Davies Liu > RDD caching stops working in pyspark after some time >

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3731: --- Target Version/s: 1.1.1, 1.2.0 > RDD caching stops working in pyspark after some time > --

[jira] [Updated] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3731: --- Priority: Critical (was: Major) > RDD caching stops working in pyspark after some time >

[jira] [Commented] (SPARK-3731) RDD caching stops working in pyspark after some time

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160002#comment-14160002 ] Patrick Wendell commented on SPARK-3731: [~davies] any chance you can take a look

[jira] [Commented] (SPARK-3034) [HIve] java.sql.Date cannot be cast to java.sql.Timestamp

2014-10-05 Thread Venkata Ramana G (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159998#comment-14159998 ] Venkata Ramana G commented on SPARK-3034: - Requires adding another data type DateT

[jira] [Updated] (SPARK-3765) Add test information to sbt build docs

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3765: --- Summary: Add test information to sbt build docs (was: add testing with sbt to doc) > Add tes

[jira] [Updated] (SPARK-3174) Provide elastic scaling within a Spark application

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3174: --- Component/s: YARN Spark Core > Provide elastic scaling within a Spark applica

[jira] [Updated] (SPARK-3805) Enable Standalone worker cleanup by default

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3805: --- Component/s: Deploy > Enable Standalone worker cleanup by default > --

[jira] [Updated] (SPARK-1190) Do not initialize log4j if slf4j log4j backend is not being used

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1190: --- Assignee: Patrick Wendell (was: Patrick Cogan) > Do not initialize log4j if slf4j log4j backe

[jira] [Updated] (SPARK-3694) Allow printing object graph of tasks/RDD's with a debug flag

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3694: --- Assignee: (was: Patrick Wendell) > Allow printing object graph of tasks/RDD's with a debug

[jira] [Resolved] (SPARK-3314) Script creation of AMIs

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3314. Resolution: Fixed Assignee: Patrick Wendell So I think the original goal of this JIRA

[jira] [Updated] (SPARK-3782) Direct use of log4j in AkkaUtils interferes with certain logging configurations

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3782: --- Summary: Direct use of log4j in AkkaUtils interferes with certain logging configurations (wa

[jira] [Commented] (SPARK-3782) AkkaUtils directly using log4j

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159846#comment-14159846 ] Patrick Wendell commented on SPARK-3782: Would you mind pasting the exact exceptio

[jira] [Resolved] (SPARK-3783) The type parameters for SparkContext.accumulable are inconsistent Accumulable itself

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3783. Resolution: Fixed Fix Version/s: 1.2.0 https://github.com/apache/spark/pull/2637 > T

[jira] [Updated] (SPARK-3783) The type parameters for SparkContext.accumulable are inconsistent Accumulable itself

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3783: --- Assignee: Nathan Kronenfeld > The type parameters for SparkContext.accumulable are inconsisten

[jira] [Updated] (SPARK-3806) minor bug in CliSuite

2014-10-05 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangfei updated SPARK-3806: --- Summary: minor bug in CliSuite (was: minor bug and exception in CliSuite) > minor bug in CliSuite > -

[jira] [Updated] (SPARK-3761) Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3761: --- Priority: Critical (was: Major) > Class anonfun$1 not found exception / sbt 13.x / Scala 2.10

[jira] [Updated] (SPARK-3761) Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3761: --- Component/s: Spark Core > Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4 >

[jira] [Updated] (SPARK-3761) Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.4

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3761: --- Priority: Major (was: Blocker) > Class anonfun$1 not found exception / sbt 13.x / Scala 2.10.

[jira] [Commented] (SPARK-3806) minor bug and exception in CliSuite

2014-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159842#comment-14159842 ] Apache Spark commented on SPARK-3806: - User 'scwf' has created a pull request for this

[jira] [Comment Edited] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159840#comment-14159840 ] Patrick Wendell edited comment on SPARK-3561 at 10/6/14 3:55 AM: ---

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159840#comment-14159840 ] Patrick Wendell commented on SPARK-3561: I also changed the title here that reflec

[jira] [Updated] (SPARK-3561) Allow for pluggable execution contexts in Spark

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3561: --- Summary: Allow for pluggable execution contexts in Spark (was: Native Hadoop/YARN integration

[jira] [Commented] (SPARK-3561) Native Hadoop/YARN integration for batch/ETL workloads.

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159838#comment-14159838 ] Patrick Wendell commented on SPARK-3561: [~ozhurakousky] do you have a timeline fo

[jira] [Commented] (SPARK-3805) Enable Standalone worker cleanup by default

2014-10-05 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159837#comment-14159837 ] Patrick Wendell commented on SPARK-3805: I commented on the PR - but because this

[jira] [Created] (SPARK-3806) minor bug and exception in CliSuite

2014-10-05 Thread wangfei (JIRA)
wangfei created SPARK-3806: -- Summary: minor bug and exception in CliSuite Key: SPARK-3806 URL: https://issues.apache.org/jira/browse/SPARK-3806 Project: Spark Issue Type: Bug Components: S

[jira] [Updated] (SPARK-3568) Add metrics for ranking algorithms

2014-10-05 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-3568: -- Description: Include common metrics for ranking algorithms(http://www-nlp.stanford.edu/IR-book/), inclu

[jira] [Updated] (SPARK-3568) Add metrics for ranking algorithms

2014-10-05 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-3568: -- Description: Include common metrics for ranking algorithms (http://www-nlp.stanford.edu/IR-book/), incl

[jira] [Updated] (SPARK-3568) Add metrics for ranking algorithms

2014-10-05 Thread Shuo Xiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-3568: -- Description: Include widely-used metrics for ranking algorithms, including: - Mean Average Precision -

[jira] [Resolved] (SPARK-3794) Building spark core fails due to inadvertent dependency on Commons IO

2014-10-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3794. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2662 [https:/

[jira] [Resolved] (SPARK-3776) Wrong conversion to Catalyst for Option[Product]

2014-10-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3776. - Resolution: Fixed Issue resolved by pull request 2641 [https://github.com/apache/spark/pul

[jira] [Resolved] (SPARK-3645) Make caching using SQL commands eager by default, with the option of being lazy

2014-10-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3645. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2513 [https:/

[jira] [Resolved] (SPARK-3792) enable JavaHiveQLSuite

2014-10-05 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-3792. - Resolution: Fixed Fix Version/s: 1.2.0 Issue resolved by pull request 2652 [https:/

[jira] [Updated] (SPARK-3369) Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3369: -- Labels: breaking_change (was: ) > Java mapPartitions Iterator->Iterable is inconsistent with Scala's >

[jira] [Commented] (SPARK-3755) Do not bind port 1 - 1024 to server in spark

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159737#comment-14159737 ] Andrew Ash commented on SPARK-3755: --- I think you can only edit the title and description

[jira] [Updated] (SPARK-3166) Custom serialisers can't be shipped in application jars

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3166: -- Target Version/s: 1.2.0 > Custom serialisers can't be shipped in application jars >

[jira] [Commented] (SPARK-3464) Graceful decommission of executors

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159712#comment-14159712 ] Andrew Ash commented on SPARK-3464: --- [~pwendell] what JIRA replaced this one? > Gracefu

[jira] [Commented] (SPARK-3805) Enable Standalone worker cleanup by default

2014-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159711#comment-14159711 ] Apache Spark commented on SPARK-3805: - User 'ash211' has created a pull request for th

[jira] [Created] (SPARK-3805) Enable Standalone worker cleanup by default

2014-10-05 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-3805: - Summary: Enable Standalone worker cleanup by default Key: SPARK-3805 URL: https://issues.apache.org/jira/browse/SPARK-3805 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159707#comment-14159707 ] Andrew Ash commented on SPARK-1860: --- Filed as SPARK-3805 > Standalone Worker cleanup sh

[jira] [Created] (SPARK-3804) Output of Generator expressions is not stable after serialization.

2014-10-05 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-3804: --- Summary: Output of Generator expressions is not stable after serialization. Key: SPARK-3804 URL: https://issues.apache.org/jira/browse/SPARK-3804 Project: Spark

[jira] [Commented] (SPARK-2445) MesosExecutorBackend crashes in fine grained mode

2014-10-05 Thread Brenden Matthews (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159602#comment-14159602 ] Brenden Matthews commented on SPARK-2445: - [~drexin] can you confirm that https://

[jira] [Comment Edited] (SPARK-2445) MesosExecutorBackend crashes in fine grained mode

2014-10-05 Thread Brenden Matthews (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159602#comment-14159602 ] Brenden Matthews edited comment on SPARK-2445 at 10/5/14 5:45 PM: --

[jira] [Commented] (SPARK-3561) Native Hadoop/YARN integration for batch/ETL workloads.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159593#comment-14159593 ] Oleg Zhurakousky commented on SPARK-3561: - After giving it some thought, I am chan

[jira] [Commented] (SPARK-3597) MesosSchedulerBackend does not implement `killTask`

2014-10-05 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159591#comment-14159591 ] Andrew Or commented on SPARK-3597: -- Fixed by https://github.com/apache/spark/pull/2453 >

[jira] [Updated] (SPARK-3561) Native Hadoop/YARN integration for batch/ETL workloads.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Summary: Native Hadoop/YARN integration for batch/ETL workloads. (was: Expose pluggable arc

[jira] [Closed] (SPARK-3597) MesosSchedulerBackend does not implement `killTask`

2014-10-05 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3597. Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.1.1, 1.2.0 > MesosSchedulerBacke

[jira] [Updated] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Description: Currently Spark provides integration with external resource-managers such as A

[jira] [Updated] (SPARK-3597) MesosSchedulerBackend does not implement `killTask`

2014-10-05 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3597: - Assignee: Brenden Matthews > MesosSchedulerBackend does not implement `killTask` > ---

[jira] [Commented] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159587#comment-14159587 ] Sean Owen commented on SPARK-3803: -- I agree with your assessment. It would take some work

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masaru Dobashi updated SPARK-3803: -- Description: When I executed computePrincipalComponents method of RowMatrix, I got java.lang.Ar

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masaru Dobashi updated SPARK-3803: -- Description: GrWhen I executed computePrincipalComponents method of RowMatrix, I got java.lang.

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masaru Dobashi updated SPARK-3803: -- Description: When I executed computePrincipalComponents method of RowMatrix, I got java.lang.Ar

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masaru Dobashi updated SPARK-3803: -- Description: GrWhen I executed computePrincipalComponents method of RowMatrix, I got java.lang.

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masaru Dobashi updated SPARK-3803: -- Description: When I executed computePrincipalComponents method of RowMatrix, I got java.lang.Ar

[jira] [Updated] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Description: Currently Spark _integrates with external resource-managing platforms_ such as

[jira] [Updated] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Description: Currently Spark _integrates with external resource-managing platforms_ such as

[jira] [Commented] (SPARK-3007) Add "Dynamic Partition" support to Spark Sql hive

2014-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159578#comment-14159578 ] Apache Spark commented on SPARK-3007: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159573#comment-14159573 ] Sean Owen commented on SPARK-3561: -- I'd be interested to see a more specific motivating u

[jira] [Commented] (SPARK-3801) Make app dir cleanup more efficient

2014-10-05 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159561#comment-14159561 ] Liang-Chi Hsieh commented on SPARK-3801: I think so. Thanks. > Make app dir clean

[jira] [Closed] (SPARK-3801) Make app dir cleanup more efficient

2014-10-05 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh closed SPARK-3801. -- Resolution: Duplicate > Make app dir cleanup more efficient > --

[jira] [Comment Edited] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159559#comment-14159559 ] Oleg Zhurakousky edited comment on SPARK-3561 at 10/5/14 3:26 PM: --

[jira] [Commented] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159559#comment-14159559 ] Oleg Zhurakousky commented on SPARK-3561: - Sandy, one other thing: While I underst

[jira] [Updated] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Description: Currently Spark _integrates with external resource-managing platforms_ such as

[jira] [Updated] (SPARK-3561) Expose pluggable architecture to facilitate native integration with third-party execution environments.

2014-10-05 Thread Oleg Zhurakousky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated SPARK-3561: Summary: Expose pluggable architecture to facilitate native integration with third-party exe

[jira] [Commented] (SPARK-3794) Building spark core fails due to inadvertent dependency on Commons IO

2014-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159538#comment-14159538 ] Apache Spark commented on SPARK-3794: - User 'srowen' has created a pull request for th

[jira] [Updated] (SPARK-3794) Building spark core fails due to inadvertent dependency on Commons IO

2014-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-3794: - Summary: Building spark core fails due to inadvertent dependency on Commons IO (was: Building spark core

[jira] [Commented] (SPARK-3801) Make app dir cleanup more efficient

2014-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159536#comment-14159536 ] Sean Owen commented on SPARK-3801: -- I think this change will have to be subsumed in a sli

[jira] [Commented] (SPARK-3794) Building spark core fails with specific hadoop version

2014-10-05 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159524#comment-14159524 ] Sean Owen commented on SPARK-3794: -- The real problem here is that commons-io is not a dep

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masaru Dobashi updated SPARK-3803: -- Description: When I executed computePrincipalComponents method of RowMatrix, I got java.lang.Ar

[jira] [Updated] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masaru Dobashi updated SPARK-3803: -- Description: When I executed computePrincipalComponents method of RowMatrix, I got java.lang.Ar

[jira] [Created] (SPARK-3803) ArrayIndexOutOfBoundsException found in executing computePrincipalComponents

2014-10-05 Thread Masaru Dobashi (JIRA)
Masaru Dobashi created SPARK-3803: - Summary: ArrayIndexOutOfBoundsException found in executing computePrincipalComponents Key: SPARK-3803 URL: https://issues.apache.org/jira/browse/SPARK-3803 Project:

[jira] [Commented] (SPARK-3802) Scala version is wrong in dev/audit-release/blank_sbt_build/build.sbt

2014-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14159501#comment-14159501 ] Apache Spark commented on SPARK-3802: - User 'sarutak' has created a pull request for t

[jira] [Created] (SPARK-3802) Scala version is wrong in dev/audit-release/blank_sbt_build/build.sbt

2014-10-05 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3802: - Summary: Scala version is wrong in dev/audit-release/blank_sbt_build/build.sbt Key: SPARK-3802 URL: https://issues.apache.org/jira/browse/SPARK-3802 Project: Spark