[jira] [Updated] (SPARK-10646) Bivariate Statistics: Pearson's Chi-Squared Test for categorical vs. categorical

2015-09-16 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihong MA updated SPARK-10646: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-10385 > Bivariate Statistics: Pearson's

[jira] [Created] (SPARK-10646) Bivariate Statistics: Pearson's Chi-Squared Test for categorical vs. categorical

2015-09-16 Thread Jihong MA (JIRA)
Jihong MA created SPARK-10646: - Summary: Bivariate Statistics: Pearson's Chi-Squared Test for categorical vs. categorical Key: SPARK-10646 URL: https://issues.apache.org/jira/browse/SPARK-10646 Project:

[jira] [Commented] (SPARK-10320) Kafka Support new topic subscriptions without requiring restart of the streaming context

2015-09-16 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14790843#comment-14790843 ] Cody Koeninger commented on SPARK-10320: I don't think there's much benefit to multiple dstreams

[jira] [Resolved] (SPARK-1304) Job fails with spot instances (due to IllegalStateException: Shutdown in progress)

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-1304. --- Resolution: Won't Fix > Job fails with spot instances (due to IllegalStateException: Shutdown in >

[jira] [Resolved] (SPARK-3603) InvalidClassException on a Linux VM - probably problem with serialization

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3603. --- Resolution: Cannot Reproduce Resolving as "Cannot Reproduce", since this is an old issue that hasn't

[jira] [Resolved] (SPARK-3949) Use IAMRole in lieu of static access key-id/secret

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3949. --- Resolution: Incomplete As of SPARK-8576, spark-ec2 can launch instances with IAM instance roles. I'm

[jira] [Updated] (SPARK-10602) Univariate statistics as UDAFs: single-pass continuous stats

2015-09-16 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10602: -- Assignee: Seth Hendrickson > Univariate statistics as UDAFs: single-pass continuous

[jira] [Commented] (SPARK-10642) Crash in rdd.lookup() with "java.lang.Long cannot be cast to java.lang.Integer"

2015-09-16 Thread Thouis Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14790884#comment-14790884 ] Thouis Jones commented on SPARK-10642: -- Simpler cases. Fails: {code} sc.parallelize([(('a', 'b'),

[jira] [Resolved] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname="0.0.0.0" so driver can be located behind NAT

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4389. --- Resolution: Won't Fix I'm going to resolve this as "Won't Fix": - Akka 2.4 has not shipped yet, so

[jira] [Updated] (SPARK-10645) Bivariate Statistics for continuous vs. continuous

2015-09-16 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihong MA updated SPARK-10645: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-10385 > Bivariate Statistics for

[jira] [Updated] (SPARK-10646) Bivariate Statistics: Pearson's Chi-Squared Test for categorical vs. categorical

2015-09-16 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihong MA updated SPARK-10646: -- Description: Pearson's chi-squared goodness of fit test for observed against the expected

[jira] [Updated] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10650: Description: In 1.5.0 there are some extra classes in the Spark docs - including a bunch

[jira] [Updated] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10650: Priority: Critical (was: Major) > Spark docs include test and other extra classes >

[jira] [Commented] (SPARK-10626) Create a Java friendly method for randomRDD & RandomDataGenerator on RandomRDDs.

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791128#comment-14791128 ] Apache Spark commented on SPARK-10626: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10626) Create a Java friendly method for randomRDD & RandomDataGenerator on RandomRDDs.

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10626: Assignee: (was: Apache Spark) > Create a Java friendly method for randomRDD &

[jira] [Updated] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10650: - Assignee: Michael Armbrust (was: Andrew Or) > Spark docs include test and other extra

[jira] [Created] (SPARK-10647) Rename property spark.deploy.zookeeper.dir to spark.mesos.deploy.zookeeper.dir

2015-09-16 Thread Alan Braithwaite (JIRA)
Alan Braithwaite created SPARK-10647: Summary: Rename property spark.deploy.zookeeper.dir to spark.mesos.deploy.zookeeper.dir Key: SPARK-10647 URL: https://issues.apache.org/jira/browse/SPARK-10647

[jira] [Resolved] (SPARK-6086) Exceptions in DAGScheduler.updateAccumulators

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6086. --- Resolution: Cannot Reproduce Resolving as "cannot reproduce" for now, pending updates. > Exceptions

[jira] [Updated] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10650: Affects Version/s: 1.5.0 > Spark docs include test and other extra classes >

[jira] [Created] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-10650: --- Summary: Spark docs include test and other extra classes Key: SPARK-10650 URL: https://issues.apache.org/jira/browse/SPARK-10650 Project: Spark Issue

[jira] [Commented] (SPARK-3978) Schema change on Spark-Hive (Parquet file format) table not working

2015-09-16 Thread Alex Rovner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791150#comment-14791150 ] Alex Rovner commented on SPARK-3978: [~barge.nilesh] What version of Spark have you tested with? >

[jira] [Commented] (SPARK-3978) Schema change on Spark-Hive (Parquet file format) table not working

2015-09-16 Thread Nilesh Barge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791141#comment-14791141 ] Nilesh Barge commented on SPARK-3978: - Thanks for resolving this, I also verified on my end and now it

[jira] [Updated] (SPARK-10645) Bivariate Statistics for continuous vs. continuous

2015-09-16 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihong MA updated SPARK-10645: -- Component/s: SQL ML > Bivariate Statistics for continuous vs. continuous >

[jira] [Assigned] (SPARK-10623) turning on predicate pushdown throws nonsuch element exception when RDD is empty

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10623: Assignee: Zhan Zhang (was: Apache Spark) > turning on predicate pushdown throws nonsuch

[jira] [Updated] (SPARK-10647) Rename property spark.deploy.zookeeper.dir to spark.mesos.deploy.zookeeper.dir

2015-09-16 Thread Alan Braithwaite (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Braithwaite updated SPARK-10647: - Issue Type: Improvement (was: New Feature) > Rename property spark.deploy.zookeeper.dir

[jira] [Updated] (SPARK-10050) Support collecting data of MapType in DataFrame

2015-09-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-10050: -- Assignee: Sun Rui > Support collecting data of MapType in DataFrame >

[jira] [Created] (SPARK-10649) Streaming jobs unexpectedly inherits job group, job descriptions from context starting thread

2015-09-16 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-10649: - Summary: Streaming jobs unexpectedly inherits job group, job descriptions from context starting thread Key: SPARK-10649 URL: https://issues.apache.org/jira/browse/SPARK-10649

[jira] [Commented] (SPARK-3978) Schema change on Spark-Hive (Parquet file format) table not working

2015-09-16 Thread Nilesh Barge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791171#comment-14791171 ] Nilesh Barge commented on SPARK-3978: - I tested with the latest Spark 1.5 release... I got the source

[jira] [Updated] (SPARK-9794) ISO DateTime parser is too strict

2015-09-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9794: --- Assignee: Kevin Cox > ISO DateTime parser is too strict > - > >

[jira] [Resolved] (SPARK-9794) ISO DateTime parser is too strict

2015-09-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9794. Resolution: Fixed Fix Version/s: 1.6.0 > ISO DateTime parser is too strict >

[jira] [Resolved] (SPARK-10050) Support collecting data of MapType in DataFrame

2015-09-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-10050. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request

[jira] [Updated] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-10650: Target Version/s: 1.5.1 > Spark docs include test and other extra classes >

[jira] [Resolved] (SPARK-6513) Add zipWithUniqueId (and other RDD APIs) to RDDApi

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6513. --- Resolution: Won't Fix > Add zipWithUniqueId (and other RDD APIs) to RDDApi >

[jira] [Commented] (SPARK-7841) Spark build should not use lib_managed for dependencies

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791063#comment-14791063 ] Josh Rosen commented on SPARK-7841: --- I agree that we can probably fix this, but note that we'll have to

[jira] [Resolved] (SPARK-6504) Cannot read Parquet files generated from different versions at once

2015-09-16 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6504. - Resolution: Fixed Fix Version/s: 1.3.1 This should be fixed. Please reopen if you

[jira] [Assigned] (SPARK-10648) Spark-SQL JDBC fails to set a default precision and scale when they are not defined in an oracle schema.

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10648: Assignee: Apache Spark > Spark-SQL JDBC fails to set a default precision and scale when

[jira] [Commented] (SPARK-10648) Spark-SQL JDBC fails to set a default precision and scale when they are not defined in an oracle schema.

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791081#comment-14791081 ] Apache Spark commented on SPARK-10648: -- User 'travishegner' has created a pull request for this

[jira] [Assigned] (SPARK-10648) Spark-SQL JDBC fails to set a default precision and scale when they are not defined in an oracle schema.

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10648: Assignee: (was: Apache Spark) > Spark-SQL JDBC fails to set a default precision and

[jira] [Assigned] (SPARK-10649) Streaming jobs unexpectedly inherits job group, job descriptions from context starting thread

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10649: Assignee: Tathagata Das (was: Apache Spark) > Streaming jobs unexpectedly inherits job

[jira] [Commented] (SPARK-10649) Streaming jobs unexpectedly inherits job group, job descriptions from context starting thread

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791109#comment-14791109 ] Apache Spark commented on SPARK-10649: -- User 'tdas' has created a pull request for this issue:

[jira] [Updated] (SPARK-10646) Bivariate Statistics: Pearson's Chi-Squared Test for categorical vs. categorical

2015-09-16 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jihong MA updated SPARK-10646: -- Component/s: SQL ML > Bivariate Statistics: Pearson's Chi-Squared Test for

[jira] [Assigned] (SPARK-10626) Create a Java friendly method for randomRDD & RandomDataGenerator on RandomRDDs.

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10626: Assignee: Apache Spark > Create a Java friendly method for randomRDD &

[jira] [Created] (SPARK-10648) Spark-SQL JDBC fails to set a default precision and scale when they are not defined in an oracle schema.

2015-09-16 Thread Travis Hegner (JIRA)
Travis Hegner created SPARK-10648: - Summary: Spark-SQL JDBC fails to set a default precision and scale when they are not defined in an oracle schema. Key: SPARK-10648 URL:

[jira] [Assigned] (SPARK-10649) Streaming jobs unexpectedly inherits job group, job descriptions from context starting thread

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10649: Assignee: Apache Spark (was: Tathagata Das) > Streaming jobs unexpectedly inherits job

[jira] [Commented] (SPARK-10623) turning on predicate pushdown throws nonsuch element exception when RDD is empty

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791179#comment-14791179 ] Apache Spark commented on SPARK-10623: -- User 'zhzhan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10623) turning on predicate pushdown throws nonsuch element exception when RDD is empty

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10623: Assignee: Apache Spark (was: Zhan Zhang) > turning on predicate pushdown throws nonsuch

[jira] [Updated] (SPARK-10623) turning on predicate pushdown throws nonsuch element exception when RDD is empty

2015-09-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-10623: Target Version/s: 1.6.0, 1.5.1 > turning on predicate pushdown throws nonsuch element exception

[jira] [Updated] (SPARK-10651) Flaky test: BroadcastSuite

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10651: -- Description: Saw many failures recently in master build. See attached CSV for a full list.

[jira] [Commented] (SPARK-10381) Infinite loop when OutputCommitCoordination is enabled and OutputCommitter.commitTask throws exception

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791354#comment-14791354 ] Apache Spark commented on SPARK-10381: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-10644) Applications wait even if free executors are available

2015-09-16 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791370#comment-14791370 ] Saisai Shao commented on SPARK-10644: - Does you jobs have dependencies? That is saying the 4th job

[jira] [Commented] (SPARK-10653) Remove unnecessary things from SparkEnv

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791388#comment-14791388 ] Josh Rosen commented on SPARK-10653: Note that SparkEnv is technically a developer API, but all of

[jira] [Updated] (SPARK-10647) Mesos HA mode misuses spark.deploy.zookeeper.dir property; configs should be documented

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10647: --- Affects Version/s: 1.4.1 1.5.0 > Mesos HA mode misuses

[jira] [Commented] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

2015-09-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791438#comment-14791438 ] Thomas Graves commented on SPARK-10640: --- yes 1.5 history server reading 1.5.0 logs. I'm not as

[jira] [Comment Edited] (SPARK-10644) Applications wait even if free executors are available

2015-09-16 Thread Balagopal Nair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791434#comment-14791434 ] Balagopal Nair edited comment on SPARK-10644 at 9/17/15 1:51 AM: - No.

[jira] [Commented] (SPARK-10625) Spark SQL JDBC read/write is unable to handle JDBC Drivers that adds unserializable objects into connection properties

2015-09-16 Thread Peng Cheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791252#comment-14791252 ] Peng Cheng commented on SPARK-10625: A pull request has been send that contains 2 extra unit tests

[jira] [Updated] (SPARK-10651) Flaky test: BroadcastSuite

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10651: -- Labels: flaky-test (was: ) > Flaky test: BroadcastSuite > -- > >

[jira] [Updated] (SPARK-10649) Streaming jobs unexpectedly inherits job group, job descriptions from context starting thread

2015-09-16 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-10649: -- Description: The job group, and job descriptions information is passed through thread local

[jira] [Commented] (SPARK-10381) Infinite loop when OutputCommitCoordination is enabled and OutputCommitter.commitTask throws exception

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791347#comment-14791347 ] Apache Spark commented on SPARK-10381: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-10653) Remove unnecessary things from SparkEnv

2015-09-16 Thread Andrew Or (JIRA)
Andrew Or created SPARK-10653: - Summary: Remove unnecessary things from SparkEnv Key: SPARK-10653 URL: https://issues.apache.org/jira/browse/SPARK-10653 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-10652) Set good job descriptions for streaming related jobs

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10652: Assignee: Tathagata Das (was: Apache Spark) > Set good job descriptions for streaming

[jira] [Assigned] (SPARK-10652) Set good job descriptions for streaming related jobs

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10652: Assignee: Apache Spark (was: Tathagata Das) > Set good job descriptions for streaming

[jira] [Commented] (SPARK-10652) Set good job descriptions for streaming related jobs

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791357#comment-14791357 ] Apache Spark commented on SPARK-10652: -- User 'tdas' has created a pull request for this issue:

[jira] [Commented] (SPARK-10635) pyspark - running on a different host

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791389#comment-14791389 ] Josh Rosen commented on SPARK-10635: [~davies], do you think we should support this? This seems like

[jira] [Commented] (SPARK-10644) Applications wait even if free executors are available

2015-09-16 Thread Balagopal Nair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791434#comment-14791434 ] Balagopal Nair commented on SPARK-10644: No. These are independent jobs running under different

[jira] [Updated] (SPARK-10651) Flaky test: BroadcastSuite

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10651: -- Attachment: BroadcastSuiteFailures.csv > Flaky test: BroadcastSuite >

[jira] [Updated] (SPARK-10651) Flaky test: BroadcastSuite

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10651: -- Description: Saw many failures recently in master build. See attached CSV for a full list.

[jira] [Created] (SPARK-10651) Flaky test: BroadcastSuite

2015-09-16 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-10651: - Summary: Flaky test: BroadcastSuite Key: SPARK-10651 URL: https://issues.apache.org/jira/browse/SPARK-10651 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-10058) Flaky test: HeartbeatReceiverSuite: normal heartbeat

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10058: -- Component/s: Tests > Flaky test: HeartbeatReceiverSuite: normal heartbeat >

[jira] [Assigned] (SPARK-10639) Need to convert UDAF's result from scala to sql type

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10639: Assignee: Apache Spark > Need to convert UDAF's result from scala to sql type >

[jira] [Commented] (SPARK-10639) Need to convert UDAF's result from scala to sql type

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791302#comment-14791302 ] Apache Spark commented on SPARK-10639: -- User 'yhuai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10639) Need to convert UDAF's result from scala to sql type

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10639: Assignee: (was: Apache Spark) > Need to convert UDAF's result from scala to sql type

[jira] [Updated] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10640: --- Affects Version/s: 1.3.0 1.4.0 Target Version/s: 1.5.1

[jira] [Commented] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791416#comment-14791416 ] Josh Rosen commented on SPARK-10640: This is a 1.5.0 history server reading 1.5.0 logs? In principle

[jira] [Created] (SPARK-10655) Enhance DB2 dialect to handle XML, and DECIMAL , and DECFLOAT

2015-09-16 Thread Suresh Thalamati (JIRA)
Suresh Thalamati created SPARK-10655: Summary: Enhance DB2 dialect to handle XML, and DECIMAL , and DECFLOAT Key: SPARK-10655 URL: https://issues.apache.org/jira/browse/SPARK-10655 Project: Spark

[jira] [Commented] (SPARK-10655) Enhance DB2 dialect to handle XML, and DECIMAL , and DECFLOAT

2015-09-16 Thread Suresh Thalamati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791440#comment-14791440 ] Suresh Thalamati commented on SPARK-10655: -- I am working on pull request for this issue. >

[jira] [Commented] (SPARK-10644) Applications wait even if free executors are available

2015-09-16 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791439#comment-14791439 ] Saisai Shao commented on SPARK-10644: - So what's the cluster manager you use, standalone, mesos or

[jira] [Updated] (SPARK-10058) Flaky test: HeartbeatReceiverSuite: normal heartbeat

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10058: -- Priority: Blocker (was: Critical) > Flaky test: HeartbeatReceiverSuite: normal heartbeat >

[jira] [Assigned] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10650: Assignee: Apache Spark (was: Michael Armbrust) > Spark docs include test and other extra

[jira] [Commented] (SPARK-10058) Flaky test: HeartbeatReceiverSuite: normal heartbeat

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791286#comment-14791286 ] Xiangrui Meng commented on SPARK-10058: --- Changed the priority to Blocker since this failed master

[jira] [Assigned] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10650: Assignee: Michael Armbrust (was: Apache Spark) > Spark docs include test and other extra

[jira] [Commented] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791285#comment-14791285 ] Apache Spark commented on SPARK-10650: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Updated] (SPARK-10058) Flaky test: HeartbeatReceiverSuite: normal heartbeat

2015-09-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10058: -- Issue Type: Bug (was: Test) > Flaky test: HeartbeatReceiverSuite: normal heartbeat >

[jira] [Commented] (SPARK-10654) Add columnSimilarities to IndexedRowMatrix

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791425#comment-14791425 ] Apache Spark commented on SPARK-10654: -- User 'rezazadeh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10654) Add columnSimilarities to IndexedRowMatrix

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10654: Assignee: Apache Spark > Add columnSimilarities to IndexedRowMatrix >

[jira] [Commented] (SPARK-10644) Applications wait even if free executors are available

2015-09-16 Thread Balagopal Nair (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791443#comment-14791443 ] Balagopal Nair commented on SPARK-10644: Standalone cluster manager. I've verified this behaviour

[jira] [Updated] (SPARK-10647) Mesos HA mode misuses spark.deploy.zookeeper.dir property; configs should be documented

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10647: --- Issue Type: Bug (was: Improvement) > Mesos HA mode misuses spark.deploy.zookeeper.dir property;

[jira] [Updated] (SPARK-10647) Mesos HA mode misuses spark.deploy.zookeeper.dir property; configs should be documented

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10647: --- Summary: Mesos HA mode misuses spark.deploy.zookeeper.dir property; configs should be documented

[jira] [Updated] (SPARK-10647) Rename property spark.deploy.zookeeper.dir to spark.mesos.deploy.zookeeper.dir

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10647: --- Component/s: Mesos > Rename property spark.deploy.zookeeper.dir to spark.mesos.deploy.zookeeper.dir

[jira] [Commented] (SPARK-10647) Rename property spark.deploy.zookeeper.dir to spark.mesos.deploy.zookeeper.dir

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791387#comment-14791387 ] Josh Rosen commented on SPARK-10647: The spark.deploy.zookeeper.* properties are used by the

[jira] [Updated] (SPARK-10647) Rename property spark.deploy.zookeeper.dir to spark.mesos.deploy.zookeeper.dir

2015-09-16 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10647: --- Assignee: Timothy Chen > Rename property spark.deploy.zookeeper.dir to

[jira] [Commented] (SPARK-10625) Spark SQL JDBC read/write is unable to handle JDBC Drivers that adds unserializable objects into connection properties

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791420#comment-14791420 ] Apache Spark commented on SPARK-10625: -- User 'tribbloid' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10625) Spark SQL JDBC read/write is unable to handle JDBC Drivers that adds unserializable objects into connection properties

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10625: Assignee: (was: Apache Spark) > Spark SQL JDBC read/write is unable to handle JDBC

[jira] [Created] (SPARK-10654) Add columnSimilarities to IndexedRowMatrix

2015-09-16 Thread Reza Zadeh (JIRA)
Reza Zadeh created SPARK-10654: -- Summary: Add columnSimilarities to IndexedRowMatrix Key: SPARK-10654 URL: https://issues.apache.org/jira/browse/SPARK-10654 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-10625) Spark SQL JDBC read/write is unable to handle JDBC Drivers that adds unserializable objects into connection properties

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10625: Assignee: Apache Spark > Spark SQL JDBC read/write is unable to handle JDBC Drivers that

[jira] [Updated] (SPARK-10650) Spark docs include test and other extra classes

2015-09-16 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10650: -- Target Version/s: 1.6.0, 1.5.1 (was: 1.5.1) > Spark docs include test and other extra classes >

[jira] [Created] (SPARK-10652) Set good job descriptions for streaming related jobs

2015-09-16 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-10652: - Summary: Set good job descriptions for streaming related jobs Key: SPARK-10652 URL: https://issues.apache.org/jira/browse/SPARK-10652 Project: Spark Issue

[jira] [Assigned] (SPARK-10654) Add columnSimilarities to IndexedRowMatrix

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10654: Assignee: (was: Apache Spark) > Add columnSimilarities to IndexedRowMatrix >

[jira] [Created] (SPARK-10656) select(df(*)) fails when a column has special characters

2015-09-16 Thread Nick Pritchard (JIRA)
Nick Pritchard created SPARK-10656: -- Summary: select(df(*)) fails when a column has special characters Key: SPARK-10656 URL: https://issues.apache.org/jira/browse/SPARK-10656 Project: Spark

[jira] [Commented] (SPARK-10657) Remove legacy SCP-based Jenkins log archiving code

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791549#comment-14791549 ] Apache Spark commented on SPARK-10657: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10657) Remove legacy SCP-based Jenkins log archiving code

2015-09-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10657: Assignee: Apache Spark (was: Josh Rosen) > Remove legacy SCP-based Jenkins log archiving

<    1   2   3   >