[jira] [Updated] (SPARK-10699) Support checkpointInterval can be disabled

[ https://issues.apache.org/jira/browse/SPARK-10699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10699: -- Shepherd: Joseph K. Bradley Assignee: Yanbo Liang Target

[jira] [Commented] (SPARK-10732) Starting spark streaming from a specific point in time.

[ https://issues.apache.org/jira/browse/SPARK-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903092#comment-14903092 ] Bijay Singh Bisht commented on SPARK-10732: --- I think if a stream does not support replay the

[jira] [Updated] (SPARK-10750) ML Param validate should print better error information

[ https://issues.apache.org/jira/browse/SPARK-10750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10750: -- Shepherd: Joseph K. Bradley Assignee: Yanbo Liang Target

[jira] [Commented] (SPARK-7129) Add generic boosting algorithm to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903180#comment-14903180 ] Seth Hendrickson commented on SPARK-7129: - I had some time to give this topic some thought and

[jira] [Resolved] (SPARK-10485) IF expression is not correctly resolved when one of the options have NullType

[ https://issues.apache.org/jira/browse/SPARK-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-10485. -- Resolution: Fixed I tested on 1.5 and it seems fixed to me. Please reopen if you have

[jira] [Comment Edited] (SPARK-10732) Starting spark streaming from a specific point in time.

[ https://issues.apache.org/jira/browse/SPARK-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903231#comment-14903231 ] Cody Koeninger edited comment on SPARK-10732 at 9/22/15 7:02 PM: - Yeah,

[jira] [Commented] (SPARK-10759) Missing Python code example in ML Programming guide

[ https://issues.apache.org/jira/browse/SPARK-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903263#comment-14903263 ] Lauren Moos commented on SPARK-10759: - I can work on this > Missing Python code example in ML

[jira] [Updated] (SPARK-10740) handle nondeterministic expressions correctly for set operations

[ https://issues.apache.org/jira/browse/SPARK-10740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10740: - Priority: Blocker (was: Major) > handle nondeterministic expressions correctly for set operations >

[jira] [Updated] (SPARK-10740) handle nondeterministic expressions correctly for set operations

[ https://issues.apache.org/jira/browse/SPARK-10740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10740: - Target Version/s: 1.6.0, 1.5.1 > handle nondeterministic expressions correctly for set operations >

[jira] [Updated] (SPARK-10740) handle nondeterministic expressions correctly for set operations

[ https://issues.apache.org/jira/browse/SPARK-10740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10740: - Assignee: Wenchen Fan > handle nondeterministic expressions correctly for set operations >

[jira] [Resolved] (SPARK-10740) handle nondeterministic expressions correctly for set operations

[ https://issues.apache.org/jira/browse/SPARK-10740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10740. -- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

[jira] [Commented] (SPARK-9442) java.lang.ArithmeticException: / by zero when reading Parquet

[ https://issues.apache.org/jira/browse/SPARK-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903260#comment-14903260 ] Chris Heller commented on SPARK-9442: - Curious if the issue seen here was with a parquet file created

[jira] [Commented] (SPARK-10732) Starting spark streaming from a specific point in time.

[ https://issues.apache.org/jira/browse/SPARK-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903183#comment-14903183 ] Bijay Singh Bisht commented on SPARK-10732: --- I get it. Apparently there is a discussion to have

[jira] [Resolved] (SPARK-10704) Rename HashShufflereader to BlockStoreShuffleReader

[ https://issues.apache.org/jira/browse/SPARK-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10704. - Resolution: Fixed Fix Version/s: 1.6.0 > Rename HashShufflereader to

[jira] [Commented] (SPARK-10732) Starting spark streaming from a specific point in time.

[ https://issues.apache.org/jira/browse/SPARK-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903231#comment-14903231 ] Cody Koeninger commented on SPARK-10732: Yeah, even if that gets implemented it will likely be at

[jira] [Created] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

Timothy Chen created SPARK-10749: Summary: Support multiple roles with Spark Mesos dispatcher Key: SPARK-10749 URL: https://issues.apache.org/jira/browse/SPARK-10749 Project: Spark Issue

[jira] [Created] (SPARK-10748) Log error instead of crashing Spark Mesos dispatcher when a job is misconfigured

Timothy Chen created SPARK-10748: Summary: Log error instead of crashing Spark Mesos dispatcher when a job is misconfigured Key: SPARK-10748 URL: https://issues.apache.org/jira/browse/SPARK-10748

[jira] [Resolved] (SPARK-10419) Add JDBC dialect for Microsoft SQL Server

[ https://issues.apache.org/jira/browse/SPARK-10419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10419. - Resolution: Fixed Assignee: Ewan Leith Fix Version/s: 1.6.0 > Add JDBC dialect

[jira] [Updated] (SPARK-10458) Would like to know if a given Spark Context is stopped or currently stopping

[ https://issues.apache.org/jira/browse/SPARK-10458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10458: -- Assignee: Madhusudanan Kandasamy > Would like to know if a given Spark Context is stopped or currently

[jira] [Created] (SPARK-10750) ML Param validate should print better error information

Yanbo Liang created SPARK-10750: --- Summary: ML Param validate should print better error information Key: SPARK-10750 URL: https://issues.apache.org/jira/browse/SPARK-10750 Project: Spark Issue

[jira] [Commented] (SPARK-10750) ML Param validate should print better error information

[ https://issues.apache.org/jira/browse/SPARK-10750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902029#comment-14902029 ] Yanbo Liang commented on SPARK-10750: - This is because Param.validate(value: T) use value.toString at

[jira] [Commented] (SPARK-8386) DataFrame and JDBC regression

[ https://issues.apache.org/jira/browse/SPARK-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902042#comment-14902042 ] Reynold Xin commented on SPARK-8386: [~viirya] do you have time to take a look? > DataFrame and JDBC

[jira] [Updated] (SPARK-10649) Streaming jobs unexpectedly inherits job group, job descriptions from context starting thread

[ https://issues.apache.org/jira/browse/SPARK-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10649: -- Fix Version/s: 1.5.1 > Streaming jobs unexpectedly inherits job group, job descriptions from context

[jira] [Updated] (SPARK-10695) spark.mesos.constraints documentation uses "=" to separate value instead ":" as parser and mesos expects.

[ https://issues.apache.org/jira/browse/SPARK-10695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10695: -- Assignee: Akash Mishra > spark.mesos.constraints documentation uses "=" to separate value instead ":"

[jira] [Resolved] (SPARK-10695) spark.mesos.constraints documentation uses "=" to separate value instead ":" as parser and mesos expects.

[ https://issues.apache.org/jira/browse/SPARK-10695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10695. --- Resolution: Fixed Fix Version/s: 1.6.0 Target Version/s: 1.6.0, 1.5.1 (was: 1.5.1)

[jira] [Resolved] (SPARK-10458) Would like to know if a given Spark Context is stopped or currently stopping

[ https://issues.apache.org/jira/browse/SPARK-10458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10458. --- Resolution: Fixed Fix Version/s: 1.6.0 Target Version/s: 1.6.0 > Would like to know

[jira] [Updated] (SPARK-10695) spark.mesos.constraints documentation uses "=" to separate value instead ":" as parser and mesos expects.

[ https://issues.apache.org/jira/browse/SPARK-10695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10695: -- Component/s: Mesos > spark.mesos.constraints documentation uses "=" to separate value instead ":" >

[jira] [Resolved] (SPARK-9821) pyspark reduceByKey should allow a custom partitioner

[ https://issues.apache.org/jira/browse/SPARK-9821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9821. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8569

[jira] [Created] (SPARK-10751) ML Param validate should print better error information

Yanbo Liang created SPARK-10751: --- Summary: ML Param validate should print better error information Key: SPARK-10751 URL: https://issues.apache.org/jira/browse/SPARK-10751 Project: Spark Issue

[jira] [Resolved] (SPARK-10716) spark-1.5.0-bin-hadoop2.6.tgz file doesn't uncompress on OS X due to hidden file

[ https://issues.apache.org/jira/browse/SPARK-10716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10716. - Resolution: Fixed Assignee: Sean Owen Fix Version/s: 1.5.1

[jira] [Resolved] (SPARK-10577) [PySpark] DataFrame hint for broadcast join

[ https://issues.apache.org/jira/browse/SPARK-10577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10577. - Resolution: Fixed Fix Version/s: 1.6.0 > [PySpark] DataFrame hint for broadcast join >

[jira] [Commented] (SPARK-10577) [PySpark] DataFrame hint for broadcast join

[ https://issues.apache.org/jira/browse/SPARK-10577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902049#comment-14902049 ] Reynold Xin commented on SPARK-10577: - [~maver1ck] the patch is now merged - you can just create a

[jira] [Commented] (SPARK-10577) [PySpark] DataFrame hint for broadcast join

[ https://issues.apache.org/jira/browse/SPARK-10577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902073#comment-14902073 ] Maciej Bryński commented on SPARK-10577: [~rxin] As I wrote before. I already tested this patch

[jira] [Resolved] (SPARK-8567) Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-8567. -- Resolution: Fixed Target Version/s: 1.5.0, 1.4.1, 1.6.0 (was: 1.4.1, 1.5.0, 1.6.0) > Flaky

[jira] [Updated] (SPARK-8567) Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8567: - Fix Version/s: 1.5.1 1.6.0 > Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars >

[jira] [Closed] (SPARK-10751) ML Param validate should print better error information

[ https://issues.apache.org/jira/browse/SPARK-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang closed SPARK-10751. --- Resolution: Duplicate > ML Param validate should print better error information >

[jira] [Commented] (SPARK-10750) ML Param validate should print better error information

[ https://issues.apache.org/jira/browse/SPARK-10750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902044#comment-14902044 ] Apache Spark commented on SPARK-10750: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10750) ML Param validate should print better error information

[ https://issues.apache.org/jira/browse/SPARK-10750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10750: Assignee: (was: Apache Spark) > ML Param validate should print better error

[jira] [Assigned] (SPARK-10750) ML Param validate should print better error information

[ https://issues.apache.org/jira/browse/SPARK-10750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10750: Assignee: Apache Spark > ML Param validate should print better error information >

[jira] [Resolved] (SPARK-10446) Support to specify join type when calling join with usingColumns

[ https://issues.apache.org/jira/browse/SPARK-10446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10446. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 1.6.0 > Support to

[jira] [Updated] (SPARK-10324) MLlib 1.6 Roadmap

[ https://issues.apache.org/jira/browse/SPARK-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10324: -- Description: Following SPARK-8445, we created this master list for MLlib features we

[jira] [Commented] (SPARK-10734) DirectKafkaInputDStream uses the OffsetRequest.LatestTime to find the latest offset, however using the batch time would be more desireable.

[ https://issues.apache.org/jira/browse/SPARK-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903106#comment-14903106 ] Cody Koeninger commented on SPARK-10734: as I explained in SPARK-10732 , kafka's getOffsetsBefore

[jira] [Resolved] (SPARK-10756) DataFrame write to teradata using jdbc not working, tries to create table each time irrespective of table existence

[ https://issues.apache.org/jira/browse/SPARK-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10756. --- Resolution: Duplicate > DataFrame write to teradata using jdbc not working, tries to create table >

[jira] [Resolved] (SPARK-10750) ML Param validate should print better error information

[ https://issues.apache.org/jira/browse/SPARK-10750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-10750. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8863

[jira] [Comment Edited] (SPARK-5152) Let metrics.properties file take an hdfs:// path

[ https://issues.apache.org/jira/browse/SPARK-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902883#comment-14902883 ] Yongjia Wang edited comment on SPARK-5152 at 9/22/15 6:00 PM: -- I voted for

[jira] [Resolved] (SPARK-10593) sql lateral view same name gives wrong value

[ https://issues.apache.org/jira/browse/SPARK-10593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10593. -- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

[jira] [Commented] (SPARK-10739) Add attempt window for long running Spark application on Yarn

[ https://issues.apache.org/jira/browse/SPARK-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902937#comment-14902937 ] Marcelo Vanzin commented on SPARK-10739: Sean might be talking about SPARK-6735. > Add attempt

[jira] [Created] (SPARK-10760) SparkR glm: the documentation in examples - family argument is missing

Narine Kokhlikyan created SPARK-10760: - Summary: SparkR glm: the documentation in examples - family argument is missing Key: SPARK-10760 URL: https://issues.apache.org/jira/browse/SPARK-10760

[jira] [Assigned] (SPARK-10760) SparkR glm: the documentation in examples - family argument is missing

[ https://issues.apache.org/jira/browse/SPARK-10760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10760: Assignee: (was: Apache Spark) > SparkR glm: the documentation in examples - family

[jira] [Resolved] (SPARK-9585) HiveHBaseTableInputFormat can'be cached

[ https://issues.apache.org/jira/browse/SPARK-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9585. Resolution: Fixed Fix Version/s: 1.6.0 > HiveHBaseTableInputFormat can'be cached >

[jira] [Commented] (SPARK-10760) SparkR glm: the documentation in examples - family argument is missing

[ https://issues.apache.org/jira/browse/SPARK-10760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903113#comment-14903113 ] Apache Spark commented on SPARK-10760: -- User 'NarineK' has created a pull request for this issue:

[jira] [Commented] (SPARK-7129) Add generic boosting algorithm to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903136#comment-14903136 ] Joseph K. Bradley commented on SPARK-7129: -- It's not really on the roadmap for 1.6, so I

[jira] [Commented] (SPARK-10638) spark streaming stop gracefully keeps the spark context

[ https://issues.apache.org/jira/browse/SPARK-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903018#comment-14903018 ] Mamdouh Alramadan commented on SPARK-10638: --- Any updates on this issue? > spark streaming stop

[jira] [Commented] (SPARK-10691) Make LogisticRegressionModel.evaluate() method public

[ https://issues.apache.org/jira/browse/SPARK-10691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903070#comment-14903070 ] Joseph K. Bradley commented on SPARK-10691: --- We could document that `evaluate` calls

[jira] [Commented] (SPARK-10739) Add attempt window for long running Spark application on Yarn

[ https://issues.apache.org/jira/browse/SPARK-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902967#comment-14902967 ] Sean Owen commented on SPARK-10739: --- Yessir that's the one, thanks. [~jerryshao] it looks pretty

[jira] [Created] (SPARK-10759) Missing Python code example in ML Programming guide

Raela Wang created SPARK-10759: -- Summary: Missing Python code example in ML Programming guide Key: SPARK-10759 URL: https://issues.apache.org/jira/browse/SPARK-10759 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10739) Add attempt window for long running Spark application on Yarn

[ https://issues.apache.org/jira/browse/SPARK-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902982#comment-14902982 ] Sandy Ryza commented on SPARK-10739: That's the one I was referring to as well. That's about

[jira] [Commented] (SPARK-10739) Add attempt window for long running Spark application on Yarn

[ https://issues.apache.org/jira/browse/SPARK-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902996#comment-14902996 ] Saisai Shao commented on SPARK-10739: - Yes, as Sandy mentioned about, SPARK-6735 is focused on

[jira] [Resolved] (SPARK-9962) Decision Tree training: prevNodeIdsForInstances.unpersist() at end of training

[ https://issues.apache.org/jira/browse/SPARK-9962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-9962. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8541

[jira] [Updated] (SPARK-10729) word2vec model save for python

[ https://issues.apache.org/jira/browse/SPARK-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10729: -- Target Version/s: 1.6.0 > word2vec model save for python >

[jira] [Commented] (SPARK-10732) Starting spark streaming from a specific point in time.

[ https://issues.apache.org/jira/browse/SPARK-10732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903110#comment-14903110 ] Cody Koeninger commented on SPARK-10732: As I already said, kafka's implementation of

[jira] [Assigned] (SPARK-10760) SparkR glm: the documentation in examples - family argument is missing

[ https://issues.apache.org/jira/browse/SPARK-10760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10760: Assignee: Apache Spark > SparkR glm: the documentation in examples - family argument is

[jira] [Commented] (SPARK-10756) DataFrame write to teradata using jdbc not working, tries to create table each time irrespective of table existence

[ https://issues.apache.org/jira/browse/SPARK-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903100#comment-14903100 ] Suresh Thalamati commented on SPARK-10756: -- This issue is similar to

[jira] [Updated] (SPARK-10381) Infinite loop when OutputCommitCoordination is enabled and OutputCommitter.commitTask throws exception

[ https://issues.apache.org/jira/browse/SPARK-10381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10381: --- Fix Version/s: 1.3.2 > Infinite loop when OutputCommitCoordination is enabled and >

[jira] [Updated] (SPARK-8447) Test external shuffle service with all shuffle managers

[ https://issues.apache.org/jira/browse/SPARK-8447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-8447: -- Target Version/s: 1.6.0 (was: 1.5.1) > Test external shuffle service with all shuffle managers >

[jira] [Assigned] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

[ https://issues.apache.org/jira/browse/SPARK-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10749: Assignee: (was: Apache Spark) > Support multiple roles with Spark Mesos dispatcher >

[jira] [Commented] (SPARK-8418) Add single- and multi-value support to ML Transformers

[ https://issues.apache.org/jira/browse/SPARK-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903350#comment-14903350 ] Joseph K. Bradley commented on SPARK-8418: -- New idea: We could allow transformers to leverage

[jira] [Updated] (SPARK-6701) Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application

[ https://issues.apache.org/jira/browse/SPARK-6701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6701: -- Target Version/s: (was: 1.5.1) > Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application >

[jira] [Updated] (SPARK-7420) Flaky test: o.a.s.streaming.JobGeneratorSuite "Do not clear received block data too soon"

[ https://issues.apache.org/jira/browse/SPARK-7420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-7420: -- Target Version/s: (was: 1.5.1) > Flaky test: o.a.s.streaming.JobGeneratorSuite "Do not clear received

[jira] [Updated] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

[ https://issues.apache.org/jira/browse/SPARK-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6484: -- Target Version/s: (was: 1.5.1) I'm going to untarget this from 1.5.1 because, as far as I know, this

[jira] [Commented] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903512#comment-14903512 ] Dan Brown commented on SPARK-10685: --- Thanks for fixing the python udf part of the issue! What about

[jira] [Commented] (SPARK-10663) Change test.toDF to test in Spark ML Programming Guide

[ https://issues.apache.org/jira/browse/SPARK-10663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903749#comment-14903749 ] Apache Spark commented on SPARK-10663: -- User 'hagenhaus' has created a pull request for this issue:

[jira] [Updated] (SPARK-10705) Stop converting internal rows to external rows in DataFrame.toJSON

[ https://issues.apache.org/jira/browse/SPARK-10705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10705: --- Assignee: Liang-Chi Hsieh > Stop converting internal rows to external rows in DataFrame.toJSON >

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903740#comment-14903740 ] Yi Zhou commented on SPARK-10733: - yes. i still got error after applying the commit. >

[jira] [Assigned] (SPARK-10663) Change test.toDF to test in Spark ML Programming Guide

[ https://issues.apache.org/jira/browse/SPARK-10663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10663: Assignee: (was: Apache Spark) > Change test.toDF to test in Spark ML Programming

[jira] [Commented] (SPARK-7442) Spark 1.3.1 / Hadoop 2.6 prebuilt pacakge has broken S3 filesystem access

[ https://issues.apache.org/jira/browse/SPARK-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903750#comment-14903750 ] Amey Ghadigaonkar commented on SPARK-7442: -- Getting the same error with Spark 1.4.1 running on

[jira] [Assigned] (SPARK-10663) Change test.toDF to test in Spark ML Programming Guide

[ https://issues.apache.org/jira/browse/SPARK-10663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10663: Assignee: Apache Spark > Change test.toDF to test in Spark ML Programming Guide >

[jira] [Commented] (SPARK-10731) The head() implementation of dataframe is very slow

[ https://issues.apache.org/jira/browse/SPARK-10731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903818#comment-14903818 ] Apache Spark commented on SPARK-10731: -- User 'rxin' has created a pull request for this issue:

[jira] [Updated] (SPARK-8882) A New Receiver Scheduling Mechanism to solve unbalanced receivers

[ https://issues.apache.org/jira/browse/SPARK-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-8882: - Summary: A New Receiver Scheduling Mechanism to solve unbalanced receivers (was: A New Receiver

[jira] [Updated] (SPARK-8882) A New Receiver Scheduling Mechanism

[ https://issues.apache.org/jira/browse/SPARK-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-8882: - Description: There are some problems in the current mechanism: - If a task fails more than

[jira] [Resolved] (SPARK-10672) We should not fail to create a table If we cannot persist metadata of a data source table to metastore in a Hive compatible way

[ https://issues.apache.org/jira/browse/SPARK-10672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10672. -- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-10737) When using UnsafeRows, SortMergeJoin may return wrong results

[ https://issues.apache.org/jira/browse/SPARK-10737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-10737. -- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

[jira] [Commented] (SPARK-10761) Refactor DiskBlockObjectWriter to not require BlockId

[ https://issues.apache.org/jira/browse/SPARK-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903450#comment-14903450 ] Apache Spark commented on SPARK-10761: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Created] (SPARK-10761) Refactor DiskBlockObjectWriter to not require BlockId

Josh Rosen created SPARK-10761: -- Summary: Refactor DiskBlockObjectWriter to not require BlockId Key: SPARK-10761 URL: https://issues.apache.org/jira/browse/SPARK-10761 Project: Spark Issue

[jira] [Commented] (SPARK-10333) Add user guide for linear-methods.md columns

[ https://issues.apache.org/jira/browse/SPARK-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903448#comment-14903448 ] Lauren Moos commented on SPARK-10333: - I'd be happy to work on this > Add user guide for

[jira] [Updated] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10685: --- Assignee: Reynold Xin > Misaligned data with RDD.zip and DataFrame.withColumn after repartition >

[jira] [Resolved] (SPARK-10685) Misaligned data with RDD.zip and DataFrame.withColumn after repartition

[ https://issues.apache.org/jira/browse/SPARK-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-10685. Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull

[jira] [Commented] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

[ https://issues.apache.org/jira/browse/SPARK-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903474#comment-14903474 ] Apache Spark commented on SPARK-10749: -- User 'tnachen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

[ https://issues.apache.org/jira/browse/SPARK-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10749: Assignee: Apache Spark > Support multiple roles with Spark Mesos dispatcher >

[jira] [Created] (SPARK-10762) GenericRowWithSchema exception in casting ArrayBuffer to HashSet in DataFrame to RDD from Hive table

Glenn Strycker created SPARK-10762: -- Summary: GenericRowWithSchema exception in casting ArrayBuffer to HashSet in DataFrame to RDD from Hive table Key: SPARK-10762 URL:

[jira] [Created] (SPARK-10763) Update Java MLLIB/ML tests to use simplified dataframe construction

holdenk created SPARK-10763: --- Summary: Update Java MLLIB/ML tests to use simplified dataframe construction Key: SPARK-10763 URL: https://issues.apache.org/jira/browse/SPARK-10763 Project: Spark

[jira] [Commented] (SPARK-2737) ClassCastExceptions when collect()ing JavaRDDs' underlying Scala RDDs

[ https://issues.apache.org/jira/browse/SPARK-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903542#comment-14903542 ] Glenn Strycker commented on SPARK-2737: --- I am getting a similar error in Spark 1.3.0... see a new

[jira] [Commented] (SPARK-10409) Multilayer perceptron regression

[ https://issues.apache.org/jira/browse/SPARK-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903480#comment-14903480 ] Xiangrui Meng commented on SPARK-10409: --- [~lmoos] This is a major feature. Could you work on

[jira] [Commented] (SPARK-10607) Scheduler should include defensive measures against infinite loops due to task commit denial

[ https://issues.apache.org/jira/browse/SPARK-10607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903476#comment-14903476 ] Josh Rosen commented on SPARK-10607: Retargeting; this enhancement doesn't need to be targeted at a

[jira] [Updated] (SPARK-10607) Scheduler should include defensive measures against infinite loops due to task commit denial

[ https://issues.apache.org/jira/browse/SPARK-10607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-10607: --- Target Version/s: (was: 1.3.2, 1.4.2, 1.5.1) > Scheduler should include defensive measures against

[jira] [Updated] (SPARK-10333) Add user guide for linear-methods.md columns

[ https://issues.apache.org/jira/browse/SPARK-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10333: -- Assignee: Lauren Moos > Add user guide for linear-methods.md columns >

[jira] [Updated] (SPARK-10759) Missing Python code example in ML Programming guide

[ https://issues.apache.org/jira/browse/SPARK-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10759: -- Assignee: Lauren Moos > Missing Python code example in ML Programming guide >

[jira] [Commented] (SPARK-10409) Multilayer perceptron regression

[ https://issues.apache.org/jira/browse/SPARK-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903553#comment-14903553 ] Lauren Moos commented on SPARK-10409: - no problem! > Multilayer perceptron regression >

[jira] [Resolved] (SPARK-10714) Refactor PythonRDD to decouple iterator computation from PythonRDD

[ https://issues.apache.org/jira/browse/SPARK-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-10714. Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull

[jira] [Resolved] (SPARK-8632) Poor Python UDF performance because of RDD caching

[ https://issues.apache.org/jira/browse/SPARK-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-8632. --- Resolution: Fixed Fix Version/s: 1.5.1 1.6.0 Issue resolved by pull request

  1   2   >