[jira] [Resolved] (SPARK-2067) Spark logo in application UI uses absolute path

[ https://issues.apache.org/jira/browse/SPARK-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2067. Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 Issue resolved by

[jira] [Updated] (SPARK-2067) Spark logo in application UI uses absolute path

[ https://issues.apache.org/jira/browse/SPARK-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2067: --- Assignee: Neville Li Spark logo in application UI uses absolute path

[jira] [Created] (SPARK-2077) Log serializer in use on application startup

Andrew Ash created SPARK-2077: - Summary: Log serializer in use on application startup Key: SPARK-2077 URL: https://issues.apache.org/jira/browse/SPARK-2077 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2077) Log serializer in use on application startup

[ https://issues.apache.org/jira/browse/SPARK-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021670#comment-14021670 ] Andrew Ash commented on SPARK-2077: --- https://github.com/apache/spark/pull/1017 Log

[jira] [Commented] (SPARK-2078) Use ISO8601 date formats in logging

[ https://issues.apache.org/jira/browse/SPARK-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021709#comment-14021709 ] Andrew Ash commented on SPARK-2078: --- https://github.com/apache/spark/pull/1018 Use

[jira] [Created] (SPARK-2078) Use ISO8601 date formats in logging

Andrew Ash created SPARK-2078: - Summary: Use ISO8601 date formats in logging Key: SPARK-2078 URL: https://issues.apache.org/jira/browse/SPARK-2078 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2044) Pluggable interface for shuffles

[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021716#comment-14021716 ] Weihua Jiang commented on SPARK-2044: - Hi Matei, Thanks a lot for your reply. 1. I

[jira] [Commented] (SPARK-2044) Pluggable interface for shuffles

[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021714#comment-14021714 ] Weihua Jiang commented on SPARK-2044: - Hi Matei, Thanks a lot for your reply. 1. I

[jira] [Resolved] (SPARK-1308) Add partitions() method to PySpark RDDs

[ https://issues.apache.org/jira/browse/SPARK-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-1308. Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Reynold Xin Add partitions()

[jira] [Updated] (SPARK-1308) Add getNumPartitions() method to PySpark RDDs

[ https://issues.apache.org/jira/browse/SPARK-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1308: --- Assignee: Syed A. Hashmi (was: Reynold Xin) Add getNumPartitions() method to PySpark RDDs

[jira] [Comment Edited] (SPARK-2044) Pluggable interface for shuffles

[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021716#comment-14021716 ] Weihua Jiang edited comment on SPARK-2044 at 6/9/14 7:10 AM: -

[jira] [Updated] (SPARK-1308) Add getNumPartitions() method to PySpark RDDs

[ https://issues.apache.org/jira/browse/SPARK-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-1308: --- Summary: Add getNumPartitions() method to PySpark RDDs (was: Add partitions() method to PySpark

[jira] [Comment Edited] (SPARK-2044) Pluggable interface for shuffles

[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021716#comment-14021716 ] Weihua Jiang edited comment on SPARK-2044 at 6/9/14 7:11 AM: -

[jira] [Commented] (SPARK-1944) Document --verbose in spark-shell -h

[ https://issues.apache.org/jira/browse/SPARK-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021726#comment-14021726 ] Andrew Ash commented on SPARK-1944: --- https://github.com/apache/spark/pull/1020

[jira] [Commented] (SPARK-1977) mutable.BitSet in ALS not serializable with KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14021797#comment-14021797 ] Santiago M. Mola commented on SPARK-1977: - Xiangrui Meng, I can't reproduce it at

[jira] [Updated] (SPARK-1719) spark.executor.extraLibraryPath isn't applied on yarn

[ https://issues.apache.org/jira/browse/SPARK-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-1719: --- Fix Version/s: 1.1.0 spark.executor.extraLibraryPath isn't applied on yarn

[jira] [Commented] (SPARK-2071) Package private classes that are deleted from an older version of Spark trigger errors

[ https://issues.apache.org/jira/browse/SPARK-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025239#comment-14025239 ] Prashant Sharma commented on SPARK-2071: Or manually place the jar of the older

[jira] [Commented] (SPARK-1291) Link the spark UI to RM ui in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025244#comment-14025244 ] Thomas Graves commented on SPARK-1291: -- https://github.com/apache/spark/pull/1002

[jira] [Commented] (SPARK-2075) Hadoop1 distribution of 1.0.0 does not contain classes expected by the Maven 1.0.0 artifact

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025255#comment-14025255 ] Paul R. Brown commented on SPARK-2075: -- The job is run by a Java client that connects

[jira] [Comment Edited] (SPARK-2075) Hadoop1 distribution of 1.0.0 does not contain classes expected by the Maven 1.0.0 artifact

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025296#comment-14025296 ] Paul R. Brown edited comment on SPARK-2075 at 6/9/14 3:36 PM: --

[jira] [Commented] (SPARK-2075) Hadoop1 distribution of 1.0.0 does not contain classes expected by the Maven 1.0.0 artifact

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025296#comment-14025296 ] Paul R. Brown commented on SPARK-2075: -- As food for thought,

[jira] [Comment Edited] (SPARK-2075) Hadoop1 distribution of 1.0.0 does not contain classes expected by the Maven 1.0.0 artifact

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025296#comment-14025296 ] Paul R. Brown edited comment on SPARK-2075 at 6/9/14 3:37 PM: --

[jira] [Comment Edited] (SPARK-2075) Hadoop1 distribution of 1.0.0 does not contain classes expected by the Maven 1.0.0 artifact

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025296#comment-14025296 ] Paul R. Brown edited comment on SPARK-2075 at 6/9/14 3:54 PM: --

[jira] [Created] (SPARK-2080) Yarn: history UI link missing, wrong reported user

Marcelo Vanzin created SPARK-2080: - Summary: Yarn: history UI link missing, wrong reported user Key: SPARK-2080 URL: https://issues.apache.org/jira/browse/SPARK-2080 Project: Spark Issue

[jira] [Created] (SPARK-2079) Skip unnecessary wrapping in List when serializing SchemaRDD to Python

Kan Zhang created SPARK-2079: Summary: Skip unnecessary wrapping in List when serializing SchemaRDD to Python Key: SPARK-2079 URL: https://issues.apache.org/jira/browse/SPARK-2079 Project: Spark

[jira] [Commented] (SPARK-2080) Yarn: history UI link missing, wrong reported user

[ https://issues.apache.org/jira/browse/SPARK-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025346#comment-14025346 ] Marcelo Vanzin commented on SPARK-2080: --- Patrick / someone, I can't seem to be able

[jira] [Commented] (SPARK-2079) Skip unnecessary wrapping in List when serializing SchemaRDD to Python

[ https://issues.apache.org/jira/browse/SPARK-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025363#comment-14025363 ] Kan Zhang commented on SPARK-2079: -- PR: https://github.com/apache/spark/pull/1023 Skip

[jira] [Updated] (SPARK-2079) Removing unnecessary wrapping when serializing SchemaRDD to Python

[ https://issues.apache.org/jira/browse/SPARK-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kan Zhang updated SPARK-2079: - Summary: Removing unnecessary wrapping when serializing SchemaRDD to Python (was: Skip unnecessary

[jira] [Commented] (SPARK-1944) Document --verbose in spark-shell -h

[ https://issues.apache.org/jira/browse/SPARK-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025408#comment-14025408 ] Patrick Wendell commented on SPARK-1944: Accidental edit - my bad! Document

[jira] [Updated] (SPARK-1944) Document --verbose in spark-shell -h

[ https://issues.apache.org/jira/browse/SPARK-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1944: --- Target Version/s: 1.0.1, 1.1.0 Fix Version/s: (was: 1.0.1)

[jira] [Created] (SPARK-2081) Undefine output() from the abstract class Command and implement it in concrete subclasses

Zongheng Yang created SPARK-2081: Summary: Undefine output() from the abstract class Command and implement it in concrete subclasses Key: SPARK-2081 URL: https://issues.apache.org/jira/browse/SPARK-2081

[jira] [Updated] (SPARK-2075) Hadoop1 distribution of 1.0.0 does not contain classes expected by the Maven 1.0.0 artifact

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2075: --- Priority: Critical (was: Major) Hadoop1 distribution of 1.0.0 does not contain classes

[jira] [Updated] (SPARK-2075) Hadoop1 distribution of 1.0.0 does not contain classes expected by the Maven 1.0.0 artifact

[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2075: --- Fix Version/s: 1.0.1 Hadoop1 distribution of 1.0.0 does not contain classes expected by the

[jira] [Updated] (SPARK-2081) Undefine output() from the abstract class Command and implement it in concrete subclasses

[ https://issues.apache.org/jira/browse/SPARK-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2081: Assignee: Zongheng Yang Undefine output() from the abstract class Command and implement

[jira] [Updated] (SPARK-2034) KafkaInputDStream doesn't close resources and may prevent JVM shutdown

[ https://issues.apache.org/jira/browse/SPARK-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2034: Assignee: (was: Kan Zhang) KafkaInputDStream doesn't close resources and may prevent

[jira] [Commented] (SPARK-2044) Pluggable interface for shuffles

[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025580#comment-14025580 ] Matei Zaharia commented on SPARK-2044: -- Hey Weihua, I'll look into the sorting flag;

[jira] [Created] (SPARK-2082) Stratified sampling implementation in PairRDDFunctions

Doris Xin created SPARK-2082: Summary: Stratified sampling implementation in PairRDDFunctions Key: SPARK-2082 URL: https://issues.apache.org/jira/browse/SPARK-2082 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-1522) YARN ClientBase will throw a NPE if there is no YARN application specific classpath.

[ https://issues.apache.org/jira/browse/SPARK-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1522. -- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Bernardo Gomez Palacio YARN

[jira] [Created] (SPARK-2083) Allow local task to retry after failure.

Peng Cheng created SPARK-2083: - Summary: Allow local task to retry after failure. Key: SPARK-2083 URL: https://issues.apache.org/jira/browse/SPARK-2083 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-2084) Mention SPARK_JAR in env var section on configuration page

Sandy Ryza created SPARK-2084: - Summary: Mention SPARK_JAR in env var section on configuration page Key: SPARK-2084 URL: https://issues.apache.org/jira/browse/SPARK-2084 Project: Spark Issue

[jira] [Updated] (SPARK-2085) Apply user-specific regularization instead of uniform regularization in Alternating Least Squares (ALS)

[ https://issues.apache.org/jira/browse/SPARK-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-2085: -- Description: The current implementation of ALS takes a single regularization parameter and apply it

[jira] [Created] (SPARK-2085) Apply user-specific regularization instead of uniform regularization in Alternating Least Squares (ALS)

Shuo Xiang created SPARK-2085: - Summary: Apply user-specific regularization instead of uniform regularization in Alternating Least Squares (ALS) Key: SPARK-2085 URL: https://issues.apache.org/jira/browse/SPARK-2085

[jira] [Updated] (SPARK-2085) Apply user-specific regularization instead of uniform regularization in Alternating Least Squares (ALS)

[ https://issues.apache.org/jira/browse/SPARK-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuo Xiang updated SPARK-2085: -- Description: The current implementation of ALS takes a single regularization parameter and apply it

[jira] [Commented] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025895#comment-14025895 ] Erik Erlandson commented on SPARK-1493: --- RAT itself appears to preclude exclusion

[jira] [Comment Edited] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025895#comment-14025895 ] Erik Erlandson edited comment on SPARK-1493 at 6/9/14 11:13 PM:

[jira] [Updated] (SPARK-1704) Support EXPLAIN in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1704: Assignee: Zongheng Yang (was: Michael Armbrust) Support EXPLAIN in Spark SQL

[jira] [Assigned] (SPARK-1704) Support EXPLAIN in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-1704: --- Assignee: Michael Armbrust Support EXPLAIN in Spark SQL

[jira] [Resolved] (SPARK-1704) Support EXPLAIN in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-1704. - Resolution: Fixed Fix Version/s: 1.0.1 Support EXPLAIN in Spark SQL

[jira] [Updated] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1493: --- Fix Version/s: (was: 1.1.0) Apache RAT excludes don't work with file path (instead of

[jira] [Commented] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025954#comment-14025954 ] Patrick Wendell commented on SPARK-1493: Thanks for looking into this Erik. It

[jira] [Updated] (SPARK-2086) Improve output of toDebugString to make shuffle boundaries more clear

[ https://issues.apache.org/jira/browse/SPARK-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2086: --- Description: It would be nice if the toDebugString method of an RDD did a better job of

[jira] [Created] (SPARK-2088) NPE in toString when creationSiteInfo is null after deserialization

Doris Xin created SPARK-2088: Summary: NPE in toString when creationSiteInfo is null after deserialization Key: SPARK-2088 URL: https://issues.apache.org/jira/browse/SPARK-2088 Project: Spark

[jira] [Commented] (SPARK-1305) Support persisting RDD's directly to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026037#comment-14026037 ] Henry Saputra commented on SPARK-1305: -- Sorry to comment on old JIRA but does anyone

[jira] [Commented] (SPARK-1305) Support persisting RDD's directly to Tachyon

[ https://issues.apache.org/jira/browse/SPARK-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026038#comment-14026038 ] Henry Saputra commented on SPARK-1305: -- Never mind, Found it, it was when Spark in

[jira] [Commented] (SPARK-2071) Package private classes that are deleted from an older version of Spark trigger errors

[ https://issues.apache.org/jira/browse/SPARK-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026045#comment-14026045 ] Patrick Wendell commented on SPARK-2071: Yes we could use sbt to retreive them and

[jira] [Updated] (SPARK-2088) NPE in toString when creationSiteInfo is null after deserialization

[ https://issues.apache.org/jira/browse/SPARK-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2088: --- Assignee: Doris Xin NPE in toString when creationSiteInfo is null after deserialization

[jira] [Updated] (SPARK-2088) NPE in toString when creationSiteInfo is null after deserialization

[ https://issues.apache.org/jira/browse/SPARK-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2088: --- Target Version/s: 1.0.0, 1.0.1 NPE in toString when creationSiteInfo is null after

[jira] [Updated] (SPARK-2088) NPE in toString when creationSiteInfo is null after deserialization

[ https://issues.apache.org/jira/browse/SPARK-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2088: --- Affects Version/s: 1.0.0 NPE in toString when creationSiteInfo is null after

[jira] [Commented] (SPARK-2000) cannot connect to cluster in Standalone mode when run spark-shell in one of the cluster node without specify master

[ https://issues.apache.org/jira/browse/SPARK-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026052#comment-14026052 ] Chen Chao commented on SPARK-2000: -- Hi, Patrick, I just thought it was the same problem

[jira] [Commented] (SPARK-1998) SparkFlumeEvent with body bigger than 1020 bytes are not read properly

[ https://issues.apache.org/jira/browse/SPARK-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026053#comment-14026053 ] sunshangchun commented on SPARK-1998: - I've pulled a request

[jira] [Created] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

Sandy Ryza created SPARK-2089: - Summary: With YARN, preferredNodeLocalityData isn't honored Key: SPARK-2089 URL: https://issues.apache.org/jira/browse/SPARK-2089 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2044) Pluggable interface for shuffles

[ https://issues.apache.org/jira/browse/SPARK-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026109#comment-14026109 ] Weihua Jiang commented on SPARK-2044: - Hi Matei, Thanks for the reply. I am glad

[jira] [Resolved] (SPARK-1416) Add support for SequenceFiles in PySpark

[ https://issues.apache.org/jira/browse/SPARK-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1416. -- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 Implemented in

[jira] [Updated] (SPARK-1416) Add support for SequenceFiles in PySpark

[ https://issues.apache.org/jira/browse/SPARK-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-1416: - Assignee: Nick Pentreath Add support for SequenceFiles in PySpark

[jira] [Created] (SPARK-2090) spark-shell input text entry not showing on REPL

Richard Conway created SPARK-2090: - Summary: spark-shell input text entry not showing on REPL Key: SPARK-2090 URL: https://issues.apache.org/jira/browse/SPARK-2090 Project: Spark Issue Type: