[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173413#comment-14173413 ] Mridul Muralidharan commented on SPARK-3948: Not exactly, what I was

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173427#comment-14173427 ] Saisai Shao commented on SPARK-3948: Hi [~mridulm80], thanks a lot for your

[jira] [Commented] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173428#comment-14173428 ] Davis Shepherd commented on SPARK-3882: --- This is also a serious memory leak that

[jira] [Commented] (SPARK-3807) SparkSql does not work for tables created using custom serde

[ https://issues.apache.org/jira/browse/SPARK-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173430#comment-14173430 ] Apache Spark commented on SPARK-3807: - User 'adrian-wang' has created a pull request

[jira] [Issue Comment Deleted] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-3882: -- Comment: was deleted (was: This is also a serious memory leak that will cause long running

[jira] [Commented] (SPARK-3965) Spark assembly for hadoop2 contains avro-mapred for hadoop1

[ https://issues.apache.org/jira/browse/SPARK-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173445#comment-14173445 ] Apache Spark commented on SPARK-3965: - User 'dajac' has created a pull request for

[jira] [Commented] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173451#comment-14173451 ] Reynold Xin commented on SPARK-3963: Would it make sense to support arbitrary data

[jira] [Updated] (SPARK-3873) Scala style: check import ordering

[ https://issues.apache.org/jira/browse/SPARK-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3873: --- Assignee: (was: Marcelo Vanzin) Scala style: check import ordering

[jira] [Commented] (SPARK-3888) Limit the memory used by python worker

[ https://issues.apache.org/jira/browse/SPARK-3888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173454#comment-14173454 ] Apache Spark commented on SPARK-3888: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173489#comment-14173489 ] Mridul Muralidharan commented on SPARK-3948: Damn, this sucks : the

[jira] [Comment Edited] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173489#comment-14173489 ] Mridul Muralidharan edited comment on SPARK-3948 at 10/16/14 7:39 AM:

[jira] [Commented] (SPARK-3814) Bitwise does not work in Hive

[ https://issues.apache.org/jira/browse/SPARK-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173526#comment-14173526 ] Ravindra Pesala commented on SPARK-3814: Added support for Bitwise AND(), OR(|)

[jira] [Updated] (SPARK-3814) Support for Bitwise AND(), OR(|) ,XOR(^), NOT(~) in Spark HQL and SQL

[ https://issues.apache.org/jira/browse/SPARK-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated SPARK-3814: --- Summary: Support for Bitwise AND(), OR(|) ,XOR(^), NOT(~) in Spark HQL and SQL (was: Bitwise

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173528#comment-14173528 ] Saisai Shao commented on SPARK-3948: Hi [~mridulm80], Thanks a lot for your

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173541#comment-14173541 ] qiaohaijun commented on SPARK-2706: --- sh make-distribution.sh --tgz -Phadoop-provided

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173542#comment-14173542 ] Praveen Seluka commented on SPARK-3174: --- [~vanzin] Regarding your question related

[jira] [Commented] (SPARK-1479) building spark on 2.0.0-cdh4.4.0 failed

[ https://issues.apache.org/jira/browse/SPARK-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173543#comment-14173543 ] qiaohaijun commented on SPARK-1479: --- sh make-distribution.sh --tgz -Phadoop-provided

[jira] [Comment Edited] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173542#comment-14173542 ] Praveen Seluka edited comment on SPARK-3174 at 10/16/14 8:55 AM:

[jira] [Comment Edited] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173542#comment-14173542 ] Praveen Seluka edited comment on SPARK-3174 at 10/16/14 8:56 AM:

[jira] [Created] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

Christophe PRÉAUD created SPARK-3967: Summary: Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions Key: SPARK-3967

[jira] [Updated] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christophe PRÉAUD updated SPARK-3967: - Attachment: spark-1.1.0-yarn_cluster_tmpdir.patch Ensure that the temporary file which

[jira] [Commented] (SPARK-3967) Spark applications fail in yarn-cluster mode when the directories configured in yarn.nodemanager.local-dirs are located on different disks/partitions

[ https://issues.apache.org/jira/browse/SPARK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173566#comment-14173566 ] Christophe PRÉAUD commented on SPARK-3967: -- After investigating, it turns out

[jira] [Commented] (SPARK-3431) Parallelize execution of tests

[ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173618#comment-14173618 ] Sean Owen commented on SPARK-3431: -- Yes that should be what scalatest does. It is a fork

[jira] [Commented] (SPARK-2321) Design a proper progress reporting event listener API

[ https://issues.apache.org/jira/browse/SPARK-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173619#comment-14173619 ] Dev Lakhani commented on SPARK-2321: There are some issues and bugs under the webui

[jira] [Created] (SPARK-3968) Using parquet-mr filter2 api in spark sql, add a custom filter for InSet clause

Yash Datta created SPARK-3968: - Summary: Using parquet-mr filter2 api in spark sql, add a custom filter for InSet clause Key: SPARK-3968 URL: https://issues.apache.org/jira/browse/SPARK-3968 Project:

[jira] [Updated] (SPARK-3968) Using parquet-mr filter2 api in spark sql, add a custom filter for InSet clause

[ https://issues.apache.org/jira/browse/SPARK-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Datta updated SPARK-3968: -- Shepherd: Yash Datta Using parquet-mr filter2 api in spark sql, add a custom filter for InSet clause

[jira] [Commented] (SPARK-3629) Improvements to YARN doc

[ https://issues.apache.org/jira/browse/SPARK-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173731#comment-14173731 ] ssj commented on SPARK-3629: need someone to verifity this patch Improvements to YARN doc

[jira] [Commented] (SPARK-3948) Sort-based shuffle can lead to assorted stream-corruption exceptions

[ https://issues.apache.org/jira/browse/SPARK-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173757#comment-14173757 ] Apache Spark commented on SPARK-3948: - User 'jerryshao' has created a pull request for

[jira] [Commented] (SPARK-3907) add truncate table support

[ https://issues.apache.org/jira/browse/SPARK-3907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173806#comment-14173806 ] Apache Spark commented on SPARK-3907: - User 'wangxiaojing' has created a pull request

[jira] [Commented] (SPARK-3969) Optimizer should have a super class as an interface.

[ https://issues.apache.org/jira/browse/SPARK-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173854#comment-14173854 ] Apache Spark commented on SPARK-3969: - User 'ueshin' has created a pull request for

[jira] [Created] (SPARK-3970) Remove duplicate removal of local dirs

Liang-Chi Hsieh created SPARK-3970: -- Summary: Remove duplicate removal of local dirs Key: SPARK-3970 URL: https://issues.apache.org/jira/browse/SPARK-3970 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3969) Optimizer should have a super class as an interface.

Takuya Ueshin created SPARK-3969: Summary: Optimizer should have a super class as an interface. Key: SPARK-3969 URL: https://issues.apache.org/jira/browse/SPARK-3969 Project: Spark Issue

[jira] [Commented] (SPARK-3970) Remove duplicate removal of local dirs

[ https://issues.apache.org/jira/browse/SPARK-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173856#comment-14173856 ] Apache Spark commented on SPARK-3970: - User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-2365) Add IndexedRDD, an efficient updatable key-value store

[ https://issues.apache.org/jira/browse/SPARK-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173904#comment-14173904 ] Akshat Aranya commented on SPARK-2365: -- This looks great! I have been using

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173926#comment-14173926 ] Dev Lakhani commented on SPARK-3957: Here is my thoughts on a possible approach. Hi

[jira] [Commented] (SPARK-3174) Provide elastic scaling within a Spark application

[ https://issues.apache.org/jira/browse/SPARK-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173951#comment-14173951 ] Marcelo Vanzin commented on SPARK-3174: --- bq. Lets say we start the Spark

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173954#comment-14173954 ] Shivaram Venkataraman commented on SPARK-3957: -- I think it needs to be

[jira] [Commented] (SPARK-3883) Provide SSL support for Akka and HttpServer based connections

[ https://issues.apache.org/jira/browse/SPARK-3883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174005#comment-14174005 ] Marcelo Vanzin commented on SPARK-3883: --- FYI, any PR here should make sure the

[jira] [Commented] (SPARK-2750) Add Https support for Web UI

[ https://issues.apache.org/jira/browse/SPARK-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174003#comment-14174003 ] Marcelo Vanzin commented on SPARK-2750: --- FYI, any PR here should make sure the

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174019#comment-14174019 ] Andrew Or commented on SPARK-3957: -- Yeah my understanding is that broadcast blocks aren't

[jira] [Closed] (SPARK-1761) Add broadcast information on SparkUI storage tab

[ https://issues.apache.org/jira/browse/SPARK-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-1761. Resolution: Duplicate Add broadcast information on SparkUI storage tab

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174024#comment-14174024 ] Andrew Or commented on SPARK-3957: -- Hey [~devl.development] are you planning to work on

[jira] [Commented] (SPARK-1761) Add broadcast information on SparkUI storage tab

[ https://issues.apache.org/jira/browse/SPARK-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174026#comment-14174026 ] Andrew Or commented on SPARK-1761: -- Closing in favor of SPARK-3957, which is more

[jira] [Created] (SPARK-3971) Failed to deserialize Vector in cluster mode

Davies Liu created SPARK-3971: - Summary: Failed to deserialize Vector in cluster mode Key: SPARK-3971 URL: https://issues.apache.org/jira/browse/SPARK-3971 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-2585) Remove special handling of Hadoop JobConf

[ https://issues.apache.org/jira/browse/SPARK-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2585. --- Resolution: Fixed Due to the CONFIGURATION_INSTANTIATION_LOCK thread-safety issue, I think that

[jira] [Created] (SPARK-3972) PySpark Error on Windows with sc.wholeTextFiles

Michael Griffiths created SPARK-3972: Summary: PySpark Error on Windows with sc.wholeTextFiles Key: SPARK-3972 URL: https://issues.apache.org/jira/browse/SPARK-3972 Project: Spark Issue

[jira] [Commented] (SPARK-3736) Workers should reconnect to Master if disconnected

[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174049#comment-14174049 ] Apache Spark commented on SPARK-3736: - User 'mccheah' has created a pull request for

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174053#comment-14174053 ] Nan Zhu commented on SPARK-3957: I agree with [~andrewor14], I was also thinking about

[jira] [Created] (SPARK-3973) Print callSite information for broadcast variables

Shivaram Venkataraman created SPARK-3973: Summary: Print callSite information for broadcast variables Key: SPARK-3973 URL: https://issues.apache.org/jira/browse/SPARK-3973 Project: Spark

[jira] [Commented] (SPARK-3973) Print callSite information for broadcast variables

[ https://issues.apache.org/jira/browse/SPARK-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174066#comment-14174066 ] Apache Spark commented on SPARK-3973: - User 'shivaram' has created a pull request for

[jira] [Commented] (SPARK-2706) Enable Spark to support Hive 0.13

[ https://issues.apache.org/jira/browse/SPARK-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174073#comment-14174073 ] Zhan Zhang commented on SPARK-2706: --- The code does not go to upstream yet. To build

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174107#comment-14174107 ] Dev Lakhani commented on SPARK-3957: Hi For now I am happy for [~CodingCat] to take

[jira] [Commented] (SPARK-3972) PySpark Error on Windows with sc.wholeTextFiles

[ https://issues.apache.org/jira/browse/SPARK-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174110#comment-14174110 ] Michael Griffiths commented on SPARK-3972: -- This issue does NOT occur if I build

[jira] [Updated] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3882: --- Description: A long running spark context (non-streaming) will eventually start throwing the

[jira] [Created] (SPARK-3974) Block matrix abstracitons and partitioners

Reza Zadeh created SPARK-3974: - Summary: Block matrix abstracitons and partitioners Key: SPARK-3974 URL: https://issues.apache.org/jira/browse/SPARK-3974 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3975) Block Matrix addition and multiplication

Reza Zadeh created SPARK-3975: - Summary: Block Matrix addition and multiplication Key: SPARK-3975 URL: https://issues.apache.org/jira/browse/SPARK-3975 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3976) Detect block matrix partitioning schemes

Reza Zadeh created SPARK-3976: - Summary: Detect block matrix partitioning schemes Key: SPARK-3976 URL: https://issues.apache.org/jira/browse/SPARK-3976 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-3977) Conversions between {Row, Coordinate}Matrix - BlockMatrix

Reza Zadeh created SPARK-3977: - Summary: Conversions between {Row, Coordinate}Matrix - BlockMatrix Key: SPARK-3977 URL: https://issues.apache.org/jira/browse/SPARK-3977 Project: Spark Issue

[jira] [Commented] (SPARK-3971) Failed to deserialize Vector in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174146#comment-14174146 ] Apache Spark commented on SPARK-3971: - User 'davies' has created a pull request for

[jira] [Commented] (SPARK-3466) Limit size of results that a driver collects for each action

[ https://issues.apache.org/jira/browse/SPARK-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174148#comment-14174148 ] Matt Cheah commented on SPARK-3466: --- I'll look into this. Someone please assign to me!

[jira] [Created] (SPARK-3978) Schema change on Spark-Hive (Parquet file format) table not working

Nilesh Barge created SPARK-3978: --- Summary: Schema change on Spark-Hive (Parquet file format) table not working Key: SPARK-3978 URL: https://issues.apache.org/jira/browse/SPARK-3978 Project: Spark

[jira] [Updated] (SPARK-3466) Limit size of results that a driver collects for each action

[ https://issues.apache.org/jira/browse/SPARK-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3466: -- Assignee: Matthew Cheah Limit size of results that a driver collects for each action

[jira] [Updated] (SPARK-3971) Failed to deserialize Vector in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3971: - Assignee: Davies Liu Failed to deserialize Vector in cluster mode

[jira] [Updated] (SPARK-3971) Failed to deserialize Vector in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3971: - Target Version/s: 1.2.0 Affects Version/s: 1.2.0 Failed to deserialize Vector in cluster

[jira] [Created] (SPARK-3979) Yarn backend's default file replication should match HDFS's default one

Marcelo Vanzin created SPARK-3979: - Summary: Yarn backend's default file replication should match HDFS's default one Key: SPARK-3979 URL: https://issues.apache.org/jira/browse/SPARK-3979 Project:

[jira] [Commented] (SPARK-3979) Yarn backend's default file replication should match HDFS's default one

[ https://issues.apache.org/jira/browse/SPARK-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174165#comment-14174165 ] Marcelo Vanzin commented on SPARK-3979: --- BTW, this would avoid issues like this:

[jira] [Updated] (SPARK-3979) Yarn backend's default file replication should match HDFS's default one

[ https://issues.apache.org/jira/browse/SPARK-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-3979: -- Description: This code in ClientBase.scala sets the replication used for files uploaded to

[jira] [Commented] (SPARK-3979) Yarn backend's default file replication should match HDFS's default one

[ https://issues.apache.org/jira/browse/SPARK-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174224#comment-14174224 ] Apache Spark commented on SPARK-3979: - User 'vanzin' has created a pull request for

[jira] [Created] (SPARK-3980) GraphX Performance Issue

Jarred Li created SPARK-3980: Summary: GraphX Performance Issue Key: SPARK-3980 URL: https://issues.apache.org/jira/browse/SPARK-3980 Project: Spark Issue Type: Bug Components: GraphX

[jira] [Created] (SPARK-3981) Consider a better approach to initialize SerDe on executors

Xiangrui Meng created SPARK-3981: Summary: Consider a better approach to initialize SerDe on executors Key: SPARK-3981 URL: https://issues.apache.org/jira/browse/SPARK-3981 Project: Spark

[jira] [Updated] (SPARK-3980) GraphX Performance Issue

[ https://issues.apache.org/jira/browse/SPARK-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarred Li updated SPARK-3980: - Description: I run 4 workes in AWS (c3.xlarge), 4g memory for executor, 85,331,846 edges

[jira] [Created] (SPARK-3982) receiverStream in Python API

Davies Liu created SPARK-3982: - Summary: receiverStream in Python API Key: SPARK-3982 URL: https://issues.apache.org/jira/browse/SPARK-3982 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-3980) GraphX Performance Issue

[ https://issues.apache.org/jira/browse/SPARK-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarred Li updated SPARK-3980: - Description: I run 4 workes in AWS (c3.xlarge), 4g memory for executor, 85,331,846 edges

[jira] [Created] (SPARK-3983) Scheduler delay (shown in the UI) is incorrect

Kay Ousterhout created SPARK-3983: - Summary: Scheduler delay (shown in the UI) is incorrect Key: SPARK-3983 URL: https://issues.apache.org/jira/browse/SPARK-3983 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-3983) Scheduler delay (shown in the UI) is incorrect

[ https://issues.apache.org/jira/browse/SPARK-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-3983: -- Comment: was deleted (was: This is especially problematic when debugging performance of short

[jira] [Commented] (SPARK-3983) Scheduler delay (shown in the UI) is incorrect

[ https://issues.apache.org/jira/browse/SPARK-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174452#comment-14174452 ] Kay Ousterhout commented on SPARK-3983: --- This is especially problematic when

[jira] [Updated] (SPARK-3983) Scheduler delay (shown in the UI) is incorrect

[ https://issues.apache.org/jira/browse/SPARK-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-3983: -- Description: The reported scheduler delay includes time to get a new thread (from a

[jira] [Comment Edited] (SPARK-3877) The exit code of spark-submit is still 0 when an yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174507#comment-14174507 ] Marcelo Vanzin edited comment on SPARK-3877 at 10/17/14 12:08 AM:

[jira] [Commented] (SPARK-3877) The exit code of spark-submit is still 0 when an yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174507#comment-14174507 ] Marcelo Vanzin commented on SPARK-3877: --- [~tgraves] this can be seen as a subset of

[jira] [Created] (SPARK-3984) Display finer grained metrics about task launch overhead in the UI

Kay Ousterhout created SPARK-3984: - Summary: Display finer grained metrics about task launch overhead in the UI Key: SPARK-3984 URL: https://issues.apache.org/jira/browse/SPARK-3984 Project: Spark

[jira] [Commented] (SPARK-3983) Scheduler delay (shown in the UI) is incorrect

[ https://issues.apache.org/jira/browse/SPARK-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174574#comment-14174574 ] Kay Ousterhout commented on SPARK-3983: --- https://github.com/apache/spark/pull/2832

[jira] [Updated] (SPARK-3983) Scheduler delay (shown in the UI) is incorrect

[ https://issues.apache.org/jira/browse/SPARK-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-3983: -- Component/s: Web UI Scheduler delay (shown in the UI) is incorrect

[jira] [Issue Comment Deleted] (SPARK-3984) Display finer grained metrics about task launch overhead in the UI

[ https://issues.apache.org/jira/browse/SPARK-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-3984: -- Comment: was deleted (was: https://github.com/apache/spark/pull/2832) Display finer grained

[jira] [Commented] (SPARK-3984) Display finer grained metrics about task launch overhead in the UI

[ https://issues.apache.org/jira/browse/SPARK-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174576#comment-14174576 ] Apache Spark commented on SPARK-3984: - User 'kayousterhout' has created a pull request

[jira] [Commented] (SPARK-3983) Scheduler delay (shown in the UI) is incorrect

[ https://issues.apache.org/jira/browse/SPARK-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174577#comment-14174577 ] Apache Spark commented on SPARK-3983: - User 'kayousterhout' has created a pull request

[jira] [Commented] (SPARK-3984) Display finer grained metrics about task launch overhead in the UI

[ https://issues.apache.org/jira/browse/SPARK-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174575#comment-14174575 ] Kay Ousterhout commented on SPARK-3984: --- https://github.com/apache/spark/pull/2832

[jira] [Commented] (SPARK-3982) receiverStream in Python API

[ https://issues.apache.org/jira/browse/SPARK-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174580#comment-14174580 ] Apache Spark commented on SPARK-3982: - User 'davies' has created a pull request for

[jira] [Resolved] (SPARK-3874) Provide stable TaskContext API

[ https://issues.apache.org/jira/browse/SPARK-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3874. Resolution: Fixed Fix Version/s: 1.2.0 Provide stable TaskContext API

[jira] [Updated] (SPARK-3975) Block Matrix addition and multiplication

[ https://issues.apache.org/jira/browse/SPARK-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3975: --- Component/s: MLlib Block Matrix addition and multiplication

[jira] [Updated] (SPARK-3974) Block matrix abstracitons and partitioners

[ https://issues.apache.org/jira/browse/SPARK-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Zadeh updated SPARK-3974: -- Component/s: MLlib Block matrix abstracitons and partitioners

[jira] [Updated] (SPARK-3976) Detect block matrix partitioning schemes

[ https://issues.apache.org/jira/browse/SPARK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Zadeh updated SPARK-3976: -- Component/s: MLlib Detect block matrix partitioning schemes

[jira] [Updated] (SPARK-3977) Conversions between {Row, Coordinate}Matrix - BlockMatrix

[ https://issues.apache.org/jira/browse/SPARK-3977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Zadeh updated SPARK-3977: -- Component/s: MLlib Conversions between {Row, Coordinate}Matrix - BlockMatrix

[jira] [Commented] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174629#comment-14174629 ] Patrick Wendell commented on SPARK-3882: This is a known issue (SPARK-2316) that

[jira] [Updated] (SPARK-3973) Print callSite information for broadcast variables

[ https://issues.apache.org/jira/browse/SPARK-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3973: --- Component/s: Spark Core Print callSite information for broadcast variables

[jira] [Closed] (SPARK-3923) All Standalone Mode services time out with each other

[ https://issues.apache.org/jira/browse/SPARK-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3923. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Aaron Davidson Target

[jira] [Closed] (SPARK-3941) _remainingMem should not increase twice when updateBlockInfo

[ https://issues.apache.org/jira/browse/SPARK-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3941. Resolution: Fixed Fix Version/s: 1.2.0 Assignee: Zhang, Liye Target Version/s:

[jira] [Updated] (SPARK-3890) remove redundant spark.executor.memory in doc

[ https://issues.apache.org/jira/browse/SPARK-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3890: - Affects Version/s: (was: 1.2.0) 1.1.0 remove redundant spark.executor.memory

[jira] [Closed] (SPARK-3890) remove redundant spark.executor.memory in doc

[ https://issues.apache.org/jira/browse/SPARK-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3890. Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Assignee:

[jira] [Updated] (SPARK-3890) remove redundant spark.executor.memory in doc

[ https://issues.apache.org/jira/browse/SPARK-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-3890: - Affects Version/s: 1.2.0 remove redundant spark.executor.memory in doc

[jira] [Commented] (SPARK-3963) Support getting task-scoped properties from TaskContext

[ https://issues.apache.org/jira/browse/SPARK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174660#comment-14174660 ] Patrick Wendell commented on SPARK-3963: In the initial version of this - I don't

  1   2   >