[jira] [Commented] (SPARK-4673) Optimizing limit using coalesce

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229510#comment-14229510 ] Apache Spark commented on SPARK-4673: - User 'scwf' has created a pull request for this

[jira] [Updated] (SPARK-4672) Cut off the super long serialization chain in GraphX to avoid the StackOverflow error

2014-11-30 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijie Xu updated SPARK-4672: Description: While running iterative algorithms in GraphX, a StackOverflow error will stably occur in the s

[jira] [Created] (SPARK-4673) Optimizing limit using coalesce

2014-11-30 Thread wangfei (JIRA)
wangfei created SPARK-4673: -- Summary: Optimizing limit using coalesce Key: SPARK-4673 URL: https://issues.apache.org/jira/browse/SPARK-4673 Project: Spark Issue Type: Bug Components: SQL

[jira] [Created] (SPARK-4672) Cut off the super long serialization chain in GraphX to avoid the StackOverflow error

2014-11-30 Thread Lijie Xu (JIRA)
Lijie Xu created SPARK-4672: --- Summary: Cut off the super long serialization chain in GraphX to avoid the StackOverflow error Key: SPARK-4672 URL: https://issues.apache.org/jira/browse/SPARK-4672 Project: Sp

[jira] [Created] (SPARK-4671) Streaming block need not to replicate 2 copies when WAL is enabled

2014-11-30 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-4671: -- Summary: Streaming block need not to replicate 2 copies when WAL is enabled Key: SPARK-4671 URL: https://issues.apache.org/jira/browse/SPARK-4671 Project: Spark

[jira] [Commented] (SPARK-4397) Reorganize 'implicit's to improve the API convenience

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229497#comment-14229497 ] Apache Spark commented on SPARK-4397: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-4670) bitwise NOT has a wrong `toString` output

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229460#comment-14229460 ] Apache Spark commented on SPARK-4670: - User 'adrian-wang' has created a pull request f

[jira] [Created] (SPARK-4670) bitwise NOT has a wrong `toString` output

2014-11-30 Thread Adrian Wang (JIRA)
Adrian Wang created SPARK-4670: -- Summary: bitwise NOT has a wrong `toString` output Key: SPARK-4670 URL: https://issues.apache.org/jira/browse/SPARK-4670 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4669) Allow users to set arbitrary akka configurations via property file

2014-11-30 Thread WangTaoTheTonic (JIRA)
WangTaoTheTonic created SPARK-4669: -- Summary: Allow users to set arbitrary akka configurations via property file Key: SPARK-4669 URL: https://issues.apache.org/jira/browse/SPARK-4669 Project: Spark

[jira] [Commented] (SPARK-4668) Fix documentation typos

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229447#comment-14229447 ] Apache Spark commented on SPARK-4668: - User 'ryan-williams' has created a pull request

[jira] [Created] (SPARK-4668) Fix documentation typos

2014-11-30 Thread Ryan Williams (JIRA)
Ryan Williams created SPARK-4668: Summary: Fix documentation typos Key: SPARK-4668 URL: https://issues.apache.org/jira/browse/SPARK-4668 Project: Spark Issue Type: Bug Components: D

[jira] [Commented] (SPARK-4667) Spillable can request more than twice its current memory from pool

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229444#comment-14229444 ] Apache Spark commented on SPARK-4667: - User 'ryan-williams' has created a pull request

[jira] [Created] (SPARK-4667) Spillable can request more than twice its current memory from pool

2014-11-30 Thread Ryan Williams (JIRA)
Ryan Williams created SPARK-4667: Summary: Spillable can request more than twice its current memory from pool Key: SPARK-4667 URL: https://issues.apache.org/jira/browse/SPARK-4667 Project: Spark

[jira] [Created] (SPARK-4666) "executor.memoryOverhead" config should take a "memory string"

2014-11-30 Thread Ryan Williams (JIRA)
Ryan Williams created SPARK-4666: Summary: "executor.memoryOverhead" config should take a "memory string" Key: SPARK-4666 URL: https://issues.apache.org/jira/browse/SPARK-4666 Project: Spark

[jira] [Commented] (SPARK-4665) Config value for setting yarn container overhead to a fraction of executor memory

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229442#comment-14229442 ] Apache Spark commented on SPARK-4665: - User 'ryan-williams' has created a pull request

[jira] [Created] (SPARK-4665) Config value for setting yarn container overhead to a fraction of executor memory

2014-11-30 Thread Ryan Williams (JIRA)
Ryan Williams created SPARK-4665: Summary: Config value for setting yarn container overhead to a fraction of executor memory Key: SPARK-4665 URL: https://issues.apache.org/jira/browse/SPARK-4665 Proje

[jira] [Commented] (SPARK-4664) Overflow of `maxFrameSizeBytes`

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229430#comment-14229430 ] Apache Spark commented on SPARK-4664: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-4664) Overflow of `maxFrameSizeBytes`

2014-11-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4664: --- Summary: Overflow of `maxFrameSizeBytes` Key: SPARK-4664 URL: https://issues.apache.org/jira/browse/SPARK-4664 Project: Spark Issue Type: Bug Compone

[jira] [Resolved] (SPARK-4632) Upgrade MQTT dependency to use latest mqtt-client

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4632. Resolution: Fixed Fix Version/s: 1.3.0 > Upgrade MQTT dependency to use latest mqtt-c

[jira] [Updated] (SPARK-4632) Upgrade MQTT dependency to use mqtt-client 1.0.1

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4632: --- Summary: Upgrade MQTT dependency to use mqtt-client 1.0.1 (was: Upgrade MQTT dependency to us

[jira] [Commented] (SPARK-4663) close() function is not surrounded by finally in ParquetTableOperations.scala

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229410#comment-14229410 ] Apache Spark commented on SPARK-4663: - User 'baishuo' has created a pull request for t

[jira] [Comment Edited] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229407#comment-14229407 ] Patrick Wendell edited comment on SPARK-4630 at 12/1/14 4:41 AM: ---

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229407#comment-14229407 ] Patrick Wendell commented on SPARK-4630: Thanks Sandy - that's useful context. I t

[jira] [Created] (SPARK-4663) close() function is not surrounded by finally in ParquetTableOperations.scala

2014-11-30 Thread baishuo (JIRA)
baishuo created SPARK-4663: -- Summary: close() function is not surrounded by finally in ParquetTableOperations.scala Key: SPARK-4663 URL: https://issues.apache.org/jira/browse/SPARK-4663 Project: Spark

[jira] [Updated] (SPARK-4653) DAGScheduler refactoring and cleanup

2014-11-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-4653: -- Description: This is an umbrella JIRA for DAGScheduler refactoring and cleanup. Please comment or open

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-30 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229371#comment-14229371 ] Sandy Ryza commented on SPARK-4630: --- Hey [~pwendell], Spark deals much better with large

[jira] [Commented] (SPARK-4662) Whitelist more Hive unittest

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229370#comment-14229370 ] Apache Spark commented on SPARK-4662: - User 'chenghao-intel' has created a pull reques

[jira] [Commented] (SPARK-4644) Implement skewed join

2014-11-30 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229369#comment-14229369 ] Shixiong Zhu commented on SPARK-4644: - {quote} User 'zsxwing' has created a pull reque

[jira] [Commented] (SPARK-4661) Minor code and docs cleanup

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229366#comment-14229366 ] Apache Spark commented on SPARK-4661: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-4662) Whitelist more Hive unittest

2014-11-30 Thread Cheng Hao (JIRA)
Cheng Hao created SPARK-4662: Summary: Whitelist more Hive unittest Key: SPARK-4662 URL: https://issues.apache.org/jira/browse/SPARK-4662 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-4644) Implement skewed join

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229357#comment-14229357 ] Apache Spark commented on SPARK-4644: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-4661) Minor code and docs cleanup

2014-11-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4661: --- Summary: Minor code and docs cleanup Key: SPARK-4661 URL: https://issues.apache.org/jira/browse/SPARK-4661 Project: Spark Issue Type: Improvement Com

[jira] [Updated] (SPARK-1517) Publish nightly snapshots of documentation, maven artifacts, and binary builds

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1517: --- Priority: Blocker (was: Critical) > Publish nightly snapshots of documentation, maven artifac

[jira] [Closed] (SPARK-4651) Adding -Phadoop-2.4+ to compile Spark with newer versions of Hadoop

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell closed SPARK-4651. -- Resolution: Won't Fix > Adding -Phadoop-2.4+ to compile Spark with newer versions of Hadoop > --

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

2014-11-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229318#comment-14229318 ] Josh Rosen commented on SPARK-3926: --- Just saw a bug report on the mailing list that look

[jira] [Resolved] (SPARK-4656) Typo in Programming Guide markdown

2014-11-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4656. --- Resolution: Fixed Fix Version/s: 1.2.1 Issue resolved by pull request 3412 [https://github.com/

[jira] [Updated] (SPARK-4623) Add the some error infomation if using spark-sql in yarn-cluster mode

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4623: --- Fix Version/s: (was: 1.2.0) 1.3.0 > Add the some error infomation if us

[jira] [Closed] (SPARK-4623) Add the some error infomation if using spark-sql in yarn-cluster mode

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell closed SPARK-4623. -- Resolution: Fixed Fix Version/s: 1.2.0 > Add the some error infomation if using spark-sql

[jira] [Commented] (SPARK-732) Recomputation of RDDs may result in duplicated accumulator updates

2014-11-30 Thread Daniel Siegmann (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229277#comment-14229277 ] Daniel Siegmann commented on SPARK-732: --- This is very disappointing. Essentially, Spa

[jira] [Commented] (SPARK-3278) Isotonic regression

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229271#comment-14229271 ] Apache Spark commented on SPARK-3278: - User 'zapletal-martin' has created a pull reque

[jira] [Commented] (SPARK-4002) KafkaStreamSuite "Kafka input stream" case fails on OSX

2014-11-30 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229208#comment-14229208 ] Ryan Williams commented on SPARK-4002: -- my hostname is just "mbp", so I don't think t

[jira] [Commented] (SPARK-3694) Allow printing object graph of tasks/RDD's with a debug flag

2014-11-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229189#comment-14229189 ] Apache Spark commented on SPARK-3694: - User 'ilganeli' has created a pull request for

[jira] [Updated] (SPARK-4660) JavaSerializer uses wrong classloader

2014-11-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Kołaczkowski updated SPARK-4660: -- Attachment: spark-serializer-classloader.patch Attaching a patch against 1.1 branch. >

[jira] [Created] (SPARK-4660) JavaSerializer uses wrong classloader

2014-11-30 Thread JIRA
Piotr Kołaczkowski created SPARK-4660: - Summary: JavaSerializer uses wrong classloader Key: SPARK-4660 URL: https://issues.apache.org/jira/browse/SPARK-4660 Project: Spark Issue Type: Bug

[jira] [Closed] (SPARK-4451) force to kill process after 5 seconds

2014-11-30 Thread WangTaoTheTonic (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] WangTaoTheTonic closed SPARK-4451. -- Resolution: Won't Fix it is better for catching bugs in future to keep not forcing to kill daemo