[jira] [Commented] (SPARK-542) Cache Miss when machine have multiple hostname

2014-04-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13963926#comment-13963926 ] Shixiong Zhu commented on SPARK-542: using ip has similar problem. A machine can have

[jira] [Commented] (SPARK-1482) Potential resource leaks in saveAsHadoopDataset and saveAsNewAPIHadoopDataset

2014-04-13 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13967782#comment-13967782 ] Shixiong Zhu commented on SPARK-1482: - PR: https://github.com/apache/spark/pull/400

[jira] [Commented] (SPARK-1583) Use java.util.HashMap.remove by mistake in BlockManagerMasterActor.removeBlockManager

2014-04-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13977895#comment-13977895 ] Shixiong Zhu commented on SPARK-1583: - PR: https://github.com/apache/spark/pull/500

[jira] [Commented] (SPARK-1589) Compare Option[Partitioner] and Partitioner directly

2014-04-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978262#comment-13978262 ] Shixiong Zhu commented on SPARK-1589: - PR: https://github.com/apache/spark/pull/508

[jira] [Created] (SPARK-1590) Recommend to use FindBugs

2014-04-23 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-1590: --- Summary: Recommend to use FindBugs Key: SPARK-1590 URL: https://issues.apache.org/jira/browse/SPARK-1590 Project: Spark Issue Type: Question

[jira] [Comment Edited] (SPARK-1590) Recommend to use FindBugs

2014-04-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13978291#comment-13978291 ] Shixiong Zhu edited comment on SPARK-1590 at 4/23/14 3:18 PM: --

[jira] [Updated] (SPARK-1590) Recommend to use FindBugs

2014-04-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-1590: Attachment: findbugs.png FindBugs in IDEA Recommend to use FindBugs -

[jira] [Commented] (SPARK-1590) Recommend to use FindBugs

2014-04-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979505#comment-13979505 ] Shixiong Zhu commented on SPARK-1590: - Agree that a period scanning is better then

[jira] [Created] (SPARK-1611) Incorrect initialization order in AppendOnlyMap

2014-04-24 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-1611: --- Summary: Incorrect initialization order in AppendOnlyMap Key: SPARK-1611 URL: https://issues.apache.org/jira/browse/SPARK-1611 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-1583) Use java.util.HashMap.remove by mistake in BlockManagerMasterActor.removeBlockManager

2014-04-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1583. - Resolution: Fixed Fix Version/s: 1.0.0 Use java.util.HashMap.remove by mistake in

[jira] [Created] (SPARK-1612) Potential resource leaks in Utils.copyStream and Utils.offsetBytes

2014-04-24 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-1612: --- Summary: Potential resource leaks in Utils.copyStream and Utils.offsetBytes Key: SPARK-1612 URL: https://issues.apache.org/jira/browse/SPARK-1612 Project: Spark

[jira] [Updated] (SPARK-1612) Potential resource leaks in Utils.copyStream and Utils.offsetBytes

2014-04-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-1612: Labels: easyfix (was: ) Potential resource leaks in Utils.copyStream and Utils.offsetBytes

[jira] [Commented] (SPARK-1612) Potential resource leaks in Utils.copyStream and Utils.offsetBytes

2014-04-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979752#comment-13979752 ] Shixiong Zhu commented on SPARK-1612: - PR: https://github.com/apache/spark/pull/535

[jira] [Resolved] (SPARK-1611) Incorrect initialization order in AppendOnlyMap

2014-04-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1611. - Resolution: Fixed Fix Version/s: 1.0.0 Incorrect initialization order in AppendOnlyMap

[jira] [Created] (SPARK-1628) Missing hashCode methods in Partitioner subclasses

2014-04-25 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-1628: --- Summary: Missing hashCode methods in Partitioner subclasses Key: SPARK-1628 URL: https://issues.apache.org/jira/browse/SPARK-1628 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1382) NullPointerException when calling DStream.slice() before StreamingContext.start()

2014-04-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982332#comment-13982332 ] Shixiong Zhu commented on SPARK-1382: - PR: https://github.com/apache/spark/pull/562

[jira] [Resolved] (SPARK-1382) NullPointerException when calling DStream.slice() before StreamingContext.start()

2014-04-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1382. - Resolution: Fixed Fix Version/s: 1.0.0 Assignee: Shixiong Zhu

[jira] [Created] (SPARK-1878) Incorrect initialization order in JavaStreamingContext

2014-05-19 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-1878: --- Summary: Incorrect initialization order in JavaStreamingContext Key: SPARK-1878 URL: https://issues.apache.org/jira/browse/SPARK-1878 Project: Spark Issue

[jira] [Commented] (SPARK-1878) Incorrect initialization order in JavaStreamingContext

2014-05-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001436#comment-14001436 ] Shixiong Zhu commented on SPARK-1878: - PR: https://github.com/apache/spark/pull/822

[jira] [Commented] (SPARK-1925) Typo in org.apache.spark.mllib.tree.DecisionTree.isSampleValid

2014-05-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008584#comment-14008584 ] Shixiong Zhu commented on SPARK-1925: - PR: https://github.com/apache/spark/pull/879

[jira] [Updated] (SPARK-1925) Typo in org.apache.spark.mllib.tree.DecisionTree.isSampleValid

2014-05-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-1925: Description: I believe this is a typo: if ((level 0) (parentFilters.length == 0)) {

[jira] [Resolved] (SPARK-1925) Typo in org.apache.spark.mllib.tree.DecisionTree.isSampleValid

2014-05-26 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1925. - Resolution: Fixed Fix Version/s: 1.0.0 Typo in

[jira] [Created] (SPARK-1932) Race conditions in BlockManager.cachedPeers and ConnectionManager.onReceiveCallback

2014-05-26 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-1932: --- Summary: Race conditions in BlockManager.cachedPeers and ConnectionManager.onReceiveCallback Key: SPARK-1932 URL: https://issues.apache.org/jira/browse/SPARK-1932

[jira] [Commented] (SPARK-1932) Race conditions in BlockManager.cachedPeers and ConnectionManager.onReceiveCallback

2014-05-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009330#comment-14009330 ] Shixiong Zhu commented on SPARK-1932: - PR: https://github.com/apache/spark/pull/887

[jira] [Resolved] (SPARK-1932) Race conditions in BlockManager.cachedPeers and ConnectionManager.onReceiveCallback

2014-05-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1932. - Resolution: Fixed Fix Version/s: 1.0.0 Race conditions in BlockManager.cachedPeers and

[jira] [Created] (SPARK-2634) MapOutputTrackerWorker.mapStatuses should be thread-safe

2014-07-22 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-2634: --- Summary: MapOutputTrackerWorker.mapStatuses should be thread-safe Key: SPARK-2634 URL: https://issues.apache.org/jira/browse/SPARK-2634 Project: Spark Issue

[jira] [Created] (SPARK-3741) ConnectionManager.sendMessage may not propagate errors to MessageStatus

2014-09-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-3741: --- Summary: ConnectionManager.sendMessage may not propagate errors to MessageStatus Key: SPARK-3741 URL: https://issues.apache.org/jira/browse/SPARK-3741 Project: Spark

[jira] [Created] (SPARK-3777) Display Executor ID for Tasks in Stage page

2014-10-03 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-3777: --- Summary: Display Executor ID for Tasks in Stage page Key: SPARK-3777 URL: https://issues.apache.org/jira/browse/SPARK-3777 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-1656) Potential resource leak in HttpBroadcast, SparkSubmitArguments, FileSystemPersistenceEngine and DiskStore

2014-10-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-1656. - Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Already merged into master

[jira] [Created] (SPARK-3877) The exit code of spark-submit is still 0 when an yarn application fails

2014-10-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-3877: --- Summary: The exit code of spark-submit is still 0 when an yarn application fails Key: SPARK-3877 URL: https://issues.apache.org/jira/browse/SPARK-3877 Project: Spark

[jira] [Commented] (SPARK-3922) A global UTF8 constant for Spark

2014-10-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176615#comment-14176615 ] Shixiong Zhu commented on SPARK-3922: - PR: https://github.com/apache/spark/pull/2781

[jira] [Created] (SPARK-4008) Fix kryo with fold in KryoSerializerSuite

2014-10-20 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4008: --- Summary: Fix kryo with fold in KryoSerializerSuite Key: SPARK-4008 URL: https://issues.apache.org/jira/browse/SPARK-4008 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-4008) g

2014-10-20 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4008: Summary: g (was: Fix kryo with fold in KryoSerializerSuite) g - Key:

[jira] [Created] (SPARK-4035) Wrong format specifier in BlockerManager.scala

2014-10-21 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4035: --- Summary: Wrong format specifier in BlockerManager.scala Key: SPARK-4035 URL: https://issues.apache.org/jira/browse/SPARK-4035 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4097) Race condition in org.apache.spark.ComplexFutureAction.cancel

2014-10-27 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4097: --- Summary: Race condition in org.apache.spark.ComplexFutureAction.cancel Key: SPARK-4097 URL: https://issues.apache.org/jira/browse/SPARK-4097 Project: Spark

[jira] [Updated] (SPARK-4097) Race condition in org.apache.spark.ComplexFutureAction.cancel

2014-10-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4097: Description: There is a chance that `thread` is null when calling `thread.interrupt()`.

[jira] [Resolved] (SPARK-4008) Fix kryo with fold in KryoSerializerSuite

2014-10-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-4008. - Resolution: Fixed Fix Version/s: 1.2.0 Fix kryo with fold in KryoSerializerSuite

[jira] [Created] (SPARK-4153) All columns of the application table in the history page are sorted in alphabetical order

2014-10-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4153: --- Summary: All columns of the application table in the history page are sorted in alphabetical order Key: SPARK-4153 URL: https://issues.apache.org/jira/browse/SPARK-4153

[jira] [Created] (SPARK-4163) When fetching blocks unsuccessfully, Web UI only displays Fetch failure

2014-10-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4163: --- Summary: When fetching blocks unsuccessfully, Web UI only displays Fetch failure Key: SPARK-4163 URL: https://issues.apache.org/jira/browse/SPARK-4163 Project: Spark

[jira] [Created] (SPARK-4166) Display the executor ID in the Web UI when ExecutorLostFailure happens

2014-10-31 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4166: --- Summary: Display the executor ID in the Web UI when ExecutorLostFailure happens Key: SPARK-4166 URL: https://issues.apache.org/jira/browse/SPARK-4166 Project: Spark

[jira] [Created] (SPARK-4204) Utils.exceptionString only return the information for the outermost exception

2014-11-03 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4204: --- Summary: Utils.exceptionString only return the information for the outermost exception Key: SPARK-4204 URL: https://issues.apache.org/jira/browse/SPARK-4204 Project:

[jira] [Created] (SPARK-4265) Better extensibility for TaskEndReason

2014-11-05 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4265: --- Summary: Better extensibility for TaskEndReason Key: SPARK-4265 URL: https://issues.apache.org/jira/browse/SPARK-4265 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-4268) Use #::: to get benefit from Stream in SqlLexical.allCaseVersions

2014-11-06 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4268: --- Summary: Use #::: to get benefit from Stream in SqlLexical.allCaseVersions Key: SPARK-4268 URL: https://issues.apache.org/jira/browse/SPARK-4268 Project: Spark

[jira] [Created] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

2014-11-07 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4296: --- Summary: Throw Expression not in GROUP BY when using same expression in group by clause and select clause Key: SPARK-4296 URL: https://issues.apache.org/jira/browse/SPARK-4296

[jira] [Comment Edited] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

2014-11-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201934#comment-14201934 ] Shixiong Zhu edited comment on SPARK-4296 at 11/7/14 11:21 AM:

[jira] [Commented] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

2014-11-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201934#comment-14201934 ] Shixiong Zhu commented on SPARK-4296: - Stack trace: {code:java} Aggregate

[jira] [Commented] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

2014-11-07 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14201937#comment-14201937 ] Shixiong Zhu commented on SPARK-4296: - Original reported by Tridib Samanta at

[jira] [Created] (SPARK-4310) Submitted column in Stage page doesn't sort by time

2014-11-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4310: --- Summary: Submitted column in Stage page doesn't sort by time Key: SPARK-4310 URL: https://issues.apache.org/jira/browse/SPARK-4310 Project: Spark Issue Type:

[jira] [Created] (SPARK-4313) Thread Dump link is broken in yarn-cluster mode

2014-11-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4313: --- Summary: Thread Dump link is broken in yarn-cluster mode Key: SPARK-4313 URL: https://issues.apache.org/jira/browse/SPARK-4313 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4361) SparkContext HadoopRDD is not clear about how to use a Hadoop Configuration

2014-11-12 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4361: --- Summary: SparkContext HadoopRDD is not clear about how to use a Hadoop Configuration Key: SPARK-4361 URL: https://issues.apache.org/jira/browse/SPARK-4361 Project:

[jira] [Created] (SPARK-4363) The Broadcast example is out of date

2014-11-12 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4363: --- Summary: The Broadcast example is out of date Key: SPARK-4363 URL: https://issues.apache.org/jira/browse/SPARK-4363 Project: Spark Issue Type: Documentation

[jira] [Created] (SPARK-4364) Some variable types in org.apache.spark.streaming.JavaAPISuite are wrong

2014-11-12 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4364: --- Summary: Some variable types in org.apache.spark.streaming.JavaAPISuite are wrong Key: SPARK-4364 URL: https://issues.apache.org/jira/browse/SPARK-4364 Project: Spark

[jira] [Closed] (SPARK-4364) Some variable types in org.apache.spark.streaming.JavaAPISuite are wrong

2014-11-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu closed SPARK-4364. --- Resolution: Duplicate Sorry. Didn't notice SPARK-4297 Some variable types in

[jira] [Created] (SPARK-4379) RDD.checkpoint throws an Exception

2014-11-13 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4379: --- Summary: RDD.checkpoint throws an Exception Key: SPARK-4379 URL: https://issues.apache.org/jira/browse/SPARK-4379 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-4397) Reorganize 'implicit's to improve the API convenience

2014-11-13 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4397: --- Summary: Reorganize 'implicit's to improve the API convenience Key: SPARK-4397 URL: https://issues.apache.org/jira/browse/SPARK-4397 Project: Spark Issue

[jira] [Created] (SPARK-4472) Print Spark context available as sc. only when SparkContext is created successfully

2014-11-18 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4472: --- Summary: Print Spark context available as sc. only when SparkContext is created successfully Key: SPARK-4472 URL: https://issues.apache.org/jira/browse/SPARK-4472

[jira] [Created] (SPARK-4481) Some comments for `updateStateByKey` are wrong

2014-11-18 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4481: --- Summary: Some comments for `updateStateByKey` are wrong Key: SPARK-4481 URL: https://issues.apache.org/jira/browse/SPARK-4481 Project: Spark Issue Type:

[jira] [Created] (SPARK-4505) Reduce the memory usage of CompactBuffer[T] when T is a primitive type

2014-11-19 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4505: --- Summary: Reduce the memory usage of CompactBuffer[T] when T is a primitive type Key: SPARK-4505 URL: https://issues.apache.org/jira/browse/SPARK-4505 Project: Spark

[jira] [Created] (SPARK-4644) Implement skewed join

2014-11-27 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4644: --- Summary: Implement skewed join Key: SPARK-4644 URL: https://issues.apache.org/jira/browse/SPARK-4644 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-4644) Implement skewed join

2014-11-27 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4644: Attachment: Skewed Join Design Doc.pdf The design doc of skewed join Implement skewed join

[jira] [Commented] (SPARK-4644) Implement skewed join

2014-11-28 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228308#comment-14228308 ] Shixiong Zhu commented on SPARK-4644: - I disagree to use `broadcast join` because: 1.

[jira] [Created] (SPARK-4661) Minor code and docs cleanup

2014-11-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4661: --- Summary: Minor code and docs cleanup Key: SPARK-4661 URL: https://issues.apache.org/jira/browse/SPARK-4661 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4644) Implement skewed join

2014-11-30 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14229369#comment-14229369 ] Shixiong Zhu commented on SPARK-4644: - {quote} User 'zsxwing' has created a pull

[jira] [Created] (SPARK-4664) Overflow of `maxFrameSizeBytes`

2014-11-30 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4664: --- Summary: Overflow of `maxFrameSizeBytes` Key: SPARK-4664 URL: https://issues.apache.org/jira/browse/SPARK-4664 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

2014-12-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14229553#comment-14229553 ] Shixiong Zhu commented on SPARK-3926: - {noformat} scala :javap -private -c

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

2014-12-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14229599#comment-14229599 ] Shixiong Zhu commented on SPARK-3926: - {quote} To allow subtypes of non-serializable

[jira] [Issue Comment Deleted] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

2014-12-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-3926: Comment: was deleted (was: {noformat} scala :javap -private -c

[jira] [Commented] (SPARK-4644) Implement skewed join

2014-12-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14230920#comment-14230920 ] Shixiong Zhu commented on SPARK-4644: - [~pwendell] Fair enough. I'll see what I can do

[jira] [Commented] (SPARK-3926) result of JavaRDD collectAsMap() is not serializable

2014-12-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231098#comment-14231098 ] Shixiong Zhu commented on SPARK-3926: - I checked the usage of

[jira] [Commented] (SPARK-4644) Implement skewed join

2014-12-02 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232522#comment-14232522 ] Shixiong Zhu commented on SPARK-4644: - Looks `groupByKey` is really different from

[jira] [Created] (SPARK-4715) ShuffleMemoryManager.tryToAcquire may return a negative value

2014-12-03 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4715: --- Summary: ShuffleMemoryManager.tryToAcquire may return a negative value Key: SPARK-4715 URL: https://issues.apache.org/jira/browse/SPARK-4715 Project: Spark

[jira] [Created] (SPARK-4795) Redesign the primitive type = Writable implicit APIs to make them be activated automatically

2014-12-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4795: --- Summary: Redesign the primitive type = Writable implicit APIs to make them be activated automatically Key: SPARK-4795 URL: https://issues.apache.org/jira/browse/SPARK-4795

[jira] [Created] (SPARK-4812) SparkPlan.codegenEnabled may be initialized to a wrong value

2014-12-09 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4812: --- Summary: SparkPlan.codegenEnabled may be initialized to a wrong value Key: SPARK-4812 URL: https://issues.apache.org/jira/browse/SPARK-4812 Project: Spark

[jira] [Created] (SPARK-4813) ContextWaiter didn't handle 'spurious wakeup'

2014-12-10 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4813: --- Summary: ContextWaiter didn't handle 'spurious wakeup' Key: SPARK-4813 URL: https://issues.apache.org/jira/browse/SPARK-4813 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4812) SparkPlan.codegenEnabled may be initialized to a wrong value

2014-12-10 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4812: Description: The problem is `codegenEnabled` is `val`, but it uses a `val` `sqlContext`, which can

[jira] [Created] (SPARK-4824) Join should use `Iterator` rather than `Iterable`

2014-12-10 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4824: --- Summary: Join should use `Iterator` rather than `Iterable` Key: SPARK-4824 URL: https://issues.apache.org/jira/browse/SPARK-4824 Project: Spark Issue Type:

[jira] [Updated] (SPARK-4812) SparkPlan.codegenEnabled may be initialized to a wrong value

2014-12-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4812: Description: The problem is `codegenEnabled` is `val`, but it uses a `val` `sqlContext`, which can

[jira] [Created] (SPARK-4859) Improve StreamingListenerBus

2014-12-15 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4859: --- Summary: Improve StreamingListenerBus Key: SPARK-4859 URL: https://issues.apache.org/jira/browse/SPARK-4859 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2075) Anonymous classes are missing from Spark distribution

2014-12-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252906#comment-14252906 ] Shixiong Zhu commented on SPARK-2075: - Please take a look at my PR Anonymous classes

[jira] [Created] (SPARK-4918) Reuse Text in saveAsTextFile

2014-12-22 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4918: --- Summary: Reuse Text in saveAsTextFile Key: SPARK-4918 URL: https://issues.apache.org/jira/browse/SPARK-4918 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-4918) Reuse Text in saveAsTextFile

2014-12-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4918: Component/s: Spark Core Reuse Text in saveAsTextFile

[jira] [Created] (SPARK-4931) Fix the messy format about log4j in running-on-yarn.md

2014-12-22 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4931: --- Summary: Fix the messy format about log4j in running-on-yarn.md Key: SPARK-4931 URL: https://issues.apache.org/jira/browse/SPARK-4931 Project: Spark Issue

[jira] [Updated] (SPARK-4931) Fix the messy format about log4j in running-on-yarn.md

2014-12-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4931: Attachment: log4j.png Fix the messy format about log4j in running-on-yarn.md

[jira] [Created] (SPARK-4951) A busy executor may be killed when dynamicAllocation is enabled

2014-12-23 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-4951: --- Summary: A busy executor may be killed when dynamicAllocation is enabled Key: SPARK-4951 URL: https://issues.apache.org/jira/browse/SPARK-4951 Project: Spark

[jira] [Updated] (SPARK-4873) Use `Future.zip` instead of `Future.flatMap`(for-loop) in WriteAheadLogBasedBlockHandler

2014-12-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4873: Description: Use Future.zip instead of Future.flatMap(for-loop). zip implies these two Futures will

[jira] [Commented] (SPARK-4389) Set akka.remote.netty.tcp.bind-hostname=0.0.0.0 so driver can be located behind NAT

2014-12-26 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259020#comment-14259020 ] Shixiong Zhu commented on SPARK-4389: - Try to make sure I understand this issue: The

[jira] [Updated] (SPARK-5379) Add awaitTerminationOrTimeout

2015-01-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5379: Description: Add `awaitTerminationOrTimeout` to return if the waiting time elapsed: 1 `true` if

[jira] [Created] (SPARK-5379) Add awaitTerminationOrTimeout

2015-01-22 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-5379: --- Summary: Add awaitTerminationOrTimeout Key: SPARK-5379 URL: https://issues.apache.org/jira/browse/SPARK-5379 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-4296) Throw Expression not in GROUP BY when using same expression in group by clause and select clause

2015-01-11 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4296: Target Version/s: 1.3.0, 1.2.1 (was: 1.2.0) Affects Version/s: 1.1.1

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14273560#comment-14273560 ] Shixiong Zhu commented on SPARK-5124: - {quote} 1. Let's not rely on the property of

[jira] [Updated] (SPARK-3019) Pluggable block transfer (data plane communication) interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-3019: Attachment: Pluggable RPC - draft 1.pdf Pluggable block transfer (data plane communication)

[jira] [Updated] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5124: Attachment: Pluggable RPC - draft 1.pdf Standardize internal RPC interface

[jira] [Updated] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5124: Attachment: (was: Pluggable RPC - draft 1.pdf) Standardize internal RPC interface

[jira] [Updated] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5172: Description: Steps to check it: 1. Download spark-1.2.0-bin-hadoop2.4.tgz from

[jira] [Updated] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-5124: Attachment: Pluggable RPC - draft 1.pdf Standardize internal RPC interface

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270809#comment-14270809 ] Shixiong Zhu commented on SPARK-5124: - {quote} 1. For DAGScheduler, we are probably OK

[jira] [Created] (SPARK-5126) No error log for a typo master url

2015-01-06 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-5126: --- Summary: No error log for a typo master url Key: SPARK-5126 URL: https://issues.apache.org/jira/browse/SPARK-5126 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-4859) Refactor LiveListenerBus and StreamingListenerBus

2015-01-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4859: Description: [#4006|https://github.com/apache/spark/pull/4006] refactors LiveListenerBus and

[jira] [Created] (SPARK-5219) Race condition in TaskSchedulerImpl and TaskSetManager

2015-01-13 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-5219: --- Summary: Race condition in TaskSchedulerImpl and TaskSetManager Key: SPARK-5219 URL: https://issues.apache.org/jira/browse/SPARK-5219 Project: Spark Issue

[jira] [Commented] (SPARK-5124) Standardize internal RPC interface

2015-01-13 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14275014#comment-14275014 ] Shixiong Zhu commented on SPARK-5124: - For 1) I prefer to finish it before this JIRA.

  1   2   3   4   5   6   7   8   9   10   >