[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4878#issuecomment-77522081 I wasn't expecting this to be closed. Although parts of the changes didn't seem particularly useful, I believe the changes in `ExternalSorter` could potentially lead t

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-05 Thread ravipesala
Github user ravipesala commented on the pull request: https://github.com/apache/spark/pull/4878#issuecomment-77407789 I am sorry, the comments are valid. I am closing this PR. Thank you for reviewing it. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-05 Thread ravipesala
Github user ravipesala closed the pull request at: https://github.com/apache/spark/pull/4878 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-04 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/4878#discussion_r25821421 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/FileSystemPersistenceEngine.scala --- @@ -68,7 +68,7 @@ private[spark] class FileSystemPer

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-04 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/4878#discussion_r25821370 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -106,7 +106,7 @@ class IndexShuffleBlockManager(conf: Spark

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-04 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/4878#issuecomment-77264094 Sorry to chime in late, but have you done performance tests with this to see if it makes a difference? There two issues I see here: (1) This isn't the plac

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/4878#discussion_r25748317 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala --- @@ -721,13 +721,13 @@ private[spark] class ExternalSorter[K, V, C](

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4878#discussion_r25747285 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/FileSystemPersistenceEngine.scala --- @@ -58,7 +58,7 @@ private[spark] class FileSystemPersis

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4878#issuecomment-77083944 LGTM pending @srowen's comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4878#issuecomment-77025584 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4878#issuecomment-77025569 [Test build #28236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28236/consoleFull) for PR 4878 at commit [`c7a739e`](https://gith

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4878#discussion_r25714850 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/FileSystemPersistenceEngine.scala --- @@ -58,7 +58,7 @@ private[spark] class FileSystemPersistenc

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4878#issuecomment-77009074 [Test build #28236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28236/consoleFull) for PR 4878 at commit [`c7a739e`](https://githu

[GitHub] spark pull request: [SPARK-5920][CORE]BufferedInputStream is added...

2015-03-03 Thread ravipesala
GitHub user ravipesala opened a pull request: https://github.com/apache/spark/pull/4878 [SPARK-5920][CORE]BufferedInputStream is added at required places BufferedInputStream and BufferedOutputStream is added at required places. You can merge this pull request into a Git repository b