Github user ash211 commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-70440912
Sounds good, I concur. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2310
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-70212481
@ash211 I think we can close this issue now that we have merged #3120
---
If your project is set up for it, you can reply to this email and have your
reply appear on Gi
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-63892481
@ash211 Hopefully this could be merged into 1.2, once you rebase it to
master and keep the API unchanged, thanks!
---
If your project is set up for it, you can reply to t
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/2310#discussion_r20682906
--- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala
---
@@ -167,7 +169,7 @@ case class InputMetrics(readMethod:
DataReadMethod.Value) {
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-56322813
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20637/consoleFull)
for PR 2310 at commit
[`576483b`](https://github.com/a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-56320019
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20637/consoleFull)
for PR 2310 at commit
[`576483b`](https://github.com/ap
Github user ash211 commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-56319874
Switched to an AtomicLong. Also needed to move instantiation of metrics
outside of compute() This now works in my testing for the HadoopRDD ->
CoalescedRDD pipeline like
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-56290971
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20616/consoleFull)
for PR 2310 at commit
[`df9ce44`](https://github.com/a
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-56290108
@ash211 this particular fix will only work for HadoopRDD but not other
cases where we track input bytes read. A more general fix is pretty simple
though, I think.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-56289949
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20616/consoleFull)
for PR 2310 at commit
[`df9ce44`](https://github.com/ap
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-56289925
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user ash211 commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-54741169
Failed tests seem to be Jenkins broken-ness
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-54741057
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19946/consoleFull)
for PR 2310 at commit
[`df9ce44`](https://github.com/a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2310#issuecomment-54739987
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19946/consoleFull)
for PR 2310 at commit
[`df9ce44`](https://github.com/ap
GitHub user ash211 opened a pull request:
https://github.com/apache/spark/pull/2310
SPARK-2630 Input data size of CoalescedRDD counts only one partition
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ash211/spark SPARK-2630
Alt
16 matches
Mail list logo
