Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4150
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-75131653
Hi @shenh062326 since this is a duplicate would you mind closing this PR?
The associated JIRA is already closed. Thanks.
---
If your project is set up for it, you can
Github user ksakellis commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-73275084
I agree with @srowen and @sryza. Also given #4067 this metric should really
just report size
---
If your project is set up for it, you can reply to this email and have
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-73271526
@shenh062326 Sandy is saying that in those other cases, the values you are
getting are not even in the same units, and so would be invalid. I believe we
should close this
Github user shenh062326 commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-71347933
If we use a inputFormat that donât instanc of
org.apache.hadoop.mapreduce.lib.input.{CombineFileSplit, FileSplit}, then we
can't get information of input metrics.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-71177075
Given this reasoning, it does seem like this is a duplicate of SPARK-5199
---
If your project is set up for it, you can reply to this email and have your
reply appear on G
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-71055028
I think this is a duplicate of #4050, which only adds support for
`CombineFileSplit`s. We shouldn't add support for generic `InputSplit`s
because many input formats do not
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-70999157
My only question was whether `getLength()` is indeed defined in the
`InputSplit` interface in older Hadoop versions, but it looks like it is. This
change compiles with def
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-70960330
[Test build #25937 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25937/consoleFull)
for PR 4150 at commit
[`9e04a54`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-70960337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4150#issuecomment-70953515
[Test build #25937 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25937/consoleFull)
for PR 4150 at commit
[`9e04a54`](https://githu
GitHub user shenh062326 opened a pull request:
https://github.com/apache/spark/pull/4150
[SPARK-5347][CORE] Change FileSplit to InputSplit in update inputMetrics
When inputFormatClass is set to CombineFileInputFormat, input metrics show
that input is empty. It don't appear is spark-
12 matches
Mail list logo
