[
https://issues.apache.org/jira/browse/TEZ-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4499:
------------------------------
Summary: Move shuffle counters to a common class (was: Refactor shuffle
counters to a common class)
> Move shuffle counters to a common class
> ---------------------------------------
>
> Key: TEZ-4499
> URL: https://issues.apache.org/jira/browse/TEZ-4499
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: László Bodor
> Priority: Major
>
> Both ShuffleManager and ShuffleScheduler has fields like:
> {code}
> this.approximateInputRecords =
> inputContext.getCounters().findCounter(TaskCounter.APPROXIMATE_INPUT_RECORDS);
> this.shuffledInputsCounter =
> inputContext.getCounters().findCounter(TaskCounter.NUM_SHUFFLED_INPUTS);
> this.failedShufflesCounter =
> inputContext.getCounters().findCounter(TaskCounter.NUM_FAILED_SHUFFLE_INPUTS);
> this.bytesShuffledCounter =
> inputContext.getCounters().findCounter(TaskCounter.SHUFFLE_BYTES);
> this.decompressedDataSizeCounter =
> inputContext.getCounters().findCounter(TaskCounter.SHUFFLE_BYTES_DECOMPRESSED);
> this.bytesShuffledToDiskCounter =
> inputContext.getCounters().findCounter(TaskCounter.SHUFFLE_BYTES_TO_DISK);
> this.bytesShuffledToMemCounter =
> inputContext.getCounters().findCounter(TaskCounter.SHUFFLE_BYTES_TO_MEM);
> this.bytesShuffledDirectDiskCounter =
> inputContext.getCounters().findCounter(TaskCounter.SHUFFLE_BYTES_DISK_DIRECT);
> {code}
> we should collect then into a separate class like "ShuffleCounters"
> initialized by an inputContext and increment them through an interface
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
