[
https://issues.apache.org/jira/browse/TEZ-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4498:
------------------------------
Description:
This problem is present for a long time ago, we maintain 2 completely different
groups of classes for shuffling:
UnorderedKVInput -> *ShuffleManager -> Fetcher*
OrderedGroupedKVInput -> *Shuffle -> ShuffleScheduler -> FetcherOrderedGrouped*
I don't think any of us would start to merge them in a single patch (and if so,
we can wait for a complete review and fight with conflicts all the time),
instead, we can make smaller refactors that make sense on their own.
(I'm 100% this is not the first ticket addressing this but I cannot find the
earlier ones at the moment)
was:
This problem is present for a long time ago, we maintain 2 completely different
groups of classes for shuffling:
UnorderedKVInput -> *ShuffleManager -> Fetcher*
OrderedGroupedKVInput -> *Shuffle -> ShuffleScheduler -> FetcherOrderedGrouped*
I don't think any of us would start to merge them in a single patch, instead,
we can make smaller refactors that makes sense on their own.
(I'm 100% this is not the first ticket addressing this but I cannot find the
earlier ones)
> Refactor/merge ordered and unordered shuffle
> --------------------------------------------
>
> Key: TEZ-4498
> URL: https://issues.apache.org/jira/browse/TEZ-4498
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Priority: Major
>
> This problem is present for a long time ago, we maintain 2 completely
> different groups of classes for shuffling:
> UnorderedKVInput -> *ShuffleManager -> Fetcher*
> OrderedGroupedKVInput -> *Shuffle -> ShuffleScheduler ->
> FetcherOrderedGrouped*
> I don't think any of us would start to merge them in a single patch (and if
> so, we can wait for a complete review and fight with conflicts all the time),
> instead, we can make smaller refactors that make sense on their own.
> (I'm 100% this is not the first ticket addressing this but I cannot find the
> earlier ones at the moment)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
