[
https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal Vijayaraghavan reassigned TEZ-4208:
-----------------------------------------
Assignee: Rajesh Balamohan
> Pipelinesorter uses single SortSpan after spill
> -----------------------------------------------
>
> Key: TEZ-4208
> URL: https://issues.apache.org/jira/browse/TEZ-4208
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Major
> Attachments: TEZ-4208.1.patch, TEZ-4208.2.patch, q67_sorter.log
>
>
> Though it could have created multiple spans, tez always uses the first span
> after spill. It is quite possible that other spans are bigger compared to the
> first one, due to progressive space allocation. Fixing this would help in
> reducing the number of spills (depending on the jobs) and lesser load for
> indexcache entries (as lesser number of files have to be opened).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
