Stamatis Zampetakis created TEZ-4407:
----------------------------------------
Summary: Misleading split info in TezSplitGrouper logs when
adjusting small splits
Key: TEZ-4407
URL: https://issues.apache.org/jira/browse/TEZ-4407
Project: Apache Tez
Issue Type: Bug
Affects Versions: 0.10.1
Reporter: Stamatis Zampetakis
The log message in [TezSplitGrouper.getGroupedSplits|
https://github.com/apache/tez/blob/627f33077480afdcefc0611fbde87d6be0010176/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/grouper/TezSplitGrouper.java#L272]
is misleading and quite often it may show that the "Desired splits" is the
same with "New desired splits" which does not make much sense.
{noformat}
2022-04-19 01:59:05,064 [INFO] [App Shared Pool - #18]
|grouper.TezSplitGrouper|: Desired splits: 4 too large. Desired splitLength:
7589213 Min splitLength: 268435456 New desired splits: 4 Final desired splits:
4 All splits have localhost: false Total length: 1047311531 Original splits: 18
{noformat}
Due to the above it is difficult/impossible to see what was the initial desired
splits without reading the code.
This was caused by TEZ-3291.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
