[ https://issues.apache.org/jira/browse/TEZ-1993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481929#comment-14481929 ]
Gopal V commented on TEZ-1993: ------------------------------ Committed to trunk, thanks [~bikassaha]. This is going to be plugged in via HIVE-10114 split strategies implementation > Implement a pluggable InputSizeEstimator for grouping fairly > ------------------------------------------------------------ > > Key: TEZ-1993 > URL: https://issues.apache.org/jira/browse/TEZ-1993 > Project: Apache Tez > Issue Type: Bug > Affects Versions: 0.7.0 > Reporter: Gopal V > Assignee: Gopal V > Fix For: 0.7.0 > > Attachments: TEZ-1993.1.patch, TEZ-1993.2.patch, TEZ-1993.3.patch > > > Split grouping is currently done using a file size measurement which is the > exact size of the split as it stays at rest on HDFS. > This is not valid for columnar formats and especially suffers from highly > compressible data skews. -- This message was sent by Atlassian JIRA (v6.3.4#6332)