[ https://issues.apache.org/jira/browse/HIVE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297178#comment-14297178 ]
Owen O'Malley commented on HIVE-9451: ------------------------------------- We should also record the stripe size that was used as the file was written. That gives a strict upper bound on the size of memory in the writer. > Add max size of column dictionaries to ORC metadata > --------------------------------------------------- > > Key: HIVE-9451 > URL: https://issues.apache.org/jira/browse/HIVE-9451 > Project: Hive > Issue Type: Improvement > Reporter: Owen O'Malley > > To predict the amount of memory required to read an ORC file we need to know > the size of the dictionaries for the columns that we are reading. I propose > adding the number of bytes for each column's dictionary to the stripe's > column statistics. The file's column statistics would have the maximum > dictionary size for each column. -- This message was sent by Atlassian JIRA (v6.3.4#6332)