Owen O'Malley created HIVE-11807: ------------------------------------ Summary: Set ORC buffer size in relation to set stripe size Key: HIVE-11807 URL: https://issues.apache.org/jira/browse/HIVE-11807 Project: Hive Issue Type: Improvement Components: File Formats Reporter: Owen O'Malley Assignee: Owen O'Malley
A customer produced ORC files with very small stripe sizes (10k rows/stripe) by setting a small 64MB stripe size and 256K buffer size for a 54 column table. At that size, each of the streams only get a buffer or two before the stripe size is reached. The current code uses the available memory instead of the stripe size and thus doesn't shrink the buffer size if the JVM has much more memory than the stripe size. -- This message was sent by Atlassian JIRA (v6.3.4#6332)