[
https://issues.apache.org/jira/browse/HIVE-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133458#comment-16133458
]
Sergey Shelukhin commented on HIVE-17344:
-----------------------------------------
I don't think there's actually a bug since everywhere this buffer is passed in
it's either allocated exactly to size, or read from disk the same way. Reads
duplicate it so remaining will not change.
Looking at capacity is not correct in all cases either, e.g. if someone were to
cache multiple slices of the same large buffer, like LLAP allocator does in
other places, it would be accounted for multiple times. So, I dunno if this
change is needed. Perhaps instead at caching time it should check capacity and
remaining are the same.
> LocalCache element memory usage is not calculated properly.
> -----------------------------------------------------------
>
> Key: HIVE-17344
> URL: https://issues.apache.org/jira/browse/HIVE-17344
> Project: Hive
> Issue Type: Bug
> Reporter: Janos Gub
> Assignee: Janos Gub
> Attachments: HIVE-17344.patch
>
>
> Orc footer cache has a calculation of memory usage:
> {code:java}
> public int getMemoryUsage() {
> return bb.remaining() + 100; // 100 is for 2 longs, BB and java overheads
> (semi-arbitrary).
> }
> {code}
> ByteBuffer.remaining returns the remaining space in the bytebuffer, thus
> allowing this cache have elements MAXWEIGHT/100 of arbitrary size. I think
> the correct solution would be bb.capacity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
