Ádám Szita created HIVE-24297:
---------------------------------
Summary: LLAP buffer collision causes NPE
Key: HIVE-24297
URL: https://issues.apache.org/jira/browse/HIVE-24297
Project: Hive
Issue Type: Bug
Reporter: Ádám Szita
Assignee: Ádám Szita
HIVE-23741 introduced an optimization so that CacheTags are not stored on
buffer level, but rather on file level, as one cache tag can only relate to one
file. With this change a buffer->filecache reference was introduced so that the
buffer's tag can be calculated with an extra indirection i.e.
buffer.filecache.tag.
However during buffer collision in putFileData method, we don't set the
filecache reference of the collided (new) buffer:
[https://github.com/apache/hive/commit/2e18a7408a8dd49beecad8d66bfe054b7dc474da#diff-d2ccd7cf3042845a0812a5e118f82db49253d82fc86449ffa408903bf434fb6dR309-R311]
Later this cases NPE when the new (instantly decRef'ed) buffer is evicted:
{code:java}
Caused by: java.lang.NullPointerException
at
java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:778)
at
java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1546)
at
org.apache.hadoop.hive.llap.cache.CacheContentsTracker.getTagState(CacheContentsTracker.java:129)
at
org.apache.hadoop.hive.llap.cache.CacheContentsTracker.getTagState(CacheContentsTracker.java:125)
at
org.apache.hadoop.hive.llap.cache.CacheContentsTracker.reportRemoved(CacheContentsTracker.java:109)
at
org.apache.hadoop.hive.llap.cache.CacheContentsTracker.notifyEvicted(CacheContentsTracker.java:238)
at
org.apache.hadoop.hive.llap.cache.LowLevelLrfuCachePolicy.evictSomeBlocks(LowLevelLrfuCachePolicy.java:276)
at
org.apache.hadoop.hive.llap.cache.CacheContentsTracker.evictSomeBlocks(CacheContentsTracker.java:177)
at
org.apache.hadoop.hive.llap.cache.LowLevelCacheMemoryManager.reserveMemory(LowLevelCacheMemoryManager.java:98)
at
org.apache.hadoop.hive.llap.cache.LowLevelCacheMemoryManager.reserveMemory(LowLevelCacheMemoryManager.java:65)
at
org.apache.hadoop.hive.llap.cache.BuddyAllocator.allocateMultiple(BuddyAllocator.java:323)
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.allocateMultiple(EncodedReaderImpl.java:1302)
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedStream(EncodedReaderImpl.java:930)
at
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:506)
... 16 more {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
