Ádám Szita created HIVE-26428:
---------------------------------
Summary: Limit usage of LLAP BPWrapper to threads of IO threadpools
Key: HIVE-26428
URL: https://issues.apache.org/jira/browse/HIVE-26428
Project: Hive
Issue Type: Improvement
Reporter: Ádám Szita
Assignee: Ádám Szita
BPWrapper is used in LRFU cache eviction policy to decrease the time spent
waiting for lock on the heap. This is done by adding a buffer as threadlocal
and accumulating CacheableBuffer instances there before trying to acquire a
lock. This works well when we have threads from pools such as IO-Elevator
threads or OrcEncode threads.
For ephemeral threads there's no advantage of doing this as the buffers in
threadlocals may never reach the heap or list structures of LRFU, thereby also
making evictions less efficient. This can happen e.g. LLAPCacheAwareFS is used
with Parquet, where we're using the Tez threads for both execution and IO.
We should disable BPWrappers for such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
