[
https://issues.apache.org/jira/browse/HIVE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389534#comment-14389534
]
Sergey Shelukhin commented on HIVE-10092:
-----------------------------------------
Separate jira for cache bypassing is HIVE-10170
> LLAP: improve how buffers are locked for split
> ----------------------------------------------
>
> Key: HIVE-10092
> URL: https://issues.apache.org/jira/browse/HIVE-10092
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Fix For: llap
>
> Attachments: HIVE-10092.patch
>
>
> Right now, for simplicity, entire split of decompressed buffers is locked in
> cache, in case some buffers are shared between RGs, to avoid dealing with
> situations where we uncompress some data, pass it on to processor for RG N,
> then processor processes and unlocks it, and before we can pass it on for RG
> N+1 it's evicted.
> However, if split is too big, and cache is small, or many splits are
> processed at the same time, this can result in a deadlock as entire cache is
> locked. We need to improve locking to be more granular and probably also try
> to avoid deadlocks in general (bypass cache?)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
