[ 
https://issues.apache.org/jira/browse/HIVE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HIVE-10092:
------------------------------------
    Attachment: HIVE-10092.patch

Committed to branch... attaching patch here for reference since bugs are 
possible

> LLAP: improve how buffers are locked for split
> ----------------------------------------------
>
>                 Key: HIVE-10092
>                 URL: https://issues.apache.org/jira/browse/HIVE-10092
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-10092.patch
>
>
> Right now, for simplicity, entire split of decompressed buffers is locked in 
> cache, in case some buffers are shared between RGs, to avoid dealing with 
> situations where we uncompress some data, pass it on to processor for RG N, 
> then processor processes and unlocks it, and before we can pass it on for RG 
> N+1 it's evicted. 
> However, if split is too big, and cache is small, or many splits are 
> processed at the same time, this can result in a deadlock as entire cache is 
> locked. We need to improve locking to be more granular and probably also try 
> to avoid deadlocks in general (bypass cache?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to