[
https://issues.apache.org/jira/browse/IMPALA-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-6638.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.12.0
> File handle cache shows contention when cold
> --------------------------------------------
>
> Key: IMPALA-6638
> URL: https://issues.apache.org/jira/browse/IMPALA-6638
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.12.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
> Fix For: Impala 2.12.0
>
>
> Performance tests show that when the file handle cache is cold, there is
> contention on the file handle cache partition lock. This added contention is
> particularly severe when there are multiple IO threads accessing the same
> file (e.g. when there is a query accessing multiple Parquet columns). This is
> because the IO threads all map to the same partition because they are
> accessing the same file.
> The contention is due to the fact that FileHandleCache::GetFileHandle() holds
> the lock while it opens the file handle. This lengthens the critical section
> considerably, because opening a file handle involves network traffic to the
> NameNode. This contention does not exist when the cache is hot.
> FileHandleCache::GetFileHandle() should drop the lock while it is opening the
> file handle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
