[
https://issues.apache.org/jira/browse/IMPALA-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572642#comment-17572642
]
Michael Smith edited comment on IMPALA-10214 at 7/28/22 8:38 PM:
-----------------------------------------------------------------
Local file handles are already cached because Ozone shares the HDFS
implementation.
The file handle cache now supports local file handles, as well as remote
HDFS/S3/ABFS handles. In evaluating Ozone, [~weichiu] identified that memory
used by the file handle cache was substantially larger than HDFS when using
remote handles. The original measurements behind setting
{{max_cached_file_handles=20000}} probably no longer apply, but I'm not sure
it's safe to change it.
was (Author: JIRAUSER288956):
Local file handles are already cached because Ozone shares the HDFS
implementation. We likely want to assign a separate cache for Ozone local file
handles ({{DiskIoMgr::file_handle_cache_}}).
> Ozone support for file handle cache
> -----------------------------------
>
> Key: IMPALA-10214
> URL: https://issues.apache.org/jira/browse/IMPALA-10214
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Sahil Takiar
> Priority: Major
>
> This is dependent on the Ozone input streams supporting the {{CanUnbuffer}}
> interface first (last I checked, the input streams don't implement the
> interface).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
