[
https://issues.apache.org/jira/browse/IMPALA-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-6232.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.11.0
commit 1f1bff8e8d35b66308a1e865cdc8bce41ce89873
Author: Joe McDonnell <joemcdonn...@cloudera.com>
Date: Mon Dec 4 10:21:33 2017 -0800
IMPALA-6232: Disable file handle cache by default
There are scenarios where HDFS file appends or HDFS file
overwrites can lead to HDFS disabling short circuit reads.
Since this can be a performance regression, this changes
the default value for max_cached_file_handles to 0 to
disable the file handle cache by default. This also changes
the default value for unused_file_handle_timeout_sec to 270.
If users enable the file handle cache, this setting will
prevent some of the scenarios that disable short circuit
reads.
Ran existing file handle cache tests to verify that there
is no impact.
Change-Id: Iea7f943f63b72b42286a9e8b9987308baa79d7b0
Reviewed-on: http://gerrit.cloudera.org:8080/8750
Reviewed-by: Joe McDonnell <joemcdonn...@cloudera.com>
Tested-by: Impala Public Jenkins
> Short circuit reads disabled when using Impala HDFS file handle cache
> ---------------------------------------------------------------------
>
> Key: IMPALA-6232
> URL: https://issues.apache.org/jira/browse/IMPALA-6232
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.10.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Fix For: Impala 2.11.0
>
>
> In Impala 2.10, the HDFS file handle cache was enabled by default. However,
> testing has revealed that in cases where files are overwritten or appended,
> the file handle can encounter an error that causes HDFS to disable short
> circuit reads for 10 minutes. See
> [HDFS-12528|https://issues.apache.org/jira/browse/HDFS-12528].
> Due to this performance impact and the associated unpredictability, Impala
> should disable the file handle cache by default until this issue is resolved.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
