[ https://issues.apache.org/jira/browse/IMPALA-6232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe McDonnell resolved IMPALA-6232. ----------------------------------- Resolution: Fixed Fix Version/s: Impala 2.11.0 commit 1f1bff8e8d35b66308a1e865cdc8bce41ce89873 Author: Joe McDonnell <joemcdonn...@cloudera.com> Date: Mon Dec 4 10:21:33 2017 -0800 IMPALA-6232: Disable file handle cache by default There are scenarios where HDFS file appends or HDFS file overwrites can lead to HDFS disabling short circuit reads. Since this can be a performance regression, this changes the default value for max_cached_file_handles to 0 to disable the file handle cache by default. This also changes the default value for unused_file_handle_timeout_sec to 270. If users enable the file handle cache, this setting will prevent some of the scenarios that disable short circuit reads. Ran existing file handle cache tests to verify that there is no impact. Change-Id: Iea7f943f63b72b42286a9e8b9987308baa79d7b0 Reviewed-on: http://gerrit.cloudera.org:8080/8750 Reviewed-by: Joe McDonnell <joemcdonn...@cloudera.com> Tested-by: Impala Public Jenkins > Short circuit reads disabled when using Impala HDFS file handle cache > --------------------------------------------------------------------- > > Key: IMPALA-6232 > URL: https://issues.apache.org/jira/browse/IMPALA-6232 > Project: IMPALA > Issue Type: Bug > Components: Backend > Affects Versions: Impala 2.10.0 > Reporter: Joe McDonnell > Assignee: Joe McDonnell > Priority: Blocker > Fix For: Impala 2.11.0 > > > In Impala 2.10, the HDFS file handle cache was enabled by default. However, > testing has revealed that in cases where files are overwritten or appended, > the file handle can encounter an error that causes HDFS to disable short > circuit reads for 10 minutes. See > [HDFS-12528|https://issues.apache.org/jira/browse/HDFS-12528]. > Due to this performance impact and the associated unpredictability, Impala > should disable the file handle cache by default until this issue is resolved. -- This message was sent by Atlassian JIRA (v6.4.14#64029)