Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13522
to look at the new patch set (#3).
Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
......................................................................
IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
The file handle cache uses the mtime to distinguish
different versions of a file separate. For example,
if a file at mtime=1 is overwritten with a version
at mtime=2, the old file handle from mtime=1 will
not be used for the mtime=2 version.
In some codepaths, for legacy reasons, the mtime
would be unconditionally set to -1, and this
eliminates the ability to distinguish between different
versions of files. There is no need to set the mtime to
-1. It seems to be a legacy bit of cruft.
This removes the mtime=-1 behavior for HDFS scan ranges.
It removes mtime from BufferOpts and plumbs the mtime
through the scan range codepaths separately. Local
non-HDFS files do not use the mtime, so those continue
to use mtime=-1.
Testing:
- Passed core tests
Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672
---
M be/src/exec/base-sequence-scanner.cc
M be/src/exec/hdfs-orc-scanner.cc
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/parquet-column-readers.cc
M be/src/exec/parquet/parquet-page-index.cc
M be/src/exec/scanner-context.cc
M be/src/runtime/io/disk-io-mgr-stress.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/handle-cache.inline.h
M be/src/runtime/io/request-ranges.h
M be/src/runtime/io/scan-range.cc
M be/src/runtime/tmp-file-mgr.cc
16 files changed, 62 insertions(+), 59 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/13522/3
--
To view, visit http://gerrit.cloudera.org:8080/13522
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672
Gerrit-Change-Number: 13522
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
