Hello Michael Ho, Dan Hecht, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7408 to look at the new patch set (#8). Change subject: IMPALA-5417: make I/O buffer queue fixed-size ...................................................................... IMPALA-5417: make I/O buffer queue fixed-size This removes the dynamically-varying queue size behaviour in the I/O manager. The motivation is to bound resource consumption of scans and make it possible to reserve memory for I/O buffers upfront. Does some cleanup/documentation of the locking policy. Fix some cases in ScanRange::GetNext() where members documented as being protected by ScanRange::lock_ were accessed without holding it. I think the races were either benign or prevented by holding DiskIoRequestContext::lock_ in practice. Testing: Ran exhaustive build. Perf: Ran the full set of workloads (TPC-H, TPC-DS, targeted) on a 16 node cluster. Everything was within normal variance. Change-Id: If7cc3f7199f5320db00b7face97a96cdadb6f83f --- M be/src/runtime/disk-io-mgr-internal.h M be/src/runtime/disk-io-mgr-reader-context.cc M be/src/runtime/disk-io-mgr-scan-range.cc M be/src/runtime/disk-io-mgr-test.cc M be/src/runtime/disk-io-mgr.cc M be/src/runtime/disk-io-mgr.h 6 files changed, 289 insertions(+), 342 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/7408/8 -- To view, visit http://gerrit.cloudera.org:8080/7408 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: If7cc3f7199f5320db00b7face97a96cdadb6f83f Gerrit-PatchSet: 8 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com> Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>