Github user theburge commented on the issue:
https://github.com/apache/couchdb-couch/pull/185
@jaydoane Thanks for running and documenting those experiments, I think
those results look very promising. Reducing time taken and user-space memory
usage can only be good. :)
FWIW, I'm not convinced major faults is the metric of interest for
identifying file reads on newly opened file in an empty cache scenario. (My
thinking is that it shows faults for explicitly mapped ranges that are only
available on-disk; whereas page ins show pages being pulled in for whatever
reason via the VM itself. For laughs and demonstration purposes, run `sar -B 1`
and try some experiments, like dropping cache and loading a `man` page. You'll
typically see page in rate exceed major fault rate, once you do the
conversation from pages/s to kB/s. In truth, it's probably most comprehensive
to look at both fault types and page in/page out rates, but that gets trickier.)
The big surprise here for me is the difference in physical disk operations,
since my understanding would be that the VM would fill complete OS pages to
satisfy reads, so I can't see why reading N bytes on page boundaries would be
much different (from an OS perspective) to reading 4kB. I'd love an explanation
for that (although the user-space benefits are clear to see and very positive),
just to be sure it's not some other effect.
My final suggestion would be to evaluate a more serious MT-esque scenario:
how does the performance improvement of the new algorithm vary with with lots
(10s to 100s) of files opening concurrently? (And maybe with higher background
CPU/disk utilisation, if you can set-up the experiment.) Remember that that's
the scenario where we see this as most relevant, and timeouts there are a big
issue (file opening times out, compaction keels over, file handle remains,
etc.), so if there's little or no wall clock time improvement in a highly
concurrent setting, that'd be a shame and undermine some of the practical
benefit. Obviously, I'd expect only positive things compared to the status quo
based on the evaluation above, but performance expectations in varying
scenarios don't always comply with common sense... :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
