On Mon, 18 Feb 2013, Linda Walsh wrote:
Hi dag, I really appreciate your working on this,
but it is really annoying hard and tedious.
_I_ am not certain about all the requirements of Direct I/O,
I.e. would have to research (goog/kernel source...etc).
It may be different on different platforms, I _vaguely_ remember
'talking'(email) with someone working on 'dd', and they were telling
me how they had to compensate for a change in the kernel which used
to handle the buffering of partial sector reads/writes for those
who did directio on a device. Then they decided that much hand-holding
was wrong because, IMO, they basically wanted people to use the buffer
cache -- since for most people, and most things it's a good thing.
It is different on some platforms, iirc the iozone source code has
provisions for three (VX_DIRECT, O_DIRECT or O_DIRECTIO), albeit
manageable. I would be already happy providing the functionality for
systems supporting O_DIRECT (can't test the others anyway).
Sorry about my example below -- I'd already replaced the --directio in
the shell script with --drop-cache -- which I'd forgotten I already
had in the script (memory for these things is completely gone!)..
But really, it did have the direct-io in the statement before I
changed it into a drop cache.
Ok, the "Invalid argument" error could be because the filesystem did not
support direct I/O (e.g. NTFS) or there was a misalignment. On the system
I wrote this patch for, it simply works out of the box on NFS. (RHEL5 64bit)
Anyway, the first thing I'd want to find out is why it is writing to a socket
for a local file copy?
Because even when running rsync locally, it works as client-server over a
socket ?
It's going to be hard for direct I/O to make a difference (if it is workable,
the fact that they move a 'window' over the source emulating a memory-mapped
file isn't real helpful lining up memory with the sectors, but the minimums
we need to go for a minimum read size of 4096 (am pretty sure that we have to
do that even on short files, and the kernel will just tell us we got less).
2nd thing -- need to make sure is to have a source of memory aligned
boundaries.
I'd look at the implementation in fio, it's well documented.
First improvement to the patch would be to bail out when we get EINVAL so
it's clear that direct-io as-is does not work/alignment is not
implemented.
--
-- dag wieers, d...@wieers.com, http://dag.wieers.com/
-- dagit linux solutions, i...@dagit.net, http://dagit.net/
[Any errors in spelling, tact or fact are transmission errors]
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
