On Sunday, 22 April 2012 at 09:33:59 UTC, Marco Leise wrote:
So when you did your first measurements, with 160 seconds for
rmd, did you wait for the I/O to complete? Sorry if that's a
stupid question :p but that's the obvious difference when using
write-through from what the documentation says.
You are probably right that the buffers are just not flushed. I
unzipped with 7zip, then waited 5 min, then used rmdir below,
which still took 144 secs.
G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
11:47:32.40
11:49:56.70
If I use 7zip to unzip, then reboot prior to rmdir, the rmdir
takes only 27 sec
G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
12:47:38.13
12:48:05.60
If I use 7zip to unzip, then run sync on G:, rmdir takes 2:36
http://technet.microsoft.com/en-us/sysinternals/bb897438
C:\Ntutils>sync g:
Sync 2.2: Disk Flusher for Windows 9x/Me/NT/2K/XP
Copyright (C) 1997-2004 Mark Russinovich
Sysinternals - www.sysinternals.com
Flushing: G
G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
13:10:11.93
13:12:47.75
according to that ntfs document, the buffers should have been
scheduled for flush within about 8 secs.
http://msdn.microsoft.com/en-us/library/windows/desktop/aa364218(v=vs.85).aspx
"To ensure that the right amount of flushing occurs, the cache
manager spawns a process every second called a lazy writer. The
lazy writer process queues one-eighth of the pages that have not
been flushed recently to be written to disk. It constantly
reevaluates the amount of data being flushed for optimal system
performance, and if more data needs to be written it queues more
data. Lazy writers do not flush temporary files, because the
assumption is that they will be deleted by the application or
system.
"
This is measurement below is for rmdir immediately following
unzip with the writeThrough code ... about 6 secs.
G:\>cmd /v:on /c "echo !TIME! & rmdir /q /s tz & echo !TIME!"
12:07:58.23
12:08:04.26
I can't explain why the writeThrough enables faster rmdir
operation, but I saw similar improvement following myDefrag
sortByName on the folder, which is what I was attempting to
duplicate with the sorted unzip. At this time it provides the
only reasonable way to get consistent results ... the lazy
flushing doesn't seem to work, myDefrag takes too long, the sync
utility doesn't seem to improve anything, and reboot isn't really
an option.
