On 01/27/12 16:12, Brock Pytlik wrote:
On 01/27/12 11:25, Shawn Walker wrote:
Greetings,
The following set of changes contains fixes for the following issue:
7135470 conflict checking should use mmap'd file operations
webrev:
https://cr.opensolaris.org/action/browse/pkg/swalker/pkg-7135470/webrev/
mmap'ing the (two) files used into the pkg process reduces the number
of system calls and I/O during planning operations with a worst-case
scenario (such as change-facet) where all actions must be processed
for conflict checking.
On my system, this consistently reduced total operation time from ~60s
to ~55s; on kodiak from ~1m53s to ~1m50s.
The numbers from truss also support this:
# current gate tip
$ truss -c pkg change-facet --no-refresh -nv facet.wombat=False >
/dev/null
...
syscall seconds calls errors
...
read 4.395 183172
...
brk .048 15182
...
llseek .408 180115 22
...
-------- ------ ----
sys totals: 5.221 454598 17286
usr time: 48.949
elapsed: 68.740
# current gate tip + mmap changes
$ truss -c pkg change-facet --no-refresh -nv facet.wombat=False >
/dev/null
...
syscall seconds calls errors
...
read .090 9569
...
brk .033 11410
...
llseek .020 9425 22
...
-------- ------ ----
sys totals: .554 113199 17337
usr time: 46.168
elapsed: 52.590
Note, however, that this is a worst-case scenario where the entire
conflict database file is processed.
For a normal 'pkg update -n' case where files are only being updated,
this has essentially no effect.
As for memory usage:
Before mmap'ing:
PID VSZ SZ RSS COMMAND
20025 496880 124220 487140 /usr/bin/python2.6 ...
After mmap'ing:
PID VSZ SZ RSS COMMAND
11913 496524 124131 486788 /usr/bin/python2.6 ...
pmap -x total usage went up by the size of the largest file mmap'd
(about 36MB in my case).
So, a slight decrease by those numbers. Of the five seconds saved, it
appears ~4.2 were due to the reduced number of read()s/llseek()s.
-Shawn
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
image.py:
3582-3595: If this is designed to reduce the seeks, then I'm confused as
to why we're mmaping the offsets file which we just read through linearly.
You're right; I missed that. I've reverted this particular change.
(Oddly enough, it did consistently save about 0.5-1 second though.)
imageplan.py:
1588: putting the rstrip here means that we do call it a few more times
than we used to. I doubt that it has much of an impact but is there a
way to put it back below the break on line 1591?
Hmm, I can separate the two break conditions, but beyond that, there's
still going to be at least one extra call in the EOF case.
It doesn't seem to make any performance difference, but that's not
surprising.
1982-1984: Since this isn't using msf, shouldn't it go outside the
context of the with statement on line 1962?
Oh, I didn't notice that. Fixed.
Thanks,
-Shawn
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
