gzip is faster with -O3
dd if=/dev/random of=testfile bs=1m count=5000 gzip compiled with -O3: # date ; nice -10 ./gzip -c9 testfile > testfile.gz ; date Wed Aug 9 08:01:21 CDT 2006 Wed Aug 9 08:09:06 CDT 2006 465 Seconds. gzip compiled with -O2: # date ; nice -10 ./gzip -c9 testfile > testfile.gz ; date Wed Aug 9 08:19:14 CDT 2006 Wed Aug 9 08:27:06 CDT 2006 472 Seconds. 7 second difference, it's not much but I still wanted to share it with the group. -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gzip is faster with -O3
"Nikolas Britton" <[EMAIL PROTECTED]> writes: > dd if=/dev/random of=testfile bs=1m count=5000 1. gzip isn't usually used to compress incompressible data. 2. use "time" to figure out how much CPU time it actually burns. 5 GB are somewhat I/O bound, but gcc options don't help with that, so CPU time is better than wallclock time. > gzip compiled with -O3: > # date ; nice -10 ./gzip -c9 testfile > testfile.gz ; date > Wed Aug 9 08:01:21 CDT 2006 -- Matthias Andree ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gzip is faster with -O3
Nikolas Britton wrote: > dd if=/dev/random of=testfile bs=1m count=5000 > > gzip compiled with -O3: > # date ; nice -10 ./gzip -c9 testfile > testfile.gz ; date > Wed Aug 9 08:01:21 CDT 2006 > Wed Aug 9 08:09:06 CDT 2006 > 465 Seconds. > > gzip compiled with -O2: > # date ; nice -10 ./gzip -c9 testfile > testfile.gz ; date > Wed Aug 9 08:19:14 CDT 2006 > Wed Aug 9 08:27:06 CDT 2006 > 472 Seconds. > > 7 second difference, it's not much but I still wanted to share it with > the group. > > You should use /bin/time for measuring. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gzip is faster with -O3
On 8/9/06, Matthias Andree <[EMAIL PROTECTED]> wrote: 1. gzip isn't usually used to compress incompressible data. 2. use "time" to figure out how much CPU time it actually burns. 5 GB are somewhat I/O bound, but gcc options don't help with that, so CPU time is better than wallclock time. dd if=/dev/zero of=testfile bs=1m count=5000 gzip comiled with -O3 # time nice -10 ./gzip -c9 testfile > /dev/null 73.187u 8.682s 2:08.41 63.7%70+617k 40161+0io 0pf+0w gzip compiled with -O2 # time nice -10 ./gzip -c9 testfile > /dev/null 61.183u 8.468s 2:00.14 57.9%58+609k 40162+0io 0pf+0w Now... what do all of those numbers mean, I've never used time before... thanks for the tip btw? -- BSD Podcasts @: http://bsdtalk.blogspot.com/ http://freebsdforall.blogspot.com/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gzip is faster with -O3
On Wed, Aug 09, 2006 at 05:31:58PM -0500, Nikolas Britton wrote: > On 8/9/06, Matthias Andree <[EMAIL PROTECTED]> wrote: > > > >1. gzip isn't usually used to compress incompressible data. > > > >2. use "time" to figure out how much CPU time it actually burns. > > 5 GB are somewhat I/O bound, but gcc options don't help with that, so > > CPU time is better than wallclock time. > > > > dd if=/dev/zero of=testfile bs=1m count=5000 > > gzip comiled with -O3 > # time nice -10 ./gzip -c9 testfile > /dev/null > 73.187u 8.682s 2:08.41 63.7%70+617k 40161+0io 0pf+0w > > gzip compiled with -O2 > # time nice -10 ./gzip -c9 testfile > /dev/null > 61.183u 8.468s 2:00.14 57.9%58+609k 40162+0io 0pf+0w > > Now... what do all of those numbers mean, I've never used time > before... thanks for the tip btw? In this case the used a similar amount of system time (the number ending in s), but the -O3 case took 8 seconds more user time (the number ending in u) and real time (the third number.) If this were statisticaly meaningful, -O3 would be slower in this case. If you want to do a meaningful test you need to do several runs each way, probably ignoring the first one due to cache effects and then run the results through the program you can build in src/tools/tools/ministat/ so see if there is a measurable difference. Poul-Henning Kamp has a nice (if probably somewhat overkill for this case) writeup on doing benchmarking here: http://lists.freebsd.org/pipermail/freebsd-current/2004-January/019595.html It's a tricky business even for something simple. :) You might also consider using /usr/bin/time instead of the csh builtin time. It's output is a little more readable. -- Brooks pgp820W2lpVvK.pgp Description: PGP signature
Re: gzip is faster with -O3
On Wed, 09 Aug 2006, Nikolas Britton wrote: > On 8/9/06, Matthias Andree <[EMAIL PROTECTED]> wrote: > > > >1. gzip isn't usually used to compress incompressible data. > > > >2. use "time" to figure out how much CPU time it actually burns. > > 5 GB are somewhat I/O bound, but gcc options don't help with that, so > > CPU time is better than wallclock time. > > > > dd if=/dev/zero of=testfile bs=1m count=5000 That's the other extreme. Reading /dev/zero bears no surprises and has thus no entropy, i. e. no information - that's also a case where you don't usually use gzip because the dd command is probably shorter than the gzipped file :-) The truth is in between, a tarball of /usr/doc, /usr/src and /usr/ports (without distfiles and packages) is quite compressible, or a tarball of /usr/local if you've got some interesting ports installed - many of them. While I wouldn't claim that were typical gzip fodder, it's at least not degenerated either way. Degenerated input (zeros or truly random data) usually triggers worst-case behavior which, albeit necessary to know, is less interesting to the average user. If someone found a faster bzip2 compression algorithm or pull good levers to tune it, that would perhaps be interesting :-) > gzip comiled with -O3 > # time nice -10 ./gzip -c9 testfile > /dev/null > 73.187u 8.682s 2:08.41 63.7%70+617k 40161+0io 0pf+0w > > gzip compiled with -O2 > # time nice -10 ./gzip -c9 testfile > /dev/null > 61.183u 8.468s 2:00.14 57.9%58+609k 40162+0io 0pf+0w If you want a fast compressor, install lzo2, then lzop and use the latter :-) > Now... what do all of those numbers mean, I've never used time > before... thanks for the tip btw? You're welcome. What you've got looks like the (t)csh time output, which is documented in the csh(1) manual page - and it documents (what I suspected but didn't utter) that -O3 isn't faster in all circumstances. To compare file sizes, run "size gzip" which prints actual code size. There are cases when code locality can win over unrolling, for instance. Anyways, to answer your question (which is buried in the csh(1) man page in a language that is hard to understand for people unfamiliar with the C programming language) are, in order of appearance: 1 user time in seconds (CPU time spent in the program) 2 system time in s (CPU time spent in the kernel, I/O overhead and such) 3 wallclock time - roughly 2 min (elapsed time without regard to the CPU use) - The difference from 3 - (1 + 2) is waiting for hardware or spent in concurrently running processes 4 average CPU use (how much of the elapsed time was gzip allowed to have the CPU) - This should be (1+2) * 100% / 3 5 shared/unshared memory in kByte 6 input and output operations 7 major page faults and voluntary wait operations Different shells (bash for instance) cause different output, as does running the external time utility explicitly, for instance, by running "\time", "command time" or "/path/to/time". Kind regards, -- Matthias Andree ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gzip is faster with -O3
On 2006.08.09 17:53:01 -0500, Brooks Davis wrote: > [...] Poul-Henning Kamp has a nice (if probably > somewhat overkill for this case) writeup on doing benchmarking here: > > http://lists.freebsd.org/pipermail/freebsd-current/2004-January/019595.html It should be mentioned that this is in the Developers Handbook along with some more tips/comments from Robert Watson: http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/testing.html -- Simon L. Nielsen ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
