Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Hi again, Wow does that mean cryptocompress is almost ready for primetime (next months or so)?! I've been waiting for something like this for ages, it would be so great if that could turn into reality soon. I am quite enthusiastic how it will perform on the Dual-Athlon64 + 4x Software-Raid-5 (PATA) I plan to buy. Thank you many, many times!! lg Clemens
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Clemens Eisserer wrote: Hi again, Wow does that mean cryptocompress is almost ready for primetime (next months or so)?! I've been waiting for something like this for ages, it would be so great if that could turn into reality soon. I am quite enthusiastic how it will perform on the Dual-Athlon64 + 4x Software-Raid-5 (PATA) I plan to buy. Thank you many, many times!! lg Clemens It should work well for that hardware configuration. See our website for new benchmarks using cryptcompress.
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Hi, Am Donnerstag, den 26.01.2006, 21:41 +0300 schrieb Edward Shishkin: Danny Milosavljevic wrote: Hi, Nice :) Just curious, is there a description how to enable cryptcompress for files somewhere? (or is it still bleeding-edge ? :)) cheers, Danny Hello. If you have a free partition and a time to report about possible bugs, we'll send you a setup against 2.6.15-mm4 with the description. yes, now I do :) Thanks, Edward. cheers, Danny
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Hello, On Thursday 26 January 2006 21:56, Jens Axboe wrote: On Thu, Jan 26 2006, Edward Shishkin wrote: Jens Axboe wrote: On Wed, Jan 25 2006, Hans Reiser wrote: Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s Just curious - does your crypt plugin reside in user space? Nop. This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting. Then why is the sys time lower than the plain writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others. I guess the compression was done by the background writeout daemon. CPU utilization numbers would say more than sys time. -- Alex. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
On Thu, Jan 26 2006, Edward Shishkin wrote: Jens Axboe wrote: On Thu, Jan 26 2006, Edward Shishkin wrote: Jens Axboe wrote: On Wed, Jan 25 2006, Hans Reiser wrote: Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s Just curious - does your crypt plugin reside in user space? Nop. This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting. Then why is the sys time lower than the plain writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others. I guess this is because real compression is going in background flush, not in sys_write-write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time. So the systime quoted above is basically useless, it doesn't reflect the real time spent in the kernel by far. I think you should note that when you post these scores, otherwise you're really showing a skewed picture. -- Jens Axboe
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
On Thu, Jan 26 2006, Hans Reiser wrote: Edward Shishkin wrote: I guess this is because real compression is going in background flush, not in sys_write-write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time. Edward. Which means that only real time is a meaningful measurement. Indeed. I guess the compression stuff cost is hard to quantify, since it has cache effects on the rest of the system in addition to costing CPU cycles on its own. A profile of, say, dbench with and without compression would be interesting to see. And the actual dbench reults, naturally :-) -- Jens Axboe
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Jens Axboe wrote: So the systime quoted above is basically useless, it doesn't reflect the real time spent in the kernel by far. I think you should note that when you post these scores, otherwise you're really showing a skewed picture. He wasn't expecting me to post the benchmark, and I frankly only looked at the real time and forgot there was a systime in there that needed cutting out when I posted it. My error, not his. I must say though, the real time makes me quite happy, especially considering how CPUs are just going to keep getting faster. Best, Hans
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
On Fri, Jan 27 2006, Hans Reiser wrote: Jens Axboe wrote: So the systime quoted above is basically useless, it doesn't reflect the real time spent in the kernel by far. I think you should note that when you post these scores, otherwise you're really showing a skewed picture. He wasn't expecting me to post the benchmark, and I frankly only looked at the real time and forgot there was a systime in there that needed cutting out when I posted it. My error, not his. I must say though, the real time makes me quite happy, especially considering how CPUs are just going to keep getting faster. Yeah and that's ok, I was just interested in seeing some more interesting compression benchmarks so I wondered if you had done that. -- Jens Axboe
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
On Wed, Jan 25 2006, Hans Reiser wrote: Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s Just curious - does your crypt plugin reside in user space? -- Jens Axboe
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Hi, Nice :) Just curious, is there a description how to enable cryptcompress for files somewhere? (or is it still bleeding-edge ? :)) cheers, Danny
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Jens Axboe wrote: On Wed, Jan 25 2006, Hans Reiser wrote: Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s Just curious - does your crypt plugin reside in user space? Nop. This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting. Edward.
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Danny Milosavljevic wrote: Hi, Nice :) Just curious, is there a description how to enable cryptcompress for files somewhere? (or is it still bleeding-edge ? :)) cheers, Danny Hello. If you have a free partition and a time to report about possible bugs, we'll send you a setup against 2.6.15-mm4 with the description. Thanks, Edward.
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
On Thu, Jan 26 2006, Edward Shishkin wrote: Jens Axboe wrote: On Wed, Jan 25 2006, Hans Reiser wrote: Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s Just curious - does your crypt plugin reside in user space? Nop. This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting. Then why is the sys time lower than the plain writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others. -- Jens Axboe
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Jens Axboe wrote: On Thu, Jan 26 2006, Edward Shishkin wrote: Jens Axboe wrote: On Wed, Jan 25 2006, Hans Reiser wrote: Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s Just curious - does your crypt plugin reside in user space? Nop. This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting. Then why is the sys time lower than the plain writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others. I guess this is because real compression is going in background flush, not in sys_write-write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time. Edward.
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Jens Axboe wrote: On Wed, Jan 25 2006, Hans Reiser wrote: Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s Just curious - does your crypt plugin reside in user space? No, kernel. It would have to encrypt+compress with every write to be in user space, we encrypt+compress only at flush time, and that is a key optimization (encryption is disabled at the moment due to needing a little API work, but) for file sets that are cachable.
Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Edward Shishkin wrote: I guess this is because real compression is going in background flush, not in sys_write-write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time. Edward. Which means that only real time is a meaningful measurement.
random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)
Notice how CPU speed (and number of cpus) completely determines compression performance. cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin. Edward Shishkin wrote: Here are the tests that vs asked for: Creation (dd) of 20 tarfiles (the original 200M file is in ramfs) Kernel: 2.6.15-mm4 + current git snapshot of reiser4 -- Laputa workstation Uni Intel Pentium 4 (2.26 GHz) 512M RAM ext2: real 2m, 15s sys 0m, 14s reiser4(unix file) real 2m, 7s sys 0m, 23s reiser4(cryptcompress, lzo1, 64K) real 2m, 13s sys 0m, 11s -- Belka workstation Dual Intel Xeon (2.4GHz) 1G RAM ext2: real 2m, 16s sys 0m, 10s reiser4(unix file) real 2m, 14s sys 0m, 17s reiser4(cryptcompress, lzo1, 64K) real 1m, 35s sys 0m, 14s
