On Sat, 26 Jul 2008, Tharindu Rukshan Bamunuarachchi wrote: > It is impossible to simulate my scenario with iozone. iozone performs very > well for ZFS. OTOH, > iozone does not measure latency. > > Please find attached tool (Solaris x86), which we have written to measure > latency.
Very interesting software. I ran it for a little while in a ZFS filesystem configured with 8K zfs blocksize and produced a 673MB file. This is with a full graphical login environment running. I did a short run using 128K zfs blocksize and notice that the average peak latencies are about 2X as high as with 8K blocks, but the maximum peak latencies are similar (i.e. somewhat under 10,000 us). I suspect that the maximum peak latencies have something to do with zfs itself (or something in the test program) rather than the pool configuration. Here is the text output with 8k filesystem blocks: % ./DTool -W -i 1 -s 700 -r 10000 -f file System Tick = 100 usecs Clock resolution 10 HR Timer created for 100usecs z_FileName = file i_Rate = 10000 l_BlockSize = 700 i_SyncInterval = 0 l_TickInterval = 100 i_TicksPerIO = 1 i_NumOfIOsPerSlot = 1 Max (us)| Min (us) | Avg (us) | MB/S | File Freq Distribution 80 | 5 | 6.5637 | 3.3371 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 2116 | 4 | 7.2277 | 6.6429 | file 50(99.88), 200(0.10), 500(0.00), 2000(0.01), 5000(0.01), 10000(0.00), 100000(0.00), 200000(0.00), 60 | 5 | 6.7522 | 6.6733 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 64 | 5 | 6.6542 | 6.6753 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 46 | 4 | 6.5489 | 6.6753 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 68 | 5 | 6.5236 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8694 | 4 | 8.7859 | 6.4859 | file 50(99.39), 200(0.54), 500(0.03), 2000(0.03), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 70 | 4 | 6.5669 | 6.6753 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 48 | 5 | 6.5907 | 6.6733 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 49 | 5 | 6.5948 | 6.6753 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 47 | 4 | 6.5437 | 6.6753 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 7991 | 4 | 8.7452 | 6.5043 | file 50(99.45), 200(0.45), 500(0.06), 2000(0.03), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 57 | 4 | 6.7606 | 6.6753 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 49 | 5 | 6.6358 | 6.6753 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 46 | 5 | 6.4603 | 6.6726 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 60 | 5 | 6.4511 | 6.6727 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 9099 | 4 | 9.0321 | 6.4891 | file 50(99.37), 200(0.51), 500(0.07), 2000(0.04), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 48 | 5 | 6.5132 | 6.6727 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 72 | 5 | 6.5453 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 44 | 5 | 6.5788 | 6.6753 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 71 | 5 | 6.5554 | 6.6727 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 9138 | 4 | 8.9271 | 6.5061 | file 50(99.43), 200(0.48), 500(0.03), 2000(0.04), 5000(0.01), 10000(0.01), 100000(0.00), 200000(0.00), 45 | 5 | 6.5028 | 6.6753 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 51 | 5 | 6.5297 | 6.6733 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 47 | 5 | 6.6340 | 6.6727 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 49 | 4 | 6.6172 | 6.6753 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8671 | 4 | 8.5149 | 6.4436 | file 50(99.58), 200(0.35), 500(0.03), 2000(0.02), 5000(0.01), 10000(0.01), 100000(0.00), 200000(0.00), 51 | 5 | 6.5969 | 6.6754 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 523 | 4 | 6.6691 | 6.6753 | file 50(99.98), 200(0.01), 500(0.00), 2000(0.01), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 68 | 5 | 6.6747 | 6.6726 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 41 | 5 | 6.5438 | 6.6734 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8713 | 4 | 8.8272 | 6.4589 | file 50(99.36), 200(0.50), 500(0.11), 2000(0.02), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 50 | 4 | 6.4549 | 6.6754 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 49 | 5 | 6.5445 | 6.6726 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 69 | 5 | 6.4427 | 6.6753 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 71 | 5 | 6.5174 | 6.6727 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8745 | 4 | 9.0431 | 6.4960 | file 50(99.34), 200(0.55), 500(0.08), 2000(0.01), 5000(0.01), 10000(0.01), 100000(0.00), 200000(0.00), 61 | 5 | 6.6875 | 6.6727 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 62 | 5 | 6.5137 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 59 | 5 | 6.6640 | 6.6727 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 206 | 4 | 6.8049 | 6.6353 | file 50(99.86), 200(0.13), 500(0.01), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 9237 | 4 | 8.3959 | 6.5524 | file 50(99.59), 200(0.30), 500(0.07), 2000(0.03), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 63 | 5 | 6.4380 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 44 | 5 | 6.3695 | 6.6754 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 54 | 5 | 6.5120 | 6.6753 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 1166 | 4 | 7.2351 | 6.5967 | file 50(99.67), 200(0.29), 500(0.03), 2000(0.01), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 9103 | 4 | 7.9335 | 6.5807 | file 50(99.83), 200(0.12), 500(0.03), 2000(0.01), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 60 | 5 | 6.5613 | 6.6727 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 66 | 5 | 6.4940 | 6.6733 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 51 | 5 | 6.5826 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 1549 | 4 | 7.8373 | 6.4979 | file 50(99.48), 200(0.43), 500(0.06), 2000(0.03), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 9221 | 4 | 7.6834 | 6.6720 | file 50(99.96), 200(0.03), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 64 | 4 | 6.5624 | 6.6733 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 47 | 5 | 6.5555 | 6.6747 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 47 | 5 | 6.5979 | 6.6725 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 9219 | 4 | 9.0427 | 6.4955 | file 50(99.45), 200(0.43), 500(0.08), 2000(0.03), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 71 | 5 | 6.4686 | 6.6725 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 59 | 5 | 6.6367 | 6.6727 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 49 | 4 | 6.6251 | 6.6752 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 59 | 5 | 6.5028 | 6.6727 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8722 | 4 | 8.7847 | 6.5003 | file 50(99.21), 200(0.72), 500(0.02), 2000(0.04), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 46 | 5 | 6.4410 | 6.6727 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 45 | 5 | 6.5116 | 6.6726 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 70 | 5 | 6.5629 | 6.6753 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 41 | 5 | 6.4026 | 6.6727 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8728 | 4 | 8.5549 | 6.4809 | file 50(99.47), 200(0.46), 500(0.05), 2000(0.01), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 64 | 5 | 6.6568 | 6.6753 | file 50(99.98), 200(0.02), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 59 | 5 | 6.3614 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 89 | 5 | 6.5293 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 47 | 5 | 6.4494 | 6.6733 | file 50(100.00), 200(0.00), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8752 | 4 | 8.7413 | 6.4929 | file 50(99.51), 200(0.40), 500(0.04), 2000(0.04), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 73 | 5 | 6.5893 | 6.6726 | file 50(99.97), 200(0.03), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 51 | 5 | 6.6581 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 62 | 5 | 6.6587 | 6.6726 | file 50(99.99), 200(0.01), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 77 | 5 | 7.2512 | 6.6753 | file 50(99.94), 200(0.06), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 9169 | 4 | 9.7338 | 6.4627 | file 50(99.04), 200(0.88), 500(0.04), 2000(0.02), 5000(0.01), 10000(0.01), 100000(0.00), 200000(0.00), 69 | 5 | 7.2407 | 6.6727 | file 50(99.96), 200(0.04), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 97 | 4 | 7.3353 | 6.6753 | file 50(99.93), 200(0.07), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 382 | 4 | 7.2239 | 6.6753 | file 50(99.93), 200(0.06), 500(0.01), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 109 | 4 | 7.2756 | 6.6753 | file 50(99.94), 200(0.06), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8015 | 4 | 9.7219 | 6.4942 | file 50(98.94), 200(0.92), 500(0.10), 2000(0.03), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 70 | 5 | 7.2720 | 6.6753 | file 50(99.92), 200(0.08), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 99 | 5 | 7.3499 | 6.6753 | file 50(99.90), 200(0.10), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 64 | 4 | 7.2259 | 6.6753 | file 50(99.96), 200(0.04), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 69 | 4 | 7.2479 | 6.6753 | file 50(99.94), 200(0.06), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8778 | 4 | 10.0523 | 6.4904 | file 50(98.99), 200(0.88), 500(0.08), 2000(0.03), 5000(0.01), 10000(0.01), 100000(0.00), 200000(0.00), 72 | 5 | 7.3874 | 6.6753 | file 50(99.90), 200(0.10), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 69 | 5 | 7.4387 | 6.6726 | file 50(99.94), 200(0.06), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 69 | 5 | 7.4582 | 6.6726 | file 50(99.95), 200(0.05), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 71 | 4 | 7.4296 | 6.6753 | file 50(99.91), 200(0.09), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8756 | 4 | 9.4746 | 6.4835 | file 50(99.08), 200(0.85), 500(0.04), 2000(0.02), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 70 | 5 | 7.4215 | 6.6753 | file 50(99.96), 200(0.04), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 66 | 4 | 7.3560 | 6.6753 | file 50(99.94), 200(0.06), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 65 | 4 | 7.3263 | 6.6754 | file 50(99.94), 200(0.06), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 87 | 4 | 7.2929 | 6.6753 | file 50(99.95), 200(0.05), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 8753 | 4 | 9.4104 | 6.4477 | file 50(99.16), 200(0.77), 500(0.03), 2000(0.03), 5000(0.00), 10000(0.01), 100000(0.00), 200000(0.00), 71 | 4 | 7.4860 | 6.6754 | file 50(99.90), 200(0.10), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 66 | 5 | 7.2805 | 6.6753 | file 50(99.96), 200(0.04), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 78 | 5 | 7.4491 | 6.6740 | file 50(99.92), 200(0.08), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), 75 | 5 | 7.4095 | 6.6753 | file 50(99.95), 200(0.05), 500(0.00), 2000(0.00), 5000(0.00), 10000(0.00), 100000(0.00), 200000(0.00), ====================================== Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
