Dear Mark/All, Our trading system is writing to local and/or array volume at 10k messages per second. Each message is about 700bytes in size.
Before ZFS, we used UFS. Even with UFS, there was evey 5 second peak due to fsflush invocation. However each peak is about ~5ms. Our application can not recover from such higher latency. So we used several tuning parameters (tune_r_* and autoup) to decrease the flush interval. As a result peaks came down to ~1.5ms. But it is still too high for our application. I believe, if we could reduce ZFS sync interval down to ~1s, peaks will be reduced to ~1ms or less. We like <1ms peaks per second than 5ms peak per 5 second :-) Are there any tunable, so i can reduce ZFS sync interval. If there is no any tunable, can not I use "mdb" for the job ...? This is not general and we are ok with increased I/O rate. Please advice/help. Thankx in advance. tharindu Mark Maybee wrote: > ZFS is designed to "sync" a transaction group about every 5 seconds > under normal work loads. So your system looks to be operating as > designed. Is there some specific reason why you need to reduce this > interval? In general, this is a bad idea, as there is somewhat of a > "fixed overhead" associated with each sync, so increasing the sync > frequency could result in increased IO. > > -Mark > > Tharindu Rukshan Bamunuarachchi wrote: >> Dear ZFS Gurus, >> >> We are developing low latency transaction processing systems for >> stock exchanges. >> Low latency high performance file system is critical component of our >> trading systems. >> >> We have choose ZFS as our primary file system. >> But we saw periodical disk write peaks every 4-5 second. >> >> Please refer first column of below output. (marked in bold) >> Output is generated from our own Disk performance measuring tool. i.e >> DTool (please find attachment) >> >> Compared UFS/VxFS , ZFS is performing very well, but we could not >> minimize periodical peaks. >> We used autoup and tune_r_fsflush flags for UFS tuning. >> >> Are there any ZFS specific tuning, which will reduce file system >> flush interval of ZFS. >> >> I have tried all parameters specified in "solarisinternals" and >> google.com. >> I would like to go for ZFS code change/recompile if necessary. >> >> Please advice. >> >> Cheers >> Tharindu >> >> >> >> cpu4600-100 /tantan >./*DTool -f M -s 1000 -r 10000 -i 1 -W* >> System Tick = 100 usecs >> Clock resolution 10 >> HR Timer created for 100usecs >> z_FileName = M >> i_Rate = 10000 >> l_BlockSize = 1000 >> i_SyncInterval = 0 >> l_TickInterval = 100 >> i_TicksPerIO = 1 >> i_NumOfIOsPerSlot = 1 >> Max (us)| Min (us) | Avg (us) | MB/S | >> File Freq Distribution >> 336 | 4 | 10.5635 | 4.7688 | M >> 50(98.55), 200(1.09), 500(0.36), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> *1911 * | 4 | 10.3152 | 9.4822 | M >> 50(98.90), 200(0.77), 500(0.32), 2000(0.01), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 307 | 4 | 9.9386 | 9.5324 | M >> 50(99.03), 200(0.66), 500(0.31), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 331 | 4 | 9.9465 | 9.5332 | M >> 50(99.04), 200(0.72), 500(0.24), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 318 | 4 | 10.1241 | 9.5309 | M >> 50(99.07), 200(0.66), 500(0.27), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 303 | 4 | 9.9236 | 9.5296 | M >> 50(99.13), 200(0.59), 500(0.28), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 560 | 4 | 10.2604 | 9.4565 | M >> 50(98.82), 200(0.86), 500(0.31), 2000(0.01), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 376 | 4 | 9.9975 | 9.5176 | M >> 50(99.05), 200(0.63), 500(0.32), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> *9783 * | 4 | 10.8216 | 9.5301 | M >> 50(99.05), 200(0.58), 500(0.36), 2000(0.00), 5000(0.00), 10000(0.01), >> 100000(0.00), 200000(0.00), >> 332 | 4 | 9.9345 | 9.5252 | M >> 50(99.06), 200(0.61), 500(0.33), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 355 | 4 | 9.9906 | 9.5315 | M >> 50(99.01), 200(0.69), 500(0.30), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 356 | 4 | 10.2341 | 9.5207 | M >> 50(98.96), 200(0.76), 500(0.28), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> 320 | 4 | 9.8893 | 9.5279 | M >> 50(99.10), 200(0.59), 500(0.31), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> *10005* | 4 | 10.8956 | 9.5258 | M >> 50(99.07), 200(0.63), 500(0.29), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.01), 200000(0.00), >> 308 | 4 | 9.8417 | 9.5312 | M >> 50(99.07), 200(0.64), 500(0.29), 2000(0.00), 5000(0.00), 10000(0.00), >> 100000(0.00), 200000(0.00), >> >> >> ------------------------------------------------------------------------ >> >> ******************************************************************************************************************************************************************* >> >> >> >> "The information contained in this email including in any attachment >> is confidential and is meant to be read only by the person to whom it >> is addressed. If you are not the intended recipient(s), you are >> prohibited from printing, forwarding, saving or copying this email. >> If you have received this e-mail in error, please immediately notify >> the sender and delete this e-mail and its attachments from your >> computer." >> >> ******************************************************************************************************************************************************************* >> >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> zfs-code mailing list >> [EMAIL PROTECTED] >> http://mail.opensolaris.org/mailman/listinfo/zfs-code > ******************************************************************************************************************************************************************* "The information contained in this email including in any attachment is confidential and is meant to be read only by the person to whom it is addressed. If you are not the intended recipient(s), you are prohibited from printing, forwarding, saving or copying this email. If you have received this e-mail in error, please immediately notify the sender and delete this e-mail and its attachments from your computer." ******************************************************************************************************************************************************************* _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
