On Wed, 23 Jul 2008, Tharindu Rukshan Bamunuarachchi wrote:
> 10,000 x 700 = 7MB per second ......
>
> We have this rate for whole day ....
>
> 10,000 orders per second is minimum requirments of modern day stock exchanges
> ...
>
> Cache still help us for ~1 hours, but after that who will help us ...
>
> We are using 2540 for current testing ...
> I have tried same with 6140, but no significant improvement ... only one or
> two hours ...
It might not be exactly what you have in mind, but this "how do I get
latency down at all costs" thing reminded me of this old paper:
http://www.sun.com/blueprints/1000/layout.pdf
I'm not a storage architect, someone with more experience in the area care
to comment on this ? With huge disks as we have these days, the "wide
thin" idea has gone under a bit - but how to replace such setups with
modern arrays, if the workload is such that caches eventually must get
blown and you're down to spindle speed ?
FrankH.
>
> Robert Milkowski wrote:
>
> Hello Tharindu,
>
> Wednesday, July 23, 2008, 6:35:33 AM, you wrote:
>
> TRB> Dear Mark/All,
>
> TRB> Our trading system is writing to local and/or array volume at 10k
> TRB> messages per second.
> TRB> Each message is about 700bytes in size.
>
> TRB> Before ZFS, we used UFS.
> TRB> Even with UFS, there was evey 5 second peak due to fsflush invocation.
>
> TRB> However each peak is about ~5ms.
> TRB> Our application can not recover from such higher latency.
>
> TRB> So we used several tuning parameters (tune_r_* and autoup) to decrease
> TRB> the flush interval.
> TRB> As a result peaks came down to ~1.5ms. But it is still too high for our
> TRB> application.
>
> TRB> I believe, if we could reduce ZFS sync interval down to ~1s, peaks will
> TRB> be reduced to ~1ms or less.
> TRB> We like <1ms peaks per second than 5ms peak per 5 second :-)
>
> TRB> Are there any tunable, so i can reduce ZFS sync interval.
> TRB> If there is no any tunable, can not I use "mdb" for the job ...?
>
> TRB> This is not general and we are ok with increased I/O rate.
> TRB> Please advice/help.
>
> txt_time/D
>
> btw:
> 10,000 * 700 = ~7MB
>
> What's your storage subsystem? Any, even small, raid device with write
> cache should help.
>
>
>
>
>
>
------------------------------------------------------------------------------
No good can come from selling your freedom, not for all the gold in the world,
for the value of this heavenly gift far exceeds that of any fortune on earth.
------------------------------------------------------------------------------
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
