so what you are saying is that if we were using NFS v4 things should be
dramatically better?
do you think this applies to any NFS v4 client or only Suns?
-----Original Message-----
From: [EMAIL PROTECTED] on behalf of Erblichs
Sent: Sun 4/22/2007 4:50 AM
To: Leon Koll
Cc: zfs-discuss@opensolaris.org
Subject: Re: [zfs-discuss] Re: ZFS+NFS on storedge 6120 (sun t4)
Leon Koll,
As a knowldegeable outsider I can say something.
The benchbark (SFS) page specifies NFSv3,v2 support, so I question
whether you ra n NFSv4. I would expect a major change in
performance just to version 4 NFS version and ZFS.
The benchmark seems to stress your configuration enough that
the latency to service NFS ops increases to the point of non
serviced NFS requests. However, you don't know what is the
byte count per IO op. Reads are bottlenecked against rtt of
the connection and writes are normally sub 1K with a later
commit. However, many ops are probably just file handle
verifications which again are limited to your connection
rtt (round trip time). So, my initial guess is that the number
of NFS threads are somewhat related to the number of non
state (v4 now has state) per file handle op. Thus, if a 64k
ZFS block is being modified by 1 byte, COW would require a
64k byte read, 1 byte modify, and then allocation of another
64k block. So, for every write op, you COULD be writing a
full ZFS block.
This COW philosphy works best with extending delayed writes, etc
where later reads would make the trade-off of increased
latency of the larger block on a read op versus being able
to minimize the number of seeks on the write and read. Basicly
increasing the block size from say 8k to 64K. Thus, your
read latency goes up just to get the data off the disk
and minimizing the number of seeks, and dropping the read
ahead logic for the needed 8k to 64k file offset.
I do NOT know that "THAT" 4000 IO OPS load would match your maximal
load and that your actual load would never increase past 2000 IO ops.
Secondly, jumping from 2000 to 4000 seems to be too big of a jump
for your environment. Going to 2500 or 3000 might be more
appropriate. Lastly wrt the benchmark, some remnants (NFS and/or ZFS
and/or benchmark) seem to remain that have a negative impact.
Lastly, my guess is that this NFS and the benchark are stressing small
partial block writes and that is probably one of the worst case
scenarios for ZFS. So, my guess is the proper analogy is trying to
kill a nat with a sledgehammer. Each write IO OP really needs to be
equal
to a full size ZFS block to get the full benefit of ZFS on a per byte
basis.
Mitchell Erblich
Sr Software Engineer
-----------------
Leon Koll wrote:
>
> Welcome to the club, Andy...
>
> I tried several times to attract the attention of the community to the
> dramatic performance degradation (about 3 times) of NFZ/ZFS vs. ZFS/UFS
> combination - without any result : <a
> href="http://www.opensolaris.org/jive/thread.jspa?messageID=98592";>[1]</a> ,
> <a href="http://www.opensolaris.org/jive/thread.jspa?threadID=24015";>[2]</a>.
>
> Just look at two graphs in my <a
> href="http://napobo3.blogspot.com/2006/08/spec-sfs-bencmark-of-zfsufsvxfs.html";>posting
> dated August, 2006</a> to see how bad the situation was and, unfortunately,
> this situation wasn't changed much recently:
> http://photos1.blogger.com/blogger/7591/428/1600/sfs.1.png
>
> I don't think the storage array is a source of the problems you reported.
> It's somewhere else...
>
> [i]-- leon[/i]
>
>
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
