Hi Seigfried, just making sure you had seen this:
http://blogs.sun.com/roch/entry/nfs_and_zfs_a_fine
You have very fast NFS to non-ZFS runs.
That seems only possible if the hosting OS did not sync the
data when NFS required it or the drive in question had some
fast write caches. If the drive did have some FWC and ZFS
was still slow using them, that would be the issue with
flushing mention in the blog entry.
but also maybe there is something to be learned from the
Samba and AFP results...
Takeaways:
ZFS and NFS just work together.
ZFS has an open issue with some storage array (the
issue is *not* related to NFS); it's being worked
on. Will need collaboration from storage vendors.
NFS is slower than direct attached. Can be very very
much slower on single threaded loads.
There are many ways to workaround the slowness but most
are just not safe for your data.
-r
Siegfried Nikolaivich writes:
> This is an old topic, discussed many times at length. However, I
> still wonder if there are any workarounds to this issue except
> disabling ZIL, since it makes ZFS over NFS almost unusable (it's a
> whole magnitude slower). My understanding is that the ball is in the
> hands of NFS due to ZFS's design. The testing results are below.
>
>
> Solaris 10u3 AMD64 server with Mac client over gigabit ethernet. The
> filesystem is on a 6 disk raidz1 pool, testing the performance of
> untarring (with bzip2) the Linux 2.6.21 source code. The archive is
> stored locally and extracted remotely.
>
> Locally
> -------
> tar xfvj linux-2.6.21.tar.bz2
> real 4m4.094s, user 0m44.732s, sys 0m26.047s
>
> star xfv linux-2.6.21.tar.bz2
> real 1m47.502s, user 0m38.573s, sys 0m22.671s
>
> Over NFS
> --------
> tar xfvj linux-2.6.21.tar.bz2
> real 48m22.685s, user 0m45.703s, sys 0m59.264s
>
> star xfv linux-2.6.21.tar.bz2
> real 49m13.574s, user 0m38.996s, sys 0m35.215s
>
> star -no-fsync -x -v -f linux-2.6.21.tar.bz2
> real 49m32.127s, user 0m38.454s, sys 0m36.197s
>
>
> The performance seems pretty bad, lets see how other protocols fare.
>
> Over Samba
> ----------
> tar xfvj linux-2.6.21.tar.bz2
> real 4m34.952s, user 0m44.325s, sys 0m27.404s
>
> star xfv linux-2.6.21.tar.bz2
> real 4m2.998s, user 0m44.121s, sys 0m29.214s
>
> star -no-fsync -x -v -f linux-2.6.21.tar.bz2
> real 4m13.352s, user 0m44.239s, sys 0m29.547s
>
> Over AFP
> --------
> tar xfvj linux-2.6.21.tar.bz2
> real 3m58.405s, user 0m43.132s, sys 0m40.847s
>
> star xfv linux-2.6.21.tar.bz2
> real 19m44.212s, user 0m38.535s, sys 0m38.866s
>
> star -no-fsync -x -v -f linux-2.6.21.tar.bz2
> real 3m21.976s, user 0m42.529s, sys 0m39.529s
>
>
> Samba and AFP are much faster, except the fsync'ed star over AFP. Is
> this a ZFS or NFS issue?
>
> Over NFS to non-ZFS drive
> -------------------------
> tar xfvj linux-2.6.21.tar.bz2
> real 5m0.211s, user 0m45.330s, sys 0m50.118s
>
> star xfv linux-2.6.21.tar.bz2
> real 3m26.053s, user 0m43.069s, sys 0m33.726s
>
> star -no-fsync -x -v -f linux-2.6.21.tar.bz2
> real 3m55.522s, user 0m42.749s, sys 0m35.294s
>
> It looks like ZFS is the culprit here. The untarring is much faster
> to a single 80 GB UFS drive than a 6 disk raid-z array over NFS.
>
>
> Cheers,
> Siegfried
>
>
> PS. Getting netatalk to compile on amd64 Solaris required some
> changes since i386 wasn't being defined anymore, and somehow it
> thought the architecture was sparc64 for some linking steps.
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
