On Oct 20, 2009, at 5:28 PM, Trevor Pretty <trevor_pretty at eagle.co.nz> wrote:
> > No it concerns the difference between reads and writes. > > The write performance may be being over stated! > The clients are Linux, the server is Solaris. True the mounts on the Linux clients were async, but so are typically the mounts on Solaris clients. The OP was measuring the page cache performance of the client more then the actual disk io. If the Linux client runs an app that does fsync() on the io on an async mount then the io will be synchronous. You are thinking of the Linux NFS server export option 'async' which is unsafe. -Ross > > Ross Walker wrote: >> >> >> But this is concerning reads not writes. >> >> -Ross >> >> >> On Oct 20, 2009, at 4:43 PM, Trevor Pretty >> <trevor_pretty at eagle.co.nz> wrote: >> >>> Gary >>> >>> Where you measuring the Linux NFS write performance? It's well >>> know that Linux can use NFS in a very "unsafe" mode and report the >>> write complete when it is not all the way to safe storage. This is >>> often reported as Solaris has slow NFS write performance. This >>> link does not mention NFS v4 but you might want to check. >>> http://nfs.sourceforge.net/ >>> >>> What's the write performance like between the two OpenSolaris >>> systems? >>> >>> >>> Richard Elling wrote: >>>> >>>> cross-posting to nfs-discuss >>>> >>>> On Oct 20, 2009, at 10:35 AM, Gary Gogick wrote: >>>> >>>> >>>>> Heya all, >>>>> >>>>> I'm working on testing ZFS with NFS, and I could use some >>>>> guidance - >>>>> read speeds are a bit less than I expected. >>>>> >>>>> Over a gig-e line, we're seeing ~30 MB/s reads on average - >>>>> doesn't >>>>> seem to matter if we're doing large numbers of small files or >>>>> small >>>>> numbers of large files, the speed seems to top out there. We've >>>>> disabled pre-fetching, which may be having some affect on read >>>>> speads, but proved necessary due to severe performance issues on >>>>> database reads with it enabled. (Reading from the DB with pre- >>>>> fetching enabled was taking 4-5 times as long than with it >>>>> disabled.) >>>>> >>>> >>>> What is the performance when reading locally (eliminate NFS from >>>> the >>>> equation)? >>>> -- richard >>>> >>>> >>>>> Write speed seems to be fine. Testing is showing ~95 MB/s, which >>>>> seems pretty decent considering there's been no real network >>>>> tuning >>>>> done. >>>>> >>>>> The NFS server we're testing is a Sun x4500, configured with a >>>>> storage pool consisting of 20x 2-disk mirrors, using separate SSD >>>>> for logging. It's running the latest version of Nexenta Core. >>>>> (We've also got a second x4500 in with a raidZ2 config, running >>>>> OpenSolaris proper, showing the same issues with reads.) >>>>> >>>>> We're using NFS v4 via TCP, serving various Linux clients (the >>>>> majority are CentOS 5.3). Connectivity is presently provided >>>>> by a >>>>> single gigabit ethernet link; entirely conventional configuration >>>>> (no jumbo frames/etc). >>>>> >>>>> Our workload is pretty read heavy; we're serving both website >>>>> assets >>>>> and databases via NFS. The majority of files being served are >>>>> small >>>>> (< 1MB). The databases are MySQL/InnoDB, with the data in >>>>> separate >>>>> zfs filesystems with a record size of 16k. The website assets/ >>>>> etc. >>>>> are in zfs filesystems with the default record size. On the >>>>> database server side of things, we've disabled InnoDB's double >>>>> write >>>>> buffer. >>>>> >>>>> I'm wondering if there's any other tuning that'd be a good idea >>>>> for >>>>> ZFS in this situation, or if there's some NFS tuning that should >>>>> be >>>>> done when dealing specifically with ZFS. Any advice would be >>>>> greatly appreciated. >>>>> >>>>> Thanks, >>>>> >>>>> -- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> ----------------------------------------------------------------- >>>>> Gary Gogick >>>>> senior systems administrator | workhabit,inc. >>>>> >>>>> // email: gary at workhabit.com | web: http://www.workhabit.com >>>>> // office: 866-workhabit | fax: 919-552-9690 >>>>> >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> --- >>>>> ----------------------------------------------------------------- >>>>> _______________________________________________ >>>>> zfs-discuss mailing list >>>>> zfs-discuss at opensolaris.org >>>>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >>>>> >>>> >>>> _______________________________________________ >>>> zfs-discuss mailing list >>>> zfs-discuss at opensolaris.org >>>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >>>> >>> www.eagle.co.nz >>> This email is confidential and may be legally privileged. If >>> received in error please destroy and immediately notify us. >>> >> >>> _______________________________________________ >>> zfs-discuss mailing list >>> zfs-discuss at opensolaris.org >>> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> > www.eagle.co.nz > This email is confidential and may be legally privileged. If > received in error please destroy and immediately notify us.
