Re: Re[2]: [zfs-discuss] HowTo: UPS + ZFS & NFS + no fsync
Wee Yeh Tan writes: > Robert, > > On 4/27/07, Robert Milkowski <[EMAIL PROTECTED]> wrote: > > Hello Wee, > > > > Thursday, April 26, 2007, 4:21:00 PM, you wrote: > > > > WYT> On 4/26/07, cedric briner <[EMAIL PROTECTED]> wrote: > > >> okay let'say that it is not. :) > > >> Imagine that I setup a box: > > >> - with Solaris > > >> - with many HDs (directly attached). > > >> - use ZFS as the FS > > >> - export the Data with NFS > > >> - on an UPS. > > >> > > >> Then after reading the : > > >> http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#ZFS_and_Complex_Storage_Considerations > > >> I wonder if there is a way to tell the OS to ignore the fsync flush > > >> commands since they are likely to survive a power outage. > > > > WYT> Cedric, > > > > WYT> You do not want to ignore syncs from ZFS if your harddisk is directly > > WYT> attached to the server. As the document mentioned, that is really for > > WYT> Complex Storage with NVRAM where flush is not necessary. > > > > > > What?? > > > > Setting zil_disable=1 has nothing to do with NVRAM in storage arrays. > > It disables ZIL in ZFS wich means that if application calls fsync() or > > opens a file with O_DSYNC, etc. then ZFS won't honor it (return > > immediatelly without commiting to stable storage). > > Wait a minute. Are we talking about zil_disable or zfs_noflush (or > zfs_nocacheflush)? > The article quoted was about configuring the array to ignore flush > commands or device specific zfs_noflush, not zil_disable. > > I agree that zil_disable is okay from FS view (correctness still > depends on the application), but zfs_noflush is dangerous. > For me, both are dangerous. zil_disable can cause immense pain to applications and NFS clients. I don't see how anyone can recommend itwithout mentioning the risk of application/NFS corruption. zfs_nocacheflush is also unsafe. It opens a risk of pool corruption ! But, if you have *all* of your pooled data on safe NVRAM protected storage, and that you don't find a way to tell the storage to ignore cache flush requests, you might want to set the variable temporarily until the SYNC_NV thing is sorted out. Then make sure, nobody imports the tunable elsewhere without full understanding and make sure noone creates a new pool with non-NVRAM storage. Since those things are not under anyones control, it's not a good idea to spread these kind of recommendations. > > -- > Just me, > Wire ... > Blog: > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Re[2]: [zfs-discuss] HowTo: UPS + ZFS & NFS + no fsync
Robert Milkowski writes: > Hello Wee, > > Thursday, April 26, 2007, 4:21:00 PM, you wrote: > > WYT> On 4/26/07, cedric briner <[EMAIL PROTECTED]> wrote: > >> okay let'say that it is not. :) > >> Imagine that I setup a box: > >> - with Solaris > >> - with many HDs (directly attached). > >> - use ZFS as the FS > >> - export the Data with NFS > >> - on an UPS. > >> > >> Then after reading the : > >> http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#ZFS_and_Complex_Storage_Considerations > >> I wonder if there is a way to tell the OS to ignore the fsync flush > >> commands since they are likely to survive a power outage. > > WYT> Cedric, > > WYT> You do not want to ignore syncs from ZFS if your harddisk is directly > WYT> attached to the server. As the document mentioned, that is really for > WYT> Complex Storage with NVRAM where flush is not necessary. > > > What?? > > Setting zil_disable=1 has nothing to do with NVRAM in storage arrays. > It disables ZIL in ZFS wich means that if application calls fsync() or > opens a file with O_DSYNC, etc. then ZFS won't honor it (return > immediatelly without commiting to stable storage). > > Once txg group closes data will be written to disks and SCSI write > cache flush commands will be send. > > Setting zil_disable to 1 is not that bad actually, and if someone > doesn't care to lose some last N seconds of data in case of server > crash (however zfs itself will be consistent) it can actually speed up > nfs operations a lot. > ...set zil_disable...speed up nfs...at the expense of a risk of corruption of the NFS client's view. We must never forget this. zil_disable is really not an option IMO. -r > > -- > Best regards, > Robertmailto:[EMAIL PROTECTED] >http://milek.blogspot.com > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Re[2]: [zfs-discuss] HowTo: UPS + ZFS & NFS + no fsync
Robert, On 4/27/07, Robert Milkowski <[EMAIL PROTECTED]> wrote: Hello Wee, Thursday, April 26, 2007, 4:21:00 PM, you wrote: WYT> On 4/26/07, cedric briner <[EMAIL PROTECTED]> wrote: >> okay let'say that it is not. :) >> Imagine that I setup a box: >> - with Solaris >> - with many HDs (directly attached). >> - use ZFS as the FS >> - export the Data with NFS >> - on an UPS. >> >> Then after reading the : >> http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#ZFS_and_Complex_Storage_Considerations >> I wonder if there is a way to tell the OS to ignore the fsync flush >> commands since they are likely to survive a power outage. WYT> Cedric, WYT> You do not want to ignore syncs from ZFS if your harddisk is directly WYT> attached to the server. As the document mentioned, that is really for WYT> Complex Storage with NVRAM where flush is not necessary. What?? Setting zil_disable=1 has nothing to do with NVRAM in storage arrays. It disables ZIL in ZFS wich means that if application calls fsync() or opens a file with O_DSYNC, etc. then ZFS won't honor it (return immediatelly without commiting to stable storage). Wait a minute. Are we talking about zil_disable or zfs_noflush (or zfs_nocacheflush)? The article quoted was about configuring the array to ignore flush commands or device specific zfs_noflush, not zil_disable. I agree that zil_disable is okay from FS view (correctness still depends on the application), but zfs_noflush is dangerous. -- Just me, Wire ... Blog: ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re[2]: [zfs-discuss] HowTo: UPS + ZFS & NFS + no fsync
Hello Wee, Thursday, April 26, 2007, 4:21:00 PM, you wrote: WYT> On 4/26/07, cedric briner <[EMAIL PROTECTED]> wrote: >> okay let'say that it is not. :) >> Imagine that I setup a box: >> - with Solaris >> - with many HDs (directly attached). >> - use ZFS as the FS >> - export the Data with NFS >> - on an UPS. >> >> Then after reading the : >> http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#ZFS_and_Complex_Storage_Considerations >> I wonder if there is a way to tell the OS to ignore the fsync flush >> commands since they are likely to survive a power outage. WYT> Cedric, WYT> You do not want to ignore syncs from ZFS if your harddisk is directly WYT> attached to the server. As the document mentioned, that is really for WYT> Complex Storage with NVRAM where flush is not necessary. What?? Setting zil_disable=1 has nothing to do with NVRAM in storage arrays. It disables ZIL in ZFS wich means that if application calls fsync() or opens a file with O_DSYNC, etc. then ZFS won't honor it (return immediatelly without commiting to stable storage). Once txg group closes data will be written to disks and SCSI write cache flush commands will be send. Setting zil_disable to 1 is not that bad actually, and if someone doesn't care to lose some last N seconds of data in case of server crash (however zfs itself will be consistent) it can actually speed up nfs operations a lot. btw: people accustomed to Linux in a way have always zil_disable set to 1... :) -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
