Re: skiplist_unsafe
On Fri, 7 Dec 2007, Janne Peltonen wrote: >> If you feel that your filesystem/buffercache will do a good job at >> writing things out to disk, and you've got battery-backed cache on >> your storage, you should be relatively well off. > > But if I were to turn skiplist_unsafe on, and the OS crashed - or, say, > the cluster system forcibly unmounted my Cyrus spool and config > filesystems - wouldn't that result in horribly unrecoverable databases > all over the place? (I have everything in skiplist, except quota and > subscriptions.) It is easy enough to find out. Take an fsync() test rig such as Brad Fitzpatrick's diskchecker.pl and comment out the fsync()s. If the disk checker moans, then updates have been lost in buffer cache. Under Linux this is only safe if the filesystem is mounted with the "sync" option, even with data=journal. Part of the point of fsync() is to make sure updates hit nonvolatile storage in the correct order. A specific example: skiplist commit records are written after an fsync(), immediately followed by another fsync() before the write lock is released. If writes get reordered before they hit disk, then there is a good chance that the database will become corrupt. -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: skiplist_unsafe
> I enabled it a few weeks ago before I left UMBC, and it certainly > seemed to help write performance on the ZFS pool -- we started seeing > the larger batched writes on those devices, instead of a constant > stream of itty bity IOs caused by the constant fsync'ing. > > If you feel that your filesystem/buffercache will do a good job at > writing things out to disk, and you've got battery-backed cache on > your storage, you should be relatively well off. But if I were to turn skiplist_unsafe on, and the OS crashed - or, say, the cluster system forcibly unmounted my Cyrus spool and config filesystems - wouldn't that result in horribly unrecoverable databases all over the place? (I have everything in skiplist, except quota and subscriptions.) --Janne -- Janne Peltonen <[EMAIL PROTECTED]> Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: skiplist_unsafe?
I enabled it a few weeks ago before I left UMBC, and it certainly seemed to help write performance on the ZFS pool -- we started seeing the larger batched writes on those devices, instead of a constant stream of itty bity IOs caused by the constant fsync'ing. If you feel that your filesystem/buffercache will do a good job at writing things out to disk, and you've got battery-backed cache on your storage, you should be relatively well off. -rob On Nov 30, 2007, at 19:59, Vincent Fox wrote: > How "unsafe" is setting in imapd.conf > > skiplist_unsafe: 1 > > Our /var/cyrus/imap filesystem is on a ZFS mirror set on > arrays with dual controllers so OS and/or hardware > corruption is remote. > > The application can scramble it but that can happen > whether we have sync or not eh? > > Anything I am missing? > > > Cyrus Home Page: http://cyrusimap.web.cmu.edu/ > Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
skiplist_unsafe?
How "unsafe" is setting in imapd.conf skiplist_unsafe: 1 Our /var/cyrus/imap filesystem is on a ZFS mirror set on arrays with dual controllers so OS and/or hardware corruption is remote. The application can scramble it but that can happen whether we have sync or not eh? Anything I am missing? Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
