On 16 Aug 2017, at 23.59, Joseph Tam <jtam.h...@gmail.com> wrote: > > > Timo Sirainen wrote: > >> There are various changes in this release that can be used to significantly >> reduce disk IO with: >> 1) NFS storage especially, but I guess also other remote filesystems and >> even some with local disks >> 2) When mail storage and INDEX storage are separated > > Thanks for these changes! Big win for my setup. My servers are not > overly stressed, but how much performance gain (using any metric you > choose) can one expect from these changes?
Depends a lot of things, but for the one specific NFS installation we reduced NFS IOPS by something like 70% And actually they haven't finished setting up the local LISTINDEX changes yet, so maybe even more. >> + mail_location can now include VOLATILEDIR=<path> parameter. This >> is used for creating lock files and in future potentially other >> files that don't need to exist permanently. The path could point to >> tmpfs for example. This is especially useful to avoid creating lock >> files to NFS or other remote filesystems. For example: >> mail_location=sdbox:~/sdbox:VOLATILEDIR=/tmp/volatile/%2.256Nu/%u > > Is "/tmp/volatile" auto-created, or must be pre-created? Autocreated. >> + mail_location's LISTINDEX=<path> can now contain a full path. >> This allows storing mailbox list index to a different storage >> than the rest of the indexes, for example to tmpfs. > > Is this in any way related to VOLATILEDIR? Can they be set to the same > value without problems? Yeah, can be set to the same directory. Maybe even a good idea. >> + mail_location can now include NO-NOSELECT parameter. This >> automatically deletes any \NoSelect mailboxes that have no children. >> These mailboxes are sometimes confusing to users. > > Sorry for my IMAP ignorance, but how can this situation come about? a CREATE noselectbox/ --> With this change it's the same as CREATE "noselectbox" = will become selectable. or a CREATE noselectbox/childbox b DELETE noselectbox/childbox --> noselectbox is left behind. With this change it will be auto-deleted. >> + mail_location can now include ITERINDEX parameter. This tells Dovecot >> to perform mailbox listing from the INDEX path instead of from the >> mail root path. It's mainly useful when the INDEX storage is on a >> faster storage. >> + If mailbox_list_index_very_dirty_syncs=yes, the list index is no >> longer refreshed against filesystem when listing mailboxes. This >> allows the mailbox listing to be done entirely by only reading the >> mailbox list index. >> + Added mailbox_list_index_include_inbox setting to control whether >> INBOX's STATUS information should be cached in the mailbox list >> index. The default is "no", but it may be useful to change it to >> "yes", especially if LISTINDEX points to tmpfs. > > So as I understand it, the optimzation comes about from segregating mail > data information into 3 parts: raw mail, indices, and volatile components, > putting them into increasingly better performing storage media. > > How do these I/O optimizations affect the client's view of a mailbox > if their mailbox is subject to modification outside the dovecot system > (e.g. procmail, mail readers directly modifies mailbox files)? Is there > a trade-off between metadata consistency and performance, or it's a > win-win all around? I think already even with mailbox_list_index=yes it won't notice external changes done to the mailboxes when doing STATUS calls, so better to avoid that. It does get fixed automatically once the folder is selected though.