Re: Dovecot cluster using GlusterFS

2015-12-06 Thread Filip Pytloun
Hello Alessio and Gordon, thank you for answers.

Dsync-based architecture looks promising, but I would preffer to stay
with GlusterFS for now as I also use it as a storage for other
components.

So director is the way to go, I don't want to setup more than two nodes
to keep this setup as simple as possible - so I will probably update to
2.2.19 and have director and backend on the same servers (and Dovecot
instance).

I asked about poolmon, because I think that Dovecot should have some
internal mechanism on how to recognize broken backend by default.
But if it works nicely, I am going to use it as well :-)

> At the moment, I cannot recognize the requirement for using lmtp over
> the directors. When using postfix for delivering e-mails to the
> backend, do this directly with an corresponding MX record.

I have two MX records of the same weight with postfix using dovecot-lmtp
for delivery. So that's why I wanted to use LMTP over directors.
Using lower weight for second MX is an option, but not truly
master-master setup :-)

Filip

On 2015/12/06 02:31, Alessio Cecchi wrote:
> Il 05.12.2015 10:42 Filip Pytloun ha scritto:
> >Hello,
> >
> >I have recently setup mailserver solution using 2-node master-master
> >setup (mainly based on MySQL M-M replication and GlusterFS with 2
> >replica volume) on Ubuntu 14.04 (Dovecot 2.2.9).
> >
> >Unfortunately even with shared-storage-aware setting:
> >
> > mail_nfs_index = yes
> > mail_nfs_storage = yes
> > mail_fsync = always
> > mmap_disable = yes
> 
> With only these setting you don't solve the problem of shared storage.
> 
> >..I have hit strange issues pretty soon especially when user was
> >manipulating same mailbox from multiple devices at the same time.
> >
> >Most issues was about corrupted indexes which was solved easily by just
> >putting them on local storage of each node:
> >
> > mail_location =
> >maildir:/srv/mail/%d/%u:INDEX=/var/lib/dovecot/index/%d/%u
> >
> >But I still hit issues like this one:
> >
> > dovecot: lmtp(6276, u...@example.com): Error: Broken file
> >/srv/mail/example.com/u...@example.com/dovecot-uidlist line 8529: UIDs
> >not ordered (8527 >= 8527)
> >
> >Which I am not sure how serious it is or if it's possible to solve or
> >workaround?
> 
> You need Director for POP/IMAP and also LMTP so you can solve all "Broken
> file" and "corrupted indexes" problems.
> 
> >
> >Anyway because of the above and high possibility of GlusterFS
> >split-brains, I have decided to setup Dovecot Director according to the
> >docs [1] but I have a couple of questions:
> >
> >- is custom monitoring still required? Poolmon [2] is 4 year old so I
> >  would suppose there's some progress since that?
> 
> For me poolmon works fine.
> 
> >- it's not possible to have same backends and directors in Dovecot
> >  <2.2.17. I can backport newer Dovecot for Ubuntu Trusty, so this is
> >  not an issue, but..
> 
> Yes is possibile (also with < 2.2.17), create two instances, like dovecot
> and director, two config directory /etc/dovecot/ and /etc/director/ and bind
> on differents IPs.
> 
> >- documentation states that it still doesn't work for LMTP [3]?
> >  Which is probably important for my setup, because both Postfix servers
> >  are using dovecot-lmtp for mail delivery so there can be still some
> >  issues (but probably less frequent?) when both servers will deliver
> >  new mails for one user at once.
> >  So do I really have to split directors from backends?
> 
> I'm running Director and backend on the same server for POP/IMAP, and in
> another configuration and Director for LMTP is on the same server (but with
> 2.2.19).
> 
> >Anyone has experience with clustered Dovecot setup?
> >Why is Dovecot behaving so bad when it pretends to be shared storage
> >friendly? Are these issues only specific for older Dovecot?
> >Or is there something wrong in my architecture design?
> 
> You need Director, Dovecot has not problems with shared storage, big
> installation are always using shared storage (like NFS).
> -- 
> Alessio Cecchi
> Postmaster AT http://www.qboxmail.it
> http://www.linkedin.com/in/alessice


signature.asc
Description: Digital signature


Dovecot cluster using GlusterFS

2015-12-05 Thread Filip Pytloun
Hello,

I have recently setup mailserver solution using 2-node master-master
setup (mainly based on MySQL M-M replication and GlusterFS with 2
replica volume) on Ubuntu 14.04 (Dovecot 2.2.9).

Unfortunately even with shared-storage-aware setting:

 mail_nfs_index = yes
 mail_nfs_storage = yes
 mail_fsync = always
 mmap_disable = yes

..I have hit strange issues pretty soon especially when user was
manipulating same mailbox from multiple devices at the same time.

Most issues was about corrupted indexes which was solved easily by just
putting them on local storage of each node:

 mail_location = maildir:/srv/mail/%d/%u:INDEX=/var/lib/dovecot/index/%d/%u

But I still hit issues like this one:

 dovecot: lmtp(6276, u...@example.com): Error: Broken file 
/srv/mail/example.com/u...@example.com/dovecot-uidlist line 8529: UIDs not 
ordered (8527 >= 8527)

Which I am not sure how serious it is or if it's possible to solve or
workaround?

Anyway because of the above and high possibility of GlusterFS
split-brains, I have decided to setup Dovecot Director according to the
docs [1] but I have a couple of questions:

- is custom monitoring still required? Poolmon [2] is 4 year old so I
  would suppose there's some progress since that?

- it's not possible to have same backends and directors in Dovecot
  <2.2.17. I can backport newer Dovecot for Ubuntu Trusty, so this is
  not an issue, but..

- documentation states that it still doesn't work for LMTP [3]?
  Which is probably important for my setup, because both Postfix servers
  are using dovecot-lmtp for mail delivery so there can be still some
  issues (but probably less frequent?) when both servers will deliver
  new mails for one user at once.
  So do I really have to split directors from backends?


Anyone has experience with clustered Dovecot setup?
Why is Dovecot behaving so bad when it pretends to be shared storage
friendly? Are these issues only specific for older Dovecot?
Or is there something wrong in my architecture design?

Thanks for any help,
Filip


---
[1] http://wiki2.dovecot.org/Director
[2] https://github.com/brandond/poolmon/
[3] "LMTP however doesn't currently support mixing recipients to both
 being proxied and store locally."

---
BTW if someone is interested in SaltStack, here are Salt formulas for
Dovecot + Postfix + GlusterFS + Roundcube + Mailman setup which we are
using:

https://github.com/tcpcloud/salt-formula-dovecot
https://github.com/tcpcloud/salt-formula-postfix
https://github.com/tcpcloud/salt-formula-roundcube
https://github.com/tcpcloud/salt-formula-glusterfs


signature.asc
Description: Digital signature