[pfx] Re: distributed email system
Jeff Peng via Postfix-users: > On 2024-06-15 21:35, Wietse Venema via Postfix-users wrote: > > > > > This is a bit off topic for Postfix, but a comnmon approach is to > > shard a global database into regional ones and limit the impact of > > outages. Some database systems support sharding out of the box (for > > example, MongoDB, supported by Postfix 3.9 and later). > > > > Hi Wietse, > > what does it mean "sharding out of the box"? "X supports Y out of the box" means epression that X has support for Y built in. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
On 2024-06-15 21:35, Wietse Venema via Postfix-users wrote: This is a bit off topic for Postfix, but a comnmon approach is to shard a global database into regional ones and limit the impact of outages. Some database systems support sharding out of the box (for example, MongoDB, supported by Postfix 3.9 and later). Hi Wietse, what does it mean "sharding out of the box"? regards. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
Joachim Lindenberg via Postfix-users: > sql databases optimize for consistency instead of availability. > And even if you design your data model not to rely on joins, to > use unique ids per node, and to replicate both directions or > disallow writes on the slave, at least MariaDB failed on partitioning, > and I didn't want or tried to use an database not supported by the > application. > Obviously you may get by without encountering issues for quite > some time, and this unfortunately is true for many replication > scenario recommendations. This is a bit off topic for Postfix, but a comnmon approach is to shard a global database into regional ones and limit the impact of outages. Some database systems support sharding out of the box (for example, MongoDB, supported by Postfix 3.9 and later). Reportedly, Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
sql databases optimize for consistency instead of availability. And even if you design your data model not to rely on joins, to use unique ids per node, and to replicate both directions or disallow writes on the slave, at least MariaDB failed on partitioning, and I didn´t want or tried to use an database not supported by the application. Obviously you may get by without encountering issues for quite some time, and this unfortunately is true for many replication scenario recommendations. Cheers, Joachim -Ursprüngliche Nachricht- Von: Jeff Peng via Postfix-users Gesendet: Samstag, 15. Juni 2024 07:20 An: postfix user Betreff: [pfx] Re: distributed email system On 2024-06-15 12:46, Jean-François Bachelet via Postfix-users wrote: > Hello folks :) > isn't it what sql databases replication is good for ? > Replication becomes bad when network partition. ;) ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
On 2024-06-15 12:46, Jean-François Bachelet via Postfix-users wrote: Hello folks :) isn't it what sql databases replication is good for ? Replication becomes bad when network partition. ;) ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
Hello folks :) Le 14/06/2024 à 22:00, Joachim Lindenberg via Postfix-users a écrit : And the really hard part is to ensure those databases remain consistent with network failures. Cheers, Joachim isn't it what sql databases replication is good for ? cheers too ^^) Jeff -Ursprüngliche Nachricht- Von: Wietse Venema via Postfix-users Gesendet: Freitag, 14. Juni 2024 16:31 An: Postfix users Betreff: [pfx] Re: distributed email system Jeff Peng via Postfix-users: Hello, Is there any guide to setup a distributed email system? there should be multiple MX, multiple IMAP/storage servers, and sasl server cluster etc. That could be a job interview question. The easier part is setting up a pool of dataless Postfix SMTP servers; sharding the users across a pool of replicated Dovecot POP/IMAP servers; and nginx proxies. The harder part is implementing the databases and (web) applications to manage the above. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
On 2024-06-15 06:32, Wietse Venema via Postfix-users wrote: There is a difference between IMAP/POP and SMTP. With IMAP/POP a front end proxy needs to connect each user to the right message store instance. With SMTP, different sessions can be handled by different servers. The servers can figure where to deliver mail. That's all right. Thanks Wietse. And I think there should be a SASL cluster for smtp servers all around the world to get authorized. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
Jeff Peng via Postfix-users: > On 2024-06-14 22:31, Wietse Venema via Postfix-users wrote: > > Jeff Peng via Postfix-users: > >> Hello, > >> > >> Is there any guide to setup a distributed email system? there should > >> be > >> multiple MX, multiple IMAP/storage servers, and sasl server cluster > >> etc. > > > > That could be a job interview question. > > > > The easier part is setting up a pool of dataless Postfix SMTP > > servers; sharding the users across a pool of replicated Dovecot > > POP/IMAP servers; and nginx proxies. > > Are IMAP and smtp stateful protocols? If so I have to use hashes based > on src IP etc. There is a difference between IMAP/POP and SMTP. With IMAP/POP a front end proxy needs to connect each user to the right message store instance. With SMTP, different sessions can be handled by different servers. The servers can figure where to deliver mail. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
On 2024-06-14 22:31, Wietse Venema via Postfix-users wrote: Jeff Peng via Postfix-users: Hello, Is there any guide to setup a distributed email system? there should be multiple MX, multiple IMAP/storage servers, and sasl server cluster etc. That could be a job interview question. The easier part is setting up a pool of dataless Postfix SMTP servers; sharding the users across a pool of replicated Dovecot POP/IMAP servers; and nginx proxies. Are IMAP and smtp stateful protocols? If so I have to use hashes based on src IP etc. Thank you. regards. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
And the really hard part is to ensure those databases remain consistent with network failures. Cheers, Joachim -Ursprüngliche Nachricht- Von: Wietse Venema via Postfix-users Gesendet: Freitag, 14. Juni 2024 16:31 An: Postfix users Betreff: [pfx] Re: distributed email system Jeff Peng via Postfix-users: > Hello, > > Is there any guide to setup a distributed email system? there should > be multiple MX, multiple IMAP/storage servers, and sasl server cluster etc. That could be a job interview question. The easier part is setting up a pool of dataless Postfix SMTP servers; sharding the users across a pool of replicated Dovecot POP/IMAP servers; and nginx proxies. The harder part is implementing the databases and (web) applications to manage the above. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
Regarding the HA cluster setup for email services, both Cyrus IMAPd and Dovecot support master-slave replication. If you're setting up a cluster in HA, you'll need to store your emails in an object storage. Dovecot Pro includes this feature. Cyrus IMAPd has some support for object storage too, but I haven't tested it personally. On 6/14/24 11:13, Jeff Peng via Postfix-users wrote: Hello, Is there any guide to setup a distributed email system? there should be multiple MX, multiple IMAP/storage servers, and sasl server cluster etc. Thanks. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: distributed email system
Jeff Peng via Postfix-users: > Hello, > > Is there any guide to setup a distributed email system? there should be > multiple MX, multiple IMAP/storage servers, and sasl server cluster etc. That could be a job interview question. The easier part is setting up a pool of dataless Postfix SMTP servers; sharding the users across a pool of replicated Dovecot POP/IMAP servers; and nginx proxies. The harder part is implementing the databases and (web) applications to manage the above. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
