Re: Please do not remove replication

2024-01-25 Thread Jörg Schulz 

and, as stated in this thread, the doveadm sync command will persist.

For a 8 user environment, it should be enough to do this via crontab, 
every some minutes.



Am 25.01.24 um 19:22 schrieb pgnd:
Currently, replication is my life saver. 


depending on your use case, and how critical that is, there _are_ 
other options available.


not least of which is,

https://www.cyrusimap.org/imap/reference/admin/sop/replication.html

it's not "keeping replication in Dovecot".  but it is well-supported 
FOSS replication.


ymmv.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Please do not remove replication

2024-01-25 Thread Narcis Garcia 

+1

El 24/1/24 a les 23:33, Gerben Wierda ha escrit:

Respectfully, I would like to ask: please do not remove replication, please
rethink this.

Currently, replication is my life saver. I run two postfix/dovecot combos (on
different operating systems), with dovecot synchronising via replication. Both
are behind a HAProxy running on the router (OPNsense), one as active, one as
backup.

If one of the two fails, the other takes over, and when it comes up again
everything works fine and is up to date. I have had these kinds of system
failures (very hard to find and turned to be hardware related) and it was the
replication that made me survive the issues (even when I was far away from my
systems). Mail for my small group of users (about 8) never went down, no mail
message was ever lost, no manual interventions to sync were ever needed.

If I want to create the same level of availability without replication, I need
those two dovecots to use shared (NFS cluster) storage. But then, I have
another single point of failure (NFS storage) again. So, I need two separate
NFS machines that synchronise, Apart from the nightmare of making NFS secure,
it means that I need to double my hardware (from two systems to four) to be
protected against hardware failure (which is my goal).

The replication service is the perfect small scale solution. Together with
HAProxy, it enables HA in the most simple and effective way. Going the 'NFS
cluster' route is not feasible for me, so if replication is removed and I am
forced to upgrade, I will lose HA.

So please, take small scale users like me into account.

Gerben Wierda (LinkedIn, Mastodon)
R_IT_Strategy (main site)
Book: Chess_and_the_Art_of_Enterprise Architecture
Book: Mastering_ArchiMate
YouTube_Channel

  On 16 Jul 2023, at 18:54, Aki Tuomi via dovecot 
  wrote:

  Hi!

  Yes, director and replicator are removed, and won't be available for
  pro users either.

  For NFS setups (or similar shared setups), we have documented a way
  to use Lua to run a director-like setup, see

  https://doc.dovecot.org/3.0/configuration_manual/howto/
  director_with_lua/

  Regards to replication, doveadm sync is not being removed. So you can
  still run doveadm sync on your system to have a primary / backup
  setup.

  Aki

   On 16/07/2023 18:34 EEST William Edwards via dovecot
wrote:


   Top posting because nothing specific to reply to, sorry.
   Not exactly sure, but there’s another thread about the
   removal of Director in favour of Dovecot Pro on 3.x.
   Perhaps this change is related.

   William Edwards

Op 16 jul. 2023 om 16:33 heeft Daniele
 het volgende geschreven:

Hello,

Just like Vladimir, I'm a bit concerned about
this change, and I'd really appreciate if someone
could let us know if the replication feature
(that works so well!) will be replaced or
removed; and, in case of removal, what would be
recommended replacement?
Thanks in advance and best regards,
Daniele

 On 09-Jul-23 9:36 PM, Vladimir Mishonov
 via dovecot wrote:
 Hello everyone.

 Just saw this commit in the official
 Github repo:

 https://github.com/dovecot/core/commit/
 4c04e4c30fd4817a8b0e11d04d9681173f696f41#diff-
 
5f643d8b0d1eea65d0f3c749d14d42b25a9d60f0f149bface862f5ff348412c8


 Looking at the commit details, it
 appears that it completely removes the
 replication feature. I'm a bit
 perplexed by this change and am not
 sure what might be the justification
 for it. Personally, I find replication
 to be very useful, as it allows me to
 maintain a synchronized mirror of all
 of my mailboxes on my home server, for
 use as backup in case the primary
 server goes down for some reason.

 Perhaps there's some sort of
 replacement being planned for this
 feature? Or maybe the relevant code is
 simply going to be refactored to a
 plugin or external program, and there's
 nothing to worry about at all?

 In any case, I'd greatly appreciate if
 one of the developers could comment on
 this change.


___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-

RE: Please do not remove replication

2024-01-25 Thread Aki Tuomi via dovecot 

> On 25/01/2024 11:57 EET Michael Grant via dovecot  wrote:
> 
>  
> I keep seeing this come up over and over.  My understanding is it’s not 
> getting removed, it’s just moving to the paid version of Dovecot.  What is 
> the cost for a small user license of dovecot that incudes replication anyway? 
>  Is the price that outrageous?  
> I keep seeing this come up over and over.  My understanding is it’s not 
> getting
> removed, it’s just moving to the paid version of Dovecot.  What is the cost 
> for
> a small user license of dovecot that incudes replication anyway?  Is the price
> that outrageous? 


Hi!

Your understanding is wrong though, we are not moving replicator or director to 
a paid version of Dovecot, they are fully removed.

Aki
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

RE: Please do not remove replication

2024-01-25 Thread Marc 

Ok, very very nice of you to make such redundant setup for these people. For 
such small amount I would just keep one server, and have everyone wait when it 
fails. ;)

Ok with such small user group you do not have any issues with performance with 
any hardware. I was creating recently a test cluster with ceph for a data 
centre move and changed on it the failure domain from host to osd. That worked 
pretty well with just 2 servers. 
So what you could do is run ceph on both servers with osd failure domain and 
use the shared rbd image as a 'replication' alternative. This way you have an 
instant backup available. You don't really need to configure 2 dovecot servers. 
You can move/start the vm from 1 host to the other. 
Actually if all disks fail on host, the vm's on that host keep running, 
connecting to the disks on the other host. Really nice.


> 
> I have two postfix/(replicating)dovecot/rspamd/etc servers, 6-8 human users,
> on a total of 15-20 devices (iOS and macOS)
> 
> Yours,
> 
> Gerben Wierda (LinkedIn  ,
> Mastodon  )
> R IT Strategy   (main site)
> Book: Chess and the Art of Enterprise Architecture  book/>
> Book: Mastering ArchiMate 
> YouTube Channel 
> 
> 
>   On 25 Jan 2024, at 10:17, Marc  wrote:
> 
>   How many servers do you have? How many active clients do you have?
> 
> 
> 
> 
>   Respectfully, I would like to ask: please do not remove
> replication, please
>   rethink this.
> 
>   Currently, replication is my life saver. I run two
> postfix/dovecot combos
>   (on different operating systems), with dovecot synchronising via
>   replication. Both are behind a HAProxy running on the router
> (OPNsense), one
>   as active, one as backup.
> 
>   If one of the two fails, the other takes over, and when it comes
> up again
>   everything works fine and is up to date. I have had these kinds
> of system
>   failures (very hard to find and turned to be hardware related)
> and it was
>   the replication that made me survive the issues (even when I was
> far away
>   from my systems). Mail for my small group of users (about 8)
> never went
>   down, no mail message was ever lost, no manual interventions to
> sync were
>   ever needed.
> 
>   If I want to create the same level of availability without
> replication, I
>   need those two dovecots to use shared (NFS cluster) storage. But
> then, I
>   have another single point of failure (NFS storage) again. So, I
> need two
>   separate NFS machines that synchronise, Apart from the nightmare
> of making
>   NFS secure, it means that I need to double my hardware (from two
> systems to
>   four) to be protected against hardware failure (which is my
> goal).
> 
>   The replication service is the perfect small scale solution.
> Together with
>   HAProxy, it enables HA in the most simple and effective way.
> Going the 'NFS
>   cluster' route is not feasible for me, so if replication is
> removed and I am
>   forced to upgrade, I will lose HA.
> 
>   So please, take small scale users like me into account.
> 
>   Gerben Wierda (LinkedIn
> , Mastodon
>   )
>   R IT Strategy  (main site)
>   Book: Chess and the Art of Enterprise Architecture
>    book/>
>   Book: Mastering ArchiMate  iii/>
>   YouTube Channel 
> 
> 
> 
>   On 16 Jul 2023, at 18:54, Aki Tuomi via dovecot
> 
> 
> 
>   wrote:
> 
> 
> 
>   Hi!
> 
>   Yes, director and replicator are removed, and won't be
> available for pro
> 
> 
>   users either.
> 
> 
> 
>   For NFS setups (or similar shared setups), we have
> documented a way to use
> 
> 
>   Lua to run a director-like setup, see
> 
> 
> 
> 
>   https://doc.dovecot.org/3.0/configuration_manual/howto/director_with_l
> ua/
> 
>   Regards to replication, doveadm sync is not being 
> removed.
> So you can
> 
> 
>   still run doveadm sync on your system to have a primary / backup
> setup.
> 
> 
> 
>   Aki
> 
> 
> 
>   On 16/07/2023 18:34 EEST William Edwards via 
> dovecot
> 
> 
>wrote:
> 
> 
> 
> 
>   Top posting because nothing specific to reply 
>

RE: Please do not remove replication

2024-01-25 Thread Michael Grant via dovecot 
I keep seeing this come up over and over.  My understanding is it’s not getting
removed, it’s just moving to the paid version of Dovecot.  What is the cost for
a small user license of dovecot that incudes replication anyway?  Is the price
that outrageous? 
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Please do not remove replication

2024-01-25 Thread Gerben Wierda 
I have two postfix/(replicating)dovecot/rspamd/etc servers, 6-8 human users, on
a total of 15-20 devices (iOS and macOS)

Yours, 

Gerben Wierda (LinkedIn, Mastodon)
R_IT_Strategy (main site)
Book: Chess_and_the_Art_of_Enterprise Architecture
Book: Mastering_ArchiMate
YouTube_Channel

 On 25 Jan 2024, at 10:17, Marc  wrote:

 How many servers do you have? How many active clients do you have?


  Respectfully, I would like to ask: please do not remove
  replication, please
  rethink this.

  Currently, replication is my life saver. I run two postfix/
  dovecot combos
  (on different operating systems), with dovecot
  synchronising via
  replication. Both are behind a HAProxy running on the
  router (OPNsense), one
  as active, one as backup.

  If one of the two fails, the other takes over, and when it
  comes up again
  everything works fine and is up to date. I have had these
  kinds of system
  failures (very hard to find and turned to be hardware
  related) and it was
  the replication that made me survive the issues (even when
  I was far away
  from my systems). Mail for my small group of users (about
  8) never went
  down, no mail message was ever lost, no manual
  interventions to sync were
  ever needed.

  If I want to create the same level of availability without
  replication, I
  need those two dovecots to use shared (NFS cluster)
  storage. But then, I
  have another single point of failure (NFS storage) again.
  So, I need two
  separate NFS machines that synchronise, Apart from the
  nightmare of making
  NFS secure, it means that I need to double my hardware
  (from two systems to
  four) to be protected against hardware failure (which is my
  goal).

  The replication service is the perfect small scale
  solution. Together with
  HAProxy, it enables HA in the most simple and effective
  way. Going the 'NFS
  cluster' route is not feasible for me, so if replication is
  removed and I am
  forced to upgrade, I will lose HA.

  So please, take small scale users like me into account.

  Gerben Wierda (LinkedIn , Mastodon
  )
  R IT Strategy  (main site)
  Book: Chess and the Art of Enterprise Architecture 
  Book: Mastering ArchiMate 
  YouTube Channel 

   On 16 Jul 2023, at 18:54, Aki Tuomi via dovecot
   
  wrote:

   Hi!

   Yes, director and replicator are removed, and
   won't be available for pro
  users either.

   For NFS setups (or similar shared setups), we
   have documented a way to use
  Lua to run a director-like setup, see

   https://doc.dovecot.org/3.0/configuration_manual/
   howto/director_with_lua/

   Regards to replication, doveadm sync is not being
   removed. So you can
  still run doveadm sync on your system to have a primary /
  backup setup.

   Aki

On 16/07/2023 18:34 EEST William
Edwards via dovecot
   wrote:


Top posting because nothing specific to
reply to, sorry. Not exactly
  sure, but there’s another thread about the removal of
  Director in favour of
  Dovecot Pro on 3.x. Perhaps this change is related.

William Edwards

 Op 16 jul. 2023 om 16:33
 heeft Daniele  het
  volgende geschreven:

 Hello,

 Just like Vladimir, I'm a bit
 concerned about this change,
 and I'd
  really appreciate if someone could let us know if the
  replication feature
  (that works so well!) will be replaced or removed; and, in
  case of removal,
  what would be recommended replacement?
 Thanks in advance and best
 regards,
 Daniele

  On 09-Jul-23 9:36
  PM, Vladimir
  Mishonov via
  dovecot wrote:
  Hello everyone.

  Just saw this
  commit in the
  official Github
  repo:

RE: Please do not remove replication

2024-01-25 Thread Marc 
How many servers do you have? How many active clients do you have?

> 
> Respectfully, I would like to ask: please do not remove replication, please
> rethink this.
> 
> Currently, replication is my life saver. I run two postfix/dovecot combos
> (on different operating systems), with dovecot synchronising via
> replication. Both are behind a HAProxy running on the router (OPNsense), one
> as active, one as backup.
> 
> If one of the two fails, the other takes over, and when it comes up again
> everything works fine and is up to date. I have had these kinds of system
> failures (very hard to find and turned to be hardware related) and it was
> the replication that made me survive the issues (even when I was far away
> from my systems). Mail for my small group of users (about 8) never went
> down, no mail message was ever lost, no manual interventions to sync were
> ever needed.
> 
> If I want to create the same level of availability without replication, I
> need those two dovecots to use shared (NFS cluster) storage. But then, I
> have another single point of failure (NFS storage) again. So, I need two
> separate NFS machines that synchronise, Apart from the nightmare of making
> NFS secure, it means that I need to double my hardware (from two systems to
> four) to be protected against hardware failure (which is my goal).
> 
> The replication service is the perfect small scale solution. Together with
> HAProxy, it enables HA in the most simple and effective way. Going the 'NFS
> cluster' route is not feasible for me, so if replication is removed and I am
> forced to upgrade, I will lose HA.
> 
> So please, take small scale users like me into account.
> 
> Gerben Wierda (LinkedIn , Mastodon
> )
> R IT Strategy  (main site)
> Book: Chess and the Art of Enterprise Architecture  book/>
> Book: Mastering ArchiMate 
> YouTube Channel 
> 
> > On 16 Jul 2023, at 18:54, Aki Tuomi via dovecot 
> wrote:
> >
> > Hi!
> >
> > Yes, director and replicator are removed, and won't be available for pro
> users either.
> >
> > For NFS setups (or similar shared setups), we have documented a way to use
> Lua to run a director-like setup, see
> >
> > https://doc.dovecot.org/3.0/configuration_manual/howto/director_with_lua/
> >
> > Regards to replication, doveadm sync is not being removed. So you can
> still run doveadm sync on your system to have a primary / backup setup.
> >
> > Aki
> >
> >> On 16/07/2023 18:34 EEST William Edwards via dovecot
>  wrote:
> >>
> >>
> >> Top posting because nothing specific to reply to, sorry. Not exactly
> sure, but there’s another thread about the removal of Director in favour of
> Dovecot Pro on 3.x. Perhaps this change is related.
> >>
> >> William Edwards
> >>
> >>> Op 16 jul. 2023 om 16:33 heeft Daniele  het
> volgende geschreven:
> >>>
> >>> Hello,
> >>>
> >>> Just like Vladimir, I'm a bit concerned about this change, and I'd
> really appreciate if someone could let us know if the replication feature
> (that works so well!) will be replaced or removed; and, in case of removal,
> what would be recommended replacement?
> >>> Thanks in advance and best regards,
> >>> Daniele
> >>>
>  On 09-Jul-23 9:36 PM, Vladimir Mishonov via dovecot wrote:
>  Hello everyone.
> 
>  Just saw this commit in the official Github repo:
> 
> 
> https://github.com/dovecot/core/commit/4c04e4c30fd4817a8b0e11d04d9681173f696
> f41#diff-5f643d8b0d1eea65d0f3c749d14d42b25a9d60f0f149bface862f5ff348412c8
> 
>  Looking at the commit details, it appears that it completely removes
> the replication feature. I'm a bit perplexed by this change and am not sure
> what might be the justification for it. Personally, I find replication to be
> very useful, as it allows me to maintain a synchronized mirror of all of my
> mailboxes on my home server, for use as backup in case the primary server
> goes down for some reason.
> 
>  Perhaps there's some sort of replacement being planned for this
> feature? Or maybe the relevant code is simply going to be refactored to a
> plugin or external program, and there's nothing to worry about at all?
> 
>  In any case, I'd greatly appreciate if one of the developers could
> comment on this change.
> 
> >>>
> >>> ___
> >>> dovecot mailing list -- dovecot@dovecot.org
> >>> To unsubscribe send an email to dovecot-le...@dovecot.org
> >>>
> >>
> >> ___
> >> dovecot mailing list -- dovecot@dovecot.org
> >> To unsubscribe send an email to dovecot-le...@dovecot.org
> > ___
> > dovecot mailing list -- dovecot@dovecot.org
> > To unsubscribe send an email to dovecot-le...@dovecot.org

___

Re: Please do not remove replication

2024-01-24 Thread Michael Peddemors 

On 2024-01-24 16:35, Steven Varco wrote:

Although I’m also a very happy dovecot replication user, I don’t think this 
decision will be reverted, sadly.

However, despite of messing with NFS, I will try setting up a three-node 
GlusterFS Cluster to give redundant storage to dovecote as mail store and hope 
it performs well enough… Has anyone else such a setup (or alternatively with 
Ceph) in production?

Steven



Seen some Gluster backends blow up spectacularly..

Always say.. keep it simple.  Every thought of NFS backend, and let the 
NetApp do the job? Scales well, and haven't seen one go down in 
production yet.. knock on wood.. and the costs have really dropped.



--
"Catch the Magic of Linux..."

Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Reg. TradeMark of Wizard Tower TechnoServices Ltd.

604-682-0300 Beautiful British Columbia, Canada

___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Please do not remove replication

2024-01-24 Thread Steven Varco 
Although I’m also a very happy dovecot replication user, I don’t think this 
decision will be reverted, sadly.

However, despite of messing with NFS, I will try setting up a three-node 
GlusterFS Cluster to give redundant storage to dovecote as mail store and hope 
it performs well enough… Has anyone else such a setup (or alternatively with 
Ceph) in production?

Steven

-- 
https://steven.varco.ch/ 
https://www.tech-island.com/ 


> Am 24.01.2024 um 23:33 schrieb Gerben Wierda :
> 
> Respectfully, I would like to ask: please do not remove replication, please
> rethink this.
> 
> Currently, replication is my life saver. I run two postfix/dovecot combos (on
> different operating systems), with dovecot synchronising via replication. Both
> are behind a HAProxy running on the router (OPNsense), one as active, one as
> backup.
> 
> If one of the two fails, the other takes over, and when it comes up again
> everything works fine and is up to date. I have had these kinds of system
> failures (very hard to find and turned to be hardware related) and it was the
> replication that made me survive the issues (even when I was far away from my
> systems). Mail for my small group of users (about 8) never went down, no mail
> message was ever lost, no manual interventions to sync were ever needed.
> 
> If I want to create the same level of availability without replication, I need
> those two dovecots to use shared (NFS cluster) storage. But then, I have
> another single point of failure (NFS storage) again. So, I need two separate
> NFS machines that synchronise, Apart from the nightmare of making NFS secure,
> it means that I need to double my hardware (from two systems to four) to be
> protected against hardware failure (which is my goal).
> 
> The replication service is the perfect small scale solution. Together with
> HAProxy, it enables HA in the most simple and effective way. Going the 'NFS
> cluster' route is not feasible for me, so if replication is removed and I am
> forced to upgrade, I will lose HA.
> 
> So please, take small scale users like me into account.
> 
> Gerben Wierda (LinkedIn, Mastodon)
> R_IT_Strategy (main site)
> Book: Chess_and_the_Art_of_Enterprise Architecture
> Book: Mastering_ArchiMate
> YouTube_Channel
> 
> On 16 Jul 2023, at 18:54, Aki Tuomi via dovecot 
> wrote:
> 
> Hi!
> 
> Yes, director and replicator are removed, and won't be available for
> pro users either.
> 
> For NFS setups (or similar shared setups), we have documented a way
> to use Lua to run a director-like setup, see
> 
> https://doc.dovecot.org/3.0/configuration_manual/howto/
> director_with_lua/
> 
> Regards to replication, doveadm sync is not being removed. So you can
> still run doveadm sync on your system to have a primary / backup
> setup.
> 
> Aki
> 
>  On 16/07/2023 18:34 EEST William Edwards via dovecot
>   wrote:
> 
> 
>  Top posting because nothing specific to reply to, sorry.
>  Not exactly sure, but there’s another thread about the
>  removal of Director in favour of Dovecot Pro on 3.x.
>  Perhaps this change is related.
> 
>  William Edwards
> 
>   Op 16 jul. 2023 om 16:33 heeft Daniele
>het volgende geschreven:
> 
>   Hello,
> 
>   Just like Vladimir, I'm a bit concerned about
>   this change, and I'd really appreciate if someone
>   could let us know if the replication feature
>   (that works so well!) will be replaced or
>   removed; and, in case of removal, what would be
>   recommended replacement?
>   Thanks in advance and best regards,
>   Daniele
> 
>On 09-Jul-23 9:36 PM, Vladimir Mishonov
>via dovecot wrote:
>Hello everyone.
> 
>Just saw this commit in the official
>Github repo:
> 
>https://github.com/dovecot/core/commit/
>4c04e4c30fd4817a8b0e11d04d9681173f696f41#diff-
>
> 5f643d8b0d1eea65d0f3c749d14d42b25a9d60f0f149bface862f5ff348412c8
> 
> 
>Looking at the commit details, it
>appears that it completely removes the
>replication feature. I'm a bit
>perplexed by this change and am not
>sure what might be the justification
>for it. Personally, I find replication
>to be very useful, as it allows me to
>maintain a synchronized mirror of all
>of my mailboxes on my home server, for
>use as backup in case the primary
>server goes down for some reason.
> 
>Perhaps there's some sort of
>replacement being planned for this
>feature? Or