Just like anything else in apache... committers/community can ask or vote features out. it doesn't even need to be a plugin.
I don't mean to hijack this discussion away from Quorum.. although it would affect the implementation.. .we can keep a plugin's area in the broker... and activate them based on the CLI / config. I just want to make sure we can unlock this going forward. On Tue, Mar 17, 2020 at 8:53 PM michael.andre.pearce <michael.andre.pea...@me.com.invalid> wrote: > > Well the point is without us having sorted the issue, then this will keep > occuring.I am for one in favour we relax a little and if someone wants to add > a plugin we simply allow it. Why should there be any filter.Sent from my > Samsung Galaxy smartphone. > -------- Original message --------From: Clebert Suconic > <clebert.suco...@gmail.com> Date: 18/03/2020 00:38 (GMT+00:00) To: > dev@activemq.apache.org Subject: Re: Re: [DISCUSSION] New Quorum vote > pluggable implementation It's on a case by case basis... What we can't do is > transform everynew core functionality (such as monitoring. .it's pretty > standard /core functionality) into a plug-in, and then block its > implementationbased on a past discussion.I say, if there's a plugin, there > should be at least oneimplementation on the broker.. that goes for > Monitoring... we cansupport more.. but this has to have an implementation.You > are a committer, if you want to have your kafka plugin again, justraise a > discussion.. but we can't block every new functionality basedon that > baggage.On Tue, Mar 17, 2020 at 7:35 PM > michael.andre.pearce<michael.andre.pea...@me.com.invalid> wrote:>> I think > the issue with plugins is that whats the rule to include one or not.You know > exactly where im coming from there... re kafka plugin.Im totally for a number > of plugins being added. Including the kafka one.But we have to have a clear > rule of what will be allowed or not and a place to maintain.Sent from my > Samsung Galaxy smartphone.> -------- Original message --------From: Clebert > Suconic <clebert.suco...@gmail.com> Date: 16/03/2020 01:20 (GMT+00:00) To: > dev@activemq.apache.org Subject: Re: Re: [DISCUSSION] New Quorum vote > pluggable implementation We can make pluggable as much as you want, but going > forward I wouldlike us to change how we have been treating plugins.Because of > other discussions, plugins are not really implemented aspart of the > community.We should elect one implementation we can make it and implement > itwell.... and bring it as part of the community. People can add > moreimplementations as part of the open source project.This is probably the > same with Micrometer. Justin implementedMicrometer and the implementation is > not part of activemq. I think weshould change that going forward.This will be > a 3.0, so we should definitely improve how we do pluginsgoing forward.On Fri, > Mar 13, 2020 at 3:42 AM > michael.andre.pearce<michael.andre.pea...@me.com.invalid> wrote:>> I think a > main thing here is a pluggable api.If someone already has etcd, zookeeper, > consul etc. In their infrastructure they should be able to reuse.Like wise if > you have multiple bulkheaded broker sets it be useful to be able to reuse the > same quorum compontent e.g. in ZK that would be a different base root on the > tree path per broker set.Likewise that said we should have some out the box > support plugins from the community. Obviously what ever we do whilst it maybe > a break change e.g. a 3.0 release there must be a migration solution > (accepted that it may need outage) for existing brokers that works via the > pluggable api not to a specific solution.Also anything from a management api > pov should also be visible in the web management console. Sent from my > Samsung Galaxy smartphone.> -------- Original message --------From: Clebert > Suconic <clebert.suco...@gmail.com> Date: 09/03/2020 14:55 (GMT+00:00) To: > dev@activemq.apache.org Subject: Re: Re: [DISCUSSION] New Quorum vote > pluggable implementation I think we should have a management component, that > runs outside ofthe broker and would manage quorum.That way you can have the > quorum running outside of the broker itself.which would improve the need of > multiple brokers to manage the quorum.You just need Quorum Managers in > distinct places.I had recently worked with a software called Ceph... And Ceph > has theconcept of managers working away from their "broker" (it's not > abroker.. it's a DB, but in a sense it's the same concept here). Ithink we > should do the same.I have talked to Franz and other guys in private.. and it > seemseverybody these days mention raft consensus algorithm. Perhaps weshould > look into it.. and make it pluggable. I believe there's JRaftworking on top > of JGroups already.If we make this pluggable and make it a manager a separate > process,this will be a big win IMO.On Tue, Mar 3, 2020 at 9:55 AM Andy Taylor > <andy.tayl...@gmail.com> wrote:>> Personally I wouldn't use Zookeeper, I > think there are better options. Also> looks like Kafka are replacing it as > well. Saying that, it doesn't really> matter what is used, the main thing is > we need to remove the burden of> providing consensus away from the broker.>> > It would make sense to make it pluggable.>> A>> On Mon, 2 Mar 2020 at 19:19, > KimmKing <kimmk...@apache.org> wrote:>> > Can't agree any more.> > The > HA&Replication is more and more important for a modern messaging> > system.> > > Other apache opensource mq, kafka/rocketmq/pulsar maybe referred to.> >> >> > > And In my experience this also bring some extra-complexity about> > > performance/brainsplitting issues when rebalance data.> >> > At 2020-03-02 > 20:33:31, "Martyn Taylor" <m...@martyntaylor.me> wrote:> > >I think this is a > great idea Franz. The HA and replication components> > have> > >been a > source of issues over the years. Two problems I see are that 1)> > >there > isn't a clean separation between the consensus mechanism and the> > > >replication, and 2) the consensus algorithm used in Artemis isn't based on> > > >any standard algorithm or a research paper. Hence, all the issues that> > > >were caught over the years due to various edge cases. Integration with> > > >ZooKeeper seems like the obvious solution (i.e. push the consensus logic> > > >off to a third party lib). I suspect though, this will be a considerable> > > >amount of work and is likely to introduce new issues, so I'd proceed with> > > >caution.> > >> > >Cheers> > >> > >> > >> > >On Mon, Mar 2, 2020 at 8:28 AM > nigro_franz <nigro....@gmail.com> wrote:> > >> > >> Hi folks,> > >>> > >> > especially due to the requirements of the current Artemis quorum vote> > >> > algorithm, we've thought to re-implementing it with a different focus in> > > >> mind:> > >> 1) to make it pluggable (eg by using the many Raft > implementations,> > >> ZooKeeper or others)> > >> 2) to cleanly separate the > election phase and cluster member states (ie> > it> > >> should be the > Topology shared between them)> > >> 3) to simplify most common setups in both > amount of configuration and> > >> requirements (eg "witness" nodes could be > implemented to get a minimum> > 2*n> > >> +1 quorum of nodes instead of > forcing 2*n + 1 master-backup pairs)> > >> 4) [OPTIONALLY] to > reduce/eliminate implicit "good practices" in term of> > >> order of actions > to be performed on nodes in "special states" eg proper> > >> restart sequence > after failover or similar cases> > >> 5) [OPTIONALLY] to make shared-store > and replication behaviour more> > >> similar:> > >> journal's presence should > be the only difference between the 2s> > >>> > >> A proposal of steps to be > followed to get this:> > >> 1) abstract away the current quorum vote: it > requires extra-care because> > >> the> > >> logic is melted together with the > replication/clustering behaviour> > >> 2) refactor it in order to separate > election phase and cluster member> > >> states> > >> 3) implement a RI > version of such APIs> > >>> > >> Post-actions to help people adopt it, but > need to be thought upfront:> > >> 1) a clean upgrade path for current HA > replication users> > >> 2) deprecate or integrate the current HA replication > into the new> > version> > >>> > >> I've opened this here because many of the > HA replication users are devs> > too> > >> and given that this isn't yet > implemented: we're stull in the> > >> design/proposal phase, so anyone that > want to express their> > >> ideas/opinions/POC on this, is invited to talk > here ;)> > >>> > >> Cheers,> > >> Franz> > >>> > >>> > >>> > >>> > >>> > >> > --> > >> Sent from:> > >> > http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html> > >>> >-- > Clebert Suconic-- Clebert Suconic-- Clebert Suconic -- Clebert Suconic
