Hello, Sorry if my question wasn't clear. Actually it's Rob that is right. Since the Broker must have a persistent configuration store to handle messages recovery I would expect the Dispatch-Router to provide something similar to have an homogeneous way of handling my cluster.
Olivier -----Original Message----- From: Rob Godfrey <rob.j.godf...@gmail.com> Sent: lundi 25 juin 2018 12:26 To: users@qpid.apache.org Cc: Keith Wall <keith.w...@gmail.com> Subject: Re: [Broker-J 7.0.3] Memory configuration store and messages recovery Olivier please correct me if I am wrong, but I took the request to be that Dispatch would maintain persistently a copy of its own configuration and make necessary updates when that configuration is mutated through its AMQP management. As far as I understand it Dispatch currently has no way of storing persistently configuration that is updated via dispatch itself. -- Rob On Mon, 25 Jun 2018 at 12:15, Oleksandr Rudyy <oru...@gmail.com> wrote: > Hi Rob, > > Potentially the problem could be solved by implementing a special > Virtual Host Node which would interact with a third party service > where configuration is stored. > If dispatcher can provide such configuration either via AMQP or other > interfaces, a new Virtual Host Node needs to be implemented on Broker > side in order to get the configuration from Dispatch Router and its > store. > > Perhaps, I misunderstood Olivier's intention here. I assumed that > under "new config stores for the dispatch-router", a new config store > for the Broker is meant which is capable of interaction with a > Dispatch Router to get the configuration. > > I am sorry for confusion here. > > Kind Regards, > Alex > > > > > On 25 June 2018 at 10:51, Rob Godfrey <rob.j.godf...@gmail.com> wrote: > > Hi Alex, > > > > On Mon, 25 Jun 2018 at 11:44, Oleksandr Rudyy <oru...@gmail.com> wrote: > > > >> Hi Olivier, > >> > >> An implementation of new non-persistent config store for the > >> dispatch-router is not on the current Qpid roadmap. > >> You are welcome to contribute such implementation. > >> Basically, you need to implement a new type of VirtualHostNode and > >> new type of DurableConfigurationStore. > >> Potentially, a new type of VirtualHostNode can extend existing > >> org.apache.qpid.server.virtualhostnode.AbstractStandardVirtualHostNode. > >> You can follow an implementation of JSON Virtual Host Node > >> (org.apache.qpid.server.virtualhostnode.JsonVirtualHostNodeImpl). > >> > > > > > > I'm not sure I understand how any of this relates to the Dispatch Router? > > All your comments above seem to be referencing Qpid Broker-J not > Dispatch. > > > > -- Rob > > > > > >> > >> Kind Regards, > >> Alex > >> > >> On 25 June 2018 at 08:39, VERMEULEN Olivier < > olivier.vermeu...@murex.com> > >> wrote: > >> > Ok so I must keep a persistent config store for the Broker. > >> > But if I'm not mistaken the Dispacth-Router only supports a > >> non-persistent config store. > >> > So are there any plans to implement new config stores for the > >> dispatch-router? > >> > > >> > Thanks, > >> > Olivier > >> > > >> > -----Original Message----- > >> > From: Rob Godfrey <rob.j.godf...@gmail.com> > >> > Sent: vendredi 22 juin 2018 13:20 > >> > To: users@qpid.apache.org > >> > Cc: Keith Wall <keith.w...@gmail.com> > >> > Subject: Re: [Broker-J 7.0.3] Memory configuration store and > >> > messages > >> recovery > >> > > >> > Sure - but at that point you really need to provide the vhost > >> > config > >> before the vhost itself starts up... would providing all the config > >> (including the queue UUIDs) in a virtualHostInitialConfiguration > provided > >> to the VirtualHostNode on creation help I wonder... > >> > > >> > -- Rob > >> > > >> > On Fri, 22 Jun 2018 at 12:14, VERMEULEN Olivier < > >> olivier.vermeu...@murex.com> > >> > wrote: > >> > > >> >> Hello Rob, > >> >> > >> >> The main problem is to keep my version of the config in sync > >> >> with the one of each broker. > >> >> And knowing that all our brokers have the same config the added > >> >> complexity seems a bit overkill. > >> >> Especially when you start thinking about broker restart, scale > >> >> up, scale down... > >> >> > >> >> Olivier > >> >> > >> >> -----Original Message----- > >> >> From: Rob Godfrey <rob.j.godf...@gmail.com> > >> >> Sent: vendredi 22 juin 2018 11:27 > >> >> To: users@qpid.apache.org > >> >> Cc: Keith Wall <keith.w...@gmail.com> > >> >> Subject: Re: [Broker-J 7.0.3] Memory configuration store and > >> >> messages recovery > >> >> > >> >> In general, in order to be able to recover the queues from a > >> >> message store, the broker needs to know details of the queue - > >> >> not only its name, but also the type of the queue (standard, > >> >> priority, LVQ, etc) > as > >> >> well as other queue properties... this is precisely the data > >> >> which is stored in the configuration store, I don't think > >> >> there's really any way around that problem. What sort of > >> >> problems are you experiencing by using a non-Memory configuration store > >> >> in your setup? > >> >> > >> >> -- Rob > >> >> > >> >> On Fri, 22 Jun 2018 at 10:47, VERMEULEN Olivier < > >> >> olivier.vermeu...@murex.com> > >> >> wrote: > >> >> > >> >> > Hello Keith, > >> >> > > >> >> > Thanks for the quick answer. > >> >> > Our target is a cluster of brokers and dispatch-routers. > >> >> > To configure it we created a management component that > >> >> > centralizes the configuration of all the Qpid components. > >> >> > With this approach, having the broker store its own > >> >> > configuration > is > >> >> > redundant with what our management component does and causes > >> >> > some > >> >> problems. > >> >> > That's why I was looking into the memory configuration store > >> >> > which would have made the broker behave like the > >> >> > dispatch-router > regarding > >> >> > dynamic configuration. > >> >> > > >> >> > Olivier > >> >> > > >> >> > -----Original Message----- > >> >> > From: Keith W <keith.w...@gmail.com> > >> >> > Sent: vendredi 22 juin 2018 10:14 > >> >> > To: users@qpid.apache.org > >> >> > Subject: Re: [Broker-J 7.0.3] Memory configuration store and > >> >> > messages recovery > >> >> > > >> >> > Hello Olivier > >> >> > > >> >> > Let me say first that the memory store's primary use-case is > >> >> > to aid development and testing of Broker-J, by speeding ups > >> >> > the test > cycle. > >> >> > It also has utility when using embedding Broker-J in the unit > >> >> > tests of another project and you want to discard all messaging > >> >> > state between > >> >> tests. > >> >> > > >> >> > You are correct in your analysis. Queues (like all other objects) > >> >> > are allocated a random UUID on creation. This UUID is stored > >> >> > in > the > >> >> > configuration store together with the queue's name and all > >> >> > other attributes. Within the message store, the message > >> >> > instance records refer to the queue's UUID rather than its > >> >> > name. So if your set-up is a persistent message store and > >> >> > volatile configuration store, > yes, > >> >> > you will lose messages. This is expected. The recovery phase > >> >> > of startup will clear the message store of the orphans. > >> >> > > >> >> > What problem are you actually trying to solve? > >> >> > > >> >> > Keith. > >> >> > > >> >> > > >> >> > > >> >> > On 22 June 2018 at 08:54, VERMEULEN Olivier > >> >> > <olivier.vermeu...@murex.com> > >> >> > wrote: > >> >> > > Hello, > >> >> > > > >> >> > > I was running some tests on a Java Broker with 'Memory' > >> >> > > configuration > >> >> > store and 'DERBY' message store. > >> >> > > Is there a way with this configuration to recover the > >> >> > > messages stored in > >> >> > DB upon a broker restart? > >> >> > > Because it looks like the messages are mapped to the queue > >> >> > > UUID and that > >> >> > they are discarded directly during the broker startup if the > >> >> > queue with the specified UUID is not found. > >> >> > > And obviously since I'm using a 'Memory' configuration > >> >> > > store, the queue > >> >> > has not been recreated yet when the broker starts... > >> >> > > Have I missed something? > >> >> > > > >> >> > > Thanks, > >> >> > > Olivier > >> >> > > ******************************* > >> >> > > > >> >> > > This e-mail contains information for the intended recipient only. > >> >> > > It may > >> >> > contain proprietary material or confidential information. If > >> >> > you > are > >> >> > not the intended recipient you are not authorised to > >> >> > distribute, copy or use this e-mail or any attachment to it. > >> >> > Murex cannot guarantee that it is virus free and accepts no > >> >> > responsibility for any loss or damage arising from its use. If > >> >> > you have received this e-mail in error please notify > >> >> > immediately the sender and delete the original email received, > >> >> > any attachments and all copies from your > >> system. > >> >> > > >> >> > > -------------------------------------------------------------------- > >> >> > - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > >> >> > For additional commands, e-mail: users-h...@qpid.apache.org > >> >> > > >> >> > ******************************* > >> >> > > >> >> > This e-mail contains information for the intended recipient only. > It > >> >> > may contain proprietary material or confidential information. > >> >> > If > you > >> >> > are not the intended recipient you are not authorised to > distribute, > >> >> > copy or use this e-mail or any attachment to it. Murex cannot > >> >> > guarantee that it is virus free and accepts no responsibility > >> >> > for any loss or damage arising from its use. If you have > >> >> > received this e-mail in error please notify immediately the > >> >> > sender and delete the original email received, any attachments > >> >> > and all copies from your > >> system. > >> >> > > >> >> > > -------------------------------------------------------------------- > >> >> > - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > >> >> > For additional commands, e-mail: users-h...@qpid.apache.org > >> >> > > >> >> > > >> >> ******************************* > >> >> > >> >> This e-mail contains information for the intended recipient > >> >> only. It may contain proprietary material or confidential > >> >> information. If you are not the intended recipient you are not > >> >> authorised to distribute, copy or use this e-mail or any > >> >> attachment to it. Murex cannot guarantee that it is virus free > >> >> and accepts no responsibility for any loss or damage arising > >> >> from its use. If you have received this e-mail in error please > >> >> notify immediately the sender and delete the original email received, > >> >> any attachments and all copies from your system. > >> >> > >> > ******************************* > >> > > >> > This e-mail contains information for the intended recipient only. > >> > It > may > >> contain proprietary material or confidential information. If you > >> are not the intended recipient you are not authorised to > >> distribute, copy or use this e-mail or any attachment to it. Murex > >> cannot guarantee that it is virus free and accepts no > >> responsibility for any loss or damage arising from its use. If you > >> have received this e-mail in error please notify immediately the > >> sender and delete the original email received, any attachments and all > >> copies from your system. > >> > >> ------------------------------------------------------------------- > >> -- To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For > >> additional commands, e-mail: users-h...@qpid.apache.org > >> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For > additional commands, e-mail: users-h...@qpid.apache.org > > ******************************* This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
