On 25 March 2013 19:57, Fraser Adams <fraser.ad...@blueyonder.co.uk> wrote:
> Hey Robbie, > Thanks for the comprehensive response - all useful stuff, always on the > bleeding edge me :-) > as in "it's not bleeding working again" :-D > > It's fun to know "I think you are the first to ever ask about a change ". > I guess that I had assumed that a plugin architecture was intended to be > used for erm plugins, I guess that I slightly wryly noted too that it had > only arrived in 0.20 so "imagine my surprise" when it changed again so > quickly :'( > Sure, its plugable to allow plugins, its just that as mentioned almost noone besides us has ever used the [undocumented] extension points, and certainly not the management one as it didnt exist until relatively recently. The contiuing change is an unfortunate side effect of the fact we actually had no intention of changing that stuff in 0.20 originally, and have basically hit the time box available for 0.22. It was our original intention to make redoing the plugins one of the last things we did, but we ended up changing our plans upon realising the true extent of the issues we would face trying to do the rest of the work around it. It is always fun spending literally hours tracing through the behaviour of a giant chunk of code to ensure you understand it and dont break it when refactoring it, only to arrive at the conclusion that the specific bit you were looking at has been effectively dead code since the point it was added. Funnily enough, thats the day we decided to remove the existing plugin stuff. > > RE "Its a shame you werent tracking trunk as it may be too late to fix > some issues for the next release " yeah I've hitherto mainly been working > against official releases and I've just started tracking truck, as you > suggest I'd clearly have picked this up sooner had I been. It's no bigee > really as the QMF Management Plugin is built as a separate target. It was > an initial release 'cause a few people had been asking about it and being > able to control the Java broker from a CLI such as qpid-config. > > I've taken a look through some of the new plugin code. I did wonder when > this all changed if it was related to the config work you've been > mentioning elsewhere. > > I had a look at the new config.json file that gets put into the working > directory - where does the default one get generated from? I did a quick > grep but couldn't see what wrote out config.json if one doesn't already > exist. > > The broker now has a configuration store (which is pluggable, selectable via either the command line or broker options when embedding..), and has an 'initial-config' file embedded in its jar (currently called initial-store.json) which gets loaded in memory as an 'initial store' containign the base configuration. That is then is then used by whichever actual configuration store is configured, which in this case is a JSON store that uses the initial store to write out the config file you are seeing since none existed at the (configurable) location it is meant to store. On subsequent restarts, the Json store will be loaded from the newly written file and the 'initial store' data is ignored. > > Re "but I think you might find the changes are far more substantial than > just the plugin interface itself and those may pose bigger problems " what > sort of things are you alluding to here? My QMF plugin doesn't do anything > especially clever and you've clearly modded the JMX and HTTP plugins to the > new API. The bulk of the code beyond the "plugin" stuff just uses the > org.apache.qpid.server.model classes which don't (at a superficial glance) > appear to have changed radically. > > I guess I'll find out when I look further. > > Thanks again for the update. > I don't have anything specific in mind, I just noticed you were reaching into the internals a bit and so might find we have removed some stuff that you were using. I haven't had a chance to look properly yet (going to be later in the week I'm afraid) so it may indeed be the case theres nothing of real issuethats change; as you say, we adapted the HTTP and JMX. > > One further question RE "remove the old XML configuration files, and make > the broker pretty much entirely configurable through the management > interfaces" what's the motivation behind this? Funnily enough one of the > things that I thought the Java broker had better than the C++ broker was > the ability to specify configuration a priori. In my experience there are > plenty of cases where one actually wants to provide a broker configuration > that has been configuration managed. We certainly do in my set up and it > rather irks me to have to essentially script qpid-config to configure our > brokers to bootstrap the system. Don't get me wrong the management > interfaces are great but the ability to have a configured config also seems > useful. So, as my comments above may have already shed some more light, I hope its becoming clear I was very much summarising things in my earlier comments, and only meant that you could configure everything through management, not that you must. If you want to you will very much still be able to pre-configure the broker in full, in a variety of ways. As mentioned, you can supply an 'initial config' location when first starting the broker that will be used as the basis of the configuration store that it goes on to use. You could thus use the Memory config store along with this to effectively revert to having a static config file that isnt altered by changes via the managemnt interface. You could also place a file directly at the configured/expected [output] location of the new JSON config store in advance, such that the broker simply starts up using the config you already defined. As Rob aluded to, we want to end up where you can extactconfig from a broker and replay it in again later. For example, one of the reasons we started with a JSON config store is that its human readable, and if you look you'll find its contents somewhat similar to what you get straight out of the REST interface. Finally, I should clear up that we really have no issue with config files, or even XML, just specifically our old configuration model and horrific use of XML :) One thing that bugs me with the C++ broker is that I can't make a queue's > configuration persistent without making the queue persistent (if you see > what I mean) what I actually want is for the queue to be re-created > automatically when the broker restarts but for messages to not be persisted > (I can't easily change the producer to mark the messages non-persistent). > I'm forced to use scripts to do this which is irksome. As I say the Java > broker supported doing this via the XML config, but it seems like you've > changed this - it seemed like a valid use-case so I'm interested to know > what the driver was. > The XML configuration files are still supported for VirtualHosts if desired, so that functionality is still there if needed. What we have done for 0.22 is make it optional to have the XML config for the virtualhost, but you actually still need to use it if you want to do any virtualhost-level configuration (i.e. not inherit certain values from the broker level, or perform any of the more elaborate mesage store configruation). In future we want to re-do the virtualhost internals similar to what we have done for the broker, and end up storing all the configuration data in its store to allow us to get rid of the virtualhosts XML configuration, but its now likely to be a good bit further down the line as other things demand some/lots of attention. Before that point comes, one of the things we have indeed discussed is having the ability to define queues which always exist but never retain persistent messages across a restart. > Frase > > > > On 25/03/13 13:53, Robbie Gemmell wrote: > >> Hi Fraser, >> >> What you have now noticed is a step in our ongoing effort to reorganise >> the >> broker internals, remove the old XML configuration files, and make the >> broker pretty much entirely configurable through the management >> interfaces. >> >> The particular plugable management interface you found and used with 0.20 >> was actually only introduced in 0.20, when we simplified the plugins from >> prior versions of the broker (which used a somewhat contorted plugin >> system >> via the XML and an embedded OSGi container). That change was a first step >> towards letting us work on removing the broker-level XML configuration >> (and >> ServerConfiguration along with it, being at the heart of that stuff). The >> core of that landed on trunk over a month ago, having been in progress on >> a >> branch since the end of October. >> >> Although the larger idea of the changes have been mentioned in general >> several times over the last year or more, I dont think we actually did >> specifically call out the difference in the pluggability, primarily >> because >> at the time the change was made (either months ago when it was, or more >> recently when it was merged) no other such plugins actually existed and >> weren't really expected to. >> >> I'm afraid to say the answer to your question around stability is no, at >> least as far as the particular bit you are probably asking about is >> concerned, i.e the way the jmx and http management plugins are currently >> loading. That happens to be something that was done as a means to make it >> work with the new configuration, but exactly how it does is a source of >> annoyance and rather likely to change again. >> >> We can possibly look to improve things for 0.22 if there is somethign >> simple that would help, but I think you might find the changes are far >> more >> substantial than just the plugin interface itself and those may pose >> bigger >> problems. Its a shame you werent tracking trunk as it may be too late to >> fix some issues for the next release. I havent managed to take a proper >> look at what you did, though a quick skim does leave me with a few >> questions so I'll try to do so. >> >> I guess the take away point is that the pluggability has generally been >> there as much as an after thought than anything else, and maintaining >> compatibility between reelases has not been a priority as we have no >> plugins that target multiple releases and so few people (besides us, I >> think I recall 3 people including you mentioning it over the years) have >> ever used the pluggability we didn't really expect it to be an issue yet >> (and I think you are the first to ever ask about a change). I see you >> noted >> in your readme the pluggable points have never been documented; thats true >> and our intention with these ongoing changes is very much to document such >> things, once they become stable. >> >> Robbie >> >> On 24 March 2013 16:53, Fraser Adams <fraser.ad...@blueyonder.co.uk**> >> wrote: >> >> I put together a QmfManagementPlugin for the Java Broker using the plugin >>> API >>> >>> org.apache.qpid.server.plugin.****ManagementFactory >>> org.apache.qpid.server.****management.plugin.****ManagementPlugin >>> >>> >>> This works nicely for Qpid 0.20 but I've just tried to build against >>> trunk >>> and those interfaces have gone :'( >>> >>> Looks like org.apache.qpid.server.****configuration.**** >>> ServerConfiguration >>> >>> has gone too. >>> >>> >>> Looks like there's a new plugin API, what gives?? It's a little bit >>> frustrating - was this change mentioned anywhere (Jira?) I must have >>> missed >>> it. >>> >>> I've not had much of a change to look at the new plugin API, is this >>> likely to be vaguely stable? >>> >>> Any info gratefully received. >>> >>> Cheers, >>> Frase >>> >>> ------------------------------****----------------------------** >>> --**--------- >>> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.****org< >>> dev-unsubscribe@qpid.**apache.org <dev-unsubscr...@qpid.apache.org>> >>> >>> For additional commands, e-mail: dev-h...@qpid.apache.org >>> >>> >>> > > ------------------------------**------------------------------**--------- > To unsubscribe, e-mail: > dev-unsubscribe@qpid.apache.**org<dev-unsubscr...@qpid.apache.org> > For additional commands, e-mail: dev-h...@qpid.apache.org > >
