Wow. Thanks for the huge amount of detail! You are correct: I am using the C++ broker. As far as language for my tool, I'm pretty agnostic; I want a tool that suits my needs and I've got the resources to do it in any viable language (though I am partial to C++).
I'm going to follow up on one paragraph in particular: "To more directly answer your question "However, I’m having a hard time making that same mapping between the connection stuff and a sender to an exchange. " yeah there's some nuance there :-) the main thing is to think in terms of UML associations, so for example a binding object has queueRef and exchangeRef properties which hold ObjectIds that enable the associated queue and exchange to be indexed (binding behaves like a UML association class really) what this means though if you want to work out how a queue links to an exchange you have to do it from the perspective of the binding." My problem is that my publishers establish a 'send only' connection, so they link to the exchange directly (without a queue). To put it another way, if I have n publishers sending to exchange 'myExchange', when I run 'qpid-stat -q' I don't see n queues bound to myExchange. So, without a queueRef, I have no binding object for those publishers, and no way to get, for example, the remote PID. Am I missing something? Thanks again, -Lance On Fri, Jan 11, 2013 at 5:44 AM, Fraser Adams <[email protected] > wrote: > Howdy Lance, > > You mention qpid-tool which uses QMF so I'm assuming that you are talking > about the C++ broker. The Java broker uses JMX and I'm not so familiar with > how that does instrumentation so the following is only really applicable to > the C++ broker. > > > Unfortunately QMF can be a bit non-trivial, it's not helped by the fact > that there are two variants - QMF1 which is a binary protocol and > accompanying API and QMF2 which is built on top of Map messages. > > The python tools bundled with qpid used to use QMF1, though I *think* that > as of qpid 0.18 they've started to use QMF2 (trying 0.18 qpid-config on a > 0.8 broker gets a method failed error which suggests it's using QMF2 create > method - though I've not actually looked at the 0.18 source). > > If you are writing your tools in python there's a whole lot more stuff > available for QMF. > > If you are writing in Java a year back I wrote a complete Java > implementation of the QMF2 API: > https://cwiki.apache.org/qpid/**qmfv2-api-proposal.html<https://cwiki.apache.org/qpid/qmfv2-api-proposal.html> > > based on the protocol: > https://cwiki.apache.org/qpid/**qmf-map-message-protocol.html<https://cwiki.apache.org/qpid/qmf-map-message-protocol.html> > > The link to my code is here: > https://issues.apache.org/**jira/browse/QPID-3675<https://issues.apache.org/jira/browse/QPID-3675> > > > Slightly frustratingly there hasn't been much feedback on this - I'd > really like to see it make it into the main code base - I put a huge amount > of effort into it. The code in that Jira link also contains tests/tools > that illustrate how to use QMF2 in Java in some non-trivial scenarios. > > > As it happens in a few days time I'm about to release a major uplift to > that, which will include a QMF2 REST API and most significantly a complete > HTML5 based web UI that will enable inspection of all QMF properties and > also queue/exchange/binding adding/deletion so that may be just the sort of > thing you're interested in - it's pretty close, I'm just doing some last > minute de-snagging trying to get it looking nice on small real-estate > mobile browsers. > > > To more directly answer your question "However, I’m having a hard time > making that same mapping between the connection stuff and a sender to an > exchange. " yeah there's some nuance there :-) the main thing is to think > in terms of UML associations, so for example a binding object has queueRef > and exchangeRef properties which hold ObjectIds that enable the associated > queue and exchange to be indexed (binding behaves like a UML association > class really) what this means though if you want to work out how a queue > links to an exchange you have to do it from the perspective of the binding. > > Similarly it's quite involved to find the subscriptions associated with a > queue because the navigation goes "the wrong way" that is to say > subscription has a queueRef property. > > What I've found is that in general the best (and by far in a way most > efficient) way to do non-trivial QMF stuff is to to the getObject() stuff > once for each of the objects that you care about then dereference those > objects into Maps indexed by objectId - if you do that then you can index > everything you want easily and cheaply. > > If you look at the code for things like qpid-config it's pretty easy to > get into the trap of doing getObjects() calls on inner loops, which is > heinously inefficient :-) because each getObjects() call does a messaging > request/response under the hood. > > To get you up and running and give a bit of insight (I hope!!) I've > attached a couple of python programs I wrote last year (these use QMF1) > > connection-audit checks when connections are made and looks up the queues > being subscribed to against a whitelist, basically if the queue being > subscribed to is not a queue that's in the whitelist it generates a log > message. > > connection-audit-dom is the same but uses DOM based XML parsing (I needed > to do that when I had to use it against an oldish python version) > > connection-logger does a lot of what I think you care about, basically it > logs all connections to a broker and provides useful info on them a la > qpid-config -b queues. > > > connection-logger-orig was my original version of this which was somewhat > erm "rushed out" :-) at face value it looks OK for a few queues etc. but if > you ramp up a load of connections - well try and add a couple of hundred > queues and you'll see what I meant by heinously inefficient above :-D. > > connection-logger and connection-logger-orig are functionally equivalent > so it's worth looking through both to give a bit of insight into the right > way and wrong way to go about things. > > I'm not a python programmer by any stretch of the imagination, but I think > that they are at least fair examples. > > Keep an eye out for my QMF2/UI update over the next few days but hopefully > this response has given you a decent leg-up. > > Best regards, > Frase > > > > > On 11/01/13 02:24, Lance D. wrote: > >> Hello all. >> >> >> I’m in the process of building QPID inspection tools for a project I’m >> working on. Now, I’m looking at the results of the qpid-tool. I see that >> the connection schema has a bunch of useful info (like remote pid, >> process, >> etc). I’ve also found ways to link that info up with the queues to figure >> out who’s subscribed to what. However, I’m having a hard time making that >> same mapping between the connection stuff and a sender to an exchange. >> >> >> >> My question is, is there a way to do that mapping (either with the >> existing >> tools, or by tapping into the qmf API? >> >> >> >> Thanks, >> -Lance >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] >
