On 11/04/13 13:50, Bill Freeman wrote:
I can see three cases in which a console would want to receive both
QMF1 and QMF2:
1. Testing of QMF itself. But there isn't much that couldn't be
tested using two separate console processes (arrival order, maybe?),
so this doesn't seem to be an important case;
possibly, but given that until Ken's update no Console would have
received V2, I'm not convinced.
2. If each version offered information that the other didn't. I
would consider this to be a case of QMF2 being incomplete (QMF1 is
legacy and need not grow features added by QMF1), in which case the
correct fix is probably to add the missing features to QMF2, unless
the data is something that wants to be deprecated;
I'm pretty sure that in terms of information from the actual management
objects QMF1 and QMF2 return the same information if not in the same
format (different callback approaches as previously discussed)
3. There is some reason to connect to multiple brokers directly, and
some of them are not V2 capable. You could, of course, talk to them
all with V1, unless you were counting on federation among V2 brokers
to allow you to reach some that would otherwise require you to have an
explicitly set up tunnel in order to connect to them.
Is that actually possible? To do that I'd have thought that a Console
would need to have multiple Connections, I thought that the Connection
was created when the Console got constructed using the options passed. I
*think* a more likely use case might be where multiple Agents are
registered with some being QMF1 and some QMF2. So say for example you
configure the broker Management Agent to only send in QMF2, but you have
a bespoke Agent built using QMF1. If you are only monitoring the broker
Management Agent though that won't hold.
The nice thing about the separate callback for V2 updates is that no
existing code will break because of receiving unexpected duplicate
(sort of) updates, since existing code still won't see the V2 updates.
But the network traffic point is well taken. I do prefer filtering to
be closer to the source.
I wasn't trying to suggest that having a separate callback wasn't a good
idea, I think it's the only really good idea, I was simply suggesting
that in itself it was sub-optimal and that in addition it might be an
idea to consider additional API calls for explicitly selecting QMF2 or
both so that the appropriate subscriptions could be created to minimise
data duplication.
I suspect that subsequent updates contain any properties WHICH HAVE
CHANGED.
Not sure what you mean here, it might just be language, the "properties"
as being described here are the largely static things and the statistics
are the things that tend to change (if you look in management-schema.xml
it illustrates what fields are what).
So subsequent QMF2 updates I believe contain all of the properties but
only the specific statistics that have changed since the last update.
I'm fairly sure about that because I'm able to do a query subscription
on things like queue name.
But since properties don't tend to change, updates after the initial
*everything* update don't carry any property *updates*. This would
make it hard to filter based on properties. One doesn't want to carry
out such filtering at the agent, because then everyone is stuck with
the same filtering.
I was talking about what is currently actually happening not what
*could* be made to happen, personally I'd simply push the entire object,
it makes application code a whole lot simpler. Sure there's some
bandwidth overhead but from what I've seen the instances where you tend
to see the most objects (Connections and Queues) if one statistic has
changed actually most of them have tended to so you don't actually save
as much as you might think by trying to optimise to only send the
changed statistic (e.g. sending a statistics "diff" plus properties is
what currently happens).
A possible approach is that, in addition to carrying changed
properties and statistics, QMF2 messages would carry a decoration
containing all of the properties, available for use by the
subscription controls, but which is stripped by the time the message
hits the wire. But that really moves away from the QMF2 messages
aren't special concept. It's hard to see how the overhead of checking
for the decoration for stripping could be avoided for bread and butter
non QMF messages.
As I say what currently happens is that in QMF2 all the (static)
properties are sent each time, but only the updated statistics get sent
- it's been a while since I checked for sure, but that certainly rings a
bell 'cause it seemed a bit weird at the time. And as I also said I
remain to be convinced that "optimisations" in terms of trying to be
clever with the statistics being sent are massively worthwhile.
Another possibility is to limit subscription filtering to being based
upon that which is carried. Apparently there is enough that the
console can figure out the object ID (perhaps it is carried itself),
from which you can parse out the object name at need (or is this only
true for queues?). Hopefully, if QMF messages can be delivered across
a federation, there is enough to identify the object's broker.
So my Java QMF2 implementation can do much more sophisticated filtering
than that, it can do boolean combinations of properties, regexes and all
sorts. In theory say I could write a query subscription to only pass say
a specific named queue whose msgDepth had exceeded a particular value
(that's only an example and I'd probably use an event for that but the
point is you can do neat things but the Management Agent only passing
back changed statistics limits some of the filtering choices)
Oh for sure I could probably update my Java QMF2 query subscription code
to retain more state before the filter gets applied, but I had assumed
that the documented QMF2 API would gain traction and that query
subscriptions would be properly implemented in the Management Agent
(doing it client side in the Console implementation is purely a
work-around) so I put it on the backburner as I've already implemented
way more of that API in the Java version than anyone else has and as I
tend to use getObjects() mostly I doubt any one else is using the really
clever bits in anger. It's a shame really as the query subscription idea
in QMF2 is quite neat and powerful.
Given some of the discussions going on on the subject of AMQP 1.0
Management I'm more inclined to invest my time trying to figure out how
to migrate to that with the minimum of disruption and to try and align
the C++ and Java Brokers a little more.
None of the previous couple of paragraphs are relevant to the current
python stuff in case that wasn't totally clear, unfortunately QMF2 the
protocol and QMF2 the (miscellaneous set of) APIs can lead to a world of
confusion, frankly it's less than ideal :-(
Frase
