On 5 Sep 2006, at 21:18, Fateev, Maxim wrote:
Messages are always put into storage - but they are additionally
pushed to subscribers for dispatching. I was going to implement a
mixed approach - where consumers that can keep up with the
producer will not be impeded by waiting for a poll, but slower/non-
active consumers would ultimately rely on a poll from the store.
My view is that this "pushing" functionality should be hidden
behind MessageStore API. CachingMessageStore could be created that
would keep messages for fast consumers in memory delegating other
requests to another permanent MessageStore.
This already happens - see the Journal Persistence Adaptor.
Having two separate dispatching code paths complicates code.
yes it does - but it's done for performance - polling is slow.
Another problem with direct dispatch is that in case of synchronous
dispatch producer performance directly affected by performance of
consumer.
The performance of the consumer is currently taken into account
whilst dispatching - in the Subscription classes - and the reality is
that the producer doesn't always do the dispatching, the thread
associated with the consumer does too - if it is slow.
I don't think that it is really a good idea to have receiving
socket thread perform write to a consumer socket directly.
This is done for performance - there are defensive mechanisms built
in to the transport layers to prevent blocked clients locking the
broker - but at the end of the day this is optional - the consumer
can decide if messages are dispatched asynchronously from within the
broker.
Yes - am changing - so the PendingMessageCursor can in effect
ignore new messages from a producer if it is full - or the usage
manager has reached it's limit - and poll for them later.
It is an example of two code paths. Only polling part would be
required if you eliminate pushing part.
But the two code paths happens currently in PrefetchSubscription
3. getNextMessage(...) should block if there are no messages in a
queue. Otherwise it would require busy polling.
depends on the implementation - more messages available can be
triggered from producer etc - providing the PendingMessageCursor
is intialized with state and maintains it.
Again blocking getNextMessage() would be much more simple if
pushing part is eliminated.
I think you've made the assumption that polling is best ?
4. I'm not sure about semantic of getNextMessage(...,
RecoveryListener) API. Will it create a thread per request? Will
single blocked listener block the whole queue? I think "List
getNextMessage(..., int count)" would have cleaner semantic.
I think a Listener allows for more flexibility - in allowing the
PendingMessageCursor to stop a request if the usage memory is at
its limit for example
I'm not sure why it has more flexibility. In case of blocking
get... call you have the same flexibility but more simple
threading. For example you just don't call getNextMessage if the
usage memory is at its limit. RecoveryListener would require
additional thread for every consumer which doesn't scale.
I don't see why additional threads are required ?
BTW do you plan to have RecoveryListener per queue consumer or
single one for the whole queue?
Difficult call - I'm trying not to change the existing code too much
- as I know there's been alot of bug fixes in the
PrefetchSubscription/DurableTopicSubscriber etc. code, for timing
issues, transactions etc - I'm >>tempted not to loose that if
possible :)
You can create different destination implementation that still uses
existing Subscription framework but relies on new MessageStore API.
I have an example of such code on my desktop.
The comment at the bottom:)
We have tried an implementation of the message store using polling
before but the reason we prefer a hybrid approach (which is what we
have in essence in ActiveMQ at the moment) - is performance - we get
dramatically better performance for push based messaging (the general
case) and pulling messages for slower consumers. I think the main
difference between the implementation your working on and the current
architecture is that currently we can optionally decide to push from
the producer socket directly to consumers - which is about as fast
as it gets for broker message delivery - and we've taken on the
additional complexity to keep this as an option.
But given the flexibility behind the architecture of ActiveMQ - there
is nothing to stop having more than one Destination implementation -
now that we've incorporated your patch for the DestinationFactory.
cheers,
Rob
