Hi, messaging is indeed one of the crucial features of a IT Platform. I once worked for a Swiss Bank that maintained over 30k of different Queues... :-)
I guess we are talking here about a generic internal API. There is already a number of messaging APIs that we can take as a basis. You mentioned already few of them. The issue is always the level of abstraction - high abstraction level leads to broad support of messaging solutions, but is not easy capable to support special feature X. Be we can use here a similar approach of extensions, like we did e.g. for the spatial support. What about to use Apache Camel as a "glue" between Zest messaging API and the concrete messaging implementation ? .. my two cents .. Cheers, jj 2015-06-17 5:46 GMT+02:00 Niclas Hedhman <[email protected]>: > Gang, > I have been thinking a bit about a "Messaging" abstraction, similar to the > "Storage" and "Indexing" abstractions, and looked at my own implementations > how they are done, and see if we can come up with an Extension mechanism > that is suitable for a whole range of Messaging Extensions, such as > > * Restlet > * Jersey > * JMS(?) > * Kafka > * Storm (?) [1] > > There are a few initial "requirements" that I have written down on my white > board; > > * UnitOfWork-like - i.e. any "result messages" are not sent until > everything is completed successfully. > * "Destination" concept - the entry point for event request processing > * Hierarchical delegation - think "paths", "routing", "intercepts" > * Request Parameters - think "headers" in HTTP or other metadata > * Request Payload > * Payload objects - converted into ValueComposites through mapping > * Binary payloads - byte[] blob > * Message Receiver is created per request. Factory implementation from > user. > * Optional Response Messages - probably part of a return type. > * Usecase metadata for Request Parameters, possibly lazy deserialization. > * Message Destination injection, for outgoing messages. Possibly a new > Composite meta type. > * Visibility - some messaging may be internal to the application, using > the same abstraction. So Visibility should be used as usual. > * Capable of distinction between Load Balanced Queues, Pub/Sub and > Peer2Peer Queues. > > In very, very broad strokes, we are looking at something like this (Kafka > used as an example); > > > ___________ +----------+ ___________ > O partition O--| Kafka ME |--O partition O > ----------- +----------+ ----------- > | ^ > v | > +--------------------+ +------------+ > | Qi4j Messaging SPI | <--serves--| MC Factory | > +--------------------+ +------------+ > | ^ | > v | | > +----------------------+ | > | MessageComposite (MC)|<--instantiates--+ > +----------------------+ > > > This picture doesn't involve the many options required to be handled, but > should serve as a mental model to help understand where I am coming from. > > Before I start looking at how this could be done, does anyone have any > input on the topic (pun intended, as Topics are part of the problem domain > somehow, but I haven't worked that out yet) > > > > [1] Getting an abstraction right, which allows Zest/Qi4j applications to be > part of the Storm topology would be awesome. > -- > Niclas Hedhman, Software Developer > http://zest.apache.org - New Energy for Java >
