James, Sorry for taking so long to get back to you.
On Fri, 2004-09-10 at 16:05, [EMAIL PROTECTED] wrote: > Hey Alex > > > From: Alex Karasulu <[EMAIL PROTECTED]> > > Also we have begun to separate out frontend plumbing from LDAP > specific code in Eve's frontend. This code is a TCP specific SEDA > framework. > > We're going to make this into a subproject on its own so we can reuse > it for a Eve integrated Kerberos server without tight coupling (it can > also be > > standalone). Trustin Lee (guy who did Netty) will be looking into > adding UDP support to this in project. > > > > So what does this all mean for integration? Well we want to write > wrappers for this SEDA framework where Eve just plugs in as another > simple > > protocol to be able to benefit from those wrappers. By writing a > geronimo wrapper for SEDA rather than an Eve specific frontend we now > allow any > > protocol to be fired up within geronimo without having to > specifically integrate that protocols frontend. WDYT? > > > > So we really want to write a SEDA framework wrapper intead of one > specifically for Eve. This also makes me think if its worth writting > wrappers for the > backend of Eve and this raises several questions I > still don't have answers to but these I'll leave for other emails. > > I guess it depends what you mean by SEDA. Mostly I tend to think of the > Channel abstraction in the concurrent.jar as being a pretty good > abstraction for SEDA, where the channels can be local in VM or remote > and you can put/take/poll objects - then the rest is just That's cool stuff for doing scatter gather like operations where a rendezvous point is needed. Looking back on Matt Welsh's paper on SEDA I can see how this is just a queue within a stage. > implementation details. Or some folks like to abstract away the low > level SEDA like protocol of sync/async send behind a dynamic proxy to > make SEDA invisible as it were. e.g. > > http://activemq.codehaus.org/maven/apidocs/org/codehaus/activemq/util/ > AsyncProxy.html This is a bit of a stretch but it does capture some of the sync/async points. > If what you mean is more of a SEDA based transport framework, well > we've been working for quite some time on one of those on the ActiveMQ > project :). Last benchmark we did was 22,000 messages/second sustained > throughput on 2 2-way linux boxes with producer, broker, consumer all > going across the network for 1-2K message sizes and reasonably low CPU > usage (YMMV). That's pretty impressive. I definately will have a look through the code there and most likely will be using ActiveMQ for the messaging abstraction in LDAP replication ;). > Currently we've transports for > > * VM (in memory both sync & true SEDA) > * TCP / SSL > * UDP & multicat > * NIO through g-networks and EmberIO > * JGroups > * JRMS > * JXTA > * HTTP (for tunnelling) > * Jabber (experimental) > * AIO4J (experimental) That's really cool. I especially need to take a look at the UDP code. <snip/> > Details of the code are here... > > http://activemq.codehaus.org/Code+Overview > Ahhh this TransportChannel abstraction is exactly what Trustin and I have been talking about using for the SEDA stuff here. Looks like we can borrow from what you've already done. See right now we only do TCP and have not needed the abstraction. Now we do. Seeing you using this makes us feel a little better about the common path we've selected. > Basically we have a really simple TransportChannel API which can > represent any kind of transport protocol. > > http://activemq.codehaus.org/maven/apidocs/org/codehaus/activemq/ > transport/package-summary.html This is excellent! > The minimal contract we support for application protocols is that they > must decide if a method call is sync (requiring a receipt such as a > commit()) or async (fire & forget) and for async, whether a flush is > required after the send. Consuming is pluggable based on the transport > - so it decides if there is a thread, is it pull/push or how threads > multiplex such as for NIO / AIO. Right following ya! We've actually put in very similar hooks into the stages for handling encoding and decoding. There are two ways to process IO as it is delivered sync or asynch. This is reflected in the interfaces use on these stages. This is very similar to the send() and asyncSend() methods you have. > Most of the transport channels can be reused as is with a pluggable > WireFormat which dictates how command objects (we use the Packet base > class) get serialized on the wire. e.g. we have a default, fast as > possible, wire format for the JMS Packets, as well as Jabber & XStream > based ones etc. > > The TransportChannel abstract represents about the 8th or 9th > generation of this kind of abstraction the team have used over the last > 5 years or so for implementing high performance messaging & SEDA > transport abstractions - so if nothing else, I'd advise you to go down > a similar route of abstraction; then at the very least you'll be able > to reuse our transports easily & we can easily plugin into your stuff - > especially if you want to take advantage of our clustering, failover & > replication protocols.. Thanks for taking the time to give us some excellent advice - we're going to make use of it for sure. This is most likely the direction we're going to take RE: abstraction. If we can find a way to use these transports then that does save us a lot of hassle as well. Alex
