Saminda Abeyruwan wrote: > May be JMS in outbound messages? i think any queue based system will do for async (including home grown/optimized) but the key issue is how to scale it when massive number of sync inbound (or outbound) connections need to be supported: for each sync HTTP request-response a socket needs to be kept open to get response and typical OS limits are about 1000 and with lot of work 10000 - this problem is related to how to maintain that many threads (a thread for each socket/connection) and it is called C10k - some background on this and related issues: http://www.kegel.com/c10k.html
(...) It's time for web servers to handle ten thousand clients simultaneously, don't you think? After all, the web is a big place now. And computers are big, too. You can buy a 1000MHz machine with 2 gigabytes of RAM and an 1000Mbit/sec Ethernet card for $1200 or so. Let's see - at 20000 clients, that's 50KHz, 100Kbytes, and 50Kbits/sec per client. It shouldn't take any more horsepower than that to take four kilobytes from the disk and send them to the network once a second for each of twenty thousand clients. (That works out to $0.08 per client, by the way. Those $100/client licensing fees some operating systems charge are starting to look a little heavy!) So hardware is no longer the bottleneck. In 1999 one of the busiest ftp sites, cdrom.com, actually handled 10000 clients simultaneously through a Gigabit Ethernet pipe. As of 2001, that same speed is now being offered by several ISPs, who expect it to become increasingly popular with large business customers. And the thin client model of computing appears to be coming back in style -- this time with the server out on the Internet, serving thousands of clients. With that in mind, here are a few notes on how to configure operating systems and write code to support thousands of clients. (...) and in particular this is the only test that i know that has some useful results (though a bit outdated): http://www.volano.com/report/ (...) At 10,000 connections, BEA JRockit 3.1 on Windows has 14 times the throughput of any other Java platform - by far, the best network scalability I have ever tested. While other Java vendors waited for better threading support in the operating system or new programming interfaces for the application, the JRockit team solved the Java threads problem right where it originated. The results are remarkable, and BEA made a wise purchase. BEA JRockit 3.1 is no longer available, so the best Java platform today is Blackdown 1.3.1 on Linux. The Blackdown 1.3.1 port scales easily to 10,000 connections, keeping pace with the latest from BEA even without a just-in-time compiler. (...) The good news is that Dan Kegel's C10K Problem has been solved in Java 1.3, even when using the original blocking Java input and output methods, and even when using modest hardware. The bad news is that the Java vendors seem to have abandoned that achievement in Java 1.4. Blackdown 1.4.1 no longer offers the -green Classic VM option that allows so many connections with the 1.3.1 release. The BEA High Performance Threading System (Thin Threads) is no longer so highly performing in JRockit versions 7.0 and 8.1 compared to the earlier 3.1 release. (...) and in general on scaling threading: http://java.sun.com/docs/hotspot/threads/threads.html best, alek > > > Saminda > > On 5/12/06, *Aleksander Slominski* <[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> wrote: > > Paul Fremantle wrote: > > Alek > > > > I agree with you. If we use non-blocking IO, it is possible to > handle > > the sync to async mapping without blocking a thread. We need to walk > > thru the model and see how it works with Synapse and Axis2. Its > > essential that if we implement this we don't block threads or it > won't > > be a useful system. > > > > As regards surviving restart of the Synapse server, I'm not sure > I see > > that as a requirement. This could be handled at a higher level by > > WS-RM or TX. > yes if all services are RM (or TX) enabled but i though one of use > cases was to have synapse provide RM capability to services that > are not > RM enabled? > > in such case for a service with sync interfaces (request-reply on the > same connection) restart of synapse means that they will loose its > outbound socket connection and will not be able to send the > response to > synapse ... now if synapse (or external sync/async service helper) > knows > that it is OK to call the service again then it may just do it > essentially "restarting" execution that may be part of longer > conversation (such as workflow). > > of course service must be coded to detect duplicate requests (for > example by checking wsa:MessageID) and in such case return the result > that was stored when outbound connection was broken. this is not > perfect > but may be the simplest way to add some reliability to a system > that has > plenty of existing sync services (like RPC). > > best, > > alek > > Paul > > > > On 5/12/06, * Aleksander Slominski* <[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>> wrote: > > > > Jaliya Ekanayake wrote: > > > Hi All, > > > > > > IMHO it is not desirable to map Sync-Async scenarios. I > can give two > > > reasons. > > > > > > 1. There should be a timeout issue and we will not have > guaranteed > > > mapping always. > > > 2. Mapping sync to two async will cause the Synapse to > hold one > > thread > > > (to keep the sync socket open) which will exhaust the > server easily. > > those issues are very hard to overcome in general unless non > > blocking IO > > is used for each sync invocation (including anon > request-reply WSA) > > otherwise each sync request-reply invocation will block a > thread that > > waits on socket to receive response and even with nio it will be > > hard to > > recover if the JVM or machine is restarted as all socket > connections > > will be lost ... > > > > however i think there is way to keep things simple by using an > > external > > service(s) that only function is to execute sync invocation > and keep > > synapse core fully async and stateless (by avoiding sync > calls all IO > > operations can complete quickly and messages can be simply > queued) - > > then it is easy to configure external async/sync mapping > services to > > block on sync invocation (they may set very long timeout on > > sockets) and > > if necessary develop them with non blocking IO API and configure > > underlying OS to allow keeping very large of simultaneous > sockets > > opened > > (including writing them in C/C++ instead of Java as all > > communication is > > just sockets with callbacks ...). > > > > of course if such mapping service is restarted all socket > related > > state > > is lost unless some kind of reliable retrying (or RM) is > deployed in > > target services to reject duplicate message (WS-RM/RX is not > required > > just making service to return idempotent results for the same > > message id > > would be enough) > > > > we have developed an example of such service that can map > > async/sync and > > aggregate/load balance intranet services in WS-Dispatcher (see > > referenced paper for details): > > http://www.extreme.indiana.edu/xgws/dispatcher/ > <http://www.extreme.indiana.edu/xgws/dispatcher/> > > > > best, > > > > alek > > > > > > > > > > > > As Glen mentiond > > > ----- Original Message ----- From: "Asankha C. Perera" > > <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > <mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>> > > > To: <[email protected] > <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>> > > > Sent: Thursday, May 11, 2006 9:15 AM > > > Subject: Chat log > > > > > > > > >> > > >> * Now talking in #apache-synapse > > >> <paulfremantle> hi > > >> <asankha> hi > > >> * saminda has joined #apache-synapse > > >> * gdaniels has joined #apache-synapse > > >> <gdaniels> good morning/afternoon all > > >> <paulfremantle> hey > > >> <paulfremantle> how are you? > > >> <gdaniels> not too bad > > >> <gdaniels> getting ready to head to SF tomorrow > > >> <gdaniels> (JavaOne next wk) > > >> <paulfremantle> ah should be fun > > >> <paulfremantle> you going to any concerts while you are > there??! > > >> <gdaniels> probably :) > > >> <gdaniels> J1 always has some cool band or other, plus > there are a > > >> few shows next wk I was eyeing if I have a chance > > >> <gdaniels> did you ever get a chance to read over that > article? > > >> <paulfremantle> oh :( > > >> <paulfremantle> can we take that offline where i can grovel > > more loudly? > > >> <gdaniels> heh > > >> <gdaniels> surely > > >> <gdaniels> so are other folks here? > > >> <asankha> yep.. :) > > >> <paulfremantle> shall we start? > > >> <paulfremantle> ant said he'd be a bit late > > >> <gdaniels> :) hi Asankha > > >> <asankha> sure > > >> <asankha> hi Glen > > >> <asankha> shall we start with the topic that Glen brought > into the > > >> mailing list? > > >> <paulfremantle> yes > > >> <asankha> Glen.. could you drive us through? > > >> <gdaniels> woo hoo sending and receiving and MEPs, oh my! > > >> <gdaniels> so basically I wanted to talk about how Synapse > > deals with > > >> sending and receiving in general > > >> <gdaniels> i.e. sanity check our notion of how sending > should > > work in > > >> various situations > > >> <paulfremantle> yep > > >> <gdaniels> and figure out what we eventually want > > >> <gdaniels> this will hopefully lead us to a design for > 1.0 that we > > >> don't have to destroy and rebuild later > > >> <gdaniels> this also relates to Asankha's recent msg about > > correlation > > >> <asankha> yes.. i am waiting for replies on the message > list.. but > > >> this is a good time to talk about that too... > > >> <gdaniels> so let's start with a simple basic assumption, > which is > > >> that <send> simply sends to the wsa:To in the message > > >> <gdaniels> problems I see with that assumption include the > > ability to > > >> send a copy of a message elsewhere > > >> * dims waves to everyone, says hi and wanders off > > >> <gdaniels> and dealing with responses > > >> <gdaniels> i.e. if Synapse wants to act as a bidirectional > > proxy, it > > >> needs to rewrite the <ReplyTo> on outgoing messages > (might need to) > > >> <paulfremantle> yep > > >> <paulfremantle> and it also needs to be able to deal with > the case > > >> where the endpoint isnt WS-A aware.... > > >> <gdaniels> there are a lot of tricky issues here which we > need > > to at > > >> least think about a little, I think > > >> <gdaniels> Paul: yes > > >> <paulfremantle> so for example where it can't add ref params > > into the > > >> reply to > > >> <gdaniels> we should also be able to map sync-async and > async-sync > > >> <paulfremantle> yes > > >> <saminda> yes > > >> <gdaniels> that should come in really handy now that > MSoft is > > >> promoting a development model which locks down whether or > not a > > >> particular channel supports sync at all > > >> <gdaniels> WCF services either MUST be anonymous ReplyTo, or > > MUST be > > >> non-anonymous. No optionality. > > >> <paulfremantle> hey i didnt know that > > >> <gdaniels> so Synapse might come in handy to provide that > > optionality > > >> <paulfremantle> interesting > > >> <paulfremantle> where's that defined? apart from in the WCF > > codebase :) > > >> <gdaniels> yeah, I think it's weird, but an 800-lb > gorilla can't be > > >> wrong, right? :) > > >> <gdaniels> look up duplex channels > > >> <paulfremantle> thanks > > >> <gdaniels> so anyhoo > > >> <saminda> glen got little problem with async-sync > > >> <asankha> glen.. so if we rewrite the replyto of outgoing > > messages to > > >> synapse, once we start getting the replies, it may turn > out to > > be a > > >> bit tricky to map them back to the correct replyto > wouldnt it? > > >> * ant_back1315 is now known as ant_ > > >> <gdaniels> asankha: if we're rewriting, it's easy to add > a ref > > param > > >> for correlation > > >> <saminda> sync-async case for mutliple <send/> can be > think of > > >> <saminda> but otherway around ?? > > >> <gdaniels> saminda: could you give an example? > > >> <saminda> i can't grasp the async-sync, pls help me with it > > >> <asankha> i guess someone wants to get an async response > from a > > >> syncronous service? is that right saminda > > >> <gdaniels> message comes in to Synapse with <replyto>A</> > > >> <gdaniels> where A is a non-anon URL > > >> <saminda> ok > > >> <saminda> right > > >> <saminda> so this is IN to Synapse right > > >> <gdaniels> Synapse then either sends on that original > message > > with an > > >> edited <replyTo>anon</>, or just calls out itself on a > totally new > > >> message - in either case to a synch service > > >> <gdaniels> saminda: yes, but then the OUT goes to a sync > service > > >> <saminda> thank you > > >> <gdaniels> Synapse would then get the response back from > the sync > > >> service and either send it back <To>A</> or send back > another > > message > > >> that it creates <To>A</> > > >> <paulfremantle> +1 > > >> <saminda> +1 > > >> <gdaniels> so if a given client wanted to use > async/callback style, > > >> and the "true" destination was a .Net "non-duplex" > channel, we > > could > > >> use something like this to make it work > > >> <asankha> right.. i guess we could capture these example > scenarios > > >> into the wiki so we could always easily refer back? > > >> <gdaniels> yes yes yes > > >> <gdaniels> +1,000,000 to use-cases in the wiki > > >> <gdaniels> so I see two architectural issues (at least) here > > >> <saminda> ok > > >> <gdaniels> 1) sending to destinations that are decided > upon at > > >> runtime, and not necessarily by rewriting the <To> header > > >> <gdaniels> 2) correlation / understanding MEPs > > >> <asankha> yes.. > > >> <gdaniels> a very simple version of (1) might be > something like > > <send > > >> destination="URL"/> or <send destProperty="propName"/> > > >> <paulfremantle> so glen.... firstly.. i think that > synapse could > > >> handle the sync->async > > >> <paulfremantle> and I think we can handle the async to > sync too > > >> <gdaniels> (destProperty would be a property containing a > > destination > > >> at runtime) > > >> <paulfremantle> what we aren't very good at handling.... > is doing > > >> something to a response based on what the request is > > >> <gdaniels> paul: yes exactly > > >> <gdaniels> despite Sanjiva's clear reservations, I think we > > need that > > >> in 1.0 > > >> <asankha> that applies to both sync and async messages > > >> <gdaniels> or at least we need to seriously consider the > > >> architectural ramifications so we don't design ourselves > into a > > corner > > >> <paulfremantle> well..... my reservation > > >> <paulfremantle> is having something that makes it hard to > cluster > > >> synapse > > >> <gdaniels> hmm > > >> <gdaniels> if we have well-modularized APIs for getting > at state, > > >> then we could always write backing stores that cluster > (i.e. dbs) > > >> <gdaniels> obviously that only works for serializable > state and not > > >> sockets, though! > > >> <paulfremantle> well... there are some things we can do > anyway tho > > >> <paulfremantle> if we are calling a service using axis2 > > >> <gdaniels> when connections are being held open, you need > to go > > back > > >> to that exact process > > >> <paulfremantle> then it lets us correlate the responses > > >> <gdaniels> sure but we need to figure out what Synapse > does WRT > > >> waiting around for the responses > > >> <gdaniels> i.e. do we block a thread, and if not we need > to do some > > >> kind of correlation > > >> <asankha> the difficulty with clustering is to keep the > correlation > > >> available to all instances is it? > > >> <paulfremantle> so there are two cases arent there.... > same as > > axis2 > > >> <paulfremantle> either the reply to is anon > > >> <paulfremantle> in which case the open socket is the > correlator > > >> <paulfremantle> or there is a real reply to > > >> <paulfremantle> in which case the WSA is the correlator > > >> <paulfremantle> i think axis2 can actually do the correlation > > we need > > >> <paulfremantle> thats different to doing a clustered > thing i think > > >> <ant_> does that mean anon replyTos are not clusterable? > > >> <paulfremantle> no > > >> <paulfremantle> because they are the easy ones > > >> <paulfremantle> they have to come back to the same instance > > >> <gdaniels> ant: it's only doable if you have sharable > sockets > > >> <paulfremantle> welcome ant > > >> <ant_> hi > > >> <gdaniels> and hi :) > > >> <paulfremantle> the real reply tos are the hard ones... > > >> <paulfremantle> because they might come back to a different > > instance > > >> if synapse isnt set up right > > >> <asankha> what if we multicast the message id's and the > replyto > > where > > >> the response should go to to all instances in a cluster > > >> <paulfremantle> well thats cool > > >> <paulfremantle> but def v2! > > >> <asankha> and once an instance received the response, it > issues > > >> another multicast to invalidate the entry from the > clusterwide > > cache > > >> <gdaniels> how the state is shared should be behind > > interfaces/APIs > > >> which simply refuse to make assumptions that state comes > from a > > >> particular place > > >> <gdaniels> the important part is that the Synapse core > writes the > > >> correlation info "somewhere" via a > > >> well-defined api > > >> <gdaniels> then we can plug in cluster support into that > > >> state-management API later > > >> <gdaniels> same with configuration > > >> <paulfremantle> i think asankhas model (in email) > > >> <paulfremantle> of using axis2's req-response correlation > > >> <asankha> yes.. that was for the 'simple' case that you > mentioned > > >> Paul... > > >> <paulfremantle> and copying from the req context to the resp > > context > > >> <paulfremantle> could handle a fair number of cases > > >> <asankha> i guess it could handle cases where incoming is > sync > > >> <asankha> do you think its something we want to implement?.. > > >> <paulfremantle> yes i think its a good first step > > >> <paulfremantle> but maybe we can implement it using a > MEPcontext > > >> <paulfremantle> in other words have a MEPContext > > >> <paulfremantle> which manages properties that are copied > across > > meps > > >> <gdaniels> hmm > > >> <gdaniels> now there's an idea > > >> <paulfremantle> and it could also be used to help with > reply tos > > >> <gdaniels> sound, oh, *familiar* to anyone? :) > > >> <paulfremantle> yeah > > >> <paulfremantle> its your idea > > >> <paulfremantle> Im not trying to steal it :) > > >> <gdaniels> oh I didn't mean that > > >> <paulfremantle> :) > > >> <gdaniels> this leads to the question of "is it the right > > decision to > > >> separate Synapse from Axis2, or would we gain a lot more from > > >> integrating them more tightly" > > >> <gdaniels> :) > > >> <paulfremantle> well thats a fine question > > >> <paulfremantle> i think... so far > > >> <paulfremantle> that we've gained from the abstraction > > >> <paulfremantle> its helped focus on whats necessary > > >> <paulfremantle> and on the simple use cases > > >> <ant_> i guess as one of the ones originally pushing for the > > >> seperation i should say it doesn't really matter so much > to me > > now. > > >> Everywhere i'm likely to want to use Synapse would also > have Axis2 > > >> <gdaniels> I'm not convinced yet either way, but I think > there > > are a > > >> number of cases where more direct reuse of Axis2 > structure might > > >> really help Synapse > > >> <paulfremantle> i'm not strongly one way or the other either > > >> <gdaniels> i.e. Modules, correlation, etc > > >> <gdaniels> then again, I'm also in favor of moving > towards the > > >> "virtual service" model for configuration as well :) > > >> <gdaniels> (i.e. multiple rulesets, with the first > "implicit" rule > > >> being a URL-based dispatch to a given set) > > >> <paulfremantle> now that we have a fairly well defined > end-user > > model > > >> <paulfremantle> thats reasonably clean > > >> <paulfremantle> i dont mind how connected the > infrastructure is > > >> <paulfremantle> but I think it has helped us keep our > thinking > > clean > > >> <gdaniels> I just wonder if anyone is really hoping to > implement > > >> Synapse over other engines... IIRC it was the IONA guys > that were > > >> really talking about that and they seem to have pretty much > > disapeared > > >> <paulfremantle> yeah i dont think thats the important use > case..... > > >> in fact i dont think I ever did > > >> <asankha> Paul.. so on what we have discussed today.. how > much > > will > > >> we consider for M2? > > >> <saminda> got to go guys, bye! > > >> * saminda has left #apache-synapse ("Leaving") > > >> <paulfremantle> how about we try to get an M2 out fairly > soon with > > >> the changes we've made > > >> <paulfremantle> plus the ability to handle responses in your > > scenario > > >> asankha > > >> <asankha> cool.. and in the meanwhile we could start > documenting > > >> whats in store for M3 and how we will implement them and > use cases > > >> <paulfremantle> i like the idea of making the startup > tied to the > > >> axis2 startup and also the ability to simply specify the > > synapse xml > > >> <paulfremantle> and a bunch of good samples > > >> <paulfremantle> and DOC > > >> <paulfremantle> that would make a nice release > > >> <asankha> i have actually tried my use case with the > approach i > > >> emailed.. but we could probably have all messagecontext > properties > > >> that are correlate/* copied over as well.. is that ok? > > >> <paulfremantle> that would be a good start > > >> <asankha> yes.. im working on the samples.. but not yet > in a good > > >> state to send a patch.. and svn was having some trouble > committing > > >> the previous patches.. > > >> <asankha> ok.. will do this and send out a patch and get it > > committed > > >> by early next week > > >> <paulfremantle> ok > > >> <ant_> does anyone know whats going on with SVN? been > donw for > > ages now > > >> <paulfremantle> well lets try and get you a committer > asankha > > so you > > >> dont have to do so much patching > > >> <asankha> that'll be great Paul.. > > >> <gdaniels> +1 > > >> <ant_> i'll send the ws pmc a note about the vote right > after > > this chat > > >> <asankha> so shall we wind up for today.. or is there > anything > > else > > >> for discussion? > > >> <paulfremantle> ant... no idea about svn... > > >> <gdaniels> I've gotta run myself > > >> <paulfremantle> yes i think we're done for today > > >> <paulfremantle> thanks everyone > > >> <paulfremantle> who will post the chat log? > > >> <asankha> i will > > >> <asankha> thanks.. see u all later.. > > >> <gdaniels> bye folks! > > >> <ant_> bye > > >> * gdaniels has quit IRC > > >> > > >> > > >> > > > --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: > [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > >> For additional commands, e-mail: > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > >> > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > > For additional commands, e-mail: > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > > > > > > > > -- > > The best way to predict the future is to invent it - Alan Kay > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > For additional commands, e-mail: > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > > > > > > > > > -- > > Paul Fremantle > > VP/Technology, WSO2 and OASIS WS-RX TC Co-chair > > > > http://bloglines.com/blog/paulfremantle > <http://bloglines.com/blog/paulfremantle> > > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > > > "Oxygenating the Web Service Platform", www.wso2.com > <http://www.wso2.com> <http://www.wso2.com> > > > -- > The best way to predict the future is to invent it - Alan Kay > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > -- The best way to predict the future is to invent it - Alan Kay --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
