Saminda
On 5/12/06, *Aleksander Slominski* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
Paul Fremantle wrote:
> Alek
>
> I agree with you. If we use non-blocking IO, it is possible to
handle
> the sync to async mapping without blocking a thread. We need to
walk
> thru the model and see how it works with Synapse and Axis2. Its
> essential that if we implement this we don't block threads or it
won't
> be a useful system.
>
> As regards surviving restart of the Synapse server, I'm not sure
I see
> that as a requirement. This could be handled at a higher level by
> WS-RM or TX.
yes if all services are RM (or TX) enabled but i though one of use
cases was to have synapse provide RM capability to services that
are not
RM enabled?
in such case for a service with sync interfaces (request-reply on the
same connection) restart of synapse means that they will loose its
outbound socket connection and will not be able to send the
response to
synapse ... now if synapse (or external sync/async service helper)
knows
that it is OK to call the service again then it may just do it
essentially "restarting" execution that may be part of longer
conversation (such as workflow).
of course service must be coded to detect duplicate requests (for
example by checking wsa:MessageID) and in such case return the result
that was stored when outbound connection was broken. this is not
perfect
but may be the simplest way to add some reliability to a system
that has
plenty of existing sync services (like RPC).
best,
alek
> Paul
>
> On 5/12/06, * Aleksander Slominski* <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
> <mailto: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>>
wrote:
>
> Jaliya Ekanayake wrote:
> > Hi All,
> >
> > IMHO it is not desirable to map Sync-Async scenarios. I
can give two
> > reasons.
> >
> > 1. There should be a timeout issue and we will not have
guaranteed
> > mapping always.
> > 2. Mapping sync to two async will cause the Synapse to
hold one
> thread
> > (to keep the sync socket open) which will exhaust the
server easily.
> those issues are very hard to overcome in general unless non
> blocking IO
> is used for each sync invocation (including anon
request-reply WSA)
> otherwise each sync request-reply invocation will block a
thread that
> waits on socket to receive response and even with nio it will
be
> hard to
> recover if the JVM or machine is restarted as all socket
connections
> will be lost ...
>
> however i think there is way to keep things simple by using an
> external
> service(s) that only function is to execute sync invocation
and keep
> synapse core fully async and stateless (by avoiding sync
calls all IO
> operations can complete quickly and messages can be simply
queued) -
> then it is easy to configure external async/sync mapping
services to
> block on sync invocation (they may set very long timeout on
> sockets) and
> if necessary develop them with non blocking IO API and
configure
> underlying OS to allow keeping very large of simultaneous
sockets
> opened
> (including writing them in C/C++ instead of Java as all
> communication is
> just sockets with callbacks ...).
>
> of course if such mapping service is restarted all socket
related
> state
> is lost unless some kind of reliable retrying (or RM) is
deployed in
> target services to reject duplicate message (WS-RM/RX is not
required
> just making service to return idempotent results for the same
> message id
> would be enough)
>
> we have developed an example of such service that can map
> async/sync and
> aggregate/load balance intranet services in WS-Dispatcher (see
> referenced paper for details):
> http://www.extreme.indiana.edu/xgws/dispatcher/
<http://www.extreme.indiana.edu/xgws/dispatcher/>
>
> best,
>
> alek
> >
> >
> >
> > As Glen mentiond
> > ----- Original Message ----- From: "Asankha C. Perera"
> <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
<mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>>
> > To: <[email protected]
<mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>>
> > Sent: Thursday, May 11, 2006 9:15 AM
> > Subject: Chat log
> >
> >
> >>
> >> * Now talking in #apache-synapse
> >> <paulfremantle> hi
> >> <asankha> hi
> >> * saminda has joined #apache-synapse
> >> * gdaniels has joined #apache-synapse
> >> <gdaniels> good morning/afternoon all
> >> <paulfremantle> hey
> >> <paulfremantle> how are you?
> >> <gdaniels> not too bad
> >> <gdaniels> getting ready to head to SF tomorrow
> >> <gdaniels> (JavaOne next wk)
> >> <paulfremantle> ah should be fun
> >> <paulfremantle> you going to any concerts while you are
there??!
> >> <gdaniels> probably :)
> >> <gdaniels> J1 always has some cool band or other, plus
there are a
> >> few shows next wk I was eyeing if I have a chance
> >> <gdaniels> did you ever get a chance to read over that
article?
> >> <paulfremantle> oh :(
> >> <paulfremantle> can we take that offline where i can grovel
> more loudly?
> >> <gdaniels> heh
> >> <gdaniels> surely
> >> <gdaniels> so are other folks here?
> >> <asankha> yep.. :)
> >> <paulfremantle> shall we start?
> >> <paulfremantle> ant said he'd be a bit late
> >> <gdaniels> :) hi Asankha
> >> <asankha> sure
> >> <asankha> hi Glen
> >> <asankha> shall we start with the topic that Glen brought
into the
> >> mailing list?
> >> <paulfremantle> yes
> >> <asankha> Glen.. could you drive us through?
> >> <gdaniels> woo hoo sending and receiving and MEPs, oh my!
> >> <gdaniels> so basically I wanted to talk about how Synapse
> deals with
> >> sending and receiving in general
> >> <gdaniels> i.e. sanity check our notion of how sending
should
> work in
> >> various situations
> >> <paulfremantle> yep
> >> <gdaniels> and figure out what we eventually want
> >> <gdaniels> this will hopefully lead us to a design for
1.0 that we
> >> don't have to destroy and rebuild later
> >> <gdaniels> this also relates to Asankha's recent msg about
> correlation
> >> <asankha> yes.. i am waiting for replies on the message
list.. but
> >> this is a good time to talk about that too...
> >> <gdaniels> so let's start with a simple basic assumption,
which is
> >> that <send> simply sends to the wsa:To in the message
> >> <gdaniels> problems I see with that assumption include the
> ability to
> >> send a copy of a message elsewhere
> >> * dims waves to everyone, says hi and wanders off
> >> <gdaniels> and dealing with responses
> >> <gdaniels> i.e. if Synapse wants to act as a bidirectional
> proxy, it
> >> needs to rewrite the <ReplyTo> on outgoing messages
(might need to)
> >> <paulfremantle> yep
> >> <paulfremantle> and it also needs to be able to deal with
the case
> >> where the endpoint isnt WS-A aware....
> >> <gdaniels> there are a lot of tricky issues here which we
need
> to at
> >> least think about a little, I think
> >> <gdaniels> Paul: yes
> >> <paulfremantle> so for example where it can't add ref params
> into the
> >> reply to
> >> <gdaniels> we should also be able to map sync-async and
async-sync
> >> <paulfremantle> yes
> >> <saminda> yes
> >> <gdaniels> that should come in really handy now that
MSoft is
> >> promoting a development model which locks down whether or
not a
> >> particular channel supports sync at all
> >> <gdaniels> WCF services either MUST be anonymous ReplyTo, or
> MUST be
> >> non-anonymous. No optionality.
> >> <paulfremantle> hey i didnt know that
> >> <gdaniels> so Synapse might come in handy to provide that
> optionality
> >> <paulfremantle> interesting
> >> <paulfremantle> where's that defined? apart from in the WCF
> codebase :)
> >> <gdaniels> yeah, I think it's weird, but an 800-lb
gorilla can't be
> >> wrong, right? :)
> >> <gdaniels> look up duplex channels
> >> <paulfremantle> thanks
> >> <gdaniels> so anyhoo
> >> <saminda> glen got little problem with async-sync
> >> <asankha> glen.. so if we rewrite the replyto of outgoing
> messages to
> >> synapse, once we start getting the replies, it may turn
out to
> be a
> >> bit tricky to map them back to the correct replyto
wouldnt it?
> >> * ant_back1315 is now known as ant_
> >> <gdaniels> asankha: if we're rewriting, it's easy to add
a ref
> param
> >> for correlation
> >> <saminda> sync-async case for mutliple <send/> can be
think of
> >> <saminda> but otherway around ??
> >> <gdaniels> saminda: could you give an example?
> >> <saminda> i can't grasp the async-sync, pls help me with it
> >> <asankha> i guess someone wants to get an async response
from a
> >> syncronous service? is that right saminda
> >> <gdaniels> message comes in to Synapse with <replyto>A</>
> >> <gdaniels> where A is a non-anon URL
> >> <saminda> ok
> >> <saminda> right
> >> <saminda> so this is IN to Synapse right
> >> <gdaniels> Synapse then either sends on that original
message
> with an
> >> edited <replyTo>anon</>, or just calls out itself on a
totally new
> >> message - in either case to a synch service
> >> <gdaniels> saminda: yes, but then the OUT goes to a sync
service
> >> <saminda> thank you
> >> <gdaniels> Synapse would then get the response back from
the sync
> >> service and either send it back <To>A</> or send back
another
> message
> >> that it creates <To>A</>
> >> <paulfremantle> +1
> >> <saminda> +1
> >> <gdaniels> so if a given client wanted to use
async/callback style,
> >> and the "true" destination was a .Net "non-duplex"
channel, we
> could
> >> use something like this to make it work
> >> <asankha> right.. i guess we could capture these example
scenarios
> >> into the wiki so we could always easily refer back?
> >> <gdaniels> yes yes yes
> >> <gdaniels> +1,000,000 to use-cases in the wiki
> >> <gdaniels> so I see two architectural issues (at least) here
> >> <saminda> ok
> >> <gdaniels> 1) sending to destinations that are decided
upon at
> >> runtime, and not necessarily by rewriting the <To> header
> >> <gdaniels> 2) correlation / understanding MEPs
> >> <asankha> yes..
> >> <gdaniels> a very simple version of (1) might be
something like
> <send
> >> destination="URL"/> or <send destProperty="propName"/>
> >> <paulfremantle> so glen.... firstly.. i think that
synapse could
> >> handle the sync->async
> >> <paulfremantle> and I think we can handle the async to
sync too
> >> <gdaniels> (destProperty would be a property containing a
> destination
> >> at runtime)
> >> <paulfremantle> what we aren't very good at handling....
is doing
> >> something to a response based on what the request is
> >> <gdaniels> paul: yes exactly
> >> <gdaniels> despite Sanjiva's clear reservations, I think we
> need that
> >> in 1.0
> >> <asankha> that applies to both sync and async messages
> >> <gdaniels> or at least we need to seriously consider the
> >> architectural ramifications so we don't design ourselves
into a
> corner
> >> <paulfremantle> well..... my reservation
> >> <paulfremantle> is having something that makes it hard to
cluster
> >> synapse
> >> <gdaniels> hmm
> >> <gdaniels> if we have well-modularized APIs for getting
at state,
> >> then we could always write backing stores that cluster
(i.e. dbs)
> >> <gdaniels> obviously that only works for serializable
state and not
> >> sockets, though!
> >> <paulfremantle> well... there are some things we can do
anyway tho
> >> <paulfremantle> if we are calling a service using axis2
> >> <gdaniels> when connections are being held open, you need
to go
> back
> >> to that exact process
> >> <paulfremantle> then it lets us correlate the responses
> >> <gdaniels> sure but we need to figure out what Synapse
does WRT
> >> waiting around for the responses
> >> <gdaniels> i.e. do we block a thread, and if not we need
to do some
> >> kind of correlation
> >> <asankha> the difficulty with clustering is to keep the
correlation
> >> available to all instances is it?
> >> <paulfremantle> so there are two cases arent there....
same as
> axis2
> >> <paulfremantle> either the reply to is anon
> >> <paulfremantle> in which case the open socket is the
correlator
> >> <paulfremantle> or there is a real reply to
> >> <paulfremantle> in which case the WSA is the correlator
> >> <paulfremantle> i think axis2 can actually do the
correlation
> we need
> >> <paulfremantle> thats different to doing a clustered
thing i think
> >> <ant_> does that mean anon replyTos are not clusterable?
> >> <paulfremantle> no
> >> <paulfremantle> because they are the easy ones
> >> <paulfremantle> they have to come back to the same instance
> >> <gdaniels> ant: it's only doable if you have sharable
sockets
> >> <paulfremantle> welcome ant
> >> <ant_> hi
> >> <gdaniels> and hi :)
> >> <paulfremantle> the real reply tos are the hard ones...
> >> <paulfremantle> because they might come back to a different
> instance
> >> if synapse isnt set up right
> >> <asankha> what if we multicast the message id's and the
replyto
> where
> >> the response should go to to all instances in a cluster
> >> <paulfremantle> well thats cool
> >> <paulfremantle> but def v2!
> >> <asankha> and once an instance received the response, it
issues
> >> another multicast to invalidate the entry from the
clusterwide
> cache
> >> <gdaniels> how the state is shared should be behind
> interfaces/APIs
> >> which simply refuse to make assumptions that state comes
from a
> >> particular place
> >> <gdaniels> the important part is that the Synapse core
writes the
> >> correlation info "somewhere" via a
> >> well-defined api
> >> <gdaniels> then we can plug in cluster support into that
> >> state-management API later
> >> <gdaniels> same with configuration
> >> <paulfremantle> i think asankhas model (in email)
> >> <paulfremantle> of using axis2's req-response correlation
> >> <asankha> yes.. that was for the 'simple' case that you
mentioned
> >> Paul...
> >> <paulfremantle> and copying from the req context to the resp
> context
> >> <paulfremantle> could handle a fair number of cases
> >> <asankha> i guess it could handle cases where incoming is
sync
> >> <asankha> do you think its something we want to implement?..
> >> <paulfremantle> yes i think its a good first step
> >> <paulfremantle> but maybe we can implement it using a
MEPcontext
> >> <paulfremantle> in other words have a MEPContext
> >> <paulfremantle> which manages properties that are copied
across
> meps
> >> <gdaniels> hmm
> >> <gdaniels> now there's an idea
> >> <paulfremantle> and it could also be used to help with
reply tos
> >> <gdaniels> sound, oh, *familiar* to anyone? :)
> >> <paulfremantle> yeah
> >> <paulfremantle> its your idea
> >> <paulfremantle> Im not trying to steal it :)
> >> <gdaniels> oh I didn't mean that
> >> <paulfremantle> :)
> >> <gdaniels> this leads to the question of "is it the right
> decision to
> >> separate Synapse from Axis2, or would we gain a lot more
from
> >> integrating them more tightly"
> >> <gdaniels> :)
> >> <paulfremantle> well thats a fine question
> >> <paulfremantle> i think... so far
> >> <paulfremantle> that we've gained from the abstraction
> >> <paulfremantle> its helped focus on whats necessary
> >> <paulfremantle> and on the simple use cases
> >> <ant_> i guess as one of the ones originally pushing for the
> >> seperation i should say it doesn't really matter so much
to me
> now.
> >> Everywhere i'm likely to want to use Synapse would also
have Axis2
> >> <gdaniels> I'm not convinced yet either way, but I think
there
> are a
> >> number of cases where more direct reuse of Axis2
structure might
> >> really help Synapse
> >> <paulfremantle> i'm not strongly one way or the other either
> >> <gdaniels> i.e. Modules, correlation, etc
> >> <gdaniels> then again, I'm also in favor of moving
towards the
> >> "virtual service" model for configuration as well :)
> >> <gdaniels> (i.e. multiple rulesets, with the first
"implicit" rule
> >> being a URL-based dispatch to a given set)
> >> <paulfremantle> now that we have a fairly well defined
end-user
> model
> >> <paulfremantle> thats reasonably clean
> >> <paulfremantle> i dont mind how connected the
infrastructure is
> >> <paulfremantle> but I think it has helped us keep our
thinking
> clean
> >> <gdaniels> I just wonder if anyone is really hoping to
implement
> >> Synapse over other engines... IIRC it was the IONA guys
that were
> >> really talking about that and they seem to have pretty much
> disapeared
> >> <paulfremantle> yeah i dont think thats the important use
case.....
> >> in fact i dont think I ever did
> >> <asankha> Paul.. so on what we have discussed today.. how
much
> will
> >> we consider for M2?
> >> <saminda> got to go guys, bye!
> >> * saminda has left #apache-synapse ("Leaving")
> >> <paulfremantle> how about we try to get an M2 out fairly
soon with
> >> the changes we've made
> >> <paulfremantle> plus the ability to handle responses in your
> scenario
> >> asankha
> >> <asankha> cool.. and in the meanwhile we could start
documenting
> >> whats in store for M3 and how we will implement them and
use cases
> >> <paulfremantle> i like the idea of making the startup
tied to the
> >> axis2 startup and also the ability to simply specify the
> synapse xml
> >> <paulfremantle> and a bunch of good samples
> >> <paulfremantle> and DOC
> >> <paulfremantle> that would make a nice release
> >> <asankha> i have actually tried my use case with the
approach i
> >> emailed.. but we could probably have all messagecontext
properties
> >> that are correlate/* copied over as well.. is that ok?
> >> <paulfremantle> that would be a good start
> >> <asankha> yes.. im working on the samples.. but not yet
in a good
> >> state to send a patch.. and svn was having some trouble
committing
> >> the previous patches..
> >> <asankha> ok.. will do this and send out a patch and get it
> committed
> >> by early next week
> >> <paulfremantle> ok
> >> <ant_> does anyone know whats going on with SVN? been
donw for
> ages now
> >> <paulfremantle> well lets try and get you a committer
asankha
> so you
> >> dont have to do so much patching
> >> <asankha> that'll be great Paul..
> >> <gdaniels> +1
> >> <ant_> i'll send the ws pmc a note about the vote right
after
> this chat
> >> <asankha> so shall we wind up for today.. or is there
anything
> else
> >> for discussion?
> >> <paulfremantle> ant... no idea about svn...
> >> <gdaniels> I've gotta run myself
> >> <paulfremantle> yes i think we're done for today
> >> <paulfremantle> thanks everyone
> >> <paulfremantle> who will post the chat log?
> >> <asankha> i will
> >> <asankha> thanks.. see u all later..
> >> <gdaniels> bye folks!
> >> <ant_> bye
> >> * gdaniels has quit IRC
> >>
> >>
> >>
>
---------------------------------------------------------------------
> >> To unsubscribe, e-mail:
[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
> <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>
> >> For additional commands, e-mail:
[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
> <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>
> >>
> >
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
> <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>
> > For additional commands, e-mail:
[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
> <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>
> >
>
>
> --
> The best way to predict the future is to invent it - Alan Kay
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
> <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>
> For additional commands, e-mail:
[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
> <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>
>
>
>
>
> --
> Paul Fremantle
> VP/Technology, WSO2 and OASIS WS-RX TC Co-chair
>
> http://bloglines.com/blog/paulfremantle
<http://bloglines.com/blog/paulfremantle>
> [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>>
>
> "Oxygenating the Web Service Platform", www.wso2.com
<http://www.wso2.com> <http://www.wso2.com>
--
The best way to predict the future is to invent it - Alan Kay
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: [EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>
