Re: Best way to realize stateful services architecture withmassivedata?
Jeff, I think you forgot to mention the concurrency issues related with stateful WS. So, at least under this point of view, stateless WS are much more easier to build. Michele On 22 Oct 2007, at 17:08, Walker, Jeff wrote: Ok, So we have moved-off topic here. My original email was to Florian Georg, who asked about stateful services and very large data set transfer, via a stateful web service, if I remember correctly. Then Michele asked my opinion on stateful vs stateless. My point here is that stateless services are far easier than stateful, but almost useless because most situations require services in front of stateful applications. That's an assertion that I'm sure many will refute! That's fine, that's what this board is for. (I'm here to learn as well). Yes, stateful services are relatively easy to build from an engineering point of view. But retrofitting them into an existing architecture is very difficult. Also difficult is engineering them into an SOA. Most SOA architectural layouts I have seen skirt the issue and assume stateless services. I haven't used WS-SecureConversation. REST to me is a completely different approach to architecture. But again, I've only read about it. I would think it also is not appropriate for large data transfer. But I think it is stateless by default, right? The unswimmable sea of WS-* specifications helps only to muddy the waters. If there are no implementations, the specs mean little. I thought Axis 2 implements only these: WS-ReliableMessaging - Supported by Apache Sandesha2 WS-Coordination and WS-AtomicTransaction - Supported by Apache Kandula2 WS-Security - Supported by Apache Rampart WS-Addressing -Module included as part of Axis2 core -jeff -Original Message- From: robert lazarski [mailto:[EMAIL PROTECTED] Sent: Monday, October 22, 2007 10:59 AM To: axis-user@ws.apache.org Subject: Re: Best way to realize stateful services architecture withmassivedata? On 10/22/07, Walker, Jeff <[EMAIL PROTECTED]> wrote: You can definitely create stateful web services. It's just that they are very difficult to use. How do you force the client to make the operation calls in the expected calling sequence? The client will likely need to keep track of the order of the operations it has called. This is complicated book-keeping and often the client unable or unwilling to do this. Errors become a big issue. What do you do when the 5th call of 7 expected calls fails? Should you abandon altogether? How do you retry? The complexity gets ugly, quickly. Its not that difficult - its actually common and standardized via WS-SecureConversation. I personally have also used my own UUID scheme where WS-SecureConversation hasn't been supported on all sides. Coding up your own scheme takes about 2-3 days depending on the programmer - I've even seen beginner .net programmers implement UUID token passing easily. Calling DB transactions often also requires the right sequence, and people get by just fine. EJB stateful web services may not be as easy as a pojo, but people do that often enough too. In short, anything is easier when its stateless. The whole idea though behind WS-* is enterprise stuff like asynchronous calls, addressing, conversations, transactions and reliability. A good case could be made that its not needed a lot - that's where REST comes in. But if you do need common usage patterns beyond simple HTTP get and post, there are seemingly endless WS-* standards that thought of just about everything - and axis2 implements most of them. Robert - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Best way to realize stateful services architecture withmassivedata?
Very useful page. Thank you. Swim easy. -jeff -Original Message- From: robert lazarski [mailto:[EMAIL PROTECTED] Sent: Monday, October 22, 2007 12:30 PM To: axis-user@ws.apache.org Subject: Re: Best way to realize stateful services architecture withmassivedata? On 10/22/07, Walker, Jeff <[EMAIL PROTECTED]> wrote: > > The unswimmable sea of WS-* specifications helps only to muddy the > waters. If there are no implementations, the specs mean little. I > thought Axis 2 implements only these: > WS-ReliableMessaging - Supported by Apache Sandesha2 > WS-Coordination and WS-AtomicTransaction - Supported by Apache Kandula2 > WS-Security - Supported by Apache Rampart > WS-Addressing -Module included as part of Axis2 core > -jeff > One man's "unswimmable sea" is a mere channel to cross in a few hours for others. As for implementations, think again. http://wiki.apache.org/ws/StackComparison HTH, Robert - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Best way to realize stateful services architecture withmassivedata?
On 10/22/07, Walker, Jeff <[EMAIL PROTECTED]> wrote: > > The unswimmable sea of WS-* specifications helps only to muddy the > waters. If there are no implementations, the specs mean little. I > thought Axis 2 implements only these: > WS-ReliableMessaging - Supported by Apache Sandesha2 > WS-Coordination and WS-AtomicTransaction - Supported by Apache Kandula2 > WS-Security - Supported by Apache Rampart > WS-Addressing -Module included as part of Axis2 core > -jeff > One man's "unswimmable sea" is a mere channel to cross in a few hours for others. As for implementations, think again. http://wiki.apache.org/ws/StackComparison HTH, Robert - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Best way to realize stateful services architecture withmassivedata?
Ok, So we have moved-off topic here. My original email was to Florian Georg, who asked about stateful services and very large data set transfer, via a stateful web service, if I remember correctly. Then Michele asked my opinion on stateful vs stateless. My point here is that stateless services are far easier than stateful, but almost useless because most situations require services in front of stateful applications. That's an assertion that I'm sure many will refute! That's fine, that's what this board is for. (I'm here to learn as well). Yes, stateful services are relatively easy to build from an engineering point of view. But retrofitting them into an existing architecture is very difficult. Also difficult is engineering them into an SOA. Most SOA architectural layouts I have seen skirt the issue and assume stateless services. I haven't used WS-SecureConversation. REST to me is a completely different approach to architecture. But again, I've only read about it. I would think it also is not appropriate for large data transfer. But I think it is stateless by default, right? The unswimmable sea of WS-* specifications helps only to muddy the waters. If there are no implementations, the specs mean little. I thought Axis 2 implements only these: WS-ReliableMessaging - Supported by Apache Sandesha2 WS-Coordination and WS-AtomicTransaction - Supported by Apache Kandula2 WS-Security - Supported by Apache Rampart WS-Addressing -Module included as part of Axis2 core -jeff -Original Message- From: robert lazarski [mailto:[EMAIL PROTECTED] Sent: Monday, October 22, 2007 10:59 AM To: axis-user@ws.apache.org Subject: Re: Best way to realize stateful services architecture withmassivedata? On 10/22/07, Walker, Jeff <[EMAIL PROTECTED]> wrote: > You can definitely create stateful web services. It's just that they are > very difficult to use. How do you force the client to make the operation > calls in the expected calling sequence? The client will likely need to > keep track of the order of the operations it has called. This is > complicated book-keeping and often the client unable or unwilling to do > this. Errors become a big issue. What do you do when the 5th call of 7 > expected calls fails? Should you abandon altogether? How do you retry? > The complexity gets ugly, quickly. > Its not that difficult - its actually common and standardized via WS-SecureConversation. I personally have also used my own UUID scheme where WS-SecureConversation hasn't been supported on all sides. Coding up your own scheme takes about 2-3 days depending on the programmer - I've even seen beginner .net programmers implement UUID token passing easily. Calling DB transactions often also requires the right sequence, and people get by just fine. EJB stateful web services may not be as easy as a pojo, but people do that often enough too. In short, anything is easier when its stateless. The whole idea though behind WS-* is enterprise stuff like asynchronous calls, addressing, conversations, transactions and reliability. A good case could be made that its not needed a lot - that's where REST comes in. But if you do need common usage patterns beyond simple HTTP get and post, there are seemingly endless WS-* standards that thought of just about everything - and axis2 implements most of them. Robert - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Best way to realize stateful services architecture withmassivedata?
On 10/22/07, Walker, Jeff <[EMAIL PROTECTED]> wrote: > You can definitely create stateful web services. It's just that they are > very difficult to use. How do you force the client to make the operation > calls in the expected calling sequence? The client will likely need to > keep track of the order of the operations it has called. This is > complicated book-keeping and often the client unable or unwilling to do > this. Errors become a big issue. What do you do when the 5th call of 7 > expected calls fails? Should you abandon altogether? How do you retry? > The complexity gets ugly, quickly. > Its not that difficult - its actually common and standardized via WS-SecureConversation. I personally have also used my own UUID scheme where WS-SecureConversation hasn't been supported on all sides. Coding up your own scheme takes about 2-3 days depending on the programmer - I've even seen beginner .net programmers implement UUID token passing easily. Calling DB transactions often also requires the right sequence, and people get by just fine. EJB stateful web services may not be as easy as a pojo, but people do that often enough too. In short, anything is easier when its stateless. The whole idea though behind WS-* is enterprise stuff like asynchronous calls, addressing, conversations, transactions and reliability. A good case could be made that its not needed a lot - that's where REST comes in. But if you do need common usage patterns beyond simple HTTP get and post, there are seemingly endless WS-* standards that thought of just about everything - and axis2 implements most of them. Robert - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Best way to realize stateful services architecture withmassivedata?
Ok, now I get your point. But solution # 3 will work *only* if the "front-end" service exposes coarse grained operations, right? If so, can't you expose coarse-grained operations in your stateful WS as well? In point (c) of your conclusions you state that the third approach would be a bit faster than the first one. However you forget to take the network delay into account. Michele On Mon, 2007-10-22 at 10:38 -0400, Walker, Jeff wrote: > Well, > > 1st Approach: Stateful Service > Most applications are finite state machines. People tend to want to put > a web service on the front of their application, to say they are "SOA > compliant" or simply because they think a web service will give their > application some advantage. But because the application itself is > stateful, most likely a first attempt at the web service will produce a > stateful service. > > You can definitely create stateful web services. It's just that they are > very difficult to use. How do you force the client to make the operation > calls in the expected calling sequence? The client will likely need to > keep track of the order of the operations it has called. This is > complicated book-keeping and often the client unable or unwilling to do > this. Errors become a big issue. What do you do when the 5th call of 7 > expected calls fails? Should you abandon altogether? How do you retry? > The complexity gets ugly, quickly. > > > 2nd Approach: Stateless Service > Here, the designer begins to see that a stateless service is far easier > to handle and create. Any operation on the web service interface can be > called at any time. No sequence of calls is necessary. Each call is > fully independent of each other. Errors are localized to that called > operation, and soap faults sent back to the client are localized to that > operation. It's all very clean. It sounds great, but if my above premise > is true (that most applications are stateful) then how can you mesh the > multiple states of the application with a set of operations in a > stateless service? What if a request comes in and is passed to the > application and the application cannot service that request because it > is no longer in a state that can satisfactorily create the right > response? > > > 3rd Approach: Intermediate Composite Service > My suggestion (and it is just one of many possible design solutions, I'm > sure) is to create an intermediate web service that communicates to the > client. This intermediate, composite service offers a stateless > interface to clients. Each stateless operation on it results in a > sequence of stateful calls to the actual web service on behalf of the > client. The composite has some intelligence to decide what to do when > one of the calls in the middle of the sequence fails. (Usually it > abandons the whole calling sequence and returns a soap fault to the > client, but not always. It could conceivably abandon the current > sequence and start a new one. But that requires memory of the calls and > data passed in previously, before the error occurred). > > a). The client is saved from knowing anything about the statefulness of > your application. > b). Your stateful application usually doesn't need altering. A big plus > if the code is old and hard to maintain. > c). The composite service takes approx. as long as the sum of stateful > calls would take. (Slightly less because the marshaling of data into XML > and back is less. It is only done on the one initial call from the > client). > d). Because the composite service is still a fairly long running service > (you don't know exactly how long it will take on each of its > operations), it is best called asynchronously, but that is another > story. > e). There are drawbacks. It means another service has to be written, > tested, maintained and deployed to production. > > The best approach is obviously to simplify and just create a fully > stateless service, if that will work with your application. But don't be > surprised if it doesn't fit well. > -jeff > > > > > > > -Original Message- > From: Michele Mazzucco [mailto:[EMAIL PROTECTED] > Sent: Monday, October 22, 2007 9:41 AM > To: axis-user@ws.apache.org > Subject: RE: Best way to realize stateful services architecture > withmassivedata? > > On Mon, 2007-10-22 at 09:09 -0400, Walker, Jeff wrote: > > 1. Web services shouldn't be stateful. (Far harder to keep that > > promise than to recite it). My own current service is stateful, and it > > breaks the SOA rules we have in place here, causing me no end to > > issues. Removing the statefulness once it creeps into your project is > > like getting
RE: Best way to realize stateful services architecture withmassivedata?
Well, 1st Approach: Stateful Service Most applications are finite state machines. People tend to want to put a web service on the front of their application, to say they are "SOA compliant" or simply because they think a web service will give their application some advantage. But because the application itself is stateful, most likely a first attempt at the web service will produce a stateful service. You can definitely create stateful web services. It's just that they are very difficult to use. How do you force the client to make the operation calls in the expected calling sequence? The client will likely need to keep track of the order of the operations it has called. This is complicated book-keeping and often the client unable or unwilling to do this. Errors become a big issue. What do you do when the 5th call of 7 expected calls fails? Should you abandon altogether? How do you retry? The complexity gets ugly, quickly. 2nd Approach: Stateless Service Here, the designer begins to see that a stateless service is far easier to handle and create. Any operation on the web service interface can be called at any time. No sequence of calls is necessary. Each call is fully independent of each other. Errors are localized to that called operation, and soap faults sent back to the client are localized to that operation. It's all very clean. It sounds great, but if my above premise is true (that most applications are stateful) then how can you mesh the multiple states of the application with a set of operations in a stateless service? What if a request comes in and is passed to the application and the application cannot service that request because it is no longer in a state that can satisfactorily create the right response? 3rd Approach: Intermediate Composite Service My suggestion (and it is just one of many possible design solutions, I'm sure) is to create an intermediate web service that communicates to the client. This intermediate, composite service offers a stateless interface to clients. Each stateless operation on it results in a sequence of stateful calls to the actual web service on behalf of the client. The composite has some intelligence to decide what to do when one of the calls in the middle of the sequence fails. (Usually it abandons the whole calling sequence and returns a soap fault to the client, but not always. It could conceivably abandon the current sequence and start a new one. But that requires memory of the calls and data passed in previously, before the error occurred). a). The client is saved from knowing anything about the statefulness of your application. b). Your stateful application usually doesn't need altering. A big plus if the code is old and hard to maintain. c). The composite service takes approx. as long as the sum of stateful calls would take. (Slightly less because the marshaling of data into XML and back is less. It is only done on the one initial call from the client). d). Because the composite service is still a fairly long running service (you don't know exactly how long it will take on each of its operations), it is best called asynchronously, but that is another story. e). There are drawbacks. It means another service has to be written, tested, maintained and deployed to production. The best approach is obviously to simplify and just create a fully stateless service, if that will work with your application. But don't be surprised if it doesn't fit well. -jeff -Original Message- From: Michele Mazzucco [mailto:[EMAIL PROTECTED] Sent: Monday, October 22, 2007 9:41 AM To: axis-user@ws.apache.org Subject: RE: Best way to realize stateful services architecture withmassivedata? On Mon, 2007-10-22 at 09:09 -0400, Walker, Jeff wrote: > 1. Web services shouldn't be stateful. (Far harder to keep that > promise than to recite it). My own current service is stateful, and it > breaks the SOA rules we have in place here, causing me no end to > issues. Removing the statefulness once it creeps into your project is > like getting blood from a stone. Best remedy is to create a composite > service that is stateless, and that talks to the client. It also > formulates a series of web service calls to your stateful service that > does the real work. The composite service then returns the result to > the client. BPEL could help here. Jeff, can you tell me about a non-trivial stateless web service? What is the difference between the first (i.e. the stateful WS talks to the client) and the second approach (i.e. the stateful WS is hidden by a stateless one)? > > 2. Web services are not suitable for large data transfer, at all. I > have found past about 20Mb of data per request/response and the > service starts to timeout. That is even when RAM is added and > processors are upgraded. On one of my previous web services gigs I was > asked to build a web service to transfer file
