hi jaliya
On Wed, Jun 4, 2008 at 10:36 PM, Jaliya Ekanayake <[EMAIL PROTECTED]>
wrote:
> I am open to sharing my knowledge in any means possible. That is the sole
> reason I did this comparision instead of discussion without any results.
>
>
> So the way I tested is as follows.
>
> As I have mentioned, i deployed Mercury in Axis2 which is deployed in
> Tomcat and tested it with Mercury samples.
> Then I deployed Sandesha in Axis2 which is deployed in Tomcat and tested it
> using Sandesha2 samples.
>
I am not sure about how Sandesha2 works. But this is how Mercury works.
In In only cases The client thread put this message to the RMSSequence
(which buffers this message) and immediately returns. Then RMSSequenceWorker
(an independent thread) start working on this and send these messages using
a separate thread for each message (MessageWorker). So if we send 1000
messages (using any number of threads at client level) which basically
creates 1000 threads to send the messages.
So here effectively you try to access a Server have 150 maxthreads with 1000
simultaneous threads.
Here I agree with you that Mercury needs a Window Approach as with TCP to
limit the number of messages with out flooding the network at once.
But if sandesha2 uses only 4 threads then it does not flood the server as
such.
If this is the case Mercury should be able to use that performance
improvement (not building messages in inmemory mode). But need to have more
investigation.
So the correct experiment is to
1. Use Mercury client --> with Mercury Sample service by changing Mercury
and Sandesha2 jars
2. Use Sandesha2 client --> with Sandesha2 Sample service by changing
Mercury and Sandesha2 jars
I am also bit busy these days and I'll do this test on next week.
> I did not do any changes to the server side codings.
>
> I am attaching the two clients that I used for testing Mercury and Sandesha
>
> HTH
>
> Btw; I checked the code but could not find a place where I can see, the
> application messages are buffered and the incoming thread is released.
> Where is the place you do this? I went upto RMDContext but could not find
> it.
>
At the server side first RMSSequence receives the message and it buffers it
in the InvokerBuffer.
See InvokerBuffer.java : 242
synchronized(rmdSequence.getInvokerBuffer()){
newMessageReceived =
rmdSequence.applicationMessageReceived(
sequence.getMessageNumber(),
mercuryMessageContext);
}
Then go to inside of this method.
See RMDSequence.java 254
this.invokerBuffer.applicatinMessageReceived(messageNumber,
mercuryMessageContext, this.persistanceDto);
it calls the invoker buffer
See InvokerBuffer.java 208
this.messageBuffer.put(new Long(messageNumber), mercuryMessageContext);
this.receivedMessageNumbers.add(new Long(messageNumber));
As you can see it stores the message and returned the thread.
Then it waits here MecuryInHandler.java 337
if (newMessageReceived) {
// We have to wait only if a new message
received
synchronized (mercuryMessageContext) {
try {
mercuryMessageContext.wait();
} catch (InterruptedException e) {
}
}
}
this is where it wait and where I did the change.
thanks,
Amila.
>
> Thanks,
> Jaliya
>
>
> ----- Original Message -----
> *From:* Amila Suriarachchi <[EMAIL PROTECTED]>
> *To:* Jaliya Ekanayake <[EMAIL PROTECTED]>
> *Sent:* Wednesday, June 04, 2008 12:03 PM
> *Subject:* Re: Mercury, a new WS-RM implementation
>
>
>
> On Wed, Jun 4, 2008 at 7:37 PM, Jaliya Ekanayake <[EMAIL PROTECTED]>
> wrote:
>
>> Hi Amila,
>>
>> I did not intend to show that Mercury is bad or anything.
>> As you have mentioned, "No one can write a Perfect RM implementation in
>> one step."
>>
>> The only reason I want to highlight was that you haven't consider the
>> lessons from past.
>> I participated few development cycles of RM and we have identified the
>> core requirements of buffering.
>>
>
> I have a closer look at this problem. Here is the problem.
> Infact Mecury do the buffering. See the Invoker Buffer.
> Mercury does not build the messages when it works in the in memory model. I
> though this is a performance improvement when considering the Data binding
> frameworks like ADB. Since it does not build the message it waits until
> finish the service invocation.
> But As chamikara once told me Sandesha2 builds the message when ever it
> receives the message. Then I changed the code to build the message and
> return the thread without waiting for service invocation.
>
> msgContext.getEnvelope().build();
> // Here if we return the thread the input stream
> may be closed.
> // So have to wait until user finish with the input
> message.
> // We make this thread wait at the invoker buffer.
> // if (newMessageReceived) {
> // // We have to wait only if a new message
> received
> // synchronized (mercuryMessageContext) {
> // try {
> // mercuryMessageContext.wait();
> // } catch (InterruptedException e) {
> // }
> // }
> // }
>
> Then I tested with two 500 message sequences (paralely) and it worked fine.
> But goes beyond that again was problematic.
>
> Could you please tell me one thing.
> Did you run Sandesha2 Client --> Sandheas2 server (sample)
> Mercury Client --> Mercury server (sample)
>
> or any other way.
>
> I agree that doing Sandesha for many years you have gathered a lot of
> knowledge. What I suggested was to share that if you don't mind :).
>
> thanks,
> Amila.
>
>
>
>> Other scenarios such as breaking the channel, removing the network cable
>> are the core tests for RM. These are easy to verify and once developed even
>> the slowest implementation will be
>> able to handle the requirements of the spec. I did not want to test those
>> to see if it works and I simply assume you have tested those.
>>
>> The problem, I tried to show you is that the single pipeline style
>> architecture in which one thread perform the entire message processing
>> without buffering the messages does not fit best with the requirements of
>> RM. InOnly operation is a very good example to test this, since it has the
>> minimum overhead for the RM implementation. As I have shown, with this
>> minimum overhead, Mercury used all the threads in Tomcat.
>>
>> Hope you understand my point.
>>
>> You mentioned:
>> So now you can understand just keeping Mercury as a dead project in a
>> Sandesha2 branch won't do this.
>>
>> Why do we need to keep it a dead project? We can consider it as the next
>> phase of the RM implementation from Apache. As Sanjiva mentioned, we can
>> build this new version by incorporating best parts from both approaches.
>> Once convinced that it is going in the right direction, people will jump in.
>>
>> Thanks,
>> Jaliya
>>
>>
>> ----- Original Message -----
>> *From:* Amila Suriarachchi <[EMAIL PROTECTED]>
>> *To:* Jaliya Ekanayake <[EMAIL PROTECTED]>
>> *Cc:* [EMAIL PROTECTED] ; Sanjiva Weerawarana <[EMAIL PROTECTED]> ;
>> [email protected]
>> *Sent:* Wednesday, June 04, 2008 1:26 AM
>> *Subject:* Re: Mercury, a new WS-RM implementation
>>
>> hi jaliya,
>>
>> Thank you very much for your comparison.
>>
>> As I guess you have tested the In Only Scenario in Duplex Mode. If you
>> just change the sequence to 10 to 250 this means you have use one sequence
>> in blocking mode. I'll have a more look at this scenario and see the exact
>> reason.
>>
>> If you following the mails I have send this is exactly match what I have
>> said. No one can write a Perfect RM implementation in one step. Like you
>> have done here if Mercury can go independent releases, a lot of people can
>> report this type of issues and which finally make the project better.
>>
>> So now you can understand just keeping Mercury as a dead project in a
>> Sandesha2 branch won't do this.
>>
>> If you going through those Mercury samples you can see there are totally
>> 12 scenarios. you have consider only 1 scenario.
>> On the other hand just running this in the Reliable conditions does not
>> mean it is rm. So we can do following test for see the reliability.
>>
>> 1. Using tcp mon to break channels
>> 2. Delaying the responses at the service
>> 3. Put a message drop module which randomly drop packets with a
>> probability.
>>
>> Then we can go with security, MTOM etc.
>>
>> As a starting point can we do this kind of comparison between two. Here I
>> am not intended to start a fight between Sandesha2 and Mercury. But to have
>> comparison which ultimately benefit both.
>>
>> What do you think jaliya?
>>
>>
>> On Tue, Jun 3, 2008 at 10:49 PM, Jaliya Ekanayake <[EMAIL PROTECTED]>
>> wrote:
>>
>>> I gave a try to compare the performance of Mercury and Sandesha2. I
>>> should say, that I am bit skeptical of the performance of the Mercury due to
>>> its state maintenance.
>>> What I believe is that the RM is a heavy duty module which needs to alter
>>> the message routing inside the axis2. So IMHO, a queuing strategy is a must.
>>>
>>> So here is what I did.
>>>
>>> I deployed Axis2(version 1.4) in Tomcat 5.5.26. Then I deployed
>>> Mercury in Axis2.
>>> The details of the machines I used
>>>
>>> ------------------------------------------------------------------------------------
>>> Server : Dual Processor (Intel Cloverton quad core) machine with 2.3GHz
>>> and 8GB of memory.
>>> Client : Intel M Processor 1.6 GHz, 1 GB of memory. (laptop)
>>>
>>> ------------------------------------------------------------------------------------
>>>
>>> I thought of comparing the InOnly operation (simple Ping operation) to
>>> see how it performs.
>>> Therefore, I use the org.wso2.mercury.sample1.client.InOnlySample and
>>> check if it works fine.
>>> Everything worked perfectly fine!
>>>
>>> Next I did a simple modification to the for loop so that it sends 250
>>> messages for the RM sequence instead of 10.
>>> I got the following error at the server.
>>>
>>> "Jun 3, 2008 12:38:44 PM org.apache.tomcat.util.threads.ThreadPool
>>> logFull
>>> SEVERE: All threads (150) are currently busy, waiting. Increase
>>> maxThreads (150) or check the servlet status"
>>>
>>> Then I performed a similar test for Sandesha2. This time, I use Sandesha
>>> 2 (version 1.3) and same Axis2 in Tomcat server.
>>> I use the sandesha2.samples.userguide.AsyncPingClient.
>>>
>>> I used that client in 4 threads (4 sequences) each thread sending 250
>>> messages(total 1000) messages, and still it worked fine.
>>>
>>> IMO the reason for this is that Mercury is trying to perform the RM
>>> operations without buffering the messages and hence block all the threads.
>>> As I have mentioned earlier, RM requirements do not suite well for a single
>>> pipeline style operation, where a single thread works till the end of the
>>> invocation, specially the INORDER delivery assurance require us to block
>>> threads.
>>>
>>> Therefore, it seems like, Mercury needs some more work in this area.
>>>
>>> So, Amila, what you need is a set of good comparisons validating the
>>> Mercury effort.
>>> Till then, I am +1 for letting Mercury to be in a separate branch of
>>> Sandesha under Apache.
>>>
>>> I also would like to clarify the following from your post.
>>>
>>> "For an example If I come up with a Sandasha2 improvement which
>>> community does not agree at first time"
>>> Is this something you assume or something that happened actually? Please
>>> let us know.
>>>
>>
>> Sorry If I have misunderstood you. Here I tried to explain my view of the
>> revolutionaries taking Sandesha2 as an example. So this is not something
>> that happened actually.
>>
>> thanks,
>> Amila.
>>
>>
>>>
>>> Thanks,
>>> Jaliya
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> I have some clarifications regarding your post.
>>>
>>> You mentioned the following
>>> "For an example If I come up with a Sandasha2 improvement which
>>> community does not agree at first time"
>>>
>>> So, is it something you think would have happen or something that
>>> actually happend?
>>>
>>>
>>>
>>>
>>> ----- Original Message -----
>>> *From:* Amila Suriarachchi <[EMAIL PROTECTED]>
>>> *To:* [EMAIL PROTECTED]
>>> *Cc:* Sanjiva Weerawarana <[EMAIL PROTECTED]> ;
>>> [email protected]
>>> *Sent:* Tuesday, June 03, 2008 7:35 AM
>>> *Subject:* Re: Mercury, a new WS-RM implementation
>>>
>>>
>>>
>>> On Tue, Jun 3, 2008 at 3:05 PM, ant elder <[EMAIL PROTECTED]> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Jun 3, 2008 at 10:10 AM, Sanjiva Weerawarana <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> ant elder wrote:
>>>>>
>>>>>>
>>>>>> While things are as they are I do think things like Mercury
>>>>>> announcements should be kept off the Apache mailing lists, so no more
>>>>>> posts
>>>>>> like: http://apache.markmail.org/message/ounhpi54rx543vqw
>>>>>>
>>>>>
>>>>> Why? Who's rules are you trying to enforce with that? Mercury is an RM
>>>>> module for Axis2 and its open source and under Apache license. Feel free
>>>>> to
>>>>> ignore it but there's nothing wrong with that email.
>>>>>
>>>>>
>>>> There are no "rules" about it, but I don't think its going to help
>>>> things. Any revolutionary rewrite like this by a subset of the community is
>>>> going to make an uncomfortable situation. It seems like everyone agrees
>>>> ideally this will end up again with one RM module being developed at Apache
>>>> in the WS project with all the existing community behind it. So with that
>>>> aim it would help if everyone could try to be sensitive.
>>>>
>>>> Maybe as a start everyone could read the "rules for revolutionaries" -
>>>> http://incubator.apache.org/learn/rules-for-revolutionaries.html
>>>>
>>>
>>> As I understood this revolutionary talks about doing a substantial
>>> improvement to an existing code base.
>>>
>>> For an example If I come up with a Sandasha2 improvement which community
>>> does not agree at first time, I can create a branch of existing Sandesha2
>>> code base and implement my improvement. Then I can ask from the community to
>>> review it and merge it to the main trunk.
>>>
>>> But here the case is different. Mercury is completely different from
>>> Sandesha2. It is not an improvement to Sandesha2.
>>>
>>> AFAIK Apache in general does not have a policy to handle this kind of
>>> situations.
>>>
>>> For an example If I came up with this idea when I wrote the Simulator
>>> People would have said This is not a real implementation and a real
>>> implementation would be more complex.
>>>
>>> On the other hand If I put a complete implementation then people would
>>> have said This has happened without telling to the community.
>>>
>>> So it becomes a chicken and egg problem.
>>>
>>> IMHO the correct solution is to keep both implementations and Let people
>>> to contribute/use to whatever they prefer. And when the time goes we can
>>> decide whether to go ahead with Sandesha2, Mercury or both.
>>>
>>> thanks,
>>> Amila.
>>>
>>>
>>>
>>>
>>> <http://incubator.apache.org/learn/rules-for-revolutionaries.html>
>>>>
>>>> ...ant
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Amila Suriarachchi,
>>> WSO2 Inc.
>>>
>>>
>>
>>
>> --
>> Amila Suriarachchi,
>> WSO2 Inc.
>>
>>
>
>
> --
> Amila Suriarachchi,
> WSO2 Inc.
>
>
--
Amila Suriarachchi,
WSO2 Inc.
