On Mon, May 03, 2010 at 09:18:02PM +0400, Sergey Tumaev wrote:
> * Ryan O'Hara <roh...@redhat.com> [Mon, 3 May 2010 09:55:30 -0500]:
>> On Sun, May 02, 2010 at 07:53:12PM +0400, Sergey Tumaev wrote:
>> > Hi,
>> >
>> > I have measured the performance of Openais messaging with the
>> primitive
>> > client/server setup:
>> > 1. Server - receives the message from client and sends it back:
>> > ...
>> > while(1){
>> > result = saMsgMessageGet (srv_queue_handle, &msg_a,
>> > &msg_time, &id, SA_TIME_ONE_MINUTE);
>> > if (result != SA_AIS_OK) {
>> > printf ("Could not receive message %d\n", result);
>> > saMsgQueueClose (srv_queue_handle);
>> > saMsgFinalize (handle);
>> > exit (1);
>> > }
>> > result = saMsgMessageReply(handle, &msg_a, &id,
>> > SA_TIME_ONE_SECOND);
>> > if (result != SA_AIS_OK) {
>> > printf ("Could not send message %d\n", result);
>> > saMsgQueueClose (srv_queue_handle);
>> > saMsgFinalize (handle);
>> > exit (1);
>> > }
>> > }
>> > ...
>> >
>> > 2. Client - sends/receives message to/from server:
>> > ...
>> > gettimeofday(&t0, NULL);
>> > while(i++<1000){
>> > result = saMsgMessageSendReceive(handle, &srv_queue_name,
>> > &msg_a, &msg_a, 0, SA_TIME_ONE_SECOND);
>> > if (result != SA_AIS_OK) {
>> > printf ("TEST-1: Could not sent message to srv_queue
>> > %d\n", result);
>> > saMsgQueueClose (clt_queue_handle);
>> > saMsgFinalize (handle);
>> > exit (1);
>> > }
>> > }
>> > gettimeofday(&t1, NULL);
>> > temps = t1.tv_sec - t0.tv_sec;
>> > if(temps && (t0.tv_usec > t1.tv_usec)){
>> > tempm = 1000000 + t1.tv_usec - t0.tv_usec;
>> > temps = temps - 1;
>> > }
>> > else{
>> > tempm = t1.tv_usec - t0.tv_usec;
>> > }
>> > printf("TEST-1: Total %d send/recieve = %dsec %dµs\n", i,
>> temps,
>> > tempm);
>> > ...
>> > Test between two host machines: is ~18sec for 1000 messages.
>> > Two processes on same host: is ~33sec (!) for 1000 messages.
>> > Each message has 32 bytes size.
>> >
>> > Both results are looking too slow for messaging.
>> > Inter-process testing even slowly that inter-host!
>> > What wrong with my test?How to improve the performance?
>> >
>> > Version:
>> > corosync-1.2.1
>> > openais-1.1.2
>> > System:
>> > Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz
>> > Linux debian 2.6.26-2-686
>>
>> We've never done any sort of performance testing with MSG service, so
>> I'm not sure that your performance measurements are reasonable. The
>> AIS specification gives no requirements for performance.
>>
>> Measuring the time to complete a saMsgMessageSendReceive is going to
>> be the worst performance of any API call. You are 1) sending a
>> message, waiting for another process to read that message, send a
>> reply, etc. I'm assuming that your "server" has a saMsgMessageGet call
>> pending at all times, correct?
>>
>> One thing worth mentioning is that both saMsgMessageGet and
>> saMsgMessageSendReceive have to create new IPC connections each time
>> they are called. As a result, there is a fair amout of overhead each
>> time one of those calls is made, especially when those calls return.
>>
>> The MSG service is still very new. In fact, you are the first person
>> that I know has used it. The primary goal duing development to
>> correctly implement the specification. Performance was secondary, as
>> we can hopefully improve performance as the code matures. That said,
>> there are some known bugs in the current implementation of the MSG
>> service. I am working on addressing some of those bugs and a new
>> release will hopefully be available soon.
>>
>> Ryan
>>
> Hi Ryan,
>
> First of all I would like to thank all developers of this project.
> Openais/Corosync is alive, great job!
>
> About my testing.
> I'm planning to use openais services in my project,
> therefore I try to specify all possible problems.
> I have specially chosen saMsgMessageSendReceive function,
> because I understand that it should be slowest (blocking mode).
> But that I need.
> The scenario is very simple:
> One process ask another process (on same or another host) to transmit
> some parameter.
> The process have to get this parameter in blocking mode and sets it in
> own RAM.
> It's very important for application that this transaction is finished as
> soon as possible.
>
> Inter process messaging is more critical for me that inter host, but it
> approximately twice more slowly.
> Though should be on the contrary.
> Would you please explain it?
I'm not sure why you are seeing slower performance when sending
messages to another process on the same host. I'll try to recreate the
problem when I have some free time.
Will your application being sending *any* messages between multiple
nodes? If not, you might get better performance from other message
passing solutions. Is there a specific reason why you need/want to use
MSG service? You might want to look at the CPG service in corosync.
Ryan
_______________________________________________
Openais mailing list
Openais@lists.linux-foundation.org
https://lists.linux-foundation.org/mailman/listinfo/openais
