Hi Folks, You may have noticed that there has been some new work going on in the Axis2 clustering implementation. Here is a summary of what has been taking place in the recent past:
1. Introduced RpcChannel ------------------------------------- This is a blocking request-response communication channel. When an Axis2 node starts up, it needs to get the latest state and configuration from a neighbor. Until this information is received, the node cannot start service, hence should block until these messages are received. A Tribes RpcChannel is ideal for this scenario. 2. Introduced SYNC_ACK & ASYNC mode transmissions at the Tribes layer -------------------------------------------------------------------------------------------------------------- Earlier, we were handling ACKs at the Axis2 layer, not the Tribes layer. This approach had a significant overhead since the ACK messages have to be serialized/deserialized. Now we do this at the Tribes layer using SYNC_ACK transmission. In this scenario, the channel.send blocks until all members have successfully processed a message and sent an ACK. If a member does not receive an ACK within a specified period, Tribes will do a restransmission. The maximum number of retries can be configured in the axis2.xml cluster initialization section (<parameter name="maxRetries">5</parameter>). As you may be already thinking, this mode of communication will have a hit on the response times since the channel has to block until all members ACK. This may be undesirable and unnecessary in some circumstances. We can use ASYNC mode of transmission to speed up things. In this case, as soon as the sender sends the message down to its OS's TCP buffer, it returns. Switching between SYNC_ACK & ASYNC modes can be done by changing the "synchronizeAll" clustering parameter in the axis2.xml (<parameter name="synchronizeAll">true</parameter>). After this change, I was able to observe a significant performance improvement. 3. Introduced AtMostOneInterceptor ---------------------------------------------------- As mentioned in #2 above, Tribes takes care of retransmissions, which can lead to the same message being received more than once by a receiver. To handle this, we have introduced an AtMostOnceInterceptor, which intercepts messages on the channel on the receiver, and drops duplicate messages. 4. Introduced OrderInterceptor -------------------------------------------- This interceptor guarantees that sender order is preserved. i.e. each correct receiver in the cluster will receive the messages in the same order that they were sent by the sender. When sending a message, we set a special bit in the Channel Options, and all messages that have this bit set will be ordered. 5. Introduced Membership awareness ------------------------------------------------------ We do not do channel.getMembers anymore. A Tribes membership listener notifies the node about view changes (new members joining and members leaving the group), and the member list is maintained on each node. This way, all knowledge about the members is in one place. Also, when we try to get the state and configuration information, we first try to get it from the longest living member first (assuming that this member will have all the relevant information) failing which, we try a random member. 6. Using ByteMessages ----------------------------------- Now we send/receive Tribes ByteMessages on the channel, instead of java.ioserialized messages. When this message is used, Tribes does not try to serialize/deserialize the messages, and sends raw bytes, which has a performance advantage. On the receiving side, we handle the deserialization at the Axis2 layer, if necessary. Also this way, redundant messages & messages not intended for this group do not get deserialized. 7. Ability to provide custom classloaders ----------------------------------------------------------- Earlier, we were able to send across only objects that can be created by classes loaded from the root classloader. Hence custom POJOs that are contained within AARs & MARs cannot be sent across. We've changed the code so that now we can get hold of the relevant classloaders and deserialize these POJOs. 8. Improved test suite -------------------------------- These are integration tests. The tests fire-up two tribes nodes. Then context replication tests are carried out. Please send you feedback & valuable suggestion on these changes. -- Thanks Afkham Azeez http://www.wso2.org http://azeez78.blogspot.com GPG Fingerprint: 643F C2AF EB78 F886 40C9 B2A2 4AE2 C887 665E 0760
