I am trying to design a messaging system with non-persistent, auto acknowledge and aysnch mode of publishing. The expected load is really high as this system primarily going to handle the huge number of social media profiles that I have harvested in my system, which is nearly 5 million. And I have a really distributed architecture, where the social activity details for all those profiles is being stored in one db, and the scoring which I am generating for all those profiles is happening and getting stores in a different place. As of today with out JMS I am facing some latency in getting my scored profile in online, which I think can be get rid off by using the JMS. SO primarily my producer going to send all profile to a particular queue, so the social activity consumer going to consume the message from this Q and pull out the fresh details and update to a DB. Hope this gives a picture on what I am trying to solve, So Here's my query, and the concern I have for which seeking a solution.
On the consumer side, in the onMesasage() method I am going to have the DB interaction to insert/update the message to mysql DB after doing some processing with the message. For me this is straight forward, the only worry is the latency in the consumer side with respect to the time taken to process the business logic, as well as DB update or insert, which I think is fine and works well when we have message cursor in the broker side. But what I am worried about is the load which is going to come on the DB side , so will some one help me to understand how I can enable batching on this context. What I am looking for is the following. - The db update/insert should happen only batch wise, how to enable it - The db batch which I was talking about should be inline with JMS message batch as well, because the broker should be acknowledged only once this batch's db interaction has been completed. - ofcource this will be transacted, so if my batch insertion/updation got failed, how the re-delivery of the message will happen? So please let me know if activemq is already having provisions to handle this kind of situation? Or I need to make my own implementation to deal with this ? Thanks -- View this message in context: http://activemq.2283324.n4.nabble.com/JMS-JDBC-Need-your-input-to-design-a-better-system-tp4567407p4567407.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
