Hi, guys. I wanted to share with you some design requirements and see if you could point me in the right direction.
I am developing a system that consists of multiple microservices that need to communicate among themselves. Microservices belong in groups. In order to not have a single point of failure and to handle a large volume of messages, each group will contain at least two (probably more) microservices. When a microservice publishes a message, it designated to a specific group and should be processed by just one microservice within that group. Note that if the processing fails (microservice dies in the middle), the same message should be picked up some other microservice in the same group. An additional requirement in my system is that the order of the messages that are being processing by a group, must be maintained because the internal state relies on it. While this is true for all the message consumers in the system, in practicality, the chance of messages getting processed out of order highly depends on the pace that they are being produced and consumed. Some messages are produced at a low rate (one message every 5-10 seconds). As the processing time for each message is just a few milliseconds, the chance that consumer A will finish processing message X *AFTER* consumer B finished processing message X+1 is slim. In such cases, we can probably lift that "maintain message order" requirement and we will still be ok. However, some producers produce messages at a much higher pace (sometimes more than 30 times a second). This means that the chance the consumers are processing those jobs out of order increases. In such cases, the "maintain message order" cannot be ignored. Does ZeroMQ address these issues at all? Thanks in advance.
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
