Hi Gourav, Sorry, I did not understand your question. Specifically I am having trouble relating "work load management" to options you suggest (RPC, message based etc.). So what exactly you mean by "workload management" ? What is work in this context ?
Also, I did not understand what you meant by "the most efficient way". Efficient interms of what ? Are you looking at speed ? As per your suggestions, it seems you are trying to find a way to communicate between micro services. RPC might be troublesome if you need to communicate with processes separated from a firewall. Thanks -Thejaka On Wed, Feb 1, 2017 at 12:52 PM, Shenoy, Gourav Ganesh <goshe...@indiana.edu > wrote: > Hello dev, arch, > > > > As part of this Spring’17 Advanced Science Gateway Architecture course, we > are working on trying to debate and find possible solutions to the issue of > managing distributed workloads in Apache Airavata. This leads to the > discussion of finding the most efficient way that different Airavata > micro-services should communicate and distribute work, in such a way that: > > 1. We maintain the ability to scale these micro-services whenever > needed (autoscale perhaps?). > > 2. Achieve fault tolerance. > > 3. We can deploy these micro-services independently, or better in a > containerized manner – keeping in mind the ability to use devops for > deployment. > > > > As of now the options we are exploring are: > > 1. RPC based communication > > 2. Message based – either master-worker, or work-queue, etc > > 3. A combination of both these approaches > > > > I am more inclined towards exploring the message based approach, but again > there arises the possibility of handling limitations/corner cases of > message broker such as downtimes (may be more). In my opinion, having > asynchronous communication will help us achieve most of the above-mentioned > points. Another debatable issue is making the micro-services implementation > stateless, such that we do not have to pass the state information between > micro-services. > > > > I would love to hear any thoughts/suggestions/comments on this topic and > open up a discussion via this mail thread. If there is anything that I have > missed which is relevant to this issue, please let me know. > > > > Thanks and Regards, > > Gourav Shenoy >
