Hi Dimuthu, This is neat design. Few questions to understand your implementation:
* Since the Async command monitor needs to be persistent high available service, is it advisable to run it as a Helix Participant or should we run this outside of helix system like a API gateway? * On a related note, any thoughts on running database also as part of the kubernetes cluster? K8s has a mysql example [1] but wondering on any other pragmatic experiences. * We need to write the event listener preferably in Python since these typically run on a compute cluster where java is not so well supported and python is more ubiquitous. * What is your suggestion on the job description (the message payload in your example) format? Can we send in a thrift binary through Kafka and have the listener parse out the required information? Suresh [1] - https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/ > On Dec 4, 2017, at 1:30 PM, DImuthu Upeksha <dimuthu.upeks...@gmail.com> > wrote: > > Hi folks, > > I have implemented the support to Async Job Submission with the callback > workflows on top of the proposed task execution framework. This supports to > both Async Job Submission in remote compute resources using Agents and event > driven job monitoring. Using this approach, I'm going to address following > issues that we are facing today > > 1. Resolve fault DoS attack detection in compute resources when doing > multiple ssh command executions in a short period of time. > 2. Optimize resource utilization and robustness of Airavata Task Execution > Framework when executing long running jobs > > Design and implementation details can be found from [1]. > Sources for the main components can be found from [2], [3], [4] > > Please share your comments and suggestions > > [1] > https://docs.google.com/document/d/1DIjrkjxZZWo9XiwkKWq9WZiOX-uRD5WO-eB6TLxagAg/edit?usp=sharing > > <https://docs.google.com/document/d/1DIjrkjxZZWo9XiwkKWq9WZiOX-uRD5WO-eB6TLxagAg/edit?usp=sharing> > [2] > https://github.com/DImuthuUpe/airavata-sandbox/tree/master/airavata-kubernetes/modules/microservices/async-event-listener > > <https://github.com/DImuthuUpe/airavata-sandbox/tree/master/airavata-kubernetes/modules/microservices/async-event-listener> > [3] > https://github.com/DImuthuUpe/airavata-sandbox/tree/master/airavata-kubernetes/modules/microservices/tasks/async-command-monitor > > <https://github.com/DImuthuUpe/airavata-sandbox/tree/master/airavata-kubernetes/modules/microservices/tasks/async-command-monitor> > [4] > https://github.com/DImuthuUpe/airavata-sandbox/tree/master/airavata-kubernetes/modules/microservices/tasks/async-command-task > > <https://github.com/DImuthuUpe/airavata-sandbox/tree/master/airavata-kubernetes/modules/microservices/tasks/async-command-task> > > Thanks > Dimuthu >
