Yes, I think these technologies will be very helpful to implement Distributed Task Execution framework, and it will be fun to play around with too. I highly recommend you watch the YouTube video I shared in my previous email (see below) to get a head-start on how a similar task execution framework design was constructed using these 3 technologies (they have shared a lot more in-depth detail, which might seem out-of-the-place at start).
Thanks and Regards, Gourav Shenoy From: Apoorv Palkar <apoorv_pal...@aol.com> Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org> Date: Tuesday, March 28, 2017 at 8:47 AM To: "dev@airavata.apache.org" <dev@airavata.apache.org> Subject: Re: [GSoC] Proposal Topics I haven't used Kafka or Cassandra. I would be interested in developing a solution using these technologies to avoid redundancies. -----Original Message----- From: Shenoy, Gourav Ganesh <goshe...@indiana.edu> To: dev <dev@airavata.apache.org> Sent: Mon, Mar 27, 2017 7:52 pm Subject: [GSoC] Proposal Topics Hello dev, I am interested in participating for GSoC this season. There are a couple of topics in my mind which could be good proposals. 1. Distributed Task Execution (Workload Management) for Apache Airavata • Apoorv has already shown interest in this, and has a fair idea of the problem. • I have been working on building a prototype to solve this problem, as part of Science Gateways course [see: https://goo.gl/CZcIIn<https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_CZcIIn&d=DwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=5igYYic94E4a_fcZkq1JiHDu3vinqn86vMjTi_lEKHs&m=j1KQyNfEyfh6g4uMiNPcAmyvazus9F9AtCHOe1ZGpUE&s=3EiIQ5VK2alrQHOhMRiItGSs3uPDQbddiG8IW_oLHHk&e=>] • There are other possible approach(s), like using Akka, Cassandra, Kafka [see: https://youtu.be/s3GfXTnzG_Y<https://urldefense.proofpoint.com/v2/url?u=https-3A__youtu.be_s3GfXTnzG-5FY&d=DwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=5igYYic94E4a_fcZkq1JiHDu3vinqn86vMjTi_lEKHs&m=j1KQyNfEyfh6g4uMiNPcAmyvazus9F9AtCHOe1ZGpUE&s=oGvRrlDHO-jQepIAvhMnNl9JwfNsdEtZzXhKYnji3w8&e=>] 2. Workflow Editor/Builder for Apache Airavata • Ajinkya had started on this topic, and I can use his inputs. • The idea is to allow modelling multiple Airavata job submissions into a workflow, using tools such as CWL (Common Workflow Language). • In addition, to integrate a workflow editor UI with the processing logic, and manage dependencies (whether 2 jobs can be run in parallel v/s waiting for one to complete since it depends on output of another). I would love to hear from you all on any suggestions, inclusions to make. Thanks and Regards, Gourav Shenoy
