[jira] [Commented] (AIRAVATA-2717) [GSoC] Resurrect User-Defined Airavata Workflows
[ https://issues.apache.org/jira/browse/AIRAVATA-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411604#comment-16411604 ] Dimuthu Upeksha commented on AIRAVATA-2717: --- [~marcuschristie] Yeah I guess that's what we want. Sorry for my misrepresentation. Let's use a higher level workflow to connect applications rather than composing everything in a single workflow. > [GSoC] Resurrect User-Defined Airavata Workflows > - > > Key: AIRAVATA-2717 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2717 > Project: Airavata > Issue Type: Epic >Affects Versions: 0.17 >Reporter: Suresh Marru >Priority: Major > Labels: gsoc2018 > > Airavata used to support user-defined workflows using an interface XBaya to > drag and drop application components to a workspace and define data flow and > control flow dependencies among the application nodes. Airavata's workflow > system was used for composing, executing, and monitoring workflow graphs > primarly web service components. The workflow description was high level > abstraction and used to be converted to lower level execution run times like > BPEL, SCUFL and Python scripts. > > Airavata has evolved significantly and the current development version is > being built-over Apache Helix for DAG orchestration. This provides an > opportunity to resurrect workflow capabilities in Airavata. > This GSoC project involves finalizing a Airavata Workflow Language; modify > the orchestrator to parse user described workflow and translate to equivalent > Helix DAG's; execute and monitor the worklfows; develop a simple UI to > demonstrate the capabilities. > To describe the workflows, you can build on this - > [https://docs.google.com/document/d/1eh7BV8CHupxyM2jeqcM2tUG5MnXFt7hNDX4PQDfxCcM/edit] > or follow other discussions like - > https://issues.apache.org/jira/browse/AIRAVATA-2555 and > User community & Impact of the software: Airavata is primarily targeted to > build science gateways using computational resources from various > disciplines. The initial targeted set of gateways include projects supporting > research and education in chemistry, biophysics, and geosciences . The goal > of airavata is to enhance productivity of these gateways to utilize > cyberinfrastructure of resources (e.g., local lab resources, the Extreme > Science and Engineering Discovery Environment (XSEDE), University Clusters, > Academic and Commercial Computational Clouds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRAVATA-2717) [GSoC] Resurrect User-Defined Airavata Workflows
[ https://issues.apache.org/jira/browse/AIRAVATA-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411558#comment-16411558 ] Marcus Christie commented on AIRAVATA-2717: --- [~dimuthuupe], do we need two levels of workflows? The static workflow that the orchestrator defines is for running an Application. Do we want a higher level workflow for chaining together multiple Applications? I guess a higher level workflow dealing with Applications would be more user friendly. > [GSoC] Resurrect User-Defined Airavata Workflows > - > > Key: AIRAVATA-2717 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2717 > Project: Airavata > Issue Type: Epic >Affects Versions: 0.17 >Reporter: Suresh Marru >Priority: Major > Labels: gsoc2018 > > Airavata used to support user-defined workflows using an interface XBaya to > drag and drop application components to a workspace and define data flow and > control flow dependencies among the application nodes. Airavata's workflow > system was used for composing, executing, and monitoring workflow graphs > primarly web service components. The workflow description was high level > abstraction and used to be converted to lower level execution run times like > BPEL, SCUFL and Python scripts. > > Airavata has evolved significantly and the current development version is > being built-over Apache Helix for DAG orchestration. This provides an > opportunity to resurrect workflow capabilities in Airavata. > This GSoC project involves finalizing a Airavata Workflow Language; modify > the orchestrator to parse user described workflow and translate to equivalent > Helix DAG's; execute and monitor the worklfows; develop a simple UI to > demonstrate the capabilities. > To describe the workflows, you can build on this - > [https://docs.google.com/document/d/1eh7BV8CHupxyM2jeqcM2tUG5MnXFt7hNDX4PQDfxCcM/edit] > or follow other discussions like - > https://issues.apache.org/jira/browse/AIRAVATA-2555 and > User community & Impact of the software: Airavata is primarily targeted to > build science gateways using computational resources from various > disciplines. The initial targeted set of gateways include projects supporting > research and education in chemistry, biophysics, and geosciences . The goal > of airavata is to enhance productivity of these gateways to utilize > cyberinfrastructure of resources (e.g., local lab resources, the Extreme > Science and Engineering Discovery Environment (XSEDE), University Clusters, > Academic and Commercial Computational Clouds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRAVATA-2717) [GSoC] Resurrect User-Defined Airavata Workflows
[ https://issues.apache.org/jira/browse/AIRAVATA-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411320#comment-16411320 ] Yasas Gunarathne commented on AIRAVATA-2717: Hi Dimuthu, Thank you for the detailed answer. I created a new thread at dev mailing list to discuss about Airavata Workflow Language. Regards > [GSoC] Resurrect User-Defined Airavata Workflows > - > > Key: AIRAVATA-2717 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2717 > Project: Airavata > Issue Type: Epic >Affects Versions: 0.17 >Reporter: Suresh Marru >Priority: Major > Labels: gsoc2018 > > Airavata used to support user-defined workflows using an interface XBaya to > drag and drop application components to a workspace and define data flow and > control flow dependencies among the application nodes. Airavata's workflow > system was used for composing, executing, and monitoring workflow graphs > primarly web service components. The workflow description was high level > abstraction and used to be converted to lower level execution run times like > BPEL, SCUFL and Python scripts. > > Airavata has evolved significantly and the current development version is > being built-over Apache Helix for DAG orchestration. This provides an > opportunity to resurrect workflow capabilities in Airavata. > This GSoC project involves finalizing a Airavata Workflow Language; modify > the orchestrator to parse user described workflow and translate to equivalent > Helix DAG's; execute and monitor the worklfows; develop a simple UI to > demonstrate the capabilities. > To describe the workflows, you can build on this - > [https://docs.google.com/document/d/1eh7BV8CHupxyM2jeqcM2tUG5MnXFt7hNDX4PQDfxCcM/edit] > or follow other discussions like - > https://issues.apache.org/jira/browse/AIRAVATA-2555 and > User community & Impact of the software: Airavata is primarily targeted to > build science gateways using computational resources from various > disciplines. The initial targeted set of gateways include projects supporting > research and education in chemistry, biophysics, and geosciences . The goal > of airavata is to enhance productivity of these gateways to utilize > cyberinfrastructure of resources (e.g., local lab resources, the Extreme > Science and Engineering Discovery Environment (XSEDE), University Clusters, > Academic and Commercial Computational Clouds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRAVATA-2717) [GSoC] Resurrect User-Defined Airavata Workflows
[ https://issues.apache.org/jira/browse/AIRAVATA-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409595#comment-16409595 ] Dimuthu Upeksha commented on AIRAVATA-2717: --- Hi Yasas, It is bit outdated but you can get an idea by looking at the document. Recently we have developed a task execution framework on top of Apache Helix which satisfies most of the requirements mentioned in the document. You can refer to the discussion thread [1] under the subject "Evaluating Helix as the task execution framework" to have a better understanding of the design. You can see currently implemented tasks from [2]. However currently, we statically bind the tasks into a workflow [3]. Workflow is just a sequence of tasks. Order of these tasks is pre defined and embedded into the current orchestrator code. For example we run in the order of Environment setup tasks -> Input Data Staging Tasks -> Job Submission Task so on. In future we should be able to have more flexibility. Rather than statically defining order of tasks inside the orchestrator code, we should be able give that order from the outside. That's why we need a workflow language to interpret it. Ideally the flow would be Create a workflow description -> submit to orchestrator -> orchestrator parses the workflow -> Submit to task execution engine To create the workflow you can simply write the workflow in a text file, or use a GUI to generate it. If you follow the mail thread that I have mentioned, there is a image that illustrate such GUI tool. If you need more information, let's move to dev mailing list [1] [http://mail-archives.apache.org/mod_mbox/airavata-dev/201711.mbox/thread] [2] [https://github.com/apache/airavata/tree/develop/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task] [3] [https://github.com/apache/airavata/blob/develop/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/workflow/PreWorkflowManager.java#L83] Thanks Dimuthu > [GSoC] Resurrect User-Defined Airavata Workflows > - > > Key: AIRAVATA-2717 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2717 > Project: Airavata > Issue Type: Epic >Affects Versions: 0.17 >Reporter: Suresh Marru >Priority: Major > Labels: gsoc2018 > > Airavata used to support user-defined workflows using an interface XBaya to > drag and drop application components to a workspace and define data flow and > control flow dependencies among the application nodes. Airavata's workflow > system was used for composing, executing, and monitoring workflow graphs > primarly web service components. The workflow description was high level > abstraction and used to be converted to lower level execution run times like > BPEL, SCUFL and Python scripts. > > Airavata has evolved significantly and the current development version is > being built-over Apache Helix for DAG orchestration. This provides an > opportunity to resurrect workflow capabilities in Airavata. > This GSoC project involves finalizing a Airavata Workflow Language; modify > the orchestrator to parse user described workflow and translate to equivalent > Helix DAG's; execute and monitor the worklfows; develop a simple UI to > demonstrate the capabilities. > To describe the workflows, you can build on this - > [https://docs.google.com/document/d/1eh7BV8CHupxyM2jeqcM2tUG5MnXFt7hNDX4PQDfxCcM/edit] > or follow other discussions like - > https://issues.apache.org/jira/browse/AIRAVATA-2555 and > User community & Impact of the software: Airavata is primarily targeted to > build science gateways using computational resources from various > disciplines. The initial targeted set of gateways include projects supporting > research and education in chemistry, biophysics, and geosciences . The goal > of airavata is to enhance productivity of these gateways to utilize > cyberinfrastructure of resources (e.g., local lab resources, the Extreme > Science and Engineering Discovery Environment (XSEDE), University Clusters, > Academic and Commercial Computational Clouds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRAVATA-2717) [GSoC] Resurrect User-Defined Airavata Workflows
[ https://issues.apache.org/jira/browse/AIRAVATA-2717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408328#comment-16408328 ] Yasas Gunarathne commented on AIRAVATA-2717: Hi All, I am a final year undergraduate from the Department of Computer Science and Engineering, University of Moratuwa, Sri Lanka. During last few weeks, I went through the documentations and the code base of Apache Airavata, and deployed it locally to understand its functionalities. I found this [1] page (last modified on Feb 29, 2016) describing about Airavata workflow requirements, and I want to know whether these requirements are changed or updated. Furthermore, kindly let me know how to proceed in order to apply for this project, since I am really interested in this project for Google Summer of Code 2018. [1] [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62685248|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62685248] Regards > [GSoC] Resurrect User-Defined Airavata Workflows > - > > Key: AIRAVATA-2717 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2717 > Project: Airavata > Issue Type: Epic >Affects Versions: 0.17 >Reporter: Suresh Marru >Priority: Major > Labels: gsoc2018 > > Airavata used to support user-defined workflows using an interface XBaya to > drag and drop application components to a workspace and define data flow and > control flow dependencies among the application nodes. Airavata's workflow > system was used for composing, executing, and monitoring workflow graphs > primarly web service components. The workflow description was high level > abstraction and used to be converted to lower level execution run times like > BPEL, SCUFL and Python scripts. > > Airavata has evolved significantly and the current development version is > being built-over Apache Helix for DAG orchestration. This provides an > opportunity to resurrect workflow capabilities in Airavata. > This GSoC project involves finalizing a Airavata Workflow Language; modify > the orchestrator to parse user described workflow and translate to equivalent > Helix DAG's; execute and monitor the worklfows; develop a simple UI to > demonstrate the capabilities. > To describe the workflows, you can build on this - > [https://docs.google.com/document/d/1eh7BV8CHupxyM2jeqcM2tUG5MnXFt7hNDX4PQDfxCcM/edit] > or follow other discussions like - > https://issues.apache.org/jira/browse/AIRAVATA-2555 and > User community & Impact of the software: Airavata is primarily targeted to > build science gateways using computational resources from various > disciplines. The initial targeted set of gateways include projects supporting > research and education in chemistry, biophysics, and geosciences . The goal > of airavata is to enhance productivity of these gateways to utilize > cyberinfrastructure of resources (e.g., local lab resources, the Extreme > Science and Engineering Discovery Environment (XSEDE), University Clusters, > Academic and Commercial Computational Clouds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
