[jira] [Commented] (AIRAVATA-2718) [GSoC] Re-architect Output Data Parsing into Airavata core
[ https://issues.apache.org/jira/browse/AIRAVATA-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409608#comment-16409608 ] Dimuthu Upeksha commented on AIRAVATA-2718: --- Hi Lahiru, Thanks for your interest. One possible architecture for generalizing data parsers can be found from [1]. However you are free to come up with your own design but try to utilize the current task execution framework. You can have a good insight of the task framework by referring to my comment in [2]. If you need further clarifications, let's discuss on dev list [1] [https://docs.google.com/presentation/d/1CiPLE6Ht9ynNC9R9Bk0U7yHlsqw2g8ONTDxxDs6R_MY/edit?usp=sharing] [2] https://issues.apache.org/jira/browse/AIRAVATA-2717 Thanks Dimuthu > [GSoC] Re-architect Output Data Parsing into Airavata core > -- > > Key: AIRAVATA-2718 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2718 > Project: Airavata > Issue Type: Epic >Reporter: Suresh Marru >Priority: Major > > As discussed in this paper [1] Airavata based SEAGrid gateway has prototyped > a data catalog system [2]. [3] and [4] are also related references. The new > airavata execution architecture in develop branch is based on Apache Helix. > This provides an opportunity to re-architect the data catalog and build it on > new Helix DAG based execution within Airavata. > This project involves > * the data parsers as Airavata tasks and execute them as Helix DAG's. > * Incorporate the MongoDB based search and catalog registry and explore > Thrift API's. > * Modify the current simple UI into the new Django portal. > * Generalize the data catalog. > * Publish a paper [optional] > [1] - > [https://pdfs.semanticscholar.org/2938/686c5c7eecb1b82ce8064b30555298bd649e.pdf] > [2] - https://github.com/SciGaP/seagrid-data > [[3] - > https://www.researchgate.net/profile/Suresh_Marru/publication/275948320_Scientific_Data_Cataloging_System/links/554a05680cf2e859ce18afb4.pdf|https://www.researchgate.net/profile/Suresh_Marru/publication/275948320_Scientific_Data_Cataloging_System/links/554a05680cf2e859ce18afb4.pdf] > [[4] - > https://www.researchgate.net/profile/Dilum_Bandara/publication/282989239_Schema-independent_scientific_data_cataloging_framework/links/5653a40508aeafc2aabb59e8/Schema-independent-scientific-data-cataloging-framework.pdf|https://www.researchgate.net/profile/Dilum_Bandara/publication/282989239_Schema-independent_scientific_data_cataloging_framework/links/5653a40508aeafc2aabb59e8/Schema-independent-scientific-data-cataloging-framework.pdf] > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRAVATA-2718) [GSoC] Re-architect Output Data Parsing into Airavata core
[ https://issues.apache.org/jira/browse/AIRAVATA-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16408827#comment-16408827 ] Lahiru Jayathilake commented on AIRAVATA-2718: -- Hi Everyone, I am Lahiru Jayathilake, a final year undergraduate from Computer Science and Engineering Department, University of Moratuwa. I am looking forward to participating in GSoC 2018, and this project looks an exciting one for me. I have already built Airavata locally and went through the documentation to get an idea as the first step. Thanks > [GSoC] Re-architect Output Data Parsing into Airavata core > -- > > Key: AIRAVATA-2718 > URL: https://issues.apache.org/jira/browse/AIRAVATA-2718 > Project: Airavata > Issue Type: Epic >Reporter: Suresh Marru >Priority: Major > > As discussed in this paper [1] Airavata based SEAGrid gateway has prototyped > a data catalog system [2]. [3] and [4] are also related references. The new > airavata execution architecture in develop branch is based on Apache Helix. > This provides an opportunity to re-architect the data catalog and build it on > new Helix DAG based execution within Airavata. > This project involves > * the data parsers as Airavata tasks and execute them as Helix DAG's. > * Incorporate the MongoDB based search and catalog registry and explore > Thrift API's. > * Modify the current simple UI into the new Django portal. > * Generalize the data catalog. > * Publish a paper [optional] > [1] - > [https://pdfs.semanticscholar.org/2938/686c5c7eecb1b82ce8064b30555298bd649e.pdf] > [2] - https://github.com/SciGaP/seagrid-data > [[3] - > https://www.researchgate.net/profile/Suresh_Marru/publication/275948320_Scientific_Data_Cataloging_System/links/554a05680cf2e859ce18afb4.pdf|https://www.researchgate.net/profile/Suresh_Marru/publication/275948320_Scientific_Data_Cataloging_System/links/554a05680cf2e859ce18afb4.pdf] > [[4] - > https://www.researchgate.net/profile/Dilum_Bandara/publication/282989239_Schema-independent_scientific_data_cataloging_framework/links/5653a40508aeafc2aabb59e8/Schema-independent-scientific-data-cataloging-framework.pdf|https://www.researchgate.net/profile/Dilum_Bandara/publication/282989239_Schema-independent_scientific_data_cataloging_framework/links/5653a40508aeafc2aabb59e8/Schema-independent-scientific-data-cataloging-framework.pdf] > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
