[
https://issues.apache.org/jira/browse/AIRAVATA-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409608#comment-16409608
]
Dimuthu Upeksha commented on AIRAVATA-2718:
-------------------------------------------
Hi Lahiru,
Thanks for your interest. One possible architecture for generalizing data
parsers can be found from [1]. However you are free to come up with your own
design but try to utilize the current task execution framework. You can have a
good insight of the task framework by referring to my comment in [2]. If you
need further clarifications, let's discuss on dev list
[1]
[https://docs.google.com/presentation/d/1CiPLE6Ht9ynNC9R9Bk0U7yHlsqw2g8ONTDxxDs6R_MY/edit?usp=sharing]
[2] https://issues.apache.org/jira/browse/AIRAVATA-2717
Thanks
Dimuthu
> [GSoC] Re-architect Output Data Parsing into Airavata core
> ----------------------------------------------------------
>
> Key: AIRAVATA-2718
> URL: https://issues.apache.org/jira/browse/AIRAVATA-2718
> Project: Airavata
> Issue Type: Epic
> Reporter: Suresh Marru
> Priority: Major
>
> As discussed in this paper [1] Airavata based SEAGrid gateway has prototyped
> a data catalog system [2]. [3] and [4] are also related references. The new
> airavata execution architecture in develop branch is based on Apache Helix.
> This provides an opportunity to re-architect the data catalog and build it on
> new Helix DAG based execution within Airavata.
> This project involves
> * the data parsers as Airavata tasks and execute them as Helix DAG's.
> * Incorporate the MongoDB based search and catalog registry and explore
> Thrift API's.
> * Modify the current simple UI into the new Django portal.
> * Generalize the data catalog.
> * Publish a paper [optional]
> [1] -
> [https://pdfs.semanticscholar.org/2938/686c5c7eecb1b82ce8064b30555298bd649e.pdf]
> [2] - https://github.com/SciGaP/seagrid-data
> [[3] -
> https://www.researchgate.net/profile/Suresh_Marru/publication/275948320_Scientific_Data_Cataloging_System/links/554a05680cf2e859ce18afb4.pdf|https://www.researchgate.net/profile/Suresh_Marru/publication/275948320_Scientific_Data_Cataloging_System/links/554a05680cf2e859ce18afb4.pdf]
> [[4] -
> https://www.researchgate.net/profile/Dilum_Bandara/publication/282989239_Schema-independent_scientific_data_cataloging_framework/links/5653a40508aeafc2aabb59e8/Schema-independent-scientific-data-cataloging-framework.pdf|https://www.researchgate.net/profile/Dilum_Bandara/publication/282989239_Schema-independent_scientific_data_cataloging_framework/links/5653a40508aeafc2aabb59e8/Schema-independent-scientific-data-cataloging-framework.pdf]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
