Hi Lahiru, Thank you for summarizing both of these, and your POCs of both approaches are helpful. The second option, if feasible, will be preferable. You already mentioned the performance. In addition, you will also get forward/backward compatibility if the underlying protobuff structures are maintained with some discipline.
The big plus side of your 1st approach is the REST-compatible javascript libraries. Other than grpc-web (https://github.com/grpc/grpc-web <https://github.com/grpc/grpc-web>) have you seen broader support? I see you are building on grpc-js how is that experience? Suresh > On Feb 13, 2023, at 2:49 PM, Lahiru Jayathilake <lahirujayathil...@gmail.com> > wrote: > > Hi All, > > I have been engaging with the SMILES project to implement the Gateway and its > necessary components. Just to give you a brief introduction, the SMILES > project has three types of data that need to be combined for publication: > Computational DB, Literature DB, and Experiment DB. There should be a > frontend to filter, create, and delete data products, with a Django app as > the backend that will communicate with Apache Airavata Data Catalog [1]. > > Mainly, I have been exploring two approaches. > > 1. > <approach1.png> > > The frontend will communicate with the Django app via REST, and the Django > app will manage the manipulation of data products through gRPC calls to the > Data Catalog API. Django models will be used to represent the Computational, > Literature, and Experiment data products, without storing the data. In the > end, these data products will reside in the Data Catalog, following its > established conventions. > > POC - https://github.com/lahirujayathilake/SEAGrid > <https://github.com/lahirujayathilake/SEAGrid> > This has been implemented to cover the data product creation > > 2. > <with-grpc.png> > > In this approach, the distinction will be a gRPC server operating within the > Django app. To represent the three data products, protobufs will be defined > that extend the DataCatalog proto messages [2]. The frontend will communicate > using gRPC calls. > The gRPC API can be used to manipulate data from other clients, resulting in > improved performance. > > POC - https://github.com/lahirujayathilake/SEAGrid/tree/with-grpc > <https://github.com/lahirujayathilake/SEAGrid/tree/with-grpc> > (The frontend is inprogress) > > I would like to hear your thoughts and feedback on the designs to improve and > to go with the right approach. > > > [1] - https://github.com/apache/airavata-data-catalog > <https://github.com/apache/airavata-data-catalog> > [2] - > https://github.com/apache/airavata-data-catalog/blob/main/data-catalog-api/stubs/src/main/proto/DataCatalogAPI.proto > > <https://github.com/apache/airavata-data-catalog/blob/main/data-catalog-api/stubs/src/main/proto/DataCatalogAPI.proto> > > Cheers! > Lahiru
