Thank you. The response clarifies most of my doubts. Replying in-line in red.
From: Suresh Marru <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Friday, May 15, 2020 at 4:31 PM To: Airavata Dev <[email protected]> Subject: [External] Re: GSoC proposal This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. On May 14, 2020, at 10:08 PM, Bandaru, Vivek Shresta <[email protected]<mailto:[email protected]>> wrote: Hi All, As part of GSoC, I will be working on the EPIC: https://issues.apache.org/jira/browse/AIRAVATA-3315 I have a few questions about the deliverables and it would be really helpful if someone can answer them. 1. The last time I had a discussion about the proposal, I was told about the use case where a user can add his/her external storage (S3/Azure/GDrive etc) if they feel the storage quota assigned by the admin isn’t enough. I do not see this use case in the EPIC description though. Is this a nice to have or do we have another EPIC to address this use case? Lets simplify this goal. Let us for a while forget multiple gateways share a storage and there is a notion of internal vs external storage. Lets think about it in simple terms as a Gateway gets created and it chooses its target storage, it could be local to the web server where portal is deployed or it could be on a cloud storage (ignoring the fact how portal access the data) (So if my understanding is right, the storage in the local webserver is considered as internal storage and storage on cloud is considered as external storage. If not, can you please explain a bit more on this or direct me to any documentation or an ISSUE). The goal of this GSoC project should be how multiple users within this single gateways gets a default quota (configurable in the portal) and to provide information for users to see storage usage and percentage burn rate against the quota. You can take inspirations from Google Drive like examples (Will do that). When you login you will see used/total storage, something like that. Do not restrict your GSoC to one single EPIC. Normally GSoC proposals get developed in the community over 3 months and things are clear. Given these happened in a rush now is the time to expand the details and create new epics and issues where needed. This<https://docs.airavata.org/en/master/user-documentation/admin-tutorials/#compute-resource-information> link provided in your previous mail also clarifies my first question. Thank you. 1. Add user interfaces to display user-specific quota and current usage. - There's no Mock UI attached to the the epic like this EPIC<https://issues.apache.org/jira/browse/AIRAVATA-3314> does. Am I supposed to follow a similar pattern of the mock UI mentioned in the aforementioned epic? Not every Airavata new feature has the luxury of having a mock UI. So I suggest you directly work through programming API’s and code examples. But if you think mock UI’s help frame your thinking you can propose them too. Sorry, looks like I did not frame my question properly. I was just asking if I should follow the same pattern as mentioned in EPIC<https://issues.apache.org/jira/browse/AIRAVATA-3314>, since we do not want a single page to deviate from the Airavata UI standard. 1. 2. Use Airavata MFT service to fetch current information on users storage to cross validate. – I’m not really sure what this line means. Is this point anyway related to my first point, i.e. a user fetching his current usage stats from his external storage(Box/S3 etc)? Again do not need to take these too literally and you can step back and look at the goal and propose your interpretations of the larger problem. I was just trying to jump ahead and think on how the storage usage can be calculated. One way is to keep track of the files going into storage and record increment the sum in a database. Otherway is to fetch the used storage space by queuing a service like MFT. The cross-validation above is a mix of both. You can pro-actively track and validate with actual used storage. I think I was over-complicating it here, so you can propose a simpler solution for a user knowing their usage. I was planning on using the first approach where I track the memory of a user through his input files every time. Both the approaches should work when the gateway stores its data locally. But when a gateway stores it’s data in an external storage, we would be needing API’s which gives us the folder size for a given path. Not really sure if S3/Azure/GCP etc. provides such API’s. I would love to hear your thoughts on this. 1. Any pointers are appreciated. Regards, Vivek.
