Hello dev,
I have made some progress on this module, here is a brief description:
The skeleton code for this micro-service is uploaded to
https://github.com/apache/airavata/tree/allocation-manager
As the use-cases were pretty exhaustive, I have started on developing
the data models for this service, this documents (
https://drive.google.com/open?id=1Dxat8ewk7GQ8qzrmEQsueKORgw6IPlYShVVByoAOAys
) will summarize the preliminary structure, Both GridChem and XRAS
allocation request models were studied to develop these, Please review
the document and feel free to comment about the choices made.
Jira Link: https://issues.apache.org/jira/browse/AIRAVATA-2320
I believe the data model will share more light on the functionalities
and requirements for this service, and will surely help any developer to
make contributions.
Thanks and best regards,
*Anuj Bhandar*
On 9/23/16 4:20 PM, Anuj Bhandar wrote:
Hello Mark,
First of all, thank you for the reply !
The design you briefed me, a very good way to control the allocations,
the insight will definitely help me address few issue's for this service.
I have derived some use-cases after discussing with Sudhakar, Are
these use-cases exhaustive enough to generalize them to CIPRES, Please
review and suggest improvements if any.
Use-cases can be found here :
https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Resource+Allocation+Service
Yes Suresh,
I have created a page to document the development phase for this
module, link is mentioned above.
Thanks and best regards,
Anuj Bhandar
On 09/23/2016 03:38 PM, Miller, Mark wrote:
Hi Anuj,
Just to follow up on Suresh’s comment.
We do not use an allocation protocol, though Sudhakar does, and
probably can help with that.
We do enforce both storage and usage limits on users.
The former is done at login, and users over a certain threshold
aren’t allowed to submit jobs until their storage levels are reduced.
The latter is enforced, but manually in our main site (at least at
present).
However, our rest site has a better and more sophisticated method.
Our policy is to allow users to access 50,000 su if they are at a US
institution, and 30,000 su if they are at a foreign institution.
When the user hits 10,20, 30, and 50 thousand SU, they receive an
email to warn them about the process.
If they go above, I can turn off their account manually, but I
usually ask them how they are doing first.
In the REST site, each time a user submits a job, we calculate the
amount of total SUs it would consume if it ran to completion.
We subtract this amount from the amount of allocation the user has
remaining. If the potential cost is greater than the amount they have
remaining, the submission is rejected. That way, a user cannot submit
100 jobs each requiring 6800 SU. Which is possible, though it would
take a long time.
The idea of holding a reserve against potential expenses by a running
job is not a new idea, but it prevents things from getting out of hand.
Let me know if you have any questions about how this works.
Best,
Mark
*From:*Suresh Marru [mailto:sma...@apache.org]
*Sent:* Friday, September 23, 2016 12:00 PM
*To:* Airavata Dev <dev@airavata.apache.org>
*Subject:* Re: Building a Resource Allocation Service For Airavata
Hi Anuj,
This is great summary. Since this is a new capability for Airavata,
it will be useful to have dedicated wiki space to record all
information. We can start with a high-level problem description (you
have some of that in the form of use cases), architecture and design
thoughts and Implementation plan (pointers to JIRA’s).
May be you can put it under -
https://cwiki.apache.org/confluence/display/AIRAVATA/Architecture+Documentation
Mark and Terri might have some input on this task since CIPRES
enforces soft quotas for users (but I guess there is no formal
allocation request process). Once you have the wiki we can gather the
feedback also there.
Thanks,
Suresh
On Sep 23, 2016, at 2:39 PM, Anuj Bhandar <bhandar.a...@gmail.com
<mailto:bhandar.a...@gmail.com>> wrote:
Hi Dev,
I have undertaken this project of designing and developing an
resource allocation service for Airavata, to give you some
context, this service takes inspiration from XRAS (XSEDE Resource
Allocation Service).
Here is the preliminary design document, which was written as a
part of GSOC-2016,
https://cwiki.apache.org/confluence/display/AIRAVATA/%5BGsoc+Proposal%5D+Resource+Allocation+Manager+for+Apache+Airavata.
I have derived some use cases by discussing with Sudhakar
Pamidighantam ( PI - GridChem), here is the link :
https://issues.apache.org/jira/browse/AIRAVATA-2108.
Any comments and improvements are gladly welcome.
Thanks and best regards,
Anuj Bhandar
