yasithdev opened a new pull request, #567:
URL: https://github.com/apache/airavata/pull/567
## π― Conceptual Model
This PR is more of a RFC to think through what the fundamental problem that
airavata should tackle. This POC is built around **6 core domain interfaces**
that represent the fundamental operations of distributed task execution. The
POC uses a **gRPC-based worker system** for task execution:
```
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Core Domain Interfaces β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β ResourceRegistry β CredentialVault β ExperimentOrch β
β β’ Register compute β β’ Secure storage β β’ Create exper β
β β’ Register storage β β’ Unix permissions β β’ Generate tasks β
β β’ Validate access β β’ Encrypt/decrypt β β’ Submit for execβ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β TaskScheduler β DataMover β WorkerLifecycle β
β β’ Cost optimization β β’ 3-hop staging β β’ Spawn workers β
β β’ Worker distrib β β’ Persistent cache β β’ gRPC workers β
β β’ Atomic assignment β β’ Lineage tracking β β’ Task execution β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β gRPC Worker System β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Worker Binary β Script Generation β Task Execution β
β β’ Standalone exec β β’ SLURM scripts β β’ Poll for tasks β
β β’ gRPC client β β’ K8s manifests β β’ Execute tasks β
β β’ Auto-deployment β β’ Bare metal scriptsβ β’ Report results β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
