yasithdev opened a new pull request, #660:
URL: https://github.com/apache/airavata/pull/660
## Summary
Reworks Airavata's orchestration onto a DB-transactional executor and a
fixed SSH/SFTP transport with SLURM/AWS provisioning, removing the
Helix/ZooKeeper/Curator/RabbitMQ stack. Bundles the proto/SDK mapping fixes and
the quickstart docs on the same branch.
### Orchestration & resource model
- DB-transactional `ProcessExecutor` walks the `PROCESS → TASK` DAG with
`SELECT ... FOR UPDATE SKIP LOCKED`; the DB (`PROCESS`/`TASK`/`EXEC_STATUS`) is
the single source of truth. No Helix/ZooKeeper/Curator/RabbitMQ on the run path.
- Transport collapsed to fixed **SSH** (compute ops) + **SFTP** (data ops);
endpoint read off the compute/storage resource; `ssh_port`/`sftp_port` folded
onto the resources; job-submission/data-movement protocol abstractions removed.
- Provisioning modeled as `ResourceType{SLURM,AWS}` via
`ProvisioningAdapter`/`ProvisioningAdapters`.
- `DbTask`/`DbTaskResult` task SPI. Removed the `messaging/` (RabbitMQ)
subsystem, Helix files, and broker/ZooKeeper config + compose services. Kafka
retained (KRaft) for the remote-agent path.
- docker-SLURM cluster added to the devstack; architecture docs (`README`,
`CLAUDE.md`) updated.
### Proto / SDK
- Explicit repeated-field proto mapping, SDK proto-enum migration, and
experiment persistence fixes.
### Docs
- Quickstart + README.
## Test plan
- `mvn install -DskipTests -T4` — BUILD SUCCESS across all modules.
- Echo experiment runs end-to-end on the docker-SLURM cluster via the DB
executor (ENV_SETUP → sbatch over SSH → sacct monitoring → SFTP output staging
→ `EXPERIMENT_STATE_COMPLETED`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
