Lahiru Jayathilake created AIRAVATA-3980:
--------------------------------------------
Summary: Integrate Custos SSH Signer with Apache Airavata Job
Submission
Key: AIRAVATA-3980
URL: https://issues.apache.org/jira/browse/AIRAVATA-3980
Project: Airavata
Issue Type: Task
Components: Custos Airavata Integration
Reporter: Lahiru Jayathilake
h3. Summary
Add SSH certificate-based authentication as an additional option in Apache
Airavata's job submission framework by integrating the Custos SSH Certificate
Signing service. This gives researchers a choice between the existing static
SSH public key approach and short-lived, identity-bound SSH certificates on a
per-compute-resource basis. The project spans backend integration (Java),
Custos signer API connectivity, and Django portal UI changes to expose the new
authentication option.
h3. Problem
Apache Airavata is a science gateway framework that submits computational jobs
to HPC clusters on behalf of researchers. To establish SSH connections to these
clusters, Airavata uses an SSH key pair stored in its credential store, with
the public key placed on the target HPC login node (configured through
Airavata's Group Resource Profile).
Custos provides an SSH Certificate Signing service that issues short-lived,
identity-bound SSH certificates. Instead of placing a public key on the HPC
node, the login node trusts a Certificate Authority (CA). When a client needs
SSH access, it requests a certificate from the signer, and the certificate is
valid only for a configured duration. This model provides automatic expiration,
centralized audit logging, and identity-bound access control.
This project adds certificate-based authentication as an additional option
alongside the existing static key approach, so administrators can choose the
best fit for each compute resource. Both methods coexist, and the choice is
made per compute resource through Airavata's Group Resource Profile
configuration.
h3. Description
This project adds the Custos SSH signer as a new authentication option in
Airavata's SSH connection layer. Airavata's existing credential store and the
static public key approach remain fully intact and continue to work as before.
For compute resources that support it, administrators can enable
certificate-based authentication as an alternative: Airavata obtains a
short-lived certificate for its key from the Custos signer before connecting,
instead of relying on a pre-placed public key.
*1. Understand the current Airavata SSH flow*
* Study how Airavata establishes SSH connections to HPC clusters for job
submission
* Understand the role of the SSH agent adaptor in the connection flow
* Understand how Group Resource Profiles configure which credentials are used
for which compute resources
* Identify the integration point where certificate-based authentication can be
added as an alternative to the static key model
*2. Integrate with the Custos Signer API*
* Build a client component in Airavata that calls the Custos signer's REST API
to request an SSH certificate
* The flow becomes: Airavata retrieves its SSH key pair from the credential
store, sends the public key to the Custos signer along with authentication
credentials (OIDC token), receives a signed certificate, and uses the
certificate together with the private key to establish the SSH connection
* Handle certificate caching: since certificates are short-lived, cache them
and re-request only when expired or about to expire
* Handle error cases: signer unavailable, certificate denied (user not
authorized for the requested principal), expired tokens
*3. Modify the SSH connection adaptor*
* Extend the SSH agent adaptor to support certificate-based authentication as
an additional option alongside the existing static key authentication
* The adaptor should select the authentication method based on the compute
resource configuration: use the signed certificate for resources configured
with Custos signer, use the existing static key approach for everything else
* Both methods must coexist cleanly so each compute resource can use whichever
approach is appropriate
*4. Django portal changes*
* Update the Airavata Django portal to expose the new authentication option
when configuring compute resources in Group Resource Profiles
* Administrators should be able to choose between static SSH key and Custos
SSH certificate for each compute resource
* When certificate-based auth is selected, the portal should capture the
necessary Custos signer configuration (signer endpoint, tenant/client ID)
* Provide appropriate UI feedback showing which authentication method is
active for each compute resource
*5. Configure the HPC side*
* Document the principal authorization model: how the signer determines which
Unix principals (usernames) are authorized for a given identity
* Provide a working end-to-end demo: Airavata submits a job to an HPC cluster
using a Custos-signed certificate instead of a pre-placed public key
*6. Testing and documentation*
* Integration tests covering the certificate request, SSH connection, and job
submission flow
* Configuration guide for setting up the Custos signer connection in Airavata
* Migration guide for transitioning existing compute resources from static
keys to certificate-based authentication
h3. Expected Deliverables
* Custos signer client component integrated into Airavata's SSH connection
layer
* Extended SSH agent adaptor supporting certificate-based authentication as an
additional option alongside static keys, selectable per compute resource
* Django portal UI changes allowing administrators to choose authentication
method (static key or Custos certificate) per compute resource in Group
Resource Profiles
* Certificate caching and lifecycle management (request, cache, re-request on
expiry)
* End-to-end working demo: Airavata job submission to an HPC cluster using
Custos-signed SSH certificates
* Configuration and migration documentation
* Integration tests
h3. Required Skills
* Java (Airavata backend is Java-based)
* Python/Django (Airavata portal is Django-based)
* Go (Custos signer is Go-based, useful for understanding the API)
* REST API integration
* SSH and certificate concepts (SSH certificates, CA trust model), or
willingness to learn
* Basic understanding of HPC job submission, or willingness to learn
h3. Resources
* {*}Custos repository{*}:
[github.com/apache/airavata-custos|https://github.com/apache/airavata-custos]
* {*}Custos signer service{*}: {{signer/}} directory for the Go REST API
(sign, revoke, CA management endpoints)
* {*}Apache Airavata repository{*}:
[github.com/apache/airavata|https://github.com/apache/airavata]
* {*}Airavata Django Portal{*}:
[github.com/apache/airavata-django-portal|https://github.com/apache/airavata-django-portal]
* {*}Airavata documentation{*}: [https://airavata.apache.org/]
* {*}OpenSSH certificate documentation{*}:
[https://man.openbsd.org/ssh-keygen#CERTIFICATES]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
