Re: Proposal to Consolidate Airavata Repositories

[Jayawardana, Yasith]

Hello Everyone,

After discussing with the Airavata community, the refined consolidation 
structure is below.


  *
airavata - consolidating all airavata middleware development, including core 
services and data related services.
  *
airavata-portals - consolidating all frontends to airavata middleware.
  *
airavata-docs - consolidating all airavata middleware and portal docs.
  *
airavata-custos - consolidating all custos related backends and frontends.
  *
airavata-cerebrum - consolidating all cerebrum related backends and frontends.

The repos can be cleanly consolidated with commit history intact. So I propose 
to drop the empty airavata repositories and archive/drop the now hanging repos 
upon consolidation.

Best Regards,
Yasith


--

Yasith Jayawardana <yas...@gatech.edu>

Research Scientist, IDEaS

Georgia Institute of Technology - Atlanta, GA

________________________________
From: Jayawardana, Yasith <yas...@gatech.edu>
Sent: Thursday, June 12, 2025 8:46:31 PM
To: dev@airavata.apache.org <dev@airavata.apache.org>
Subject: Proposal to Consolidate Airavata Repositories

You don't often get email from yas...@gatech.edu. Learn why this is 
important<https://aka.ms/LearnAboutSenderIdentification>

Hello Everyone,



Currently, there’s 22 airavata repositories scattered across the apache 
organization, making it difficult for:
(a) contributors - to find repositories, read/updated the docs, and contribute 
to.
(b) maintainers – to track dependencies, plan releases, and streamline 
development and ci/cd.



The current project structure (22 repositories) is as follows:

🔧 Core Services

  *   airavata<https://github.com/apache/airavata> – Main microservices and 
orchestration engine
  *   airavata-custos<https://github.com/apache/airavata-custos> – Identity and 
access management framework
  *   airavata-mft<https://github.com/apache/airavata-mft> – Managed file 
transfer services

🖥️ User Interfaces

  *   airavata-django-portal<https://github.com/apache/airavata-django-portal> 
– Primary reference UI
  *   
airavata-django-portal-sdk<https://github.com/apache/airavata-django-portal-sdk>
 – SDK for custom portal development
  *   
airavata-django-portal-commons<https://github.com/apache/airavata-django-portal-commons>
 – Shared Django utilities
  *   
airavata-cookiecutter-django-app<https://github.com/apache/airavata-cookiecutter-django-app>
 – Starter template for apps
  *   
airavata-cookiecutter-django-output-view<https://github.com/apache/airavata-cookiecutter-django-output-view>
 – Template for output views
  *   airavata-custos-portal<https://github.com/apache/airavata-custos-portal> 
– UI for Custos
  *   airavata-php-gateway<https://github.com/apache/airavata-php-gateway> – 
Legacy PHP gateway

📦 Data & Metadata Services

  *   airavata-data-lake<https://github.com/apache/airavata-data-lake> – Data 
lake and storage backend
  *   airavata-data-catalog<https://github.com/apache/airavata-data-catalog> – 
Metadata and search services
  *   airavata-mft-portal<https://github.com/apache/airavata-mft-portal> – Web 
UI for file transfers

📚 Documentation & Branding

  *   airavata-site<https://github.com/apache/airavata-site> – Project website
  *   airavata-docs<https://github.com/apache/airavata-docs> – Developer 
documentation
  *   airavata-user-docs<https://github.com/apache/airavata-user-docs> – 
End-user guides
  *   
airavata-admin-user-docs<https://github.com/apache/airavata-admin-user-docs> – 
Admin-focused documentation

🧪 Research

  *   airavata-sandbox<https://github.com/apache/airavata-sandbox> – Prototypes 
and early-stage work
  *   airavata-cerebrum<https://github.com/apache/airavata-cerebrum> – Airavata 
for Neuroscience

Empty

  *   
airavata-jupyter-kernel<https://github.com/apache/airavata-jupyter-kernel> – 
Jupyter integration
  *   airavata-labs<https://github.com/apache/airavata-labs> – Experimental 
projects
  *   airavata-custos-docs<https://github.com/apache/airavata-custos-docs> – 
Custos documentation



I’m proposing to consolidate these (and archive others), so that it’s easier to 
discover, contribute, and maintain.



Proposed repo structure (5 repositories):

  *   apache/airavata – central place for job/experiment orchestration and 
resource management code
     *   airavata
     *   airavata-data-catalog
     *   [airavata-portal] – will be superseded by the new react portal in 
future
        *   airavata-django-portal
        *   airavata-django-portal-sdk
        *   airavata-django-portal-commons
        *   airavata-cookiecutter-django-app
        *   airavata-cookiecutter-django-output-view
        *   airavata-php-gateway
  *   apache/airavata-custos – central place for authentication and 
authorization related code
     *   airavata-custos
     *   airavata-custos-portal
  *   apache/airavata-mft – central place for data transfer related code
     *   airavata-mft
     *   airavata-mft-portal
     *   airavata-data-lake
  *   apache/airavata-docs – central place for the landing page and docs 
(possibly deployed in one location)
     *   airavata-site
     *   airavata-docs
     *   airavata-user-docs
     *   airavata-admin-user-docs
  *   apache/airavata-cerebrum – airavata-backed sdks and utilities for 
neuroscience

airavata-cerebrum



Empty repos to archive/drop:

  *   airavata-labs
  *   airavata-sandbox
  *   airavata-jupyter-kernel
  *   airavata-custos-docs



I believe this consolidation would help maintain the project in the long term 
and looking forward to your feedback.



Best Regards,

Yasith



--

Yasith Jayawardana <yas...@gatech.edu>

Research Scientist, ARTISAN

Georgia Institute of Technology, Atlanta GA