Proposal to Consolidate Airavata Repositories

[Jayawardana, Yasith]

Hello Everyone,

Currently, there’s 22 airavata repositories scattered across the apache 
organization, making it difficult for:
(a) contributors - to find repositories, read/updated the docs, and contribute 
to.
(b) maintainers – to track dependencies, plan releases, and streamline 
development and ci/cd.

The current project structure (22 repositories) is as follows:

🔧 Core Services

  *   airavata<https://github.com/apache/airavata> – Main microservices and 
orchestration engine
  *   airavata-custos<https://github.com/apache/airavata-custos> – Identity and 
access management framework
  *   airavata-mft<https://github.com/apache/airavata-mft> – Managed file 
transfer services
🖥️ User Interfaces

  *   airavata-django-portal<https://github.com/apache/airavata-django-portal> 
– Primary reference UI
  *   
airavata-django-portal-sdk<https://github.com/apache/airavata-django-portal-sdk>
 – SDK for custom portal development
  *   
airavata-django-portal-commons<https://github.com/apache/airavata-django-portal-commons>
 – Shared Django utilities
  *   
airavata-cookiecutter-django-app<https://github.com/apache/airavata-cookiecutter-django-app>
 – Starter template for apps
  *   
airavata-cookiecutter-django-output-view<https://github.com/apache/airavata-cookiecutter-django-output-view>
 – Template for output views
  *   airavata-custos-portal<https://github.com/apache/airavata-custos-portal> 
– UI for Custos
  *   airavata-php-gateway<https://github.com/apache/airavata-php-gateway> – 
Legacy PHP gateway
📦 Data & Metadata Services

  *   airavata-data-lake<https://github.com/apache/airavata-data-lake> – Data 
lake and storage backend
  *   airavata-data-catalog<https://github.com/apache/airavata-data-catalog> – 
Metadata and search services
  *   airavata-mft-portal<https://github.com/apache/airavata-mft-portal> – Web 
UI for file transfers
📚 Documentation & Branding

  *   airavata-site<https://github.com/apache/airavata-site> – Project website
  *   airavata-docs<https://github.com/apache/airavata-docs> – Developer 
documentation
  *   airavata-user-docs<https://github.com/apache/airavata-user-docs> – 
End-user guides
  *   
airavata-admin-user-docs<https://github.com/apache/airavata-admin-user-docs> – 
Admin-focused documentation
🧪 Research

  *   airavata-sandbox<https://github.com/apache/airavata-sandbox> – Prototypes 
and early-stage work
  *   airavata-cerebrum<https://github.com/apache/airavata-cerebrum> – Airavata 
for Neuroscience
Empty

  *   
airavata-jupyter-kernel<https://github.com/apache/airavata-jupyter-kernel> – 
Jupyter integration
  *   airavata-labs<https://github.com/apache/airavata-labs> – Experimental 
projects
  *   airavata-custos-docs<https://github.com/apache/airavata-custos-docs> – 
Custos documentation

I’m proposing to consolidate these (and archive others), so that it’s easier to 
discover, contribute, and maintain.

Proposed repo structure (5 repositories):

  *   apache/airavata – central place for job/experiment orchestration and 
resource management code
     *   airavata
     *   airavata-data-catalog
     *   [airavata-portal] – will be superseded by the new react portal in 
future
        *   airavata-django-portal
        *   airavata-django-portal-sdk
        *   airavata-django-portal-commons
        *   airavata-cookiecutter-django-app
        *   airavata-cookiecutter-django-output-view
        *   airavata-php-gateway
  *   apache/airavata-custos – central place for authentication and 
authorization related code
     *   airavata-custos
     *   airavata-custos-portal
  *   apache/airavata-mft – central place for data transfer related code
     *   airavata-mft
     *   airavata-mft-portal
     *   airavata-data-lake
  *   apache/airavata-docs – central place for the landing page and docs 
(possibly deployed in one location)
     *   airavata-site
     *   airavata-docs
     *   airavata-user-docs
     *   airavata-admin-user-docs
  *   apache/airavata-cerebrum – airavata-backed sdks and utilities for 
neuroscience

airavata-cerebrum

Empty repos to archive/drop:

  *   airavata-labs
  *   airavata-sandbox
  *   airavata-jupyter-kernel
  *   airavata-custos-docs

I believe this consolidation would help maintain the project in the long term 
and looking forward to your feedback.

Best Regards,
Yasith

--
Yasith Jayawardana <yas...@gatech.edu>
Research Scientist, ARTISAN
Georgia Institute of Technology, Atlanta GA