I made decent progress on some of the tasks we discussed above 1. Data models for the core custos layer is available at [2]. This will further expand into the direction of account provisioning and auditing 2. When core data models are created, in-vm events are fired into an in-memory pub sub system. I tried to implement domain specific wrappers to minimize user error [3] 3. Also, there is a service layer for connectors to directly poll data from core https://github.com/apache/airavata-custos/tree/master/pkg/service 4. An example core events <-> connector integration is available at [5]. This subscribes to core events and pulls core data through the service layer.
[2] https://github.com/apache/airavata-custos/tree/master/pkg/models [3] https://github.com/apache/airavata-custos/tree/master/pkg/events [4] https://github.com/apache/airavata-custos/tree/master/pkg/service [5] https://github.com/apache/airavata-custos/pull/472 Thanks Dimuthu On Sun, May 10, 2026 at 8:38 PM Dimuthu Upeksha <[email protected]> wrote: > Yes, making plugins to connectors makes sense. I am also onboard with the > project structure. The key challenge is to bundle these composable > components as "Go" is not as flexible as Java when it comes to mix and > match components at runtime [1]. I am currently more focused on how to > build that framework as a pub sub system at the core. > > Some of the data models will go into the registry like "User" and > "Project", some will be native to each connector to handle its state. Final > database structure will depend on the custos server configuration and the > amount of connectors we use for the deployment. For example, ACCESS > clusters will have additional data models to handle AIMEE messages and > Campus clusters will have XRAS like connectors and related data models. > > [1] https://eli.thegreenplace.net/2021/plugins-in-go/ > > On Sun, May 10, 2026 at 9:49 AM Suresh Marru <[email protected]> wrote: > >> Thanks, DImuthu, for starting the code reorganization in PR 456. The >> changes are good and moving in the right direction. Pulling amie-specific >> functionality out of the core Custos logic is a good idea. >> >> Can we align the package structure even more explicitly? My intention >> with issue #454 is to debate these high-level modules, and for the code to >> then reflect the architecture concepts. >> >> Can we change plugins to connectors? Plugins imply an extension framework >> and I see merit in that, but for Custos, I feel connectors will make it >> more obvious that these are integration modules that allow Custos to read >> from or write to external systems. >> >> Can we iterate on this structure: >> >> gateway/ (trying to be broader then portal so can put CLI and API SDK's >> there) >> core/ >> registry/ >> policy/ >> reconciler/ >> audit/ >> provisioning/ >> connectors/ >> deployments/ >> docs/ >> >> In the architecture and documentation, we can refer to core as the Custos >> Control Plane. The registry is the persistent data storage. We need to >> imply that sources of truth own domain facts and Custos does not replace >> them. For instance, XRAS remains the source of truth for allocations. >> Custos becomes the system of record for the composed operational state that >> connects these systems and will hold who should have access, what has been >> provisioned, and what action is needed. Custos will then let the managed >> system depend upon Custos to enact. >> >> Suresh >> >> > On May 9, 2026, at 6:59 PM, Suresh Marru <[email protected]> wrote: >> > >> > Hi All, >> > >> > >> > We need a better articulation of the Custos project, its scope, and >> external dependencies. I started an issue with a draft figure. Can you >> review and provide feedback on the issue? Once we consolidate all feedback >> and address it, let's reorganize the repo to reflect the modules, making it >> easier to use and contribute to the project. >> > >> > https://github.com/apache/airavata-custos/issues/454 >> > >> > Cheers, >> > Suresh >> >>
