This project does seem like a good idea but a project with only 1 committer is not usually regarded as a good fit for the Apache Incubator. Is there any chance that we could wait until there is a group of active committers? It takes at least 3 people to vote for releases and to invite extra PPMC members. There is a good chance that you can attract the extra numbers.
On Fri, 7 Aug 2026 at 18:49, vaquar khan <[email protected]> wrote: > > Hi Greg, > > Thank you, this genuinely means a lot. I am glad to accept your offer to > Champion the effort, and I appreciate you reaching out to line up mentors. > Coming from someone with your history and leadership at the Foundation, > that is exactly the kind of guidance I was hoping to find here. > > The repository is public so you and prospective mentors can read the code > before any vote: https://github.com/vaquarkhan/aegis-mcp-gateway. It builds > green on JDK 21 with the full test suite passing, and the > design-conformance matrix marks each claim 'Done' or 'Roadmap' so the > maturity line stays explicit. To be honest: four adapters (Flink, Kafka, > Spark, Iceberg) are in-depth, and the rest are deliberately thin. I am > completely open to trimming or quarantining the thin set if the community > would rather see a smaller, sharper initial contribution during the > bootstrap phase. > > A few responses to your points: > > *On your own MCP build*: I would be happy to help on a timeline that works > for both of us. The gateway core is engine-agnostic, and adapters plug in > through a small service provider interface, so bringing up a new MCP > surface is mostly mapping tool definitions and backend calls. The core > applies the shared governance chain uniformly: identity, scope, policy, > approvals, egress defense, rate limiting, redaction, and audit. The repo > has a getting-started guide, runnable examples, and an adapters guide to > start from. > > On the Foundation's existing MCP servers: I would welcome composing with > those rather than replacing them. Where an engine or server already ships > an MCP surface, the intent is for the gateway to sit in front of it for > governance rather than reimplement it. If you point me at the ones you have > in mind, I will take a look and share a short write-up on how they might > fit. > > Since you are stepping in as Champion, I would love to defer to your > experience on how best to proceed. Please let me know how you would like to > handle mentor recruitment, and what you need from me regarding next steps > for IP clearance, naming, or codebase scope before we eventually move > toward a formal vote. > > Thanks again for stepping in. > > Best regards, > > Viquar Khan > > https://www.linkedin.com/in/vaquar-khan-b695577/ > > > On Thu, Aug 6, 2026 at 1:37 PM Greg Stein <[email protected]> wrote: > > > This is great stuff, Viquar! ... I'd be happy to Champion this effort > > through the Incubator. I already have a need to build an MCP from > > scratch/zero, and would like to use exactly this kind of framework to build > > that. > > > > We have a few existing MCP servers at the Foundation today, which might be > > able to leverage this framework, too. > > > > I'll poke some people to get some additional Mentors. > > > > My background: 28 years at Apache, current Director, past Chairman, initial > > Incubator member, current Apache RAI participant, past Infra, Subversion, > > HTTPD, and other projects. > > > > Cheers, > > -g > > > > > > On Tue, Aug 4, 2026 at 12:18 AM vaquar khan <[email protected]> > > wrote: > > > > > Hello everyone, > > > I would like to propose a project called Aegis MCP Governance Gateway for > > > the Apache Incubator, and start a discussion. It is a working reference > > > implementation, and I am bringing it here mainly because it needs a > > > community rather than a single author, which I state plainly in the > > > proposal. > > > *In short: *when an AI agent needs to operate data systems like Flink, > > > Kafka, Spark, or Iceberg, it usually drives each engine control plane > > > directly or connects to a pile of separate MCP servers, each with its own > > > security posture, no shared audit, and no common policy. An MCP server > > that > > > can reach a control plane is a privileged client, so shipping one safely > > > means shipping an access-control product. Aegis puts one governed control > > > plane in front of all of those engines, so every agent action runs > > through > > > a single fail-closed chain of identity, scope, policy, approval for > > > destructive actions, egress and request-forgery defense, rate limiting, > > > output redaction, and a tamper-evident audit trail. Engines plug in > > through > > > a small service provider interface and contribute no policy. > > > Current state, stated honestly. There is a working Java 21 build: a > > gateway > > > core, four in-depth adapters (Flink, Kafka, Spark, Iceberg), thinner > > > adapters for other Apache systems, and a runnable distribution. It builds > > > cleanly with 309 tests passing across all modules. A published > > > design-conformance matrix maps every design claim to Done or Roadmap. > > > Several capabilities ship as roadmap items and are not sold as done: > > > externalized high-availability state, CIMD and SPIFFE identity, a full > > > Cedar runtime, vault-backed credentials, and an OpenTelemetry SDK bridge. > > > Parts of the code were developed with AI-assisted tools under my > > direction > > > and review, which I disclose up front for IP clearance. > > > *Biggest risk: *The project is currently a single contributor. Growing a > > > diverse, multi-organization community is the main reason I am bringing it > > > here. I am currently in active conversations with engineers in the > > > ecosystem and expect to add a few of them as initial committers in the > > next > > > few days, prior to a formal vote > > > What I am looking for now: a Champion (an ASF Member) and mentors (three > > or > > > more IPMC members), ideally including people from the Flink, Kafka, > > Spark, > > > or Iceberg communities. The full proposal follows, and the reference > > > implementation is at https://github.com/vaquarkhan/aegis-mcp-gateway. I > > > welcome feedback on scope, name (a name search is still to be run), > > risks, > > > and anything I have missed. > > > Thank you, > > > Viquar Khan > > > https://www.linkedin.com/in/vaquar-khan-b695577/ > > > > > > Apache Incubator Proposal: Aegis MCP Governance Gateway > > > This is the proposal for entry into the Apache Incubator, written for > > > discussion on the Incubator general list. Aegis is a working codename; a > > > name search will be run and the ASF trademark process followed if the > > > project is accepted. The document follows the ASF Incubator proposal > > > template and states the maturity of the code honestly, including what is > > > implemented and what is deferred to the roadmap. > > > Field > > > Value > > > Proposal name > > > Aegis MCP Governance Gateway (codename) > > > Proposed short name > > > aegis (subject to name search) > > > Proposal date > > > to be set at submission > > > Sponsor > > > Apache Incubator PMC > > > Champion > > > to be identified (ASF Member) > > > Mentors > > > to be recruited (three or more IPMC members) > > > Initial code > > > Java 21 multi-module reference implementation, 36 Maven modules, 309 > > tests > > > passing > > > Reference repository > > > https://github.com/vaquarkhan/aegis-mcp-gateway (private for now, access > > > on > > > request) > > > License > > > Apache License 2.0 > > > Discussion list > > > general at incubator.apache.org > > > Abstract > > > Aegis MCP Governance Gateway is a vendor-neutral, secure-by-default Model > > > Context Protocol (MCP) gateway that puts one governed, auditable control > > > plane in front of many Apache data and streaming engines, so an AI agent > > > never talks to Flink, Kafka, Spark, or Iceberg directly. Every agent > > action > > > passes one fail-closed governance chain: secure-by-default tool exposure, > > > per-caller authentication and scope, a policy decision point, approval > > for > > > destructive operations, egress and server-side request forgery defense, > > > rate limiting, circuit breaking, prompt-injection screening, output > > > bounding and redaction, tool-catalog integrity, and a tamper-evident > > > hash-chained audit trail. Engines plug in through a small service > > provider > > > interface and contribute only tool definitions and backend calls. > > > Proposal > > > We propose to develop, under Apache governance, a Layer 7 policy MCP > > > gateway plus a family of engine adapters. A proxy forwards bytes and a > > > router dispatches by capability, but neither is a trust boundary. A > > gateway > > > is: it knows the tool, the principal, the budget, and the policy for > > every > > > call, and it can deny, redact, gate, and record. The gateway core is > > > stateless at the protocol edge so it can scale behind an ordinary load > > > balancer once shared governance state is externalized, and it is designed > > > to align with OAuth 2.1, SPIFFE, SLSA, OpenTelemetry, and the NIST AI > > Risk > > > Management Framework and ISO/IEC 42001. > > > The initial contribution is a working Java reference implementation: a > > > reusable gateway core, four in-depth engine adapters (Flink, Kafka, > > Spark, > > > Iceberg), and a further set of thinner adapters for the wider Apache data > > > platform. During incubation the priorities are community building, IP > > > clearance, and promoting the roadmap items to full implementations. > > > Background > > > The Model Context Protocol has become a common way for AI agents to reach > > > tools and data. As organizations connect agents to distributed Apache > > data > > > estates, an agent that needs Flink, Kafka, Spark, and Iceberg must either > > > drive each engine control plane directly or connect to a sprawl of > > > independent, mostly read-only MCP servers, each with its own security > > > posture, no shared audit, and no unified policy. > > > An MCP server that can reach a control plane (a JobManager REST API, a > > > Kafka AdminClient, a SQL gateway, an Iceberg catalog) is a privileged > > > client. Shipping one safely is effectively shipping an access-control > > > product: authentication, authorization, approval for destructive actions, > > > redaction of secrets and personal data, rate and cost limits, and > > forensic > > > audit. Many existing options lack these controls and carry > > > single-maintainer sustainability risk. A single governed gateway, > > developed > > > in the open under neutral governance, gives the ecosystem a shared > > standard > > > for how agents operate Apache systems safely. > > > Rationale > > > A single governed gateway resolves the problems of the decentralized > > > approach: > > > > > > - Security. One place enforces identity, authorization, approval, > > > redaction, egress, and audit, instead of each server reimplementing or > > > omitting them. > > > - Maintainability. Adding an engine is a small adapter, not a new > > > security model. There is one chain to review and harden, so the > > security > > > review surface stays flat as engines are added. > > > - Governance. One control plane for policy, budget, and provenance, > > with > > > a stable deny taxonomy so operators can alert on codes such as > > > POLICY_DENIED or APPROVAL_REQUIRED across every engine with one rule. > > > > > > Doing this under Apache governance keeps it neutral across vendors and > > > engines rather than leaving agent access to fragmented single-vendor > > > servers. > > > Initial goals > > > > > > 1. Establish the project and practice the Apache Way: transparency, > > > consensus, meritocracy, and community over code. > > > 2. Complete IP clearance for the existing code and land it under > > > Apache-2.0 with correct LICENSE and NOTICE files. > > > 3. Publish and stabilize the adapter service provider interface so > > > per-engine adapters are small, reviewable contributions. > > > 4. Promote the roadmap items to full implementations: externalized > > > high-availability state, CIMD and SPIFFE identity, a full Cedar > > runtime, > > > vault-backed credential exchange, and an OpenTelemetry SDK bridge. > > > 5. Produce the first Apache source releases and grow a diverse, > > > multi-organization contributor base. > > > > > > Current status > > > Aegis is a new codebase with a working multi-module reference > > > implementation. It is not yet a community. The design, threat model, > > > low-level design, and standards research are complete and public. The > > code > > > builds green and is tested. A published design-conformance matrix maps > > > every design claim to Done or Roadmap, so the maturity line is explicit. > > > Verified build and test state > > > > > > - 36 Maven modules: a gateway core, adapters for 33 Apache engines > > (four > > > in-depth today, Flink, Kafka, Spark, and Iceberg, and 29 thinner > > > integrations covering Cassandra, HBase, Hive, Pulsar, Airflow, NiFi, > > > Druid, > > > Pinot, Doris, Impala, Calcite, Hudi, Paimon, Kudu, Ignite, BookKeeper, > > > Ozone, Hadoop, ZooKeeper, Solr, Superset, Arrow Flight, Beam, Storm, > > > Flume, > > > ActiveMQ, CouchDB, Ranger, and Atlas), and a shaded runnable > > > distribution. > > > These are engine adapters the gateway supports, which is distinct from > > > external adopters. > > > - Builds with JDK 21 and runs on JDK 17 or newer with Maven 3.9 or > > > newer. > > > - 309 unit and integration tests across all modules, all passing, zero > > > failures and zero errors on a clean build. > > > > > > Implemented in 0.1.0 > > > > > > - The governance chain with ordered steps and first-denial-wins > > > semantics. > > > - Identity: shared bearer and a hashed multi-caller token file with > > > per-caller scope, and an OAuth 2.1 resource-server mode with JWKS > > > validation. Constant-time secret comparison and fail-closed startup > > > validation. > > > - Approvals: HMAC-SHA256 signed, TTL-bounded, single-use, > > > tool-and-scope-bound tokens with replay rejection. > > > - Policy: a builtin decision point, a live fail-closed OPA HTTP > > decision > > > point, and a Cedar-style deny-file or HTTP delegate. > > > - Per-caller outbound credential pass-through so a downstream engine > > can > > > see a per-caller Authorization value when one was bound at admission. > > > - Egress and server-side request forgery defense: allow list plus > > > unconditional deny of cloud metadata and non-routable ranges, > > including > > > numeric IP encodings and resolved addresses, with redirects disabled > > on > > > outbound clients. > > > - Rate limiting, circuit breaking, per-tool timeout, read-only SQL > > > guard, identifier validation, output bounding, and > > data-loss-prevention > > > redaction. > > > - Tool-catalog integrity with durable pins and a fail-closed switch, a > > > tamper-evident hash-chained audit, Prometheus metrics, and > > GenAI-shaped > > > span log lines. > > > - Two transports: stdio and authenticated Streamable HTTP with TLS, > > plus > > > health, readiness, and metrics endpoints. A CycloneDX software bill of > > > materials is produced at build time. > > > > > > Deferred to the roadmap (honestly labeled, not sold as implemented) > > > > > > - CIMD and SPIFFE authentication modes (they refuse to start today). > > > - A full Cedar policy runtime, and vault-backed or token-exchange > > > credential resolution. > > > - Externalized high-availability state (shared nonce, rate, and audit > > > stores). Until then, run a single replica; multi-replica replay > > > protection > > > and rate limits require the shared store. > > > - The MCP Tasks extension for long-running jobs, an OpenTelemetry SDK > > > bridge, a full Merkle and signature-based reconciliation proof, and > > true > > > semantic routing and caching. > > > > > > These are tracked in the roadmap and the design-conformance matrix that > > > ship with the code. > > > Meritocracy, community, core developers, alignment > > > The project will operate meritocratically from day one, with technical > > > decisions on the dev list and committership earned through contribution. > > > The community is currently centered on the initial author, Viquar Khan, > > who > > > has around 23 years in software, has worked in the open since 2013, and > > has > > > contributed to Apache Spark, Apache Iceberg, Apache Kafka, and other > > > open-source projects. The author also maintains MCP-related open-source > > > packages that have together been downloaded on the order of half a > > million > > > times, so there is an existing user base to draw contributors from. > > Growing > > > a committer base across the Flink, Kafka, Spark, and Iceberg communities > > > and across employers is the primary goal of incubation. Aegis consumes > > > public engine interfaces only and does not fork or modify any engine > > core. > > > Known risks > > > Orphaned products > > > Low to moderate. Agentic access to data systems is a growing need and the > > > governance gap is real, so demand is unlikely to vanish. The initial > > author > > > has maintained open-source projects since 2013 rather than abandoning > > them, > > > and the MCP-related packages already have a real user base, which lowers > > > the abandonment risk. Mitigation: build a multi-organization community, > > > keep the core small and standards aligned, and keep adapters independent > > so > > > the project survives the departure of any single contributor. > > > Inexperience with open source > > > Low. This is not the situation here. The initial author has about 23 > > years > > > in software, has worked in open source since 2013, and has contributed to > > > Apache Spark, Apache Iceberg, Apache Kafka, and other projects, so the > > > norms of public development, code review, and releases are familiar. > > > Mentors will still help with the ASF-specific mechanics of voting, board > > > reports, and IP clearance, which are process details rather than a gap in > > > open-source experience. > > > Homogenous developers > > > This is the real risk and we state it plainly: the podling has one > > > committer today. However, we are in active discussions with potential > > > co-committers and expect to formally add them to this proposal in the > > > coming days. What further reduces this risk is that the author is > > already > > > embedded in the exact communities this project draws from (Spark, > > Iceberg, > > > Kafka) and has an existing user base for the MCP packages, so there is a > > > concrete pool to recruit from rather than a cold start. Mitigation: > > recruit > > > committers from more than one organization before graduation, shape the > > > adapter contract so per-engine contributions are natural, and use > > > good-first-issue and adapter-sized work to onboard newcomers. Growing the > > > committer base is the main reason we are here. > > > Reliance on salaried developers > > > Low current reliance; the work to date is largely independent. The > > project > > > will attract contributors with varied motivations and employers so no > > > single company can stall it. > > > Relationships with other Apache products > > > Aegis consumes the public interfaces of Flink, Kafka, Spark, Iceberg, and > > > other Apache data systems and does not fork or modify them. We will > > > coordinate on the relevant dev lists and will not duplicate or compete > > with > > > an engine that ships its own MCP surface; where an engine has one, Aegis > > > composes with it. Aegis is not proposed for any engine core. > > > Excessive fascination with the Apache brand > > > The project seeks Apache governance for its community model and > > neutrality, > > > not for the brand. Until and unless accepted, it uses a neutral name and > > > does not use the Apache marks or the feather. The breadth of engine > > > adapters reflects a common governance surface, not an attempt to claim > > the > > > ecosystem; only four adapters are in-depth today and the rest are > > > explicitly thin. > > > Documentation > > > Public documentation exists and will be maintained in the repository: an > > > end-to-end design with architecture decision records, a low-level design > > > with architecture, sequence, and state diagrams, a threat and governance > > > model, a design-conformance matrix, standards research, and operations > > and > > > adapter guides. > > > Initial source > > > The initial source is the Java multi-module gateway described above, > > > currently in a private repository at > > > https://github.com/vaquarkhan/aegis-mcp-gateway under the initial > > author. > > > Access is available on request to anyone reviewing this proposal, and the > > > author will make it public before the vote if the community prefers to > > read > > > the code first. It includes the gateway core, the four in-depth adapters, > > > the thinner adapters, the shaded distribution, runnable examples, and the > > > documentation set. On acceptance it will be donated under a Software > > Grant > > > and made public. > > > Source and intellectual property submission plan > > > The initial author will submit the code under a Software Grant Agreement > > > and complete IP clearance early in incubation, with ICLAs from every > > > committer. Every source file will carry the standard Apache-2.0 header > > with > > > correct LICENSE and NOTICE files; any premature ASF-ownership wording > > will > > > be corrected to the standard header at donation time. > > > Transparency on tooling: parts of the codebase were developed with > > > AI-assisted coding tools under the author's direction and review. The > > > author asserts authorship and the right to contribute the work, and will > > > confirm provenance and authorship during IP clearance so the grant is > > > clean. We raise this proactively so the IPMC can review it rather than > > > discover it. > > > External dependencies > > > Runtime dependencies are Apache-2.0 or compatible, including the MCP Java > > > SDK, a servlet container for the HTTP transport, SLF4J, Logback, and > > Nimbus > > > for JOSE and JWT, with JUnit for tests. Per-engine adapters add that > > engine > > > to the official client, for example the Kafka clients library. A full > > > dependency review and a CycloneDX software bill of materials are > > produced. > > > No GPL or ASF category X dependency is introduced into the core. > > > Cryptography > > > Aegis uses standard, library-based cryptography: HMAC-SHA256 for approval > > > tokens, TLS for the HTTP transport, and JSON Web Token validation for > > > OAuth. Signature-based integrity is an optional gate. The project will > > > comply with ASF export notification requirements. > > > Required resources > > > Mailing lists > > > > > > - [email protected] > > > - [email protected] > > > - [email protected] > > > > > > Source control > > > > > > - A Git repository under the ASF (gitbox with a GitHub mirror). > > > > > > Issue tracking > > > > > > - ASF Jira or GitHub issues per current ASF practice. > > > > > > Other > > > > > > - Website and wiki space, and continuous integration through ASF > > > infrastructure. > > > > > > Initial committers > > > > > > - Viquar Khan *(Note: We are in active conversations with potential > > > contributors and expect to add a few additional initial committers > > here > > > in > > > the next few days before moving to a formal vote.)* > > > > > > - To be recruited before and during incubation: committers from the > > > Flink, Kafka, Spark, and Iceberg communities and from additional > > > organizations. Adapter-sized work is designed to make these > > > contributions > > > natural. > > > > > > Affiliations > > > To be completed as committers are confirmed. Diversity of affiliation is > > an > > > explicit goal; the current single-affiliation status is the main risk > > being > > > addressed. > > > Sponsors > > > Champion > > > To be identified. An ASF Member Champion will help finalize this > > proposal, > > > recruit mentors, and shepherd the discussion and vote on the Incubator > > > general list. > > > Nominated mentors > > > Three or more IPMC members to be recruited, ideally including > > participants > > > from the Flink, Kafka, and Iceberg communities. > > > Sponsoring entity > > > The Apache Incubator PMC. > > > Roadmap during incubation > > > Phase > > > Focus > > > Bootstrap > > > IP clearance, name search, infra, initial committers, first dev-list > > > decisions > > > 0.2 > > > Externalized HA state, vault-backed credentials, OAuth multi-issuer, OTel > > > SDK bridge, supply-chain CI (enforcer, CVE scan, signing) > > > 0.3 > > > CIMD and SPIFFE identity, full Cedar runtime, MCP Tasks, signature-based > > > reconciliation proof, semantic routing and caching > > > Graduation readiness > > > Diverse community, cadence of Apache releases, healthy governance > > > References > > > > > > - Apache Incubator proposal guide: > > > https://incubator.apache.org/guides/proposal.html > > > - Apache Incubator Cookbook: https://incubator.apache.org/cookbook/ > > > - The Apache Way: https://www.apache.org/theapacheway/ > > > - Model Context Protocol: https://modelcontextprotocol.io/ > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
