[
https://issues.apache.org/jira/browse/RANGER-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramachandran Krishnan updated RANGER-5719:
------------------------------------------
Description:
Ranger audit events are produced to a shared Kafka topic
({{{}ranger_audits{}}}). In dynamic routing mode, Admin owns a canonical
partition plan: which plugin/service maps to which topic partitions, how buffer
partitions are used for new services, and which users may POST audits per
service repo. That plan must be represented, validated, and updated
consistently across Admin and the audit ingestor.
h3. In scope
Partition plan model
* JSON representation of the audit topic name, plan version, per-plugin
partition assignments, buffer pool, and per-service allow-list metadata.
Plan lifecycle helpers
* Allocation and validation when services are onboarded, offboarded, or
reconfigured.
* Append-only partition growth, buffer promotion, and version increments.
Routing utilities
* Mapping from Admin-assigned logical partition IDs to Kafka producer
partition indices, including safe behavior when live topic metadata differs
from the plan.
Allow-list normalization
* Parsing and normalization of per-repo allow-list values stored in the plan.
* Allow-list usernames are sourced from {{service.admin.users}} (not
{{{}policy.download.auth.users{}}}).
Admin configuration helpers
* Reading partition-count defaults and per-plugin overrides used when building
or expanding a plan.
was:
h2. Summary
Introduces shared libraries in {{agents-common}} and {{common-utils}} that
Admin, the Audit Ingestor, and plugins will use in follow-up PRs. This change
adds building blocks only; no end-to-end audit routing or SPIFFE enforcement is
active until later PRs land.
h3. Audit partition plan library
Adds a reusable partition-plan module for Admin-managed dynamic Kafka routing:
* Plan model — JSON representation of the audit topic, plan version,
per-plugin partition assignments, buffer pool, and per-service allow-list
derived from {{{}policy.download.auth.users{}}}.
* Plan lifecycle helpers — allocation and validation when services are
onboarded or offboarded, including buffer promotion and version bumps.
* Routing utilities — mapping from Admin-assigned partition IDs to Kafka
producer partition indices, including safe fallbacks when topic metadata lags.
* Plugin and service resolution — mapping Ranger service types to plugin IDs
and parsing service config used for ingestor authorization.
This keeps partition-plan logic in one place so Admin, ingestor, and tests
share the same contract.
h3. SPIFFE header auth utilities
Adds shared outbound auth helpers following the RANGER-5700 trusted-header
pattern:
* Header auth config — reads enable flag and SPIFFE header name from
plugin/site configuration.
* SPIFFE identity resolution — resolves workload identity from explicit
config, identity file, or standard workload identity path.
These utilities support plugin → ingestor and ingestor → Admin SPIFFE auth in
later PRs.
h3. What this PR does not include
* Database schema or seed data
* Admin REST APIs or plan persistence
* Ingestor poller, dynamic partitioner, or inbound SPIFFE filter
* Plugin install templates or audit destination changes
* Docker or E2E smoke tests
Depends on: nothing
Blocks: DB patch, Admin, Ingestor, and Plugin PRs in the RANGER-5655 series
> Shared partition-plan library and SPIFFE header auth utilities
> --------------------------------------------------------------
>
> Key: RANGER-5719
> URL: https://issues.apache.org/jira/browse/RANGER-5719
> Project: Ranger
> Issue Type: Sub-task
> Components: Ranger
> Reporter: Ramachandran Krishnan
> Assignee: Ramachandran Krishnan
> Priority: Major
> Fix For: 3.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Ranger audit events are produced to a shared Kafka topic
> ({{{}ranger_audits{}}}). In dynamic routing mode, Admin owns a canonical
> partition plan: which plugin/service maps to which topic partitions, how
> buffer partitions are used for new services, and which users may POST audits
> per service repo. That plan must be represented, validated, and updated
> consistently across Admin and the audit ingestor.
> h3. In scope
> Partition plan model
> * JSON representation of the audit topic name, plan version, per-plugin
> partition assignments, buffer pool, and per-service allow-list metadata.
> Plan lifecycle helpers
> * Allocation and validation when services are onboarded, offboarded, or
> reconfigured.
> * Append-only partition growth, buffer promotion, and version increments.
> Routing utilities
> * Mapping from Admin-assigned logical partition IDs to Kafka producer
> partition indices, including safe behavior when live topic metadata differs
> from the plan.
> Allow-list normalization
> * Parsing and normalization of per-repo allow-list values stored in the plan.
> * Allow-list usernames are sourced from {{service.admin.users}} (not
> {{{}policy.download.auth.users{}}}).
> Admin configuration helpers
> * Reading partition-count defaults and per-plugin overrides used when
> building or expanding a plan.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
