nevzheng opened a new issue, #12677:
URL: https://github.com/apache/gravitino/issues/12677
### What would you like to be improved?
Gravitino does not have a shared security contract connecting a property's
canonical definition to every server-side consumer of that property.
Entity properties are generally represented as `Map<String, String>`. The
current metadata model can mark a `PropertyEntry` as `hidden`, but
`PropertiesMetadata.isHiddenProperty()` only protects a key when the consuming
metadata includes a matching exact or prefix definition. If no definition is
selected, the property follows ordinary compatibility behavior and remains
visible.
Fileset catalogs demonstrate this gap. Provider metadata classifies
properties such as `s3-secret-access-key`, but Fileset catalog metadata does
not select the shared S3/GCS/Azure/OSS definitions. The hidden-property lookup
therefore cannot consult the provider's classification, allowing an inline
credential to cross a normal entity response boundary.
This is broader than one Fileset property. Property maps are consumed by
REST responses, events, errors, logs, connector runtime paths, and other
server-side integrations. Every consumer currently has to select the correct
metadata and independently apply the intended handling.
Entity Secrets addresses a complementary problem. It provides opt-in secret
bindings/references, persistence, and resolution. It does not classify legacy
inline values or define how all property representations should be handled
across different runtime and disclosure contexts.
Gravitino should provide an additive property-security contract that:
- lets the canonical property definition declare security semantics once;
- keeps classification independent of inline or Entity Secrets-backed
storage;
- provides deterministic safe projections for API and observer boundaries;
- provides selected material access for trusted runtime operations;
- preserves compatibility for existing ordinary and `hidden` properties; and
- supports incremental migration by property and consumer.
The first vertical slice should apply the contract to Fileset cloud-provider
properties and close #11642.
### How should we improve?
Introduce an additive, server-side property contract with three cooperating
parts:
1. **Property definition**
Extend canonical `PropertyEntry` definitions with reviewed security
profiles such as `ORDINARY`, `SENSITIVE`, `SECRET`, and `INTERNAL`, plus an
optional coarse semantic descriptor such as `OPAQUE`.
Existing `hidden=true` declarations should retain their current behavior
through an internal legacy compatibility profile. Other existing and undeclared
custom properties should continue to behave as ordinary properties unless
explicitly classified.
2. **Context-aware access decision**
Bind loaded property values to the definitions explicitly selected by the
consumer. Evaluate each property using a structured context containing a typed
purpose and surface, with named presets for common Gravitino operations.
The decision should produce a small set of actions such as:
- expose the stored value;
- omit the property;
- return a fixed, content-independent replacement;
- report presence without material; or
- permit selected runtime materialization.
This context describes how the property is being used. It is not a new
authorization or per-field ACL system.
3. **Default execution library**
Provide a core projector that executes these actions and returns an
immutable, context-specific result suitable for serialization or observation.
Consumers should not reimplement masking or filtering.
Projection must be pure: it must never resolve Entity Secrets handles or
fall back to returning raw values when selected metadata cannot be evaluated.
Trusted runtime consumers should use a separate selected-key
materialization operation. Inline material can be supplied directly, while
recognized Entity Secrets handles can be resolved through an adapter to the
existing `SecretManager`. Existing `getSecrets()` behavior should remain
unchanged until its owner reviews any broader integration.
Implementation should proceed as independently valid vertical slices. The
first slice should:
- compose the applicable S3, GCS, Azure, and OSS definitions for Fileset
catalogs;
- review each provider property's classification rather than assuming every
provider value is secret;
- apply the shared projection to Fileset create, load, and list responses;
- preserve the existing fixed replacement and write-rejection compatibility
behavior for legacy hidden properties; and
- verify that trusted connector runtime configuration still receives the
required material.
#### Acceptance criteria
- A canonical property definition can declare one reviewed security profile
independently of its storage representation.
- Ordinary projections never return classified secret material or Entity
Secrets handles.
- Secret replacement is fixed and content-independent; it does not reveal
the value's length or structure.
- Presence, omission, replacement, absence, and projection failure remain
distinguishable states.
- Selected metadata composition failures produce value-free internal errors
rather than raw-map fallback.
- Undeclared custom properties retain ordinary compatibility behavior.
- Legacy `hidden=true` properties retain their existing outward behavior
while supporting incremental migration.
- The Fileset S3 reproduction is covered by server-side integration tests.
- Tests cover inline secrets, recognized Entity Secrets handles, legacy
hidden properties, undeclared properties, metadata composition failures,
create/load/list responses, and trusted runtime access.
#### Out of scope
- Per-principal property ACLs or new authorization rules
- Requiring all secrets to use Secret Manager
- Changing the public `getSecrets()` authorization or response contract
- Protecting the metadata database, backups, or historical disclosures
- Relying on UI or client-side redaction as the security boundary
#### Related work
- #11642 — Fileset catalog provider-property metadata gap
- #11644 — Earlier connector-specific credential exposure
- #12252 — Entity Secrets SPI, REST contract, and in-memory provider
- #12457 — `getSecrets()` for catalog, schema, and Fileset connection secrets
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
