xanderbailey opened a new pull request, #2030: URL: https://github.com/apache/iceberg-rust/pull/2030
This PR introduces table-level encryption properties to enable configuration of encryption settings for Iceberg tables. These properties lay the groundwork for future encryption implementation while maintaining compatibility with the Java implementation's property names and structure. Table-level encryption is a critical security feature in Apache Iceberg's Java implementation. To support encryption in iceberg-rust and ensure interoperability between Java and Rust implementations, we need to start by adding the configuration properties that control encryption behavior. This PR adds the property definitions and parsing logic without implementing the actual encryption, keeping the change focused and reviewable. **Modified:** `crates/iceberg/src/spec/table_properties.rs` Added encryption-related properties to the `TableProperties` struct: - `PROPERTY_ENCRYPTION_KEY_ID` (`"encryption.key-id"`) - Master key ID for encrypting data encryption keys - `PROPERTY_ENCRYPTION_DEK_LENGTH` (`"encryption.data-key-length"`) - Data encryption key length (default: 16 bytes) - `PROPERTY_ENCRYPTION_AAD_LENGTH` (`"encryption.aad-length"`) - AAD prefix length for GCM (default: 16 bytes) - `PROPERTY_ENCRYPTION_KMS_TYPE` (`"encryption.kms-type"`) - KMS type (e.g., "aws", "gcp", "azure") All `Option<T>` as encryption is optional: - `encryption_key_id: Option<String>` - `encryption_dek_length: Option<usize>` - `encryption_aad_length: Option<usize>` - `encryption_kms_type: Option<String>` Extended `TryFrom<&HashMap<String, String>>` implementation to parse encryption properties Property names match exactly with Java's implementation: - Java: `TableProperties.ENCRYPTION_TABLE_KEY` → Rust: `PROPERTY_ENCRYPTION_KEY_ID` - Java: `TableProperties.ENCRYPTION_DEK_LENGTH` → Rust: `PROPERTY_ENCRYPTION_DEK_LENGTH` - Java: `CatalogProperties.ENCRYPTION_KMS_TYPE` → Rust: `PROPERTY_ENCRYPTION_KMS_TYPE` **Note:** Java's `ENCRYPTION_KMS_IMPL` property (for custom KMS implementations via reflection) is intentionally not included since Rust doesn't support runtime reflection. KMS implementations will be selected based on the `encryption.kms-type` property with compiled-in implementations. Added comprehensive test coverage: 1. `test_table_properties_default`: Verifies encryption properties are None by default 2. `test_encryption_properties_valid`: Tests parsing all encryption properties with valid values 3. `test_encryption_properties_partial`: Tests partial encryption configuration 4. `test_encryption_properties_invalid_numeric`: Verifies invalid numeric values are handled gracefully (parsed as None) 5. `test_encryption_properties_with_other_properties`: Tests encryption properties alongside existing table properties 1. **Optional Fields**: All encryption properties are `Option<T>` since encryption is an optional feature 2. **Silent Failure for Invalid Numbers**: Invalid numeric values for `dek_length` and `aad_length` are parsed as None rather than failing, matching the pattern for optional properties 3. **No Validation**: This PR doesn't validate property values (e.g., valid key lengths), leaving that for the encryption implementation 4. **No Custom KMS**: Omitted `encryption.kms-impl` property since Rust lacks reflection - KMS type selection will use `encryption.kms-type` with a factory pattern 5. **Independent PR**: No dependencies on other encryption code, can be merged independently This PR is part of a series to implement encryption support: - ✅ [PR 1: Core encryption primitives (AES-GCM operations)](https://github.com/apache/iceberg-rust/pull/2026) - ✅ PR 2: Table properties for encryption (this PR) - PR 3: Key management interfaces - PR 4: EncryptionManager implementation - PR 5: Native Parquet encryption support - PR 6: Integration with Table and FileIO ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes #. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
