dengc367 opened a new pull request, #14352:
URL: https://github.com/apache/gravitino/pull/14352
### What changes were proposed in this pull request?
`GravitinoPaimonCatalog#open()` now copies the `fs.oss.*` entries of the
Hadoop `Configuration`
back into the inner Paimon catalog's option map after
`movePaimonStorageOptionsToConf` has moved
them out.
Paimon's `OSSFileIO` builds its Hadoop configuration only from
`CatalogContext#options()` and
never reads `CatalogContext#hadoopConf()`, so those entries have to be
visible in both places.
Only the `fs.oss.` prefix is copied; every other filesystem keeps using the
Hadoop configuration
and the connector's own option map does not gain the keys back.
### Why are the changes needed?
Fixes #`14351`
Without this, a Paimon catalog backed by Aliyun OSS cannot be opened through
the Flink connector:
`OSSFileIO` reports `Aliyun OSS endpoint should not be null or empty`
because the endpoint and
credentials were moved to a place it does not read.
### Does this PR introduce _any_ user-facing change?
No API change and no new property key. Paimon catalogs on OSS start working
again; catalogs on
HDFS, S3 and local storage are unaffected.
### How was this patch tested?
- Added assertions to `TestGravitinoPaimonCatalog` covering the OSS option
hand-off; it runs for
every supported Flink version (1.19, 1.20, 2.1, 2.2).
`./gradlew :flink-connector:flink-2.2:test --tests
"*TestGravitinoPaimonCatalog*"`
- `./gradlew :flink-connector:flink-common:spotlessCheck`
- Manually against a real OSS warehouse on Flink 2.2.1: create catalog,
create a primary-key
table, upsert, read back, drop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]