dengc367 opened a new issue, #14351:
URL: https://github.com/apache/gravitino/issues/14351

   
   ### Version
   
   main branch
   
   ### Describe what's wrong
   
   Opening a Paimon catalog whose warehouse is on Aliyun OSS through the Flink 
connector fails with
   `Aliyun OSS endpoint should not be null or empty`, even though 
`fs.oss.endpoint`,
   `fs.oss.accessKeyId` and `fs.oss.accessKeySecret` are set on the Gravitino 
catalog.
   
   `GravitinoPaimonCatalog#open()` moves every `fs.*` option into the Hadoop 
`Configuration` before
   creating the inner Paimon catalog (#12121). That is correct for 
`HadoopFileIO` and `S3FileIO`,
   which read the Hadoop configuration. Paimon's `OSSFileIO`, the default 
`FileIO` for OSS
   warehouses, instead builds its Hadoop configuration only from 
`CatalogContext#options()` and
   never reads `CatalogContext#hadoopConf()`. After the move the `fs.oss.*` 
entries are no longer in
   the options, so `OSSFileIO` sees an empty endpoint.
   
   The same warehouse works when the Paimon catalog is configured directly in 
Flink, which is what
   makes the connector look broken rather than misconfigured.
   
   ### Error message and/or stacktrace
   
   ```
   java.lang.IllegalArgumentException: Aliyun OSS endpoint should not be null 
or empty.
       at org.apache.paimon.fs.FileIO.getOptions(FileIO.java)
       at org.apache.paimon.oss.OSSFileIO.<init>(OSSFileIO.java)
   ```
   
   ### How to reproduce
   
   1. Create a Paimon catalog in Gravitino with 
`warehouse=oss://<bucket>/<path>` and the OSS
      endpoint/key options, using the Flink connector on Flink 
1.19/1.20/2.1/2.2.
   2. Open the catalog from Flink (`USE CATALOG <catalog>;` or any DDL against 
it).
   3. The catalog fails with the error above.
   
   ### Additional context
   
   Only the `fs.oss.*` options are affected. The fix is to copy them back into 
the inner catalog's
   option map after the Hadoop move, leaving every other filesystem on the 
Hadoop configuration.
   I have the patch and a test ready and will open a PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to