Hi JB, I'm also in favour of smaller configs in general. However, as far as storage integration is concerned, unfortunately, the differences among the multitude of providers require us to have some input about their feature set.
The input could come from admin users (the config suggestion, following the existing pattern in Polaris). Polaris can have reasonable defaults for most of the storage configs, but as user feedback shows, there are deployments where it may not work well, so we may still need input from the admin user in some special cases. We could also interrogate the service endpoints in runtime in order to discover what the backend is, but I'm not sure this approach is preferable. It will complicate code a lot IMHO (reminds me of PC bootloaders ;) ). Do people have other ideas about handling storage provider differences? Cheers, Dmitri. On Thu, Jan 22, 2026 at 3:29 AM Jean-Baptiste Onofré <[email protected]> wrote: > Hi Yong, > > I think it's reasonable to check using the endpoint URL. > We have to be sure it's consistent for users (to have the same check > behavior as before). If so, I don't think we need a new flag for that > (personally, I'm more in favor to limit at max the number of flags / > config). > > Regards > JB > > On Wed, Jan 21, 2026 at 4:43 AM Yong Zheng <[email protected]> wrote: > > > Hello all, > > > > While working on https://github.com/apache/polaris/issues/3440, I > noticed > > the way on how we are currently determining whether a S3-compatible > storage > > is a bit odd where we are determining this by checking if a region > property > > of catalog and account id of IAM role are being set. Now back to the > > reported issue, where the reporter is using assuming role with an > > S3-compatible backend without KMS and a catalog region property was set > (as > > we didn't mention this anywhere). By doing so, it falls back to the > > wildcard KMS policy which is not valid for certain S3-compatible storage > (I > > am assuming the reporter is using MinIO or something equivalent). > > > > In this case, both account id and region are set but they are actually > > valid settings (but according to the code, the comment said it should not > > be valid: > > > https://github.com/apache/polaris/blob/0b54f7046295ff19d434f9f0bd47b0b612be51a5/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L294 > > ). > > > > I think it may be better to determine is a S3-compatible storage is AWS > or > > not by looking at endpoint URL instead (sample PR: > > https://github.com/apache/polaris/pull/3496). Let me know what your guys > > think. > > > > Thanks, > > Yong Zheng > > >
