MonkeyCanCode commented on issue #827:
URL: https://github.com/apache/polaris/issues/827#issuecomment-2603436988
This is reproduce-able on my local as well when using backend with FILE. I
recalled this was working before (but I don't recalled if I did this one on
SaaS one or OSS one). Then to get the local OSS one working, you will need to
make the changes to the following config:
```
featureConfiguration:
...
ALLOW_UNSTRUCTURED_TABLE_LOCATION: true
...
```
This is because the following code (`PolarisStorageConfigurationInfo.java`):
```
boolean allowEscape =
CallContext.getCurrentContext()
.getPolarisCallContext()
.getConfigurationStore()
.getConfiguration(
CallContext.getCurrentContext().getPolarisCallContext(),
catalog,
PolarisConfiguration.ALLOW_UNSTRUCTURED_TABLE_LOCATION);
if (!allowEscape
&& catalog.getCatalogType() != Catalog.TypeEnum.EXTERNAL
&& baseLocation != null) {
LOGGER.debug(
"Not allowing unstructured table location for entity:
{}",
entityPathReversed.get(0).getName());
return new StorageConfigurationOverride(configInfo,
List.of(baseLocation));
} else {
LOGGER.debug(
"Allowing unstructured table location for entity: {}",
entityPathReversed.get(0).getName());
List<String> locs =
userSpecifiedWriteLocations(entityPathReversed.get(0).getPropertiesAsMap());
return new StorageConfigurationOverride(
configInfo,
ImmutableList.<String>builder()
.addAll(configInfo.getAllowedLocations())
.addAll(locs)
.build());
```
When `ALLOW_UNSTRUCTURED_TABLE_LOCATION` is not set, it won't read
additional allowed locations to begin with via this abstraction. Thus the
following will keep failing (`BasePolarisCatalog.java`):
```
private void validateLocationsForTableLike(
TableIdentifier identifier,
Set<String> locations,
PolarisResolvedPathWrapper resolvedStorageEntity) {
Optional<PolarisStorageConfigurationInfo> optStorageConfiguration =
PolarisStorageConfigurationInfo.forEntityPath(
callContext.getPolarisCallContext().getDiagServices(),
resolvedStorageEntity.getRawFullPath());
optStorageConfiguration.ifPresentOrElse(
storageConfigInfo -> {
Map<String, Map<PolarisStorageActions,
PolarisStorageIntegration.ValidationResult>>
validationResults =
InMemoryStorageIntegration.validateSubpathsOfAllowedLocations(
storageConfigInfo, Set.of(PolarisStorageActions.ALL),
locations);
validationResults
.values()
.forEach(
actionResult ->
actionResult
.values()
.forEach(
result -> {
if (!result.isSuccess()) {
throw new ForbiddenException(
"Invalid locations '%s' for identifier
'%s': %s",
locations, identifier,
result.getMessage());
} else {
LOGGER.debug(
"Validated locations '{}' for
identifier '{}'",
locations,
identifier);
}
}));
...
```
When the setting is not set, the `PolarisResolvedPathWrapper` will have the
right settings but the abstraction `optStorageConfiguration` that is building
on top won't have the right setting for allowed location. Thus failing the
check later on.
From the code, it is not storage specific (as there nothing preventing GCS
to failed....from the above, it is tested with FILE only and I recalled I got
it working with AWS S3 long time back).
Please give this a try and let me know how it goes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
