github-actions[bot] commented on code in PR #64009:
URL: https://github.com/apache/doris/pull/64009#discussion_r3456461461
##########
fe/fe-core/src/main/java/org/apache/doris/fs/FileSystemFactory.java:
##########
@@ -84,25 +89,42 @@ public static void
initPluginManager(FileSystemPluginManager manager) {
*/
public static org.apache.doris.filesystem.FileSystem
getFileSystem(Map<String, String> properties)
Review Comment:
This applies the mutable S3 deterministic-list configs only when a
filesystem is first constructed, but `SpiSwitchingFileSystem` caches the
resulting instance by `StorageProperties` for the catalog lifetime. HMS/Iceberg
catalogs create one `SpiSwitchingFileSystem`, and `forPath()` uses
`cache.computeIfAbsent(sp, props -> FileSystemFactory.getFileSystem(props))`,
so after the first S3 path is opened, later `ADMIN SET FRONTEND CONFIG
s3_skip_list_for_deterministic_path = false` or `s3_head_request_max_paths =
...` will not affect that catalog's cached filesystem. The earlier cache-key
fix only covered `FileSystemCache`; this separate per-catalog cache still
serves stale deterministic-list behavior. Please include the effective runtime
values in this cache key, clear/bypass this cache for S3-compatible filesystems
when the mutable configs change, or otherwise ensure the cached instance is
rebuilt, and add a test for the config flip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
