abstractdog opened a new issue, #27579: URL: https://github.com/apache/datafusion/issues/27579
`FileMetadataCache` is keyed by the file's `object_store::path::Path`, which is only unique within one object store, while one cache is shared by every store of a `RuntimeEnv`. Entries are validated only by size and last-modified time, so files with the same in-store path, size and mtime in different stores (e.g. `s3://bucket-a/t/x.parquet` and `s3://bucket-b/t/x.parquet`) are served each other's Parquet metadata (schema, statistics, row-group offsets), leading to wrong pruning or decode errors. This also blocks sharing one file metadata cache across runtimes, e.g. an executor-lifetime cache in Ballista executors that serve many sessions reading different buckets. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
