lisasgoh commented on issue #7075: URL: https://github.com/apache/arrow-rs/issues/7075#issuecomment-2635059503
The use case here would be that unlike Datafusion that has a `register_object_store` method that allows for custom implementations for object stores, Polars doesn't (https://github.com/pola-rs/polars/issues/20568). I'm looking to contribute to polars to add support for it, but the main blocker is the lack of an existing ObjectStore FFI. The idea I have is to have a Python API `register_object_store(path: str, method: str, scheme: str)` in Polars (similar to DF) that accepts the path to the library with the custom object store, a method name and a url scheme. This will call into Rust via PyO3 which would dynamically load that library via libloading and call the method name which would return an instance of the object store. The main issue here is that there's no ObjectStore FFI at the moment AFAIK, which makes it tricky to implement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
