crepererum commented on issue #7171: URL: https://github.com/apache/arrow-rs/issues/7171#issuecomment-2678054497
> > Depends. I would say "more low-level", but now you're in a different pickle because you now longer see the high-level methods or at least you partially have to recover them (e.g. is it an upload/put, a copy, a move). So I don't think the instrumentation on that level is universally better. > > My vague thought here is we might be able to do something with http::Extensions to propagate this context lower. That sounds like a compromise I could live with. --- That said, I don't think any of the proposals really gonna help eliminating wrappers. Looking at the hierarchy posted [here](https://github.com/apache/arrow-rs/issues/7171#issuecomment-2677992820), sure we could move the metrics into the IO layer. Chunking is already harder because now you again need to "reverse" the high-level intention from the low-level HTTP call. Racing might work. Caching not really. And putting everything into the DataFusion-specific wrappers (which might not be your only high-level consumer) just moves the problem or makes it even worse: now the entire code base uses the data fusion type instead of the `object_store` type, or you re-create the wrappers for every single high-level interface you have. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
