alamb commented on PR #17266:
URL: https://github.com/apache/datafusion/pull/17266#issuecomment-3215704465
> @alamb Awesome, I'm looking forward to hearing your feedback.
>
> Just so you can understand where my head was at for this (currently very
rough) implementation in terms of configuration:
> * I thought it would be beneficial to be able to entirely disable object
store profiling and use the default, non-instrumented, implementation by
default. Collecting profiling information takes time, additional allocations
etc. and in many cases it may not be desirable to incur the (likely quite
small, but non-zero) overhead from the profiling.
I was planning on adding a CLI flag to enable the instrumented store.
> * I haven't added any of the code around it yet, but I was going to the UX
from [Add memory profiling support to DataFusion CLI and memory pool metrics
#17021](https://github.com/apache/datafusion/pull/17021) as you suggested in
the issue for this ticket and allow users to apply either
`\object_store_profiling summary` or `\object_store_profiling trace`. The first
would print just a simple summary of calls, similar to my suggestion in the
initial issue, and the latter would print both the summary and the individual
call details as we see here.
>
> Any feedback on the above, or general code structure/implementation is
obviously welcome as well, so let me know your thoughts. This code is obviously
a minimally functional example, but I'd rather incorporate feedback early than
need to re-work a bunch of stuff!
Yes,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
