alamb commented on issue #6809: URL: https://github.com/apache/arrow-datafusion/issues/6809#issuecomment-1616600843
> Would bytes_scanned not be equal/close to the download amount? I think so > How can i get this information using the Rust API? You can access the metrics using https://docs.rs/datafusion/latest/datafusion/physical_plan/trait.ExecutionPlan.html#method.metrics You can walk the ExecutionPlan (either before/during/after execution) and find the relevant ParquetExecNode Here is some code in IOx that walks these metrics and does something with them (in this case, converts them into "tracing spans" format). Perhaps that is helpful: https://github.com/influxdata/influxdb_iox/blob/4a1f8db2546d867c759f76ab2a2b2b7c8f3dac9c/iox_query/src/exec/query_tracing.rs#L93-L214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
