comphead commented on PR #3842: URL: https://github.com/apache/datafusion-comet/pull/3842#issuecomment-4185404238
i tried options with overriding compute and `super.compute(`) returns a lazy iterator (CometExecIterator). At that point, no data has been read yet — bytes_scanned is still 0. The metrics only get populated as the iterator is consumed by downstream operators, which happens after `compute()` override returns. I had to apply the logic to `CometNativeExec` with the guard to apply metrics only for tasks started with native scans. `CometNativeScanExec.doExecuteColumnar` by some reason is not called for native scans, which might require another investigation ticket(@mbutrovich WDYT?) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
