rangareddy commented on issue #15114: URL: https://github.com/apache/hudi/issues/15114#issuecomment-5056928892
This issue was reviewed as part of the JIRA-migrated backlog triage. This describes an enhancement, not a bug: after Hudi COW relations moved from Spark's `HadoopFsRelation` to `BaseFileOnlyRelation`, Spark plans surface a generic scan node instead of a `FileScan`, so file-scanning metrics (# files scanned, total bytes scanned, etc.) are no longer reported. Spark decides FileScan vs generic DataScan based on `HadoopFsRelation`, which Hudi no longer uses. Status: no fix has been merged, and `BaseFileOnlyRelation` is still in use on current master, so the metrics gap remains. Keeping open as an unimplemented observability improvement. If anyone plans to pick this up, restoring the equivalent scan metrics on Hudi's custom relations is the ask. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
