LuciferYang commented on PR #58424: URL: https://github.com/apache/spark/pull/58424#issuecomment-5548572524
The conf doc's "proven to be the same repeatable relation" reads like a general capability, but isRowBagRepeatable only admits stock ParquetFileFormat (exact getClass match), Range and LocalRelation. ORC/CSV/JSON, DataSourceV2 and cached tables are all silently excluded: users enable the flag, EXPLAIN does not change, and nothing tells them why. The boundary comes from validation scope and deserves to be stated. Name the supported sources in the conf doc and the PR description, and consider a DEBUG log saying why the whitelist rejected the plan when the shape matches (that state only becomes observable once the shape match is moved ahead of the audit, per my other comment); the rule runs in fixedPoint batches, so log once per subquery rather than on every rejection. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
