RussellSpitzer edited a comment on pull request #2780:
URL: https://github.com/apache/iceberg/pull/2780#issuecomment-873671215
I think we probably could do better by just saying how Iceberg will handle
the predicates it gets, like for example it will use X & Y predicates in a
partition evaluation expression and Z as a Data file evaluation expression
Then we keep the "which partitions were touched" info as maybe an optional
"DEBUG" line in the logs?
So Job Description looks like
[Iceberg Scan - Pushed Partition Predicates [] - Pushed Data Predicates []]
Then in the logs you get
"Planned scan of table {} - {} X files to be read from {} Y partitions" -
INFO
"Planned scan of table {} - {} X files to be read from {partition list}" -
Debug
"Planned scan of table {} - {file list} " -- Trace
That way we only display this information at runtime in the logs, but we are
sure we don't prematurely calculate scans and read manifests or what not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
