tomtongue commented on code in PR #16859:
URL: https://github.com/apache/iceberg/pull/16859#discussion_r3819413240
##########
core/src/main/java/org/apache/iceberg/MetadataLogEntriesTable.java:
##########
@@ -119,6 +140,32 @@ private static StaticDataTask.Row metadataLogEntryToRow(
// latest snapshot in this file corresponding to the log entry
latestSnapshotId,
latestSnapshot != null ? latestSnapshot.schemaId() : null,
- latestSnapshot != null ? latestSnapshot.sequenceNumber() : null);
+ latestSnapshot != null ? latestSnapshot.sequenceNumber() : null,
+ properties);
+ }
+
+ private static Map<String, String> tablePropertiesResolver(
+ TableMetadata.MetadataLogEntry metadataLogEntry,
+ FileIO io,
+ TableMetadata current,
+ boolean skipPropertiesLoad) {
+
+ // Avoid loading metadata file when properties are not projected.
+ if (skipPropertiesLoad) {
+ return null;
+ }
+
+ // Reuse the already loaded current metadata.
+ if (metadataLogEntry.file().equals(current.metadataFileLocation())) {
+ return current.properties();
+ }
Review Comment:
This part adds two optimizations:
* First, if the metadata table query doesn't include the `properties` column
(e.g., `SELECT file FROM catalog.db.tbl.metadata_log_entries`),
`skipPropertiesLoad` is set to `true`, and no metadata files are loaded to
retrieve properties.
* Second, if the metadata log entry points to the current metadata file, the
properties from the already loaded current metadata are returned without
loading the metadata file again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
