rdblue commented on issue #359: Spec: Add file and position delete files URL: https://github.com/apache/incubator-iceberg/issues/359#issuecomment-615488965 Whatever is consuming the file can use its own object model. When Iceberg loads metadata files, we use classes that extend `IndexedRecord` so that Avro can construct them directly like a specific record. An example of this is `GenericDataFile`. However, there are cases where the metadata files are read by engines, like the metadata tables. When you read the `entries` metadata table, Iceberg creates tasks out of the manifest Avro files and Spark reads the data as InternalRow directly. Delete files would be similar to these cases. If the engine needs to read a delete file, it would be able to use its object model with Avro. But I think that we will need an `IndexedRecord` implementation for file/position deletes so that we can build a filter that merges that stream with a data file record stream.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
