danielcweeks commented on code in PR #17918: URL: https://github.com/apache/iceberg/pull/17918#discussion_r3981960207
########## format/spec.md: ########## @@ -321,6 +323,32 @@ For `geography` types, an additional parameter A specifies an algorithm for inte * `andoyer`: Thomas, Paul D. Mathematical models for navigation systems. US Naval Oceanographic Office, 1965. * `karney`: [Karney, Charles FF. "Algorithms for geodesics." Journal of Geodesy 87 (2013): 43-55](https://link.springer.com/content/pdf/10.1007/s00190-012-0578-z.pdf), and [GeographicLib](https://geographiclib.sourceforge.io/) +#### File Type + +A **`file`** represents a range of bytes that may be stored inline in the value or in an external file. The `file` type and its value semantics are defined by the `FILE` logical type in the [Parquet project](https://github.com/apache/parquet-format/pull/585). + +A `file` value has a fixed set of sub-fields. The sub-fields are implicit: they are not represented in the Iceberg schema and cannot be added, removed, reordered, or promoted. Their names, types, and field-ID offsets are: + +| Sub-field | ID offset | Type | +|----------------|-----------|----------| +| `uri` | +1 | `string` | +| `offset` | +2 | `long` | +| `size` | +3 | `long` | Review Comment: For File type, I don't think you would distinguish in-memory from on disk since it's just "bytes" effectively. How any in memory representation may inflate the contents of any arbitrary file data is entirely implementation dependent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
