Hi y'all! Obviously we just had a long discussion on the behavior of a URI within an Iceberg's File Type and I wanted to make sure we continue the discussion and try to continue making progress before we have another follow up meeting.
The TLDR for those who weren't attending: we are debating the File Type Proposal <https://docs.google.com/document/d/1BdA4G-38GoVcpX6UY4UI-QA1MvrDnH6uSKeCJBadILQ/edit?tab=t.pnp3scifr79p#heading=h.v4a92wm7mz0o> and the corresponding spec PR <https://github.com/apache/iceberg/pull/17918> The key issue we want to continue discussing UDI subfield of the File type. In parquet this field can be persisted as either a relative or absolute URI. Some community members are worried about having both representations which led to today's discussion. I want to open up the floor here for folks who have opinions on the behavior here. So the major questions (and my responses) are: *1. Can a URI be relative in an Iceberg Table?* My response to this is yes. I don't think we gain much by forbidding it, and implementation wise I don't think we can reasonably stop it. Users can apply a constraint if they need this to be true. *2. On read, should Iceberg mandate converting that persisted relative value to an absolute URI, or should it represent the value exactly as read?* I think every reader should return the value as is. I can't stop an engine from showing the user a different representation, but the bytes should be carried through as is in the file imho. *3. How should maintenance operations operate by default when a file column is present?* By default, if we lack a mechanism for indicating ownership, we should assume all path references are not owned by the table. *4. Should the Spec or the implementation notes specify how tables should express ownership of the files they reference?* I think this is probably something useful that we should lay out, but I don't think it needs to be decided for this proposal. Please follow on if you have any other questions for debate or comments on these topics.
