emkornfield commented on issue #13855: URL: https://github.com/apache/iceberg/issues/13855#issuecomment-3243764910
> As explained earlier, it might mislead us when using optional fields. Referring earlier response Can you explain how this would be misleading when using optional fields? > "However, max field id of linked schema id could help us in making lenient (with few % error rate) decisions which might not be correct for the places where decisions have to be made strictly." I'm not sure where places need to be made strictly? >> If we want to optimize the last 10% of the use case, I'd say it would like be better to link two columns: > I didn't get this two way linking. Can you please explain this? This was in response to a suggestion above about tracking "fields written". I don't think it is a good idea to pursue this. Instead I think starting with Schema ID, and then if necessary, we can add a separate field of field "fields not written" if necessary to solve the use-case. However, "field written" and fields not written", already has an overlaps with stats tracked so if it is important we might want to consider alternatives that use existing stats. In general, I think there are enough threads on in that it might pay to create a google doc to summarize the proposal they can all be tracked in one place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
