LuciferYang commented on issue #2837:
URL: https://github.com/apache/iceberg-rust/issues/2837#issuecomment-5034463205
I hit this one with a VCF → Iceberg pipeline (a column named `pos`), dug
into the root cause, and opened #2865 for it.
The fix guards `is_metadata_column_name` with a `starts_with('_')` check, so
the two bare delete-file column names `pos`/`file_path` are no longer treated
as projectable data-table metadata columns, while `get_metadata_field_id` keeps
mapping them to their reserved ids for the delete read path. This matches the
spec's `_`-prefix reservation and what the Java reader does.
One design choice I'd like your take on: this is the minimal fix, on the
scan consumer side. The root cause is that `pos`/`file_path` sit in the shared
name→id map at all, so `get_metadata_field_id("pos").is_ok()` still returns
true and the two public predicates now disagree. Removing those two names from
the name→id map would be the deeper fix and looks safe in-repo (the delete read
path uses the id→field direction, which stays untouched), but it changes a
second public function's behavior, so I left it out. I noted the tradeoff in
the PR — happy to go deeper if you'd prefer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
