dramaticlly commented on PR #10784:
URL: https://github.com/apache/iceberg/pull/10784#issuecomment-5087281600
> Was chatting with @rahulsmahadev and @amogh-jahagirdar , as there's more
interest in this, @rahulsmahadev can work on this change. I think we can start
with repair stats + dry run in the first cut, as we nail the duplicate/missing
entry problem?
Thanks @szehon-ho , do you think it would be helpful to start with a
collaborative google doc?
I think like you mentioned, the scope can creep up so we might want to
agree on it first . If we get have answers to the following after running the
given action, that would be great!
1. if table is a good state with referential integrity? can model as dry
run, with option to choose either a full table or incremental from last known
good TableMetadata. If table is in a bad state, we shall provide a collection
of missing files identifiable by the path.
2. repair the existing problematic statistics: not sure if we want to limit
to only the data files with column level stats or other manifest or partitions
granularities.
3. stretch goal for the handling the path duplicates entries: it can be
manifests in the manifest-list, data or delete files in the manifests.
I heard @Priyadarshini-Mitra is happy to work and contribute as well, so
please take all the help as needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
