ayushchauhan0811 commented on pull request #2782: URL: https://github.com/apache/iceberg/pull/2782#issuecomment-874938974
@RussellSpitzer Currently incremental scan only supports the append operations but there can be cases where we can want to include other operations too. We use a CDC job to write to an iceberg table and then use this feature to do some computation on the incremental data b/w two snapshots. I didn't understand, why will it cause issues with rewrite operations? Isn't rewrite operations like overwrite operation where there are a bunch of delete files and newly added data files. I have tested it on overwrite operations on the cdc data -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
