luoyuxia commented on issue #6104: URL: https://github.com/apache/iceberg/issues/6104#issuecomment-1314686886
> The new features of rocksdb mentioned in the PR https://github.com/apache/iceberg/pull/2680 have not found the relevant configuration items in the latest 1.0.0 version? Do you have to wait for a subsequent version to be added? Yes, we have to wait it to be merged. > In compressing data it is indeed possible to solve the OOM problem by increasing the memory of TaskManger and increasing the compression frequency of data. I agree with you, so I think use rocksdb as backend is a better solution > and the commit exception problem will be encountered when the data compression task is performed. Had a look about your exception log. The reason is the cdc contains a delete row, but the compaction for such case that contains delete files hasn't been supported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
