rdblue commented on issue #732: URL: https://github.com/apache/iceberg/issues/732#issuecomment-1028481011
@atronchi, Spark will actually retry. You can bump up the number of retries by setting the `commit.retry.num-retries` in [table properties](https://iceberg.apache.org/configuration/#table-behavior-properties). There are other settings for retries, like the wait time and max timeout as well. With several commits happening at the same time, you probably just need to bump up the number of retries. The situation that @waterlx hit here was that there was a lot of work needed for each retry so the likelihood of conflict was really large. If you run into something similar, then I recommend using the `[RewriteManifests](https://iceberg.apache.org/maintenance/#rewrite-manifests)` approach I mentioned. That is now available as a [stored procedure](https://iceberg.apache.org/spark-procedures/#rewrite_manifests) so it is very simple to rewrite your manifests and avoid conflicts so retries can reuse more work and commit faster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
