aokolnychyi commented on code in PR #6907:
URL: https://github.com/apache/iceberg/pull/6907#discussion_r1115289084
##########
api/src/main/java/org/apache/iceberg/actions/RewriteDataFiles.java:
##########
@@ -76,7 +76,7 @@
*/
String MAX_CONCURRENT_FILE_GROUP_REWRITES =
"max-concurrent-file-group-rewrites";
- int MAX_CONCURRENT_FILE_GROUP_REWRITES_DEFAULT = 1;
+ int MAX_CONCURRENT_FILE_GROUP_REWRITES_DEFAULT = 5;
Review Comment:
We would probably need to look at the scheduler implementation (e.g. FIFO)
and total resources available in the cluster vs the amount of data we need to
compact and partitions. I think it makes sense exploring but if it becomes too
tricky, I'd just default it to something more reasonable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
