WinkerDu commented on a change in pull request #3073:
URL: https://github.com/apache/iceberg/pull/3073#discussion_r703695049
##########
File path: core/src/main/java/org/apache/iceberg/util/BinPacking.java
##########
@@ -169,19 +179,21 @@ public boolean hasNext() {
private static class Bin<T> {
private final long targetWeight;
+ private final int itemsPerBin;
private final List<T> items = Lists.newArrayList();
private long binWeight = 0L;
- Bin(long targetWeight) {
+ Bin(long targetWeight, int itemsPerBin) {
this.targetWeight = targetWeight;
+ this.itemsPerBin = itemsPerBin;
}
List<T> items() {
return items;
}
boolean canAdd(long weight) {
- return binWeight + weight <= targetWeight;
+ return binWeight + weight <= targetWeight && items.size() <= itemsPerBin;
Review comment:
I think maybe it's reasonable that lookback is the only condition to
trigger bin removal.
If the bins [3, 4], [4, 3] finish off before lookback limit reached, the
'lookback' setting will be confused.
On the other hand, if lookback is the only condition to remove bin, both
'lookback' and 'itemsPerBin' settings will work fine literally.
what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
