hameizi commented on a change in pull request #2916:
URL: https://github.com/apache/iceberg/pull/2916#discussion_r685778592
##########
File path: core/src/main/java/org/apache/iceberg/io/BaseTaskWriter.java
##########
@@ -105,7 +112,11 @@ protected BaseEqualityDeltaWriter(StructLike partition,
Schema schema, Schema de
this.dataWriter = new RollingFileWriter(partition);
this.eqDeleteWriter = new RollingEqDeleteWriter(partition);
- this.posDeleteWriter = new SortedPosDeleteWriter<>(appenderFactory,
fileFactory, format, partition);
+ this.posDeleteWriter = new SortedPosDeleteWriter<>(appenderFactory,
+ fileFactory,
+ format,
+ partition,
+ getRecordsNumThreshold(properties));
Review comment:
> Sometimes certain maps made by Guava etc (ImmutableMap comes to mind)
are not serializable, particularly when using Kryo (which is not the default
but the de facto default with Spark).
I'm sorry i did't test this, i will try test these case. And pass around
table properties conveniency for develop later. If there is some bug to pass
around table properties i can fix it to just the value of this propertie.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
