RussellSpitzer commented on a change in pull request #1313:
URL: https://github.com/apache/iceberg/pull/1313#discussion_r467936640
##########
File path:
spark/src/main/java/org/apache/iceberg/actions/RemoveOrphanFilesAction.java
##########
@@ -106,6 +106,7 @@ public void accept(String file) {
this.table = table;
this.ops = ((HasTableOperations) table).operations();
this.location = table.location();
+ this.deleteFunc = new DeleteConsumer(table);
Review comment:
We set up a bit of a trap here. A serializable function is required here
when using the Parallel path but we don't type check that we have a
serializable consumer for this field. So we can end up with a user passing a
non-serializable function here and getting a runtime error. Not sure if we want
to lock this function to only serializable delete functions but I think we
should at least document this and warn users.
Another early check could be to attempt to cast the Lambda to serializable
before performing the parallelization and providing an explanation of the
failure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
