jkolash commented on issue #13297:
URL: https://github.com/apache/iceberg/issues/13297#issuecomment-2969943746
Actually I don't think any spark api changes are needed the callback and the
```MetricsBatchIterator``` are both created in the ```DataSourceRDD```
In java you could write something like this.
```java
static class HeavyCallback implements Runnable{
byte[] heavyReference = new byte[1000000];
@Override
public void run() {
}
}
static class InvokeOnceCallback implements Runnable {
final AtomicReference<Runnable> callbackHandle = new
AtomicReference<>();
InvokeOnceCallback(Runnable target) {
callbackHandle.set(target);
}
@Override
public void run() {
if (callbackHandle.get() != null) {
callbackHandle.get().run();
callbackHandle.set(null);
}
}
}
static void showcaseIndirectCallback() {
InvokeOnceCallback callback = new InvokeOnceCallback(new
HeavyCallback());
//this can be called from either the iterable exhaustion or the
original callback.
//Once invoked the reference is removed and can be GC'd
callback.run();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
