Hi All,
Need your help with deleting a positionDelete file that has been
committed before and writing a new one instead.
The use case is we produce a Position delete file and commit it to
the table using rowDelta.addDeletes(posDeleteFile). Let us say this is
snapshot 1, when we do snapshot 2 what I want to do is delete posDeleteFile
from snapshot 1 and add a new file to snapshot 2.
To plan for files I am using
CloseableIterator<CombinedScanTask> iterator = table.newScan()
.useSnapshot(table.currentSnapshot().snapshotId())
.planTasks().iterator();
StreamSupport.stream(Spliterators
.spliteratorUnknownSize(iterator,
Spliterator.ORDERED)
, false)
.flatMap(f -> f.deletes().stream())
.collect(Collectors.toSet());
however, this gives me all the delete files. My specific usecase is to keep
rolling the delete file, so on a new snapshot, I want to do is delete the
delete file from snapshot - 1 and add a new delete file to the current
snapshot.
The current snapshot should then only read the new file committed and not
the ones from snapshot - 1.
Also, can you tell me how to delete the physical file along with snapshot
and manifest entries for the older delete file.
Regards,
Taher Koitawala
