saurabhd336 commented on code in PR #3132:
URL: https://github.com/apache/celeborn/pull/3132#discussion_r1992036542
##########
client/src/main/java/org/apache/celeborn/client/read/DfsPartitionReader.java:
##########
@@ -308,4 +310,15 @@ private void closeStream() {
public PartitionLocation getLocation() {
return location;
}
+
+ @Override
+ public Optional<PartitionReaderCheckpointMetadata>
getPartitionReaderCheckpointMetadata() {
+ // TODO implement similar to {@link WorkerPartitionReader}
Review Comment:
I had planned to implement in a followup PR, after testing this for worker
partition reader. Anway, I've added the implementation for DfsPartitionReader.
For LocalPartitionReader, I think the complexity of implementation outweighs
the potential benefits (since for both the worker / dfs readers, its a network
call to fetch the chunks, while for local reader it's just a local file buffer
read)
I can take up the impl for local reader in a followup PR. wdyt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
