[
https://issues.apache.org/jira/browse/HDFS-12998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365600#comment-16365600
]
Tsz Wo Nicholas Sze edited comment on HDFS-12998 at 2/15/18 2:19 PM:
---------------------------------------------------------------------
Some more comments:
- SnapshotDiffReportListingIterator does not need generic type.
- The code will be simplified if we move all the logic to next().
{code}
private class SnapshotDiffReportListingIterator implements
RemoteIterator<SnapshotDiffReportListing> {
private final String snapshotDir;
private final String fromSnapshot;
private final String toSnapshot;
private SnapshotDiffReportListing thisListing;
private byte[] startPath;
private int index;
private boolean hasNext = true;
private SnapshotDiffReportListingIterator(String snapshotDir,
String fromSnapshot, String toSnapshot) {
this.snapshotDir = snapshotDir;
this.fromSnapshot = fromSnapshot;
this.toSnapshot = toSnapshot;
}
@Override
public boolean hasNext() {
return hasNext;
}
@Override
public SnapshotDiffReportListing next() throws IOException {
if (hasNext) {
thisListing = dfs.getSnapshotDiffReportListing(snapshotDir,
fromSnapshot, toSnapshot, startPath, index);
startPath = thisListing.getLastPath();
index = thisListing.getLastIndex();
hasNext = !(Arrays.equals(startPath, DFSUtilClient.EMPTY_BYTES)
&& index == -1);
return thisListing;
} else {
thisListing = null;
throw new java.util.NoSuchElementException(
"No more entry in SnapshotDiffReport for " + snapshotDir);
}
}
}
{code}
was (Author: szetszwo):
Some more comments:
- SnapshotDiffReportListingIterator does not need generic type.
- The code will be simplified if we move all the logic to next().
{code}
private class SnapshotDiffReportListingIterator implements
RemoteIterator<SnapshotDiffReportListing> {
private SnapshotDiffReportListing thisListing;
private final String snapshotDir;
private final String fromSnapshot;
private final String toSnapshot;
private byte[] startPath;
private int index;
private boolean hasNext = true;
private SnapshotDiffReportListingIterator(String snapshotDir,
String fromSnapshot, String toSnapshot) {
this.snapshotDir = snapshotDir;
this.fromSnapshot = fromSnapshot;
this.toSnapshot = toSnapshot;
}
@Override
public boolean hasNext() {
return hasNext;
}
@Override
public SnapshotDiffReportListing next() throws IOException {
if (hasNext) {
thisListing = dfs.getSnapshotDiffReportListing(snapshotDir,
fromSnapshot, toSnapshot, startPath, index);
startPath = thisListing.getLastPath();
index = thisListing.getLastIndex();
hasNext = !(Arrays.equals(startPath, DFSUtilClient.EMPTY_BYTES)
&& index == -1);
return thisListing;
} else {
thisListing = null;
throw new java.util.NoSuchElementException(
"No more entry in SnapshotDiffReport for " + snapshotDir);
}
}
}
{code}
> SnapshotDiff - Provide an iterator-based listing API for calculating
> snapshotDiff
> ---------------------------------------------------------------------------------
>
> Key: HDFS-12998
> URL: https://issues.apache.org/jira/browse/HDFS-12998
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Shashikant Banerjee
> Assignee: Shashikant Banerjee
> Priority: Major
> Attachments: HDFS-12998.001.patch
>
>
> Currently , SnapshotDiff computation happens over multiple rpc calls to
> namenode depending on the no of snapshotDiff entries where each rpc call
> returns at max 1000 entries by default . Each "getSnapshotDiffreportListing"
> call to namenode returns a partial snapshotDiffreportList which are all
> combined and processed at the client side to generate a final
> snapshotDiffreport. There can be cases where SnapshotDiffReport can be huge
> and in situations as such , the rpc calls to namnode should happen on demand
> at the client side.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
