[ https://issues.apache.org/jira/browse/HDDS-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954802#comment-16954802 ]
Anu Engineer commented on HDDS-2328: ------------------------------------ The Listing API interface already does that. I will take a look at why we are not paging ... Can you please provide me with repro steps and which version of branch you tried with this ? > Support large-scale listing > ---------------------------- > > Key: HDDS-2328 > URL: https://issues.apache.org/jira/browse/HDDS-2328 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Manager > Reporter: Rajesh Balamohan > Assignee: Hanisha Koneru > Priority: Major > Labels: performance > > Large-scale listing of directory contents takes a lot longer time and also > has the potential to run into OOM. I have > 1 million entries in the same > level and it took lot longer time with {{RemoteIterator}} (didn't complete as > it was stuck in RDB::seek). > S3A batches it with 5K listing per fetch IIRC. It would be good to have this > feature in ozone as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org