[ https://issues.apache.org/jira/browse/HUDI-1108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Udit Mehrotra reassigned HUDI-1108: ----------------------------------- Assignee: (was: Udit Mehrotra) > Allow parallel listing of dataset partitions for various actions during write > ----------------------------------------------------------------------------- > > Key: HUDI-1108 > URL: https://issues.apache.org/jira/browse/HUDI-1108 > Project: Apache Hudi > Issue Type: Improvement > Components: Writer Core > Reporter: Balaji Varadarajan > Priority: Blocker > Fix For: 0.6.0 > > > Currently we rely on FSUtils.getAllPartitionPaths to return all partitions of > a dataset. This implementation is slow for AWS S3 file-systems. We need to > provide option to allow the listing to be parallelizable. > GH Issue : [https://github.com/apache/hudi/issues/1837] > -- This message was sent by Atlassian Jira (v8.3.4#803005)