[ https://issues.apache.org/jira/browse/HUDI-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yue Zhang resolved HUDI-6039. ----------------------------- > Fix FS based listing in clean planner > ------------------------------------- > > Key: HUDI-6039 > URL: https://issues.apache.org/jira/browse/HUDI-6039 > Project: Apache Hudi > Issue Type: Improvement > Components: cleaning, writer-core > Reporter: sivabalan narayanan > Priority: Major > Labels: pull-request-available > > we are using FS based listing when full cleaning is invoked w/ clean planner. > {code:java} > /** * Scan and list all partitions for cleaning. * @return all partitions > paths for the dataset. */ private List<String> > getPartitionPathsForFullCleaning() { // Go to brute force mode of scanning > all partitions try { // Because the partition of BaseTableMetadata > has been deleted, // all partition information can only be obtained from > FileSystemBackedTableMetadata. FileSystemBackedTableMetadata > fsBackedTableMetadata = new FileSystemBackedTableMetadata(context, > context.getHadoopConf(), config.getBasePath(), > config.shouldAssumeDatePartitioning()); return > fsBackedTableMetadata.getAllPartitionPaths(); } catch (IOException e) { > return Collections.emptyList(); } } {code} > > we should be using Metadata based listing if metadata is enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)