umehrot2 commented on a change in pull request #2417: URL: https://github.com/apache/hudi/pull/2417#discussion_r554278832
########## File path: hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java ########## @@ -49,12 +60,48 @@ public FileSystemBackedTableMetadata(SerializableConfiguration conf, String data @Override public List<String> getAllPartitionPaths() throws IOException { - FileSystem fs = new Path(datasetBasePath).getFileSystem(hadoopConf.get()); if (assumeDatePartitioning) { + FileSystem fs = new Path(datasetBasePath).getFileSystem(hadoopConf.get()); return FSUtils.getAllPartitionFoldersThreeLevelsDown(fs, datasetBasePath); - } else { - return FSUtils.getAllFoldersWithPartitionMetaFile(fs, datasetBasePath); } + + List<Path> pathsToList = new LinkedList<>(); Review comment: As I had mentioned, that this is already tested through code paths like cleaner, and I did verify that `TestCleaner` for example fails if this would return incorrect partitions. Adding specific tests for `FileSystemBackedMetadata` is tricky because `spark context` is not available in `hudi-common`. Perhaps, this can be done in a separate PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org