Hi Balaji, not sure if I fully get it. I'm attempting to refer to this test case https://github.com/apache/hudi/blob/9bcd3221fd440081dbae70e89d08539c3b484862/hudi-hadoop-mr/src/test/java/org/apache/hudi/hadoop/TestHoodieROTablePathFilter.java#L63-L65
where a partition path is supposed to be accepted. If I change L64 to Path partitionPath = new Path(Paths.get(basePath, "2017/01/01").toUri()); Then it resulted in not being accepted due to partitionPath ending with `/` (a directory path). To me, this seems to be a corner case not being covered. Could you kindly confirm the expectation please? Thanks. On Tue, Sep 8, 2020 at 8:58 PM Balaji Varadarajan <v.bal...@ymail.com.invalid> wrote: > Hi Raymond, > IIRC, we need to give a blob path to make HoodieROTablePathFilter to work > correctly (e.g: "base/partition/*"). The path-cache is at partition level > and not at table level so we need to extract the partition-path correctly > to be used as look-up key. To extract partition-path, the challenge here is > "Path" type does not have APIs to quickly figure if a path is a directory > or not and we should avoid making RPC calls here. > Thanks,Balaji.V > On Tuesday, September 8, 2020, 09:56:49 AM PDT, Raymond Xu < > xu.shiyan.raym...@gmail.com> wrote: > > > https://github.com/apache/hudi/blob/9bcd3221fd440081dbae70e89d08539c3b484862/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieROTablePathFilter.java#L120-L121 > > As shown in the 2 lines above, it does not seem to work with directory > Path. > It should work for both `new Path("base/partition")` and `new > Path("base/partition/")`, but it only works for the former case. In the > latter case, `folder` will be "base/partition" and `path` will be > "base/partition/", which will always result in returning false. > A potential bug? >
