Hi Raymond,
IIRC, we need to give a blob path to make HoodieROTablePathFilter to work
correctly (e.g: "base/partition/*"). The path-cache is at partition level and
not at table level so we need to extract the partition-path correctly to be
used as look-up key. To extract partition-path, the challenge here is "Path"
type does not have APIs to quickly figure if a path is a directory or not and
we should avoid making RPC calls here.
Thanks,Balaji.V
On Tuesday, September 8, 2020, 09:56:49 AM PDT, Raymond Xu
<xu.shiyan.raym...@gmail.com> wrote:
https://github.com/apache/hudi/blob/9bcd3221fd440081dbae70e89d08539c3b484862/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieROTablePathFilter.java#L120-L121
As shown in the 2 lines above, it does not seem to work with directory
Path.
It should work for both `new Path("base/partition")` and `new
Path("base/partition/")`, but it only works for the former case. In the
latter case, `folder` will be "base/partition" and `path` will be
"base/partition/", which will always result in returning false.
A potential bug?
