Thanks for the proposal! It is a common scenario for multiple applications to share the same dimension table. As described in the design document, the TableQuery client will obtain the addresses of all Executors from the AddressServer and then request them through RPC. I have a question about this: How does the TableQuery client decide which Executor to request? Request all Executors in turn? Or is it restricted that the key of lookup must contain bucket-key?
Best, Ming Li Jingsong Li <[email protected]> 于2023年10月8日周日 18:35写道： > Hi all, > > I want to bring up a discussion about Paimon QueryService [1]. > > Paimon primary key table already provides LSM file structure, it is a > pity that the paimon can not provide a queryable service for lookup. > > A distributed service can download Paimon files locally and provide a > Lookup service. It does not affect the write process and read process, > it is a separate server. It can be used as: > > 1. Flink Lookup Join, reuse by multiple Flink Jobs. > 2. Online Service Lookup, this requires high stability. (it may not be > so stable in the first version) > > See more in PIP [1]. > > This PIP is a high-level design for Paimon QueryService, not including > all details. > > [1] > https://cwiki.apache.org/confluence/display/PAIMON/PIP-10%3A+Introduce+Paimon+QueryService > > Best, > Jingsong >
