Do you know which metadata? For validation it is mostly using interface Schema (by which it accesses table definitions and their columns). For optimization it might use other metadata such as existence of keys, row counts, etc.
There are various implementations of Schema - including with caching - and you can also write your own. I recommend that you measure which methods are being called, and how many times. Caching may be the solution, or it might make things worse. Julian > On May 8, 2024, at 8:16 AM, Cancai Cai <can...@apache.org> wrote: > > Hello everyone in the calcite community, I am doing some interesting things > with calcite. There is a question I would like to ask, but I have not seen > this part of the calcite code, so I don't know if it is appropriate to ask > it here. > > When I use calcite's sqlValidator.validate interface to verify the metadata > of the hive table, it takes a long time. I think this is because I have a > large number of hive tables, 300,000 tables. I would like to ask if calcite > has the function of caching metadata information, or if I can do this: > cache the metadata information and then verify it.(I have done similar > things in spark sql, I don’t know if calcite can ). > > Or do you have any good plans? > > Thank you. > > Best regards, > Cancai cai
