FANNG1 commented on issue #11027: URL: https://github.com/apache/gravitino/issues/11027#issuecomment-4456022758
> Good. > > > a more practical approach is to read the actual schema from the underlying Lance dataset when needed, and cache it to improve performance. > > I agree on this point. What have you pointed out in the cache here? I'm very interested in how we can manage the cache to keep the dataset schema consistent between the namespace implementation and the schema metadata in the filepath. Till now, there are no mechanisms like a hook to detect the changes in the file path schema metadata. I'm curious about it. Something like current Iceberg metadata cache, Iceberg checks whether the cache is latest by a lightweight metadata location, Lance could check whether the cache is latest by lance dataset version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
