Hello,

I'm working on a persistent Catalog which stores the tables, views etc in a
DB. I'd like to get some guidance on the following issue:
Firstly, it seems to be quite hard to serialize a view: I found 2 methods
to do it, first is to use DescriptorProperties which is also used by the
HiveCatalog, however this is deprecated so would be nice to avoid it.
The docs say to use CatalogPropertiesUtil , however I couldn't find an easy
way to use it with views. I could create a CatalogTable from the view and
that way I can serialize it, but this feels like a workaround and not a
good solution. After this, there is another issue, which seems like a
blocker unfortunately. If I have a view which uses for example
TUMBLE_ROWTIME , the serialized version of the TimestampType logical type
loses the metadata that it's a ROWTIME kind so when I load the views back
from the DB and run a job using this view, the job fails with an error that
row time is required but found a regular timestamp. Looking at the code of
TimestampType the constructor which allows specifying the kind is marked as
internal and the docs explicitly say that metadata like kind is not
serialized. I'm wondering what is the reason behind this, as this means I
cannot get the same type back after serialization -> deserialization. If
this is something that could be improved, I'm more than happy to take a
look and potentially raise a PR. Thanks!

Reply via email to