Hi Balaji, business metadata are all types of info related to the business where the Hudi solution is being used... from a COB (ie close of business date) related to that commit to any qualifier related to that commit that might be useful to be associated with that commit id. If we enable the timeline service metadata model to be extended we could use the service instance itself to support specialised queries that involve business qualifiers in order to return a proper set of metadata pointing to the related commits that answer a business query.
if we do not have that flexibility we might end up creating a external transaction log and then comes the hard task to make that service in sync to the timeline service. let me know if that makes sense to you, Mario. Em seg., 1 de jun. de 2020 às 06:55, Balaji Varadarajan <[email protected]> escreveu: > Hi Mario, > Timeline Server was designed to serve hudi metadata for Hudi writers and > readers. it may not be suitable to serve arbitrary data. But, it is an > interesting thought. Can you elaborate more on what kind of business > metadata are you looking. Is this something you are planning to store in > commit files ? > Balaji.V > > On Sunday, May 31, 2020, 04:22:27 PM PDT, Mario de Sá Vera < > [email protected]> wrote: > > I see a need for extending the current timeline server schema so that a > flexible model could be achieved in order to accommodate business metadata. > > let me know if that makes sense to anyone here... > > Regards, > > Mario. >
