vburenin commented on pull request #2619:
URL: https://github.com/apache/hudi/pull/2619#issuecomment-801943274
@nsivabalan There are several changes into the SchemaRegistryProvider that I
would like to include:
1. Add ability to cache the schema.
2. Inject Kafka Fields into the schema
3. Get rid of NullTargetSchemaProvider and make it so that
SchemaRegistryProvider returns null for the target schema if no target schema
URL is provided.
4. Add ability to inject custom schema post processor for the following use
cases, which technically fits nicely with kafka fields injector:
- Make every non nullable value nullable
- Add a field that would always exist for data structures that are nulls
(need to deal with some protobuf->parquet corner case)
- inject some other info schema that is not available when data is being
produced. It could be some contextual info from a third party source.
Post schema processors could be an additional set of classes provided
with Hudi that should be configured to be used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
