[
https://issues.apache.org/jira/browse/GOBBLIN-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aditya Sharma updated GOBBLIN-376:
----------------------------------
Description:
RecordWithMetadataSchemaRegistrationConverter: An Avro Schema might need to be
registered to Kafka. Using RecordWithMetadata registered schema id can be added
to record metadata. Hence adding a converter with takes RecordWithMetadata,
registers Avro schema with Kafka schema registry, adds the schema id to
metadata and returns RecordWithMetadata will be useful.
BytestoRecordWithMetadata converter: To wrap bytes along with metadata
Use case: Needed a converter which takes serialized Avro record (byte array)
and schema as the string, and returns the record along with the registered
schemaId.
RecordWithMetadata object can be used to carry the schemaId along with the
record(in this case byte array). Hence the above two converters in order of
BytestoRecordWithMetadata and RecordWithMetadataSchemaRegistrationConverter
will do the work.
was:
An Avro Schema might need to be registered to Kafka. Using RecordWithMetadata
registered schema id can be added to record metadata.
Hence adding a converter with takes RecordWithMetadata, registers Avro schema
with Kafka schema registry, adds the schema id to metadata and returns
RecordWithMetadata will be useful.
BytestoRecordWithMetadata converter: To wrap bytes along with metadata
Use case:
Needed a converter which takes serialized avro record (byte array) and schema
as string, and returns the record along with the registered schemaId.
RecordWithMetadata object can be used to carry the schemaId along with the
record(in this case byte array).
> Kafka Schema Registration for RecordWithMetadata and
> BytesToRecordWithMetadataConverter
> ---------------------------------------------------------------------------------------
>
> Key: GOBBLIN-376
> URL: https://issues.apache.org/jira/browse/GOBBLIN-376
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: misc
> Reporter: Aditya Sharma
> Assignee: Aditya Sharma
> Priority: Major
>
> RecordWithMetadataSchemaRegistrationConverter: An Avro Schema might need to
> be registered to Kafka. Using RecordWithMetadata registered schema id can be
> added to record metadata. Hence adding a converter with takes
> RecordWithMetadata, registers Avro schema with Kafka schema registry, adds
> the schema id to metadata and returns RecordWithMetadata will be useful.
> BytestoRecordWithMetadata converter: To wrap bytes along with metadata
> Use case: Needed a converter which takes serialized Avro record (byte array)
> and schema as the string, and returns the record along with the registered
> schemaId.
> RecordWithMetadata object can be used to carry the schemaId along with the
> record(in this case byte array). Hence the above two converters in order of
> BytestoRecordWithMetadata and RecordWithMetadataSchemaRegistrationConverter
> will do the work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
