[
https://issues.apache.org/jira/browse/KAFKA-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abdelhamide EL ARIB updated KAFKA-7189:
---------------------------------------
Description:
Like the
[flatten|https://docs.confluent.io/current/connect/transforms/flatten.html#flatten],
there is the need too for a merge transformer.
Example transformation :
We want to add the offset and the partition for each record, and after that
merge them into one field _metadata :
{code:java}
"transforms":"AddOffset, AddPartition, MergeFields",
"transforms.AddOffset.type":"org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.AddOffset.offset.field":"offset!",
"transforms.AddPartition.type":"org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.AddPartition.partition.field":"partition!",
"transforms.MergeFields.type":"org.apache.kafka.connect.transforms.Merge$Value",
"transforms.MergeFields.field.list":"offset,partition",
"transforms.MergeFields.field.root":"_metadata"
{code}
field.root : Field name for the root field.
field.list: The list of fields to merge under the root field. Suffix with `*`
to keep it in top level, and prefix it if it's opotional.
This is our MR for this transformer [https://github.com/apache/kafka/pull/5405]
was:
Like the
[flatten|https://docs.confluent.io/current/connect/transforms/flatten.html#flatten],
there is the need too for a merge transformer.
Example transformation :
We want to add the offset and the partition for each record, and after that
merge them into one field _metadata :
{code:java}
"transforms":"AddOffset, AddPartition, MergeFields",
"transforms.AddOffset.type":"org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.AddOffset.offset.field":"offset!",
"transforms.AddPartition.type":"org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.AddPartition.partition.field":"partition!",
"transforms.MergeFields.type":"org.apache.kafka.connect.transforms.Merge$Value",
"transforms.MergeFields.field.list":"offset,partition",
"transforms.MergeFields.field.root":"_metadata"
{code}
field.root : Field name for the root field.
field.list: The list of fields to merge under the root field. Suffix with * to
keep it in top level, and prefix it if it's opotional
This is our MR for this transformer [https://github.com/apache/kafka/pull/5405]
> Add a Merge Transformer for Kafka Connect
> -----------------------------------------
>
> Key: KAFKA-7189
> URL: https://issues.apache.org/jira/browse/KAFKA-7189
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Abdelhamide EL ARIB
> Priority: Minor
>
> Like the
> [flatten|https://docs.confluent.io/current/connect/transforms/flatten.html#flatten],
> there is the need too for a merge transformer.
> Example transformation :
> We want to add the offset and the partition for each record, and after that
> merge them into one field _metadata :
>
> {code:java}
> "transforms":"AddOffset, AddPartition, MergeFields",
> "transforms.AddOffset.type":"org.apache.kafka.connect.transforms.InsertField$Value",
> "transforms.AddOffset.offset.field":"offset!",
> "transforms.AddPartition.type":"org.apache.kafka.connect.transforms.InsertField$Value",
> "transforms.AddPartition.partition.field":"partition!",
> "transforms.MergeFields.type":"org.apache.kafka.connect.transforms.Merge$Value",
> "transforms.MergeFields.field.list":"offset,partition",
> "transforms.MergeFields.field.root":"_metadata"
> {code}
>
> field.root : Field name for the root field.
> field.list: The list of fields to merge under the root field. Suffix with `*`
> to keep it in top level, and prefix it if it's opotional.
>
>
> This is our MR for this transformer
> [https://github.com/apache/kafka/pull/5405]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
