[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943824#comment-16943824
]
ASF GitHub Bot commented on KAFKA-8523:
---
rhauch commented on pull request #6914: KAFKA-8523 Enabling InsertField
transform to be used with tombstone events
URL: https://github.com/apache/kafka/pull/6914
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
> Attachments: image-2019-09-17-15-53-44-038.png
>
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> -AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.-
> Update as per the discussion in the comments: tombstones should be left as-is
> by this SMT, as any insertion would defeat their purpose of enabling log
> compaction.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16939730#comment-16939730
]
Gunnar Morling commented on KAFKA-8523:
---
Hey [~rhauch], [~frederic.tardif], yes, we're all on the same page: tombstones
shouldn't be modified at all by this SMT. I've updated and force-pushed the PR
accordingly. It's good to go from my PoV now.
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
> Attachments: image-2019-09-17-15-53-44-038.png
>
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932686#comment-16932686
]
Randall Hauch commented on KAFKA-8523:
--
[~frederic.tardif], thanks for the comment.
[~gunnar.morling]'s last comment was:
{quote}
So I would suggest I simply update my PR so that it passes on tombstones
unmodified and that should be good enough.
{quote}
As mentioned above, I agree with this approach: *tombstones should be left
as-is and not modified*. Note that [~gunnar.morling] hasn't yet updated the PR,
but it will not be merged until that happens.
Second, any new configuration on any SMT would be prefixed in the connector
config with `transforms..`, and thus it would not clash with any
existing connector or SMT configuration. For example, *+_if_+* we were to add a
new SMT property named `behavior.on.null.values`, then it would be used in a
connector configuration like:
{code:java}
transforms=MyInsertField
transforms.MyInsertField.class=org.apache.kafka.connect.transforms.InsertField
transforms.MyInsertField.behavior.on.null.values=skip
{code}
*However, as mentioned above, both Gunnar and I agreed above that we don't
think a new configuration is necessary*.
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
> Attachments: image-2019-09-17-15-53-44-038.png
>
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931777#comment-16931777
]
Frederic Tardif commented on KAFKA-8523:
just to clarify my comment above. The `behavior.on.null.values` already exists
(at least on the elasticsearch connector), so it is important to keep the null
values when we aim to interpret it as `delete` on an elasticsearch index entry.
Applying a transform that InsertFields on null would completely defies the
purpose in this scenario. I strongly believe the transform should skip null
values or this transform behaviour should at least be configurable.
!image-2019-09-17-15-53-44-038.png!
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
> Attachments: image-2019-09-17-15-53-44-038.png
>
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931761#comment-16931761
]
Randall Hauch commented on KAFKA-8523:
--
[~gunnar.morling], good point. I think a tombstone should be left unmodified by
this SMT, which means no need for a KIP or a config change.
I think converting or dropping a tombstone are separate SMTs.
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931662#comment-16931662
]
Gunnar Morling commented on KAFKA-8523:
---
Yes, I was arriving at pretty much the same conclusion; that said: is an option
even needed: I've come to think that a tombstone should remain just that, a
tombstone, i.e. I don't see a good reason to inject any value into a tombstone
really; after all, it has specific semantics -- enabling compaction -- which
shouldn't be altered. So I would suggest I simply update my PR so that it
passes on tombstones unmodified and that should be good enought. WDYT?
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931646#comment-16931646
]
Randall Hauch commented on KAFKA-8523:
--
It seems strange to me that InsertField into a null (tombstone) record would
create an empty map / struct. Rather, it seems like we need a configuration
property that defines the behavior on null.
However, that means we need a small KIP to define this property. I'd suggest a
property like `behavior.on.null` whose values are either `empty` or `skip`. If
we consider records with null values as tombstones (meaning they represent
deleted records), defaulting to `skip` makes the most sense to me.
We do have a similar problem with `ExtractField` (see KAFKA-7052), and it'd be
great if the KIP added that configuration item there as well.
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931374#comment-16931374
]
Frederic Tardif commented on KAFKA-8523:
I believe an option should be exposed to configure the behaviour in order to
either pass the null value unmodified or to apply the insert on an empty map.
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870484#comment-16870484
]
Gavin Gilmour commented on KAFKA-8523:
--
+1 For this fix.
I'm being burned by it while trying out
[https://github.com/confluentinc/demo-scene/blob/master/no-more-silos-oracle/docker-compose/scripts/create-ora-source-debezium-xstream.sh]
Is there any workaround to prevent the connector dying on deletes?
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870450#comment-16870450
]
Jagannath commented on KAFKA-8523:
--
I am seeing something similar from the ExtractField smt. I think heartbeat
messages are not going through the standard ExtractField SMT
```
Caused by: org.apache.kafka.connect.errors.DataException: Only Map objects
supported in absence of schema for [field extraction], found:
org.apache.kafka.connect.data.Struct debezium-68745ff465-lmslc debezium at
org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
debezium-68745ff465-lmslc debezium at
org.apache.kafka.connect.transforms.util.Requirements.requireMapOrNull(Requirements.java:47)
debezium-68745ff465-lmslc debezium at
org.apache.kafka.connect.transforms.ExtractField.apply(ExtractField.java:57)
debezium-68745ff465-lmslc debezium at
org.apache.kafka.connect.runtime.TransformationChain.lambda$apply$0(TransformationChain.java:50)
debezium-68745ff465-lmslc debezium at
org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
debezium-68745ff465-lmslc debezium at
org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
```
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868422#comment-16868422
]
Gunnar Morling commented on KAFKA-8523:
---
Good question, [~rhauch]; I didn't consider it, but I could see both approaches
making sense. Should we make this an option (insert into tombstones vs. pass
them on unmodified) for that SMT?
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16867753#comment-16867753
]
Randall Hauch commented on KAFKA-8523:
--
[~gunnar.morling], thanks for the report and the proposed fix. Did you consider
whether "InsertField" for a tombstone event means the SMT should do nothing
with the tombstone but pass it along?
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860794#comment-16860794
]
Gunnar Morling commented on KAFKA-8523:
---
Filed https://github.com/apache/kafka/pull/6914.
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860793#comment-16860793
]
ASF GitHub Bot commented on KAFKA-8523:
---
gunnarmorling commented on pull request #6914: KAFKA-8523 Enabling InsertField
transform to be used with tombstone events
URL: https://github.com/apache/kafka/pull/6914
Fixes https://issues.apache.org/jira/browse/KAFKA-8523
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8523) InsertField transformation fails when encountering tombstone event
[
https://issues.apache.org/jira/browse/KAFKA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860688#comment-16860688
]
Gunnar Morling commented on KAFKA-8523:
---
I'll provide a PR shortly.
> InsertField transformation fails when encountering tombstone event
> --
>
> Key: KAFKA-8523
> URL: https://issues.apache.org/jira/browse/KAFKA-8523
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
>Reporter: Gunnar Morling
>Priority: Major
>
> When applying the {{InsertField}} transformation to a tombstone event, an
> exception is raised:
> {code}
> org.apache.kafka.connect.errors.DataException: Only Map objects supported in
> absence of schema for [field insertion], found: null
> at
> org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
> at
> org.apache.kafka.connect.transforms.InsertField.applySchemaless(InsertField.java:138)
> at
> org.apache.kafka.connect.transforms.InsertField.apply(InsertField.java:131)
> at
> org.apache.kafka.connect.transforms.InsertFieldTest.tombstone(InsertFieldTest.java:128)
> {code}
> AFAICS, the transform can still be made working in in this case by simply
> building up a new value map from scratch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
