oscerd opened a new issue, #3038: URL: https://github.com/apache/camel-kamelets/issues/3038
Follow-up to #2871, from a review note @davsclaus left on #3032. ## The gap #3032 and #3036 gave the five single-record Kafka sources a non-internal name for the record metadata, including `kafka-key` (with `ce-kafkakey`). `kafka-sink` maps the other direction, but from a different name: it reads `key` / `ce-key` into `CamelKafkaKey`. So the friendly names do not round-trip. In a source-to-sink pipe with nothing in between it does not matter, because `CamelKafkaKey` is still on the exchange and the sink's own component picks it up. It matters as soon as there is a transport hop: ``` kafka-source -> http -> kafka-sink ``` `DefaultHeaderFilterStrategy` strips `Camel*` headers across HTTP, which #2871 records explicitly: > With the migration to `CamelKafka*` names, the HTTP component now correctly filters them `CamelKafkaKey` is therefore gone after the hop. `kafka-key` survives, because it is not a Camel internal name — that is the whole point of adding it — but `kafka-sink` does not read it. The record is produced with **no key**, so it is partitioned round-robin instead of by key, and per-key ordering is silently lost. This is the same class of problem #2871 was filed about, just on the return leg. ## Suggested fix Have `kafka-sink` accept `kafka-key` / `ce-kafkakey` alongside the existing `key` / `ce-key`, so a key set by any of the five sources survives a hop. Existing bindings that set `key` keep working. ## What should *not* be wired up Worth stating, so this is not implemented too broadly: - `kafka-partition` is consumer metadata — *which partition the record came from*. The sink's `partition-key` is producer input — *a value used to choose a partition*. Different concepts; wiring one to the other would be wrong. - `kafka-topic` is the topic the record was consumed from. Feeding it back into the sink would echo records to their source topic, which is rarely what is wanted, and the sink already has a `topic` property. Only the key round-trips meaningfully. ## Scope `kafka-sink` only. There is one sink, so this is a single template change plus a line in its partial, and the existing `kafka-sink-pipe-test` could cover it. --- _Claude Code on behalf of Andrea Cosentino_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
