aliehsaeedii commented on code in PR #21676:
URL: https://github.com/apache/kafka/pull/21676#discussion_r3559843453
##########
clients/src/main/java/org/apache/kafka/clients/producer/ProducerRecord.java:
##########
@@ -81,6 +83,35 @@ public ProducerRecord(String topic, Integer partition, Long
timestamp, K key, V
this.value = value;
this.timestamp = timestamp;
this.headers = new RecordHeaders(headers);
+ this.rawSerializedHeaders = null;
+ }
+
+ /**
+ * Creates a record carrying pre-serialized header bytes. When this
constructor is used, the
+ * producer writes {@code rawSerializedHeaders} directly into the record
batch without
+ * deserializing or re-serializing individual headers. The {@code
rawSerializedHeaders} must
+ * use the standard Kafka header wire format: {@code
[count(varint)][header1][header2]...},
+ * or be empty (length 0) for zero headers.
+ *
+ * <p>This is intended for internal use (e.g., Kafka Streams changelog
writing) where headers
+ * are already available in serialized form and the deserialization
round-trip should be avoided.
+ */
+ public ProducerRecord(String topic, Integer partition, Long timestamp, K
key, V value, byte[] rawSerializedHeaders) {
Review Comment:
Oh, it needs a KIP. Also I don't like to add it for all users honestly. It's
sort of a foot-gun to be public and also we’d have to push hard to get that.
Based on my experience, this area is heavily guarded by the code owners.
Suggested alternative (no public API, likely no KIP): keep the optimization
but smuggle the bytes through the *existing* constructor via an internal
carrier:
- Add an internal `Headers` implementation (e.g.
`common.header.internals.PreSerializedHeaders`) holding the raw bytes and
lazily materializing to `RecordHeaders` on any read/iteration.
- Streams passes it into the existing `ProducerRecord(...,
Iterable<Header>)` constructor — no new constructor.
- `ProducerRecord` special-cases it internally into a **package-private**
field; `KafkaProducer.doSend` (same package) reads a package-private getter and
routes to the raw append path.
- Because the carrier materializes on read, `headers()` stays correct and
interceptors keep working — `doSend` just falls back to the normal `Header[]`
path if anything already materialized it.
This matches the mechanism's blast radius to the (niche) benefit and stays
fully reversible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
