psheets opened a new pull request, #15423:
URL: https://github.com/apache/iceberg/pull/15423
## Summary
- **New `EnvelopeTransform` SMT** — wraps each record value in an envelope
struct with three fields:
- `obj`: the original record value
- `load_ts`: wall-clock timestamp at transform time
- `event_ts`: the Kafka record timestamp (nullable)
Supports both schema and schemaless modes with an LRU schema cache.
- **Fix leader election during rebalance** —
`CommitterImpl.hasLeaderPartition()` now detects when no consumer group members
have partition assignments (transient
state during a rebalance) and defers leader election instead of making an
incorrect determination.
- **Remove GCP and Azure dependencies from runtime distribution** — only
AWS catalog (Glue) is needed. This also resolves a binary incompatibility
between GCP's
`protobuf-java 4.x` and the Confluent protobuf converter which requires
`3.x`.
- **Add Confluent Schema Registry protobuf converter**
(`kafka-connect-protobuf-converter:7.9.0`) to the runtime distribution for
Protobuf-encoded topics.
- **Add Confluent Maven repository** to the root `build.gradle`.
- **Add `Makefile`** for building the kafka-connect distribution locally.
## Changes
| Area | Files |
|------|-------|
| New SMT | `kafka-connect-transforms/.../EnvelopeTransform.java` |
| Tests | `kafka-connect-transforms/.../TestEnvelopeTransform.java` |
| SPI registration |
`META-INF/services/org.apache.kafka.connect.transforms.Transformation` |
| Rebalance fix | `kafka-connect/.../channel/CommitterImpl.java` |
| Build | `build.gradle`, `kafka-connect/build.gradle`,
`kafka-connect/Makefile` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
