Hi all, A heads-up on a larger pass over pulsar-connectors over the last couple of days: 32 PRs merged, mostly integration-test coverage plus the bugs that coverage turned up. Full detail is in the merged PRs and filed issues; this is just for visibility, with two decisions I'd like maintainer input on.
The theme: connectors that had only mock tests got real integration tests (MongoDB, MariaDB JDBC, InfluxDB, Kafka, Solr, DynamoDB, HDFS3, NSQ, Aerospike, plus a new Debezium MariaDB source). Pointing real systems at them surfaced seven production bugs — three of which meant a connector was non-functional at runtime despite compiling and packaging fine (Aerospike threw on every write, HBase NoClassDefFoundError on the first call, DynamoDB's custom-endpoint path never worked). Fixes for those are merged; the bug reports (#78, #80, #94, #97, #101) have the specifics. On the CI side: the test job is now sharded across three runners (~20 min to ~6), and every Testcontainers container has a bounded startup timeout so one stalled container can't hang a shard for the full 45-minute limit — that had been intermittently cancelling unrelated PRs. Two things I'd value input on rather than decide unilaterally: - #98 / #99 (Canal): the client can't handshake against modern protobuf. The integration test is ready, but the only way I found to make it work force-pins protobuf to 3.6.1 module-wide (2018, known CVEs). Better options might be canal 1.1.8 or shading — I've left the analysis on #99. - #61 (Debezium SQL Server): multi-partition connectors never receive https://www.google.com/url?q=http://task.id&source=gmail&ust=1783807509015000&sa=E, so open() NPEs. It touches the shared kafka-connect-adaptor startup path and blocks the Db2/Spanner source work, so I'd rather agree on the approach before implementing. Also FYI: Debezium is two minors behind with no LTS line (#63 tracks the 3.6.0 migration), and two connectors (OpenMLDB, HBase) have Apple-Silicon-only blockers I've prepared branches for on a Linux runner. Thanks to the contributors whose PRs are part of this — I rebased and merged several. Happy to discuss any of it, here or on the issues. David -- This email and any attachments are intended solely for the recipient(s) named above and may contain confidential, privileged, or proprietary information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reproduction of this information is strictly prohibited. If you have received this email in error, please notify the sender immediately by replying to this email and delete it from your system.
