nzw921rx opened a new pull request, #10878:
URL: https://github.com/apache/seatunnel/pull/10878
### Purpose of this pull request
Related: #10666
Phase 1 implementation for STIP-24 (lightweight edge collector ingress model
in Zeta), focused on source-side remote collection via EdgeSocket.
This PR builds the MVP data path for edge collection: remote edge client
connection -> authenticated ingress -> payload deserialization ->
transform/sink execution in Zeta.
No edge-side sink model is introduced in this phase; edge sink and
edge-cluster control plane are out of scope for Phase 1.
**Changes**:
- Add `EdgeSocket` source connector implementation for Zeta runtime,
including source factory, source reader, deserialization path, and source
option model.
- Add source-side configs and enums for auth mode/token, packet mode,
compression/encryption handling, reconnect/retry and timeout controls.
- Add queue/runtime support classes for edge ingress payload handling and
bounded offer behavior under load.
- Add Zeta client-server protocol support for task-group address discovery
(`getJobTaskGroupAddresses`), including codec, client API, server
operation/message task registration, and coordinator/master query path.
- Add plugin mapping entry for `seatunnel.source.EdgeSocket` so the engine
can discover and load the connector.
- Add E2E module (`connector-edge-socket-e2e`) with MySQL 8.4 sink + SQL
transform coverage, plus test container bootstrap files (`my8-4.cnf`,
`setup.sql`) and JDBC driver injection.
- Add connector documentation and changelog updates in both `docs/en` and
`docs/zh`.
### Does this PR introduce _any_ user-facing change?
Yes, but **no impact on existing jobs**.
- New source connector capability for edge ingestion via `EdgeSocket`.
- New user-facing source options for authentication, packet mode, and
connection/retry behavior.
- New plugin mapping entry for `seatunnel.source.EdgeSocket`.
- New Zeta client capability to query task-group addresses
(`getJobTaskGroupAddresses`) for edge routing and diagnostics.
- Updated connector docs and changelog in both English and Chinese.
### How was this patch tested?
- Unit tests:
- `EdgeSocketFactoryTest` for source factory/config behavior
- engine client/server tests for `getJobTaskGroupAddresses` protocol path
- E2E test (`EdgeSocketMysql8_4IT`) for:
- start Zeta job with EdgeSocket source
- query task-group address through HZ negotiation path
- send records from a simulated edge collector
- apply SQL transform
- write to MySQL sink and verify results
- Manual verification for plugin discovery and connector loading path in the
e2e container.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
