programmerloverun opened a new pull request, #10849:
URL: https://github.com/apache/seatunnel/pull/10849
## Purpose
Closes #10847
Add a native Google Cloud Bigtable connector supporting both Source (read)
and
Sink (write) operations.
## Changes
### New Module: `connector-google-bigtable`
**Sink**
- `BigtableSinkWriter`: batch writes via `BulkMutation`; configurable batch
size, null mode (SKIP/EMPTY), row key composition (single/composite with
delimiter), multi-column-family mapping, version column for cell timestamp
- `BigtableSink` / `BigtableSinkFactory`: SPI registration,
`schema_save_mode`
/ `data_save_mode` support
**Source**
- `BigtableSourceReader`: full table scan with row key range, timestamp
range
filter, max-versions filter, scan row limit
- `BigtableSourceSplitEnumerator`: single split (full table or user-defined
row key range)
- `BigtableSource` / `BigtableSourceFactory`: SPI registration, bounded
source
**Shared**
- `BigtableClient`: wraps `google-cloud-bigtable:2.39.1` Data v2 client;
supports service account JSON credentials and Application Default
Credentials
(ADC)
- `BigtableDeserializationFormat`: `ByteString` → `SeaTunnelRow` type
mapping;
numeric types as big-endian binary, string/date types as UTF-8 text
- Full `SeaTunnelErrorCode` error code definitions
### Documentation
- `docs/en/connectors/sink/GoogleBigtable.md`
- `docs/en/connectors/source/GoogleBigtable.md`
- `docs/zh/connectors/sink/GoogleBigtable.md`
- `docs/zh/connectors/source/GoogleBigtable.md`
## Test
- `BigtableSinkWriterTest`: batch flush on size threshold, flush on close,
null field skip
- `BigtableSourceSplitEnumeratorTest`: full range split, custom row key
range
split
- All 5 unit tests pass; Spotless format check passes
## Known Limitations
- `schema_save_mode=CREATE_SCHEMA_WHEN_NOT_EXIST`: table/column-family
creation via Admin API is not yet implemented; users must create them
manually
- `data_save_mode=DROP_DATA`: table truncation via Admin API is not yet
implemented
- Source currently generates a single split (no parallel read); tablet
boundary partitioning can be added in a follow-up
- TTL must be configured at column-family level in Bigtable; per-cell TTL
is
not supported by the connector
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
