DanielCarter-stack commented on issue #10355: URL: https://github.com/apache/seatunnel/issues/10355#issuecomment-3763704089
<!-- code-pr-reviewer --> Thanks for proposing this feature. SeaTunnel currently lacks a native BigQuery sink connector; users must use the JDBC connector with limited performance. A new `connector-bigquery` module would be valuable. We already have patterns to follow: - **Multi-table support**: `seatunnel-connectors-v2/connector-doris/src/main/java/org/apache/seatunnel/connectors/doris/sink/writer/DorisSinkWriter.java` (implements `SupportMultiTableSinkWriter`) - **Google Cloud integration**: `seatunnel-connectors-v2/connector-google-firestore/pom.xml` (google-cloud SDK dependencies + Service Account auth) - **Per-table config pattern**: `seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/main/java/org/apache/seatunnel/connectors/cdc/base/config/JdbcSourceTableConfig.java` (table-configs array structure) **Key questions**: 1. Should this support CDC → BigQuery with dynamic multi-table routing? 2. Streaming Insert API vs batch load via GCS — which takes priority? 3. Is schema evolution with partition/clustering DDL required? Contributions are welcome. A starting point would be creating `seatunnel-connectors-v2/connector-bigquery` with `BigQuerySinkFactory`/`BigQuerySinkWriter`, leveraging the `google-cloud-bigquery` SDK and Firestore authentication patterns. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
