This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a change to branch davidzollo_republish_10943_parquet_int96
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
from aa1a7ec424 Merge remote-tracking branch 'apache/dev' into
davidzollo_republish_10943_parquet_int96
add 60c3ef88dd [Improve][E2E][DB2] Replace Db2CDCIT wget with
DependencyJar (#12041)
add 495cf931db [Feature][Connector-V2][HugeGraph] Use engine-level timer
flush (#12020)
add 65936d6e08 [Fix][Format] Preserve Avro field case during serialization
(#12009)
add c6b363af5c [Feature][Zeta] Add checkpoint completion benchmark (#12026)
add fa001a4e47 [Fix][Connector-V2] Remove fixed sleep in AssertSinkWriter
close and assert own table only (#11995)
add 03ef9f1c00 [Improve][Connector-V2][AmazonDynamoDB] Migrate max retries
validation to OptionRule (#11821)
add 5dc71549ad [Fix][Core] Do not charge the byte rate limiter for a
zero-byte row (#11766)
add 96d8e8d39d [Feature][Connector-V2] Add Google Cloud Storage file
source (#11967)
add 520b205d43 [Fix][Format][Protobuf] Support empty Schema Registry
payloads (#12039)
add a81a312125 [Test][E2E] Guard stop-with-savepoint hang when
precondition is rejected (#12035)
add 5d7c81cdcb [Test][E2E] Guard terminal-zombie-job restore gate after
master switch (#12034)
add 30170807f8 [Test][E2E] Add regression coverage for FAILED-pipeline
metrics cleanup (#12032)
add 24a2a60c18 [Test][E2E] Add recovery lifecycle REST API coverage
(#11948)
add 6b3e37b4af [Fix][Connector-Kafka] Warm up Kafka topics before
submitting e2e jobs to avoid topic-readiness flake (#11835)
add c213d612b0 [Feature][Zeta] Add state storage and recovery benchmarks
(#12044)
add 2e2e9b8040 [Improve][Connector-V2] Use isEmpty() instead of size() > 0
in ES connectors (#11996)
add f6d230ecd0 [Feature][Format] Add Debezium JSON SerDe microbenchmarks
(#12056)
add 6c5b5b630f [Improve][Connector-V2][Neo4j] Migrate configuration
validation to OptionRule (#12047)
add 9ebb27e8f9 [Docs] Add JMH and Zeta benchmark architecture (#12062)
add 1e450e11f7 [Fix][Connector-V2][AmazonSqs] Preserve messages on
deserialization failure (#12038)
add 98182ca59c [Improve][Connector-V2] Migrate HugeGraph sink validation
to OptionRule (#12075)
add bbfc188e61 [Fix][Zeta] Stabilize CI test isolation (#12050)
add 0301f7cd57 [Improve][Benchmark] Expose JMH CV and error in comparison
report (#12089)
add aa24c7e9e5 [Improve][Connector-V2][Aerospike] Migrate data format
validation to OptionRule (#12090)
add 83454b590d [Fix][Connector-V2][AmazonSqs] Support multi-row CDC
deserialization (#12084)
add ce4cde54c8 [Improve][Core] Add reusable config validation result
(#12065)
add b05f17c317 [Feature][Connector-V2][Shopify] Add Shopify source
connector (#11028)
add 7df6c9467a [Fix][Connector-V2] Preserve HTTP source JSON body (#11535)
add 97d461bc07 [Improve][Core] Warn that the base64 config shade does not
encrypt anything (#12073)
add c2bd543ffc Connector couchbase migration (#12076)
add 76c22220d7 [Improve][Zeta] Bound the size of a config file uploaded to
the REST server (#12072)
add 0badccfbe2 [Feature][Connector-Kafka] Support Confluent-framed Avro
payloads (#12088)
add 0b559fbf7b [Improve][Zeta] Return the failure message instead of a
stack trace from REST errors (#12070)
add 04fabece50 [Fix][Connector-V2][AmazonSqs] Wrap JSON parse failures
with connector error code (#12085)
add af0a647d27 [Bug][Connector-V2][CDC] Prune removed tables from restored
incremental splits (#11271)
add a0f5a56b86 [Docs][Examples] Add batch MySQL to HDFS partitioned
Parquet example (#11953)
add d3f89cd31c [Fix][Zeta] Propagate interrupted pending job insertion
(#12052)
add c57dfaa8d0 [Feature][Connector-V2] Add Amazon DocumentDB source
connector (#12046)
add c20fefe3bf [Fix][Zeta] Add E2E coverage for telemetry startup deadlock
regression (apache/seatunnel#10840) (#12133)
add 10206bc7aa [Improve][E2E] Validate multi-arch Spark image builds
(#11836)
add 6957d0f134 [Test][E2E] Add regression coverage for
SinkAggregatedCommitterTask checkpoint-cache leak (#12095)
add a8a43cfed4 [Test][E2E] Add regression coverage for slot release after
resource shortfall (#12098)
add b0624cb85d [Test][E2E] Document healthy-pipeline cascade-cancel on
sibling failure (#12100)
add d50f1b6044 [Test][E2E] Add regression coverage for WAIT-strategy
head-of-line blocking (#12103)
add f9184b374a [Test][E2E] Add scale coverage for CoordinatorService
executor growth during mass failover (#12104)
add f977afea04 [Test][E2E] Cover classloader release on failed multi-task
TaskGroup deployment (#12028)
add 5b1f7f70f7 [Test][E2E] Add regression coverage for deploy idempotency
during master failover (#12029)
add aae183be7d [Test][E2E] Add master failover test during source close
handshake (#12031)
add 96836fc68e [Test][E2E] Add regression coverage for master-election
loop retry after transient errors (#12033)
add 2f67a4d948 [Fix][Connector-V2] Fix file content comparison across read
boundaries (#12080)
add 2f7bdb1690 [Docs][Connector-V2] Improve IoTDB Kafka Maxcompute
OceanBase and Oracle connector docs (#11800)
add f0c4590557 [Docs][Connector-V2] Improve Datahub Socket Web3j and
ActiveMQ connector docs (#11788)
add e68e285dde [Test][Zeta] Add E2E coverage for classloader release when
deployLocalTask fails before context publication (apache/seatunnel#11808)
(#12143)
add 0821d8a770 [Test][Zeta] Add E2E coverage for savepoint barrier
injection against a busy source (apache/seatunnel#11473) (#12141)
add af5c3f3101 [Docs] Document S3File credential provider support for
container environments (#11871)
add b59152ffad [Improve][Connector-V2] Validate nonblank Sentry DSN
(#12148)
add 85ed97f4c0 [Docs][Connector-V2] Improve RabbitMQ Milvus and Email
connector docs (#12155)
add 84e43a95b4 [Docs][Connector-V2] Improve InfluxDB Greenplum Mqtt and
Aerospike connector docs (#11722)
add 7a58d1005b [Fix][Connector-V2] Propagate JDBC XA commit failures
(#11569)
add 72acda5704 [Feature][Connector-V2] Add Google Cloud Storage file sink
(#12146)
add c3f79ffb7c [Improve][Connector-V2] Validate nonblank DataHub sink
options (#12174)
add d85d8cc734 [Fix][Connector-V2] Use OAuth token for Slack messages
(#12171)
add 0c7e7f417d [Improve][Connector-V2] Validate DingTalk required options
(#12170)
add 42f1f5491b [Fix][Connector-V2][JDBC][DB2] Read default values as
strings in catalog (#12157)
add d73db95679 [Fix][Connector-V2][JDBC] Use range fallback when
approximate row count is unavailable (#12159)
add e00e1ad721 [Fix][Connector-V2][JDBC][SapHana] Add NCHAR type support
in catalog conversion (#12158)
add 8bea8c681c [Feature][Connector-V2] Support PostgreSQL CDC ADD COLUMN
schema evolution (#11922)
add 4c51cb3191 Merge commit '8bea8c681cacccbd99fa642ca0853718980b9234'
into pr11928-59a984
No new revisions were added by this update.
Summary of changes:
.github/workflows/benchmarks.yml | 32 +-
.github/workflows/labeler/label-scope-conf.yml | 5 +
.github/workflows/publish-e2e-spark-images.yaml | 22 +-
config/plugin_config | 2 +
.../changelog/connector-amazondocumentdb.md | 24 +
...connector-bigquery.md => connector-file-gcs.md} | 0
...nector-cdc-db2.md => connector-http-shopify.md} | 0
docs/en/connectors/formats/avro.md | 3 +-
docs/en/connectors/sink/Activemq.md | 25 +
docs/en/connectors/sink/Aerospike.md | 42 ++
docs/en/connectors/sink/AmazonDynamoDB.md | 3 +-
docs/en/connectors/sink/Couchbase.md | 23 +-
docs/en/connectors/sink/Datahub.md | 15 +
docs/en/connectors/sink/Email.md | 20 +
docs/en/connectors/sink/GcsFile.md | 172 +++++
docs/en/connectors/sink/Greenplum.md | 8 +-
docs/en/connectors/sink/HugeGraph.md | 16 +-
docs/en/connectors/sink/InfluxDB.md | 37 +
docs/en/connectors/sink/Jdbc.md | 10 +
docs/en/connectors/sink/Milvus.md | 16 +
docs/en/connectors/sink/Neo4j.md | 10 +-
docs/en/connectors/sink/Rabbitmq.md | 16 +
docs/en/connectors/sink/S3File.md | 103 +++
docs/en/connectors/sink/Socket.md | 15 +
docs/en/connectors/source/AmazonDocumentDB.md | 231 +++++++
docs/en/connectors/source/AmazonSqs.md | 5 +
docs/en/connectors/source/GcsFile.md | 138 ++++
docs/en/connectors/source/Greenplum.md | 14 +-
docs/en/connectors/source/InfluxDB.md | 66 ++
docs/en/connectors/source/Kafka.md | 4 +-
docs/en/connectors/source/Maxcompute.md | 2 +-
docs/en/connectors/source/Milvus.md | 20 +
docs/en/connectors/source/Mqtt.md | 34 +-
docs/en/connectors/source/PostgreSQL-CDC.md | 26 +
docs/en/connectors/source/Rabbitmq.md | 21 +
docs/en/connectors/source/S3File.md | 99 +++
docs/en/connectors/source/Shopify.md | 135 ++++
docs/en/connectors/source/Socket.md | 15 +
docs/en/connectors/source/Web3j.md | 15 +
docs/en/engines/zeta/benchmark.md | 177 ++++-
docs/en/engines/zeta/checkpoint-storage.md | 2 +
docs/en/engines/zeta/rest-api-v2.md | 21 +-
docs/en/getting-started/recipes/mysql-to-hdfs.md | 215 ++++++
docs/en/getting-started/recipes/overview.md | 1 +
.../introduction/concepts/incompatible-changes.md | 11 +
docs/sidebars.js | 1 +
.../changelog/connector-amazondocumentdb.md | 24 +
.../connectors/changelog/connector-file-gcs.md} | 0
.../changelog/connector-http-shopify.md} | 0
docs/zh/connectors/formats/avro.md | 3 +-
docs/zh/connectors/sink/Activemq.md | 19 +
docs/zh/connectors/sink/AmazonDynamoDB.md | 3 +-
docs/zh/connectors/sink/Couchbase.md | 21 +-
docs/zh/connectors/sink/Datahub.md | 15 +
docs/zh/connectors/sink/Email.md | 20 +
docs/zh/connectors/sink/GcsFile.md | 165 +++++
docs/zh/connectors/sink/HugeGraph.md | 16 +-
docs/zh/connectors/sink/Jdbc.md | 8 +
docs/zh/connectors/sink/Milvus.md | 16 +
docs/zh/connectors/sink/Neo4j.md | 10 +-
docs/zh/connectors/sink/Rabbitmq.md | 16 +
docs/zh/connectors/sink/S3File.md | 103 +++
docs/zh/connectors/sink/Socket.md | 15 +
docs/zh/connectors/source/AmazonDocumentDB.md | 231 +++++++
docs/zh/connectors/source/AmazonSqs.md | 5 +
docs/zh/connectors/source/GcsFile.md | 128 ++++
docs/zh/connectors/source/Greenplum.md | 8 +-
docs/zh/connectors/source/Kafka.md | 6 +-
docs/zh/connectors/source/Maxcompute.md | 2 +-
docs/zh/connectors/source/Milvus.md | 20 +
docs/zh/connectors/source/PostgreSQL-CDC.md | 24 +
docs/zh/connectors/source/Rabbitmq.md | 20 +
docs/zh/connectors/source/S3File.md | 99 +++
docs/zh/connectors/source/Shopify.md | 131 ++++
docs/zh/connectors/source/Socket.md | 15 +
docs/zh/connectors/source/Web3j.md | 15 +
docs/zh/engines/zeta/benchmark.md | 170 ++++-
docs/zh/engines/zeta/checkpoint-storage.md | 2 +
docs/zh/engines/zeta/rest-api-v2.md | 19 +-
docs/zh/getting-started/recipes/mysql-to-hdfs.md | 215 ++++++
docs/zh/getting-started/recipes/overview.md | 1 +
.../introduction/concepts/incompatible-changes.md | 11 +
plugin-mapping.properties | 4 +
seatunnel-benchmarks/README.md | 9 +
seatunnel-benchmarks/pom.xml | 10 +
.../seatunnel/benchmark/BenchmarkTemplates.java | 79 +++
.../benchmark/CheckpointBenchmarkTrigger.java | 69 ++
.../benchmark/CheckpointStorageBenchmark.java | 124 ++++
.../benchmark/CheckpointingTimeBenchmark.java | 66 ++
.../CheckpointingTimeBenchmarkPipeline.java | 261 +++++++
.../benchmark/DebeziumJsonFormatBenchmark.java | 304 ++++++++
.../benchmark/IMapDagStorageBenchmark.java | 90 +++
.../benchmark/IMapJobStorageBenchmark.java | 145 ++++
.../benchmark/IMapWalStorageBenchmark.java | 111 +++
.../SeaTunnelCheckpointEnvironmentContext.java | 348 ++++++++++
.../benchmark/SeaTunnelEnvironmentContext.java | 52 +-
.../SeaTunnelObservabilityEnvironmentContext.java | 2 +-
...TunnelObservabilityTraceEnvironmentContext.java | 7 +-
.../SeaTunnelTraceEnvironmentContext.java | 6 +-
.../connector/source/BenchmarkSourceReader.java | 4 +-
.../benchmark/dag/JobDagFixtureFactory.java | 90 +++
.../SeaTunnelStorageEnvironmentContext.java | 116 ++++
.../storage/StorageLifecycleFixtureJob.java | 283 ++++++++
.../BenchmarkCheckpointStorageFactory.java | 148 ++++
.../CheckpointIdIncrementBenchmarkWorkload.java | 98 +++
.../CheckpointOverviewBenchmarkWorkload.java | 93 +++
.../CheckpointPersistenceBenchmarkWorkload.java | 113 +++
.../CheckpointStorageBenchmarkFixture.java | 361 ++++++++++
.../imap/IMapDagStorageBenchmarkWorkload.java | 172 +++++
.../imap/IMapJobGrowthBenchmarkWorkload.java | 298 ++++++++
.../imap/IMapJobRecoveryBenchmarkWorkload.java | 122 ++++
.../imap/IMapJobStorageBenchmarkWorkload.java | 166 +++++
.../imap/IMapMetricsReportBenchmarkWorkload.java | 124 ++++
.../imap/IMapWalAppendBenchmarkWorkload.java | 201 ++++++
.../imap/IMapWalRecoveryBenchmarkWorkload.java | 150 ++++
...checkpoint.storage.api.CheckpointStorageFactory | 11 +-
.../benchmark/engine-checkpoint.yaml.template | 33 +-
.../benchmark/engine-storage.yaml.template | 34 +-
.../hazelcast-checkpoint-master.yaml.template | 33 +-
.../hazelcast-checkpoint-worker.yaml.template | 27 +-
.../benchmark/hazelcast-storage.yaml.template | 36 +-
.../benchmark/source-sink-checkpoint.conf.template | 56 +-
.../storage-lifecycle-fixture-job.conf.template | 55 +-
.../benchmark/BenchmarkTemplatesTest.java | 107 +++
.../benchmark/CheckpointingTimeBenchmarkTest.java | 72 ++
.../benchmark/DebeziumJsonFormatBenchmarkTest.java | 74 ++
.../benchmark/dag/JobDagFixtureFactoryTest.java | 56 ++
.../imap/IMapWalAppendBenchmarkWorkloadTest.java | 43 ++
.../aerospike/sink/AerospikeSinkFactory.java | 6 +-
.../seatunnel/aerospike/AerospikeFactoryTest.java | 58 +-
.../pom.xml | 52 +-
.../config/AmazonDocumentDBConfig.java | 275 ++++++++
.../config/AmazonDocumentDBSourceOptions.java | 77 +++
.../serialize/DocumentDBItemDeserializer.java | 240 +++++++
.../source/AmazonDocumentDBSource.java | 90 +++
.../source/AmazonDocumentDBSourceFactory.java | 77 +++
.../source/AmazonDocumentDBSourceReader.java | 221 ++++++
.../source/AmazonDocumentDBSourceSplit.java | 62 ++
.../AmazonDocumentDBSourceSplitEnumerator.java | 200 ++++++
.../source/AmazonDocumentDBSourceState.java | 66 ++
.../AmazonDocumentDBSourceFactoryTest.java | 55 ++
.../DocumentDBItemDeserializerTest.java | 181 +++++
.../config/AmazonDocumentDBConfigTest.java | 134 ++++
.../source/AmazonDocumentDBSourceReaderTest.java | 301 ++++++++
.../AmazonDocumentDBSourceSplitEnumeratorTest.java | 189 +++++
.../config/AmazonDynamoDBConfig.java | 6 -
.../sink/AmazonDynamoDBSinkFactory.java | 3 +-
.../AmazonDynamoDBSinkFactoryTest.java | 104 +++
.../amazonsqs/config/AmazonSqsSourceConfig.java | 3 +
.../amazonsqs/config/AmazonSqsSourceOptions.java | 6 +
.../deserialize/AmazonSqsDeserializer.java | 98 ++-
.../deserialize/SeaTunnelRowDeserializer.java | 10 +
.../exception/AmazonSqsConnectorErrorCode.java | 28 +-
.../amazonsqs/source/AmazonSqsSource.java | 14 +-
.../amazonsqs/source/AmazonSqsSourceFactory.java | 20 +-
.../amazonsqs/source/AmazonSqsSourceReader.java | 26 +-
.../amazonsqs/AmazonSqsSourceFactoryTest.java | 6 +
.../source/AmazonSqsSourceReaderTest.java | 767 +++++++++++++++++++++
.../seatunnel/assertion/sink/AssertSinkWriter.java | 15 +-
.../assertion/sink/AssertSinkWriterCloseTest.java | 87 +++
.../cdc/base/dialect/DataSourceDialect.java | 15 +
.../cdc/base/schema/SchemaChangeResolver.java | 10 +
.../enumerator/IncrementalSplitAssigner.java | 6 +
.../source/reader/IncrementalSourceReader.java | 65 +-
.../cdc/base/source/split/IncrementalSplit.java | 66 ++
.../cdc/base/utils/SourceRecordUtils.java | 5 +-
.../row/SeaTunnelRowDebeziumDeserializeSchema.java | 6 +
.../enumerator/IncrementalSplitAssignerTest.java | 90 ++-
.../source/reader/IncrementalSourceReaderTest.java | 225 ++++++
.../base/source/split/IncrementalSplitTest.java | 189 +++++
.../seatunnel/cdc/db2/source/Db2Dialect.java | 12 +
.../source/Db2IncrementalSourceFactoryTest.java | 21 +
.../connector/postgresql/PostgresObjectUtils.java | 6 +-
.../postgresql/RelationAwarePostgresSchema.java | 96 +++
.../config/PostgresSourceConfigFactory.java | 6 +-
.../cdc/postgres/source/PostgresDialect.java | 9 +-
.../postgres/source/PostgresIncrementalSource.java | 28 +-
.../source/PostgresIncrementalSourceFactory.java | 23 +-
.../PostgresRelationSchemaChangeResolver.java | 206 ++++++
.../source/PostgresRelationSchemaRecord.java | 74 ++
.../reader/PostgresSourceFetchTaskContext.java | 76 +-
.../cdc/postgres/utils/PostgresTypeUtils.java | 20 +-
.../PostgresRelationSchemaChangeResolverTest.java | 226 ++++++
.../couchbase/config/CouchbaseSinkOptions.java | 13 -
.../couchbase/sink/CouchbaseSinkFactory.java | 3 -
.../seatunnel/couchbase/sink/CouchbaseWriter.java | 137 +---
.../couchbase/sink/CouchbaseWriterOptions.java | 8 -
.../couchbase/sink/CouchbaseWriterCloseTest.java | 256 -------
.../sink/CouchbaseWriterConstructorLeakTest.java | 1 -
.../couchbase/sink/CouchbaseWriterTest.java | 250 +++++++
.../seatunnel/datahub/sink/DataHubSinkFactory.java | 7 +-
.../seatunnel/datahub/DataHubFactoryTest.java | 72 +-
.../seatunnel/sink/DingTalkSinkFactory.java | 6 +-
.../connectors/seatunnel/DingTalkFactoryTest.java | 50 +-
.../serialize/index/IndexSerializerFactory.java | 2 +-
.../easysearch/sink/EasysearchSinkWriter.java | 2 +-
.../easysearch/source/EasysearchSourceReader.java | 2 +-
.../serialize/index/IndexSerializerFactory.java | 2 +-
.../sink/ElasticsearchSinkWriter.java | 2 +-
.../seatunnel/file/config/FileSystemType.java | 3 +-
.../file/hadoop/HadoopFileSystemProxy.java | 11 +
.../file/source/reader/AbstractReadStrategy.java | 20 +-
...uousMultipleTableFileSourceSplitEnumerator.java | 20 +-
.../hadoop/ChunkedInputHadoopFileSystemProxy.java | 120 ++++
.../file/hadoop/HadoopFileSystemProxyTest.java | 50 ++
.../file/source/reader/UpdateSyncModeTest.java | 79 +++
...MultipleTableFileSourceSplitEnumeratorTest.java | 51 ++
.../connector-file/connector-file-gcs/pom.xml | 69 ++
.../seatunnel/file/gcs/catalog/GcsFileCatalog.java | 44 ++
.../file/gcs/catalog/GcsFileCatalogFactory.java | 55 ++
.../file/gcs/config/GcsFileBaseOptions.java | 49 ++
.../file/gcs/config/GcsFileSinkOptions.java} | 9 +-
.../file/gcs/config/GcsFileSourceOptions.java} | 9 +-
.../seatunnel/file/gcs/config/GcsHadoopConf.java | 105 +++
.../seatunnel/file/gcs/sink/GcsFileSink.java | 48 ++
.../file/gcs/sink/GcsFileSinkFactory.java | 158 +++++
.../seatunnel/file/gcs/source/GcsFileSource.java | 45 ++
.../file/gcs/source/GcsFileSourceFactory.java | 164 +++++
.../gcs/source/config/GcsFileSourceConfig.java | 46 ++
.../config/MultipleTableGcsFileSourceConfig.java} | 22 +-
.../services/org.apache.hadoop.fs.FileSystem | 13 +-
.../seatunnel/file/gcs/GcsFileSinkFactoryTest.java | 188 +++++
.../file/gcs/GcsFileSourceFactoryTest.java | 95 +++
.../file/gcs/catalog/GcsFileCatalogTest.java | 106 +++
.../file/gcs/config/GcsHadoopConfTest.java | 130 ++++
seatunnel-connectors-v2/connector-file/pom.xml | 1 +
.../seatunnel/http/client/HttpClientProvider.java | 60 +-
.../http/client/HttpClientProviderTest.java | 117 ++++
.../{ => connector-http-shopify}/pom.xml | 33 +-
.../seatunnel/shopify/source/ShopifySource.java | 87 +++
.../shopify/source/ShopifySourceFactory.java | 64 ++
.../source/config/ShopifySourceOptions.java} | 30 +-
.../source/config/ShopifySourceParameter.java | 43 ++
.../seatunnel/shopify/ShopifyFactoryTest.java} | 8 +-
.../shopify/ShopifySourceParameterTest.java | 52 ++
.../seatunnel/shopify/ShopifySourceTest.java | 66 ++
seatunnel-connectors-v2/connector-http/pom.xml | 1 +
.../seatunnel/hugegraph/buffer/BatchBuffer.java | 63 +-
.../hugegraph/config/HugeGraphOptions.java | 4 +-
.../seatunnel/hugegraph/sink/HugeGraphSink.java | 2 +-
.../hugegraph/sink/HugeGraphSinkFactory.java | 25 +-
.../hugegraph/sink/HugeGraphSinkWriter.java | 49 +-
.../hugegraph/sink/HugeGraphSinkFactoryTest.java | 95 ++-
.../sink/HugeGraphSinkWriterUpdateTest.java | 50 ++
.../seatunnel/jdbc/catalog/db2/DB2Catalog.java | 2 +-
.../dialect/saphana/SapHanaTypeConverter.java | 3 +
.../seatunnel/jdbc/internal/xa/XaFacade.java | 7 +-
.../jdbc/internal/xa/XaFacadeImplAutoLoad.java | 14 +-
.../seatunnel/jdbc/internal/xa/XaGroupOps.java | 13 +-
.../seatunnel/jdbc/internal/xa/XaGroupOpsImpl.java | 9 +-
.../jdbc/sink/JdbcSinkAggregatedCommitter.java | 278 +++++++-
.../jdbc/source/DynamicChunkSplitter.java | 114 +++
.../seatunnel/jdbc/utils/ObjectUtils.java | 16 +-
.../seatunnel/jdbc/catalog/db2/DB2CatalogTest.java | 35 +
.../dialect/saphana/SapHanaTypeConverterTest.java | 14 +
.../jdbc/internal/xa/XaFacadeImplAutoLoadTest.java | 145 ++++
.../jdbc/internal/xa/XaGroupOpsImplTest.java | 122 ++++
.../jdbc/sink/JdbcSinkAggregatedCommitterTest.java | 440 ++++++++++++
.../jdbc/source/DynamicChunkSplitterTest.java | 60 +-
.../seatunnel/kafka/config/KafkaSourceOptions.java | 5 +-
.../seatunnel/kafka/source/KafkaSourceConfig.java | 17 +-
.../config/Neo4jAuthenticationConditions.java | 60 ++
.../seatunnel/neo4j/config/Neo4jQueryInfo.java | 58 +-
.../seatunnel/neo4j/config/Neo4jSinkQueryInfo.java | 31 +-
.../neo4j/config/Neo4jSourceQueryInfo.java | 4 +-
.../seatunnel/neo4j/sink/Neo4jSinkFactory.java | 26 +-
.../seatunnel/neo4j/source/Neo4jSourceFactory.java | 12 +-
.../seatunnel/neo4j/Neo4jFactoryTest.java | 149 ++++
.../python/spawn_stdout_child_then_exit.py | 4 +-
.../seatunnel/sentry/sink/SentrySinkFactory.java | 3 +-
.../seatunnel/sentry/SentryFactoryTest.java | 41 +-
.../seatunnel/slack/client/SlackClient.java | 26 +-
.../seatunnel/slack/client/SlackClientTest.java} | 18 +-
seatunnel-connectors-v2/pom.xml | 1 +
.../core/starter/flowcontrol/FlowControlGate.java | 18 +-
.../core/starter/utils/ConfigShadeUtils.java | 6 +
.../starter/validation/ConfigValidationError.java | 74 ++
.../starter/validation/ConfigValidationResult.java | 116 ++++
.../starter/flowcontrol/FlowControlGateTest.java | 18 +
.../validation/ConfigValidationResultTest.java | 66 ++
.../command/SeaTunnelConfValidateCommand.java | 90 +++
.../command/SeaTunnelConfValidateCommandTest.java | 111 ++-
seatunnel-dist/pom.xml | 18 +
seatunnel-dist/release-docs/NOTICE | 104 +++
.../licenses/LICENSE-gcs-connector.txt | 1 +
.../connector-amazondocumentdb-e2e/pom.xml | 51 ++
.../amazondocumentdb/AmazonDocumentDBIT.java | 155 +++++
.../resources/amazondocumentdb_source_basic.conf | 65 ++
.../amazondocumentdb_source_projection.conf | 60 ++
.../amazondocumentdb_source_query_filter.conf | 56 ++
.../azurecosmosdb/AbstractAzureCosmosDBIT.java | 33 +-
.../seatunnel/e2e/connector/cdc/db2/Db2CDCIT.java | 20 +-
.../seatunnel/cdc/postgres/PostgresCDCIT.java | 52 +-
.../postgrescdc_to_postgres_test_add_Filed.conf | 4 +-
.../e2e/connector/couchbase/CouchbaseIT.java | 114 ++-
.../test/resources/fake_source_to_couchbase.conf | 1 -
...f => fake_source_to_couchbase_timer_flush.conf} | 16 +-
.../e2e/connector/databend/DatabendIT.java | 2 +
.../elasticsearch/ElasticsearchAuthIT.java | 48 +-
.../connector-file-gcs-e2e/pom.xml | 58 ++
.../e2e/connector/file/gcs/GcsFileIT.java | 118 ++++
.../src/test/resources/gcs/data/e2e.json | 2 +
.../src/test/resources/gcs/gcs_file_to_assert.conf | 84 +++
.../src/test/resources/gcs/gcs_file_to_gcs.conf | 67 ++
.../src/test/resources/gcs/gcs_sink_to_assert.conf | 63 ++
.../connector-http-e2e/pom.xml | 6 +
.../seatunnel/e2e/connector/http/HttpIT.java | 4 +
.../src/test/resources/mockserver-config.json | 59 +-
.../src/test/resources/shopify_json_to_assert.conf | 97 +++
.../e2e/connector/hugegraph/HugeGraphSourceIT.java | 12 +-
.../jdbc/internal/xa/XaGroupOpsImplIT.java | 75 +-
.../e2e/connector/kafka/AbstractKafkaIT.java | 173 +++++
.../seatunnel/e2e/connector/kafka/KafkaIT.java | 18 +-
.../e2e/connector/v2/milvus/MilvusIT.java | 45 +-
.../e2e/connector/rocketmq/RocketMqContainer.java | 2 +-
.../e2e/connector/rocketmq/RocketMqIT.java | 44 +-
seatunnel-e2e/seatunnel-connector-v2-e2e/pom.xml | 2 +
.../src/test/docker/spark/Dockerfile | 3 +
.../src/test/docker/spark/docker-bake.hcl | 8 +-
.../container/seatunnel/SeaTunnelContainer.java | 24 +
.../SeaTunnelContainerThreadExemptionTest.java | 24 +-
.../e2e/CheckpointCoordinatorFailoverIT.java | 300 ++++++++
.../e2e/ClusterFaultToleranceTwoPipelineIT.java | 189 +++++
.../CoordinatorExecutorMassFailoverStormIT.java | 449 ++++++++++++
.../seatunnel/engine/e2e/JobExecutionIT.java | 105 +++
.../engine/e2e/RecoveryLifecycleRestApiIT.java | 489 +++++++++++++
.../engine/e2e/SavepointBusySourceBarrierIT.java | 368 ++++++++++
.../e2e/SavepointPreconditionRecoveryIT.java | 335 +++++++++
.../seatunnel/engine/e2e/SeaTunnelSlotIT.java | 103 +++
...gregatedCommitterTaskCheckpointCacheLeakIT.java | 267 +++++++
.../engine/e2e/SplitClusterFaultToleranceIT.java | 349 ++++++++++
.../SplitClusterPendingJobLifecycleFailoverIT.java | 640 +++++++++++++++++
.../e2e/TaskDeploymentClassLoaderLeakIT.java | 308 +++++++++
...kDeploymentPrePublicationClassLoaderLeakIT.java | 733 ++++++++++++++++++++
.../e2e/telemetry/TelemetryStartupDeadlockIT.java | 473 +++++++++++++
.../batch_fake_to_console_minimal_slot.conf | 33 +-
...ocalfile_close_handshake_failover_template.conf | 78 +++
...h_one_pipeline_permanently_failed_template.conf | 104 +++
.../pending_jobs_streaming_unschedulable.conf | 31 +-
...tream_fake_busy_split_to_console_savepoint.conf | 60 ++
..._localfile_savepoint_precondition_template.conf | 47 +-
...ing_fake_to_localfile_committer_cache_leak.conf | 59 +-
.../src/test/resources/hazelcast-client.yaml | 6 +-
.../src/test/resources/hazelcast.yaml | 4 +-
.../config/YamlSeaTunnelDomConfigProcessor.java | 18 +
.../engine/common/config/server/HttpConfig.java | 8 +
.../common/config/server/ServerConfigOptions.java | 14 +
.../engine/server/CoordinatorService.java | 10 +-
.../seatunnel/engine/server/JettyService.java | 13 +-
.../rest/filter/ExceptionHandlingFilter.java | 4 +-
.../engine/server/utils/PeekBlockingQueue.java | 10 +-
.../engine/server/CoordinatorServiceTest.java | 170 ++++-
.../engine/server/rest/BaseServletTest.java | 2 +-
.../rest/filter/ExceptionHandlingFilterTest.java | 80 +++
.../engine/server/utils/PeekBlockingQueueTest.java | 2 +-
.../format/avro/AvroDeserializationSchema.java | 35 +-
.../seatunnel/format/avro/RowToAvroConverter.java | 5 +-
.../format/avro/AvroSerializationSchemaTest.java | 84 +++
...RegistryAwareProtobufDeserializationSchema.java | 94 ++-
...stryAwareProtobufDeserializationSchemaTest.java | 210 ++++--
tools/benchmarks/regression_report.py | 38 +-
tools/benchmarks/run_benchmarks.sh | 94 ++-
.../benchmarks/suites/benchmarks_core.txt | 28 +-
tools/benchmarks/test_regression_report.py | 44 ++
tools/benchmarks/test_run_benchmarks.py | 174 +++++
365 files changed, 25961 insertions(+), 1360 deletions(-)
create mode 100644 docs/en/connectors/changelog/connector-amazondocumentdb.md
copy docs/en/connectors/changelog/{connector-bigquery.md =>
connector-file-gcs.md} (100%)
copy docs/en/connectors/changelog/{connector-cdc-db2.md =>
connector-http-shopify.md} (100%)
create mode 100644 docs/en/connectors/sink/GcsFile.md
create mode 100644 docs/en/connectors/source/AmazonDocumentDB.md
create mode 100644 docs/en/connectors/source/GcsFile.md
create mode 100644 docs/en/connectors/source/Shopify.md
create mode 100644 docs/en/getting-started/recipes/mysql-to-hdfs.md
create mode 100644 docs/zh/connectors/changelog/connector-amazondocumentdb.md
copy docs/{en/connectors/changelog/connector-bigquery.md =>
zh/connectors/changelog/connector-file-gcs.md} (100%)
copy docs/{en/connectors/changelog/connector-cdc-db2.md =>
zh/connectors/changelog/connector-http-shopify.md} (100%)
create mode 100644 docs/zh/connectors/sink/GcsFile.md
create mode 100644 docs/zh/connectors/source/AmazonDocumentDB.md
create mode 100644 docs/zh/connectors/source/GcsFile.md
create mode 100644 docs/zh/connectors/source/Shopify.md
create mode 100644 docs/zh/getting-started/recipes/mysql-to-hdfs.md
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/BenchmarkTemplates.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/CheckpointBenchmarkTrigger.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/CheckpointStorageBenchmark.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/CheckpointingTimeBenchmark.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/CheckpointingTimeBenchmarkPipeline.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/DebeziumJsonFormatBenchmark.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/IMapDagStorageBenchmark.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/IMapJobStorageBenchmark.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/IMapWalStorageBenchmark.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/SeaTunnelCheckpointEnvironmentContext.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/dag/JobDagFixtureFactory.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/SeaTunnelStorageEnvironmentContext.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/StorageLifecycleFixtureJob.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/checkpoint/BenchmarkCheckpointStorageFactory.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/checkpoint/CheckpointIdIncrementBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/checkpoint/CheckpointOverviewBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/checkpoint/CheckpointPersistenceBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/checkpoint/CheckpointStorageBenchmarkFixture.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/imap/IMapDagStorageBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/imap/IMapJobGrowthBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/imap/IMapJobRecoveryBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/imap/IMapJobStorageBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/imap/IMapMetricsReportBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/imap/IMapWalAppendBenchmarkWorkload.java
create mode 100644
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/storage/imap/IMapWalRecoveryBenchmarkWorkload.java
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast-client.yaml
=>
seatunnel-benchmarks/src/main/resources/META-INF/services/org.apache.seatunnel.engine.checkpoint.storage.api.CheckpointStorageFactory
(79%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast-client.yaml
=>
seatunnel-benchmarks/src/main/resources/benchmark/engine-checkpoint.yaml.template
(57%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast-client.yaml
=>
seatunnel-benchmarks/src/main/resources/benchmark/engine-storage.yaml.template
(56%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast.yaml =>
seatunnel-benchmarks/src/main/resources/benchmark/hazelcast-checkpoint-master.yaml.template
(63%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast-client.yaml
=>
seatunnel-benchmarks/src/main/resources/benchmark/hazelcast-checkpoint-worker.yaml.template
(65%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast.yaml =>
seatunnel-benchmarks/src/main/resources/benchmark/hazelcast-storage.yaml.template
(60%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast.yaml =>
seatunnel-benchmarks/src/main/resources/benchmark/source-sink-checkpoint.conf.template
(50%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast.yaml =>
seatunnel-benchmarks/src/main/resources/benchmark/storage-lifecycle-fixture-job.conf.template
(54%)
create mode 100644
seatunnel-benchmarks/src/test/java/org/apache/seatunnel/benchmark/BenchmarkTemplatesTest.java
create mode 100644
seatunnel-benchmarks/src/test/java/org/apache/seatunnel/benchmark/CheckpointingTimeBenchmarkTest.java
create mode 100644
seatunnel-benchmarks/src/test/java/org/apache/seatunnel/benchmark/DebeziumJsonFormatBenchmarkTest.java
create mode 100644
seatunnel-benchmarks/src/test/java/org/apache/seatunnel/benchmark/dag/JobDagFixtureFactoryTest.java
create mode 100644
seatunnel-benchmarks/src/test/java/org/apache/seatunnel/benchmark/storage/imap/IMapWalAppendBenchmarkWorkloadTest.java
copy seatunnel-connectors-v2/{connector-http =>
connector-amazondocumentdb}/pom.xml (55%)
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/config/AmazonDocumentDBConfig.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/config/AmazonDocumentDBSourceOptions.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/serialize/DocumentDBItemDeserializer.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSource.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSourceFactory.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSourceReader.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSourceSplit.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSourceSplitEnumerator.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSourceState.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/test/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/AmazonDocumentDBSourceFactoryTest.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/test/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/DocumentDBItemDeserializerTest.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/test/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/config/AmazonDocumentDBConfigTest.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/test/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSourceReaderTest.java
create mode 100644
seatunnel-connectors-v2/connector-amazondocumentdb/src/test/java/org/apache/seatunnel/connectors/seatunnel/amazondocumentdb/source/AmazonDocumentDBSourceSplitEnumeratorTest.java
create mode 100644
seatunnel-connectors-v2/connector-amazondynamodb/src/test/java/org/apache/seatunnel/connectors/seatunnel/amazondynamodb/AmazonDynamoDBSinkFactoryTest.java
copy
seatunnel-benchmarks/src/main/java/org/apache/seatunnel/benchmark/SeaTunnelObservabilityEnvironmentContext.java
=>
seatunnel-connectors-v2/connector-amazonsqs/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazonsqs/exception/AmazonSqsConnectorErrorCode.java
(56%)
create mode 100644
seatunnel-connectors-v2/connector-amazonsqs/src/test/java/org/apache/seatunnel/connectors/seatunnel/amazonsqs/source/AmazonSqsSourceReaderTest.java
create mode 100644
seatunnel-connectors-v2/connector-assert/src/test/java/org/apache/seatunnel/connectors/seatunnel/assertion/sink/AssertSinkWriterCloseTest.java
create mode 100644
seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/test/java/org/apache/seatunnel/connectors/cdc/base/source/reader/IncrementalSourceReaderTest.java
create mode 100644
seatunnel-connectors-v2/connector-cdc/connector-cdc-base/src/test/java/org/apache/seatunnel/connectors/cdc/base/source/split/IncrementalSplitTest.java
create mode 100644
seatunnel-connectors-v2/connector-cdc/connector-cdc-postgres/src/main/java/io/debezium/connector/postgresql/RelationAwarePostgresSchema.java
create mode 100644
seatunnel-connectors-v2/connector-cdc/connector-cdc-postgres/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/postgres/source/PostgresRelationSchemaChangeResolver.java
create mode 100644
seatunnel-connectors-v2/connector-cdc/connector-cdc-postgres/src/main/java/org/apache/seatunnel/connectors/seatunnel/cdc/postgres/source/PostgresRelationSchemaRecord.java
create mode 100644
seatunnel-connectors-v2/connector-cdc/connector-cdc-postgres/src/test/java/org/apache/seatunnel/connectors/seatunnel/cdc/postgres/source/PostgresRelationSchemaChangeResolverTest.java
delete mode 100644
seatunnel-connectors-v2/connector-couchbase/src/test/java/org/apache/seatunnel/connectors/seatunnel/couchbase/sink/CouchbaseWriterCloseTest.java
create mode 100644
seatunnel-connectors-v2/connector-couchbase/src/test/java/org/apache/seatunnel/connectors/seatunnel/couchbase/sink/CouchbaseWriterTest.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-base/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/hadoop/ChunkedInputHadoopFileSystemProxy.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/pom.xml
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/catalog/GcsFileCatalog.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/catalog/GcsFileCatalogFactory.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/config/GcsFileBaseOptions.java
copy
seatunnel-connectors-v2/{connector-amazonsqs/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazonsqs/deserialize/SeaTunnelRowDeserializer.java
=>
connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/config/GcsFileSinkOptions.java}
(78%)
copy
seatunnel-connectors-v2/{connector-amazonsqs/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazonsqs/deserialize/SeaTunnelRowDeserializer.java
=>
connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/config/GcsFileSourceOptions.java}
(78%)
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/config/GcsHadoopConf.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/sink/GcsFileSink.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/sink/GcsFileSinkFactory.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/source/GcsFileSource.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/source/GcsFileSourceFactory.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/source/config/GcsFileSourceConfig.java
copy
seatunnel-connectors-v2/{connector-cdc/connector-cdc-base/src/main/java/org/apache/seatunnel/connectors/cdc/base/schema/SchemaChangeResolver.java
=>
connector-file/connector-file-gcs/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/source/config/MultipleTableGcsFileSourceConfig.java}
(51%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast-client.yaml
=>
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem
(78%)
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/GcsFileSinkFactoryTest.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/GcsFileSourceFactoryTest.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/catalog/GcsFileCatalogTest.java
create mode 100644
seatunnel-connectors-v2/connector-file/connector-file-gcs/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/gcs/config/GcsHadoopConfTest.java
copy seatunnel-connectors-v2/connector-http/{ =>
connector-http-shopify}/pom.xml (55%)
create mode 100644
seatunnel-connectors-v2/connector-http/connector-http-shopify/src/main/java/org/apache/seatunnel/connectors/seatunnel/shopify/source/ShopifySource.java
create mode 100644
seatunnel-connectors-v2/connector-http/connector-http-shopify/src/main/java/org/apache/seatunnel/connectors/seatunnel/shopify/source/ShopifySourceFactory.java
copy
seatunnel-connectors-v2/{connector-amazonsqs/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazonsqs/config/AmazonSqsSourceOptions.java
=>
connector-http/connector-http-shopify/src/main/java/org/apache/seatunnel/connectors/seatunnel/shopify/source/config/ShopifySourceOptions.java}
(52%)
create mode 100644
seatunnel-connectors-v2/connector-http/connector-http-shopify/src/main/java/org/apache/seatunnel/connectors/seatunnel/shopify/source/config/ShopifySourceParameter.java
copy
seatunnel-connectors-v2/{connector-dingtalk/src/test/java/org/apache/seatunnel/connectors/seatunnel/DingTalkFactoryTest.java
=>
connector-http/connector-http-shopify/src/test/java/org/apache/seatunnel/connectors/seatunnel/shopify/ShopifyFactoryTest.java}
(78%)
create mode 100644
seatunnel-connectors-v2/connector-http/connector-http-shopify/src/test/java/org/apache/seatunnel/connectors/seatunnel/shopify/ShopifySourceParameterTest.java
create mode 100644
seatunnel-connectors-v2/connector-http/connector-http-shopify/src/test/java/org/apache/seatunnel/connectors/seatunnel/shopify/ShopifySourceTest.java
create mode 100644
seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XaFacadeImplAutoLoadTest.java
create mode 100644
seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XaGroupOpsImplTest.java
create mode 100644
seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/JdbcSinkAggregatedCommitterTest.java
create mode 100644
seatunnel-connectors-v2/connector-neo4j/src/main/java/org/apache/seatunnel/connectors/seatunnel/neo4j/config/Neo4jAuthenticationConditions.java
copy
seatunnel-connectors-v2/{connector-datahub/src/test/java/org/apache/seatunnel/connectors/seatunnel/datahub/DataHubFactoryTest.java
=>
connector-slack/src/test/java/org/apache/seatunnel/connectors/seatunnel/slack/client/SlackClientTest.java}
(60%)
create mode 100644
seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/validation/ConfigValidationError.java
create mode 100644
seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/validation/ConfigValidationResult.java
create mode 100644
seatunnel-core/seatunnel-core-starter/src/test/java/org/apache/seatunnel/core/starter/validation/ConfigValidationResultTest.java
create mode 100644
seatunnel-dist/release-docs/licenses/LICENSE-gcs-connector.txt
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-amazondocumentdb-e2e/pom.xml
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-amazondocumentdb-e2e/src/test/java/org/apache/seatunnel/e2e/connector/amazondocumentdb/AmazonDocumentDBIT.java
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-amazondocumentdb-e2e/src/test/resources/amazondocumentdb_source_basic.conf
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-amazondocumentdb-e2e/src/test/resources/amazondocumentdb_source_projection.conf
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-amazondocumentdb-e2e/src/test/resources/amazondocumentdb_source_query_filter.conf
copy
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-couchbase-e2e/src/test/resources/{fake_source_to_couchbase.conf
=> fake_source_to_couchbase_timer_flush.conf} (79%)
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-gcs-e2e/pom.xml
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-gcs-e2e/src/test/java/org/apache/seatunnel/e2e/connector/file/gcs/GcsFileIT.java
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-gcs-e2e/src/test/resources/gcs/data/e2e.json
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-gcs-e2e/src/test/resources/gcs/gcs_file_to_assert.conf
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-gcs-e2e/src/test/resources/gcs/gcs_file_to_gcs.conf
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-file-gcs-e2e/src/test/resources/gcs/gcs_sink_to_assert.conf
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-http-e2e/src/test/resources/shopify_json_to_assert.conf
create mode 100644
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-kafka-e2e/src/test/java/org/apache/seatunnel/e2e/connector/kafka/AbstractKafkaIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/CoordinatorExecutorMassFailoverStormIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/RecoveryLifecycleRestApiIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/SavepointBusySourceBarrierIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/SavepointPreconditionRecoveryIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/SinkAggregatedCommitterTaskCheckpointCacheLeakIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/TaskDeploymentClassLoaderLeakIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/TaskDeploymentPrePublicationClassLoaderLeakIT.java
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/java/org/apache/seatunnel/engine/e2e/telemetry/TelemetryStartupDeadlockIT.java
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast-client.yaml
=>
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/batch_fake_to_console_minimal_slot.conf
(67%)
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/batch_fake_to_localfile_close_handshake_failover_template.conf
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/cluster_batch_one_pipeline_permanently_failed_template.conf
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast-client.yaml
=>
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/pending_jobs_streaming_unschedulable.conf
(74%)
create mode 100644
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/stream_fake_busy_split_to_console_savepoint.conf
copy
seatunnel-connectors-v2/connector-python/src/test/resources/python/spawn_stdout_child_then_exit.py
=>
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/stream_fake_to_localfile_savepoint_precondition_template.conf
(54%)
copy
seatunnel-engine/seatunnel-engine-client/src/test/resources/hazelcast.yaml =>
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/streaming_fake_to_localfile_committer_cache_leak.conf
(50%)
create mode 100644
seatunnel-engine/seatunnel-engine-server/src/test/java/org/apache/seatunnel/engine/server/rest/filter/ExceptionHandlingFilterTest.java
copy
seatunnel-connectors-v2/connector-python/src/test/resources/python/spawn_stdout_child_then_exit.py
=> tools/benchmarks/suites/benchmarks_core.txt (56%)
create mode 100644 tools/benchmarks/test_run_benchmarks.py
