goutamadwant opened a new pull request, #13167:
URL: https://github.com/apache/seatunnel/pull/13167

   ### Purpose of this pull request
   This contribution to Apache SeaTunnel adds Amplitude Source, requested in 
#10753. It implements this source only and does not close the connector 
umbrella.
   
   - Bounded raw-event exports using inclusive UTC `server_upload_time` hours, 
US/EU endpoints and project Basic authentication.
   - Complete ZIP/gzip integrity checks before emission, with bounded archive, 
decoded-byte, record and metadata processing.
   - Single-split source and existing HTTP/JSON dependencies, plus 
repository-managed Commons Compress; no Amplitude SDK or dependency-version 
change.
   - Owned background download/validation/parsing, bounded polling-thread 
emission and cancellation without holding checkpoint locks across I/O waits.
   - Factory/plugin discovery, distribution/CI/E2E wiring, examples and 
English/Chinese documentation.
   
   ### Does this PR introduce _any_ user-facing change?
   Yes. Users can configure `Amplitude` as a batch source with credentials, an 
explicit schema, upload-time hours and optional JSON Pointer mappings.
   
   This slice requires `parallelism = 1`. The inclusive end hour must have 
fully elapsed at least two hours ago. An eligible-window 404 means no files are 
currently available, not proof against late data.
   
   Recovery replays the whole window, including earlier emitted rows: 
at-least-once replay, not exactly-once delivery or an immutable snapshot. 
Downstream deduplication/upsert keys are required where duplicates are 
unacceptable. Local disk is bounded by `max_archive_bytes`; integrity 
validation adds a full decompression pass. Archive limits and rejected variants 
are documented.
   
   No sink or continuous CDC is added. Existing connector options, shared HTTP 
behavior and public APIs are unchanged.
   
   ### How was this patch tested?
   - Missing factory discovery reproduced before implementation; failing 
regressions added before correcting review-found resource and 
checkpoint-boundary defects.
   - JDK 11 and 17: all 42 Amplitude tests passed on each, covering 
configuration/SPI, mapping and decimal precision, ZIP64/CRC/truncation, 
resource limits, retries, cancellation, cleanup and partial-emission replay.
   - HTTP base, PostHog, Asana and Mailchimp: 172 adjacent tests passed on each 
JDK.
   - The 68-module E2E prerequisite reactor compiled and installed with test 
execution skipped for that prerequisite build.
   - Docker Zeta/JDK 11: all three cases passed: retry/multi-entry export; 
archive-specific job failure; checkpoint advancement during blocked download, 
clean savepoint stop and successful full-window restore.
   - Scoped package verification and Spotless passed.
   
   Reproduction commands; repeat unit/adjacent checks with JAVA_HOME set to JDK 
11 and then 17:
   ```sh
   ./mvnw -Pci -pl :connector-amplitude-e2e -am -DskipTests -DskipIT=true 
-Dskip.ui=true install
   ./mvnw -f 
seatunnel-connectors-v2/connector-http/connector-http-amplitude/pom.xml 
spotless:check verify
   ./mvnw -f seatunnel-connectors-v2/connector-http/pom.xml -pl 
connector-http-base,connector-http-posthog,connector-http-asana,connector-http-mailchimp
 test
   RUN_ENGINES=zeta ./mvnw -f 
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-amplitude-e2e/pom.xml 
-DskipUT=true -DskipIT=false -Dit.test=AmplitudeIT verify
   ```
   
   The fixture uses dummy credentials, not a live Amplitude project. 
Flink/Spark E2E, native JDK 8, Windows, abrupt worker-crash recovery, 
production throughput and full-repository tests remain unverified. Local 
results do not imply hosted CI passed.
   
   ### Check list
   * [x] License/NOTICE reviewed: Commons Compress is already covered; no new 
third-party version or committed binary.
   * [x] English/Chinese documentation and changelogs added.
   * [ ] `incompatible-changes.md`: not applicable; existing behavior is 
unchanged.
   * [x] Connector integration updated:
     1. `plugin-mapping.properties`
     2. `seatunnel-dist/pom.xml`
     3. `.github/workflows/labeler/label-scope-conf.yml`
     4. `seatunnel-e2e` module and cases
     5. `config/plugin_config`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to