hawk9821 commented on PR #11791: URL: https://github.com/apache/seatunnel/pull/11791#issuecomment-5371164339
### Review of PR #11791 and Comparison with #9976 These two PRs propose different approaches to reduce SeaTunnel CI runtime. I've reviewed the design of #11791 and the CI logs from both runs, and I'd like to share some observations. #### Optimization Strategies **#11791 (Rebalance connector integration test shards)** - Core idea: Replace modulo‑based sharding with stable hashing to rebalance the 7 connector shards, and extract Iceberg and HBase as standalone jobs. - Design goal: Reduce the existing 64‑172 min fluctuation to about 79–90 minutes per shard. - Change granularity: Relatively fine, affecting sharding logic and job extraction. **#9976 (Extract long‑running modules and run them independently)** - Core idea: Move time‑consuming modules (ClickHouse, CDC MySQL, MongoDB, HBase, etc.) out of the main workflow into separate jobs. - Change granularity: Coarser, focusing on extracting entire modules. #### CI Run Comparison | Metric | #11791 (nzw921rx) | #9976 (hawk9821) | |--------|-------------------|-------------------| | Total duration | **6h 11m 51s** | **4h 55m 30s** | | Errors | 123 | 118 | | Warnings | 164 | 181 | | Build website | 1h 25m | 1h 17m | | Dependency licenses | 22m 1s | 15m 52s | | Final status | Cancelled | Cancelled | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
