nsivabalan commented on PR #20453: URL: https://github.com/apache/hudi/pull/20453#issuecomment-6480221894
Following up on the CI question with a concrete proposal, since the harness itself looks good to me: 1. **Own workflow file** (e.g. `.github/workflows/spark_connect_pipelines.yml`) instead of a job in `bot.yml`, so its triggers, failures and ownership stay separate from the main Java CI. 2. **Run nightly** (`schedule`) plus `workflow_dispatch` for manual runs. Optionally, PRs touching SDP could opt in through a label, so authors of the #20401 sub-tasks can run it before merge. 3. **Not a required check.** A red nightly run files or updates an issue for the SDP owners rather than blocking merges. 4. **Revisit later:** once the #20401 sub-tasks have landed and the job has a stable track record, we can consider path-filtered per-PR runs. This keeps the real CLI/Connect coverage RFC-112 asks for, without bringing back a per-PR Spark 4.1 bundle build for every change while the RFC is still under review. @danny0405 does that work for you? @yihua would like your take as well, especially on adding a nightly Spark Connect / Python lane to the project's CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
