sunchao commented on code in PR #6000: URL: https://github.com/apache/datafusion-comet/pull/6000#discussion_r4039113861
########## docs/source/contributor-guide/ci.md: ########## @@ -244,6 +244,58 @@ stays in one place. When you pick up a nightly failure: Dispatching `ci.yml` from the Actions page with **Run workflow** runs every tier, including the nightly suites, if you need a result before the next scheduled run. +## Checking that the scheduled runs are healthy + +A scheduled run has no pull request to turn red, so when one breaks, nothing puts it in front of +anyone. Comet has three daily schedules plus a weekly one, and two of the daily ones report nothing +when they fail: + +| Workflow | Cron (UTC) | Reports a failure? | +| ---------------------- | ------------ | ------------------------------------- | +| `publish_snapshot.yml` | `0 3 * * *` | no | +| `miri.yml` | `0 4 * * *` | no | +| `ci.yml` nightly tier | `0 6 * * *` | yes, a `ci-nightly-failure` issue | +| `codeql.yml` | `16 4 * * 1` | yes, to the repository's Security tab | + +So the two without a reporting step have to be looked at deliberately. Check all of them at once: + +```sh +for wf in ci.yml miri.yml publish_snapshot.yml; do + gh api "repos/apache/datafusion-comet/actions/workflows/$wf/runs?event=schedule&per_page=5" \ + --jq ".workflow_runs[] | \"$wf\t\(.created_at[0:10])\t\(.conclusion)\t\(.html_url)\"" +done +``` + +Read the output for two different things: + +- **A missing row.** If the most recent run is not from the last day or two, the schedule itself + stopped firing. GitHub drops scheduled runs under load and disables them entirely in a repository + with no activity for 60 days, and it does not announce either. Confirm the workflow is still + enabled with `gh api repos/apache/datafusion-comet/actions/workflows --jq '.workflows[] | "\(.state)\t\(.path)"'`, + and re-enable it from the Actions page if it is `disabled_inactivity`. +- **A run of failures.** One red night is a flake or a real regression, and for `ci.yml` there is an + issue open about it. Several consecutive red nights on `miri.yml` or `publish_snapshot.yml` means + nobody has looked; treat the streak, not the newest run, as the thing to explain. + +For the `ci.yml` nightly, green on its own does not mean the suites ran. Path filters and the diff +base are both allowed to select nothing — a documentation-only day legitimately runs no suite at +all — and a run that tested nothing reports exactly the same green `Required Checks` as a run that +tested everything. Confirm the suites actually ran: + +```sh +run=$(gh api "repos/apache/datafusion-comet/actions/workflows/ci.yml/runs?event=schedule&per_page=1" \ + --jq '.workflow_runs[0].id') +gh run view "$run" --json jobs \ Review Comment: ### Correctness [P2] Pin the job lookup to the same repository as the run lookup Could this call include `--repo apache/datafusion-comet`? The preceding `gh api` always obtains an upstream run ID, but `gh run view` uses the locally selected repository. With `GH_REPO=andygrove/datafusion-comet`, the documented lookup obtains upstream run `35189906693` and then fails with HTTP 404 at `/repos/andygrove/datafusion-comet/actions/runs/35189906693`. Adding `--repo apache/datafusion-comet` under that same environment succeeds and returns five skipped and 40 successful jobs. Contributors working with a fork selected cannot perform the documented coverage check as written. An explicit repository keeps this command consistent with the other checks and removes its dependency on the caller's checkout. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
