This is an automated email from the ASF dual-hosted git repository. tqchen pushed a commit to branch jenkins-cleanup-trusted-main in repository https://gitbox.apache.org/repos/asf/tvm.git
commit 578eeec92220a3a4514606874b9057f4cb7c0eb4 Author: Tianqi Chen <[email protected]> AuthorDate: Thu Oct 15 14:14:23 2026 +0000 [CI] Make native pipeline commands and artifact options explicit --- ci/README.md | 2 +- ci/jenkins/README.md | 125 --------------------------- ci/jenkins/artifacts.py | 17 ++-- ci/jenkins/generated/arm_jenkinsfile.groovy | 19 +++- ci/jenkins/generated/common.groovy | 23 +++-- ci/jenkins/generated/cpu_jenkinsfile.groovy | 29 +++++-- ci/jenkins/generated/gpu_jenkinsfile.groovy | 11 ++- ci/jenkins/generated/wasm_jenkinsfile.groovy | 22 ++++- ci/jenkins/scripts/s3.py | 30 ++++--- 9 files changed, 105 insertions(+), 173 deletions(-) diff --git a/ci/README.md b/ci/README.md index 292fdddd61..e617d7af91 100644 --- a/ci/README.md +++ b/ci/README.md @@ -92,4 +92,4 @@ Other jobs run in GitHub Actions for Windows and MacOS jobs. The native Groovy entry points in `ci/jenkins/generated/` load shared helpers from `common.groovy`. Edit these sources directly; no generation step is needed. -See [Jenkins pipeline documentation](jenkins/README.md) for trust and rollout checks. +See comments in [common.groovy](jenkins/generated/common.groovy) for shared utility options and trust constraints. diff --git a/ci/jenkins/README.md b/ci/jenkins/README.md deleted file mode 100644 index 0f8062b0d9..0000000000 --- a/ci/jenkins/README.md +++ /dev/null @@ -1,125 +0,0 @@ -<!--- Licensed to the Apache Software Foundation (ASF) under one --> -<!--- or more contributor license agreements. See the NOTICE file --> -<!--- distributed with this work for additional information --> -<!--- regarding copyright ownership. The ASF licenses this file --> -<!--- to you under the Apache License, Version 2.0 (the --> -<!--- "License"); you may not use this file except in compliance --> -<!--- with the License. You may obtain a copy of the License at --> - -<!--- http://www.apache.org/licenses/LICENSE-2.0 --> - -<!--- Unless required by applicable law or agreed to in writing, --> -<!--- software distributed under the License is distributed on an --> -<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY --> -<!--- KIND, either express or implied. See the License for the --> -<!--- specific language governing permissions and limitations --> -<!--- under the License. --> - -# TVM CI - -TVM runs CI jobs on every commit to an open pull request and to branches in the apache/tvm repo (such as `main`). These jobs are essential to keeping the TVM project in a healthy state and preventing breakages. - -## Jenkins - -Jenkins runs all of the linux-based TVM CI-enabled regression tests. This includes tests against accelerated hardware such as GPUs. It excludes those regression tests that run against hardware not available in the cloud (those tests aren't currently exercised in TVM CI). The tests run by Jenkins represent most of the merge-blocking tests (and passing Jenkins should mostly correlate with passing the remaining Windows/Mac builds). - -## GitHub Actions - -GitHub Actions is used to run Windows jobs, MacOS jobs, and various on-GitHub automations. These are defined in [`.github/workflows`](../../.github/workflows/). These automations include bots to: -* [allow non-committers to merge approved / CI passing PRs](https://discuss.tvm.apache.org/t/rfc-allow-merging-via-pr-comments/12220) - -https://github.com/apache/tvm/actions has the logs for each of these workflows. Note that when debugging these workflows changes from PRs from forked repositories won't be reflected in the PR. These should be tested in the forked repository first and linked in the PR body. - -# Jenkins CI - -TVM uses Jenkins for running Linux continuous integration (CI) tests on -[branches](https://ci.tlcpack.ai/job/tvm/) and -[pull requests](https://ci.tlcpack.ai/job/tvm/view/change-requests/) through a -build configuration specified in a [`Jenkinsfile`](generated/). -Other jobs run in GitHub Actions for Windows and MacOS jobs. - -## Pipeline entry points - -The five `generated/*_jenkinsfile.groovy` paths remain Jenkins job entry points. -Despite the historical directory name, these are native Groovy sources: edit -and review them directly. `generated/common.groovy` shares preparation, checkout, -Docker setup, artifacts and spot fallback policy. Platform commands and deployment -stay visible in their respective job files. Image tags remain in -`docker-images.ini`; artifact bundles remain in `artifacts.py`. Bundle entries may -contain repository-relative globs such as `build/dist/*.whl`. Upload expands -patterns locally, sorts and deduplicates the paths, and keeps their relative -paths under the S3 prefix. Unmatched patterns produce a warning; missing literal -optional artifacts retain their warning-and-skip behavior. Directories keep -recursive upload behavior. Download still copies the complete prefix without -expanding local or remote globs. -Use `s3.py --artifacts NAME...` to select registry entries or `--paths PATH...` -to select raw paths; the selections are mutually exclusive. Existing Jenkinsfiles -can keep `--bundle` and `--items`, which are aliases with the same semantics. -`upload_artifacts(tag, options)` passes those options unchanged, as does the -retained legacy `upload(tag, options)`. -`artifacts.py` exposes the declarative `ARTIFACTS` registry; there is no `data.py` -compatibility module. - -The shared helper groups internal dependencies first, public entry points next, -and retained legacy functions last. Existing function signatures, shared variables, -startup and retry behavior remain available to existing Jenkinsfiles. - -Native build and test stages use `with_node(options, body)`. Callers provide exact -`instance` and optional `retry_instance` labels, a stage `name`, a workspace -`folder`, and optional environment `variables`. Skipped CI returns before node -allocation or setup. Each attempt checks out sources in its workspace and runs -the body under a timeout; callers explicitly run `docker_init(image)` in the body. -When a retry label is supplied, build and test failures use the same fallback -policy, retaining the newest-build guard. User cancellation and milestone interruptions propagate without retry, as does -failure of the fallback attempt. Agent loss and attempt timeouts remain eligible -for fallback. Omit `retry_instance` for a single attempt. -Stage and parallel layout remain in the platform files. - -Host-side commands live in `scripts/`. GitHub bot and release API helpers live -in `../github/`. The retained `determine_docker_images.py` supports both -Jenkins image selection and local `--lookup-only` lookup; `should_rebuild_docker.py` -and the standalone skip commands retain their command-line interfaces. - -### Trust and rollout - -Each entry point evaluates the shared helper returned by `readTrusted` at -Jenkins' selected SCM revision. Untrusted PRs that modify or delete the helper -fail immediately, before node allocation, checkout, or credentials. Missing -helpers, interrupted reads, and evaluation errors also propagate directly. -No alternative helper is fetched or evaluated. Trusted named branches retain -their selected helper revision. - -Cancellation runs once before preparation. During preparation, the normal source -merge pins the PR target revision for subsequent stages, including release -targets. Host scripts and artifact metadata follow that pinned target revision -for PR authors outside `CONTRIBUTORS.md`. - -The pinned target must contain the coordinated `artifacts.py` and `s3.py` update. -Before that update reaches a target, a Jenkins-trusted PR from an author outside -its contributor list cannot restore the renamed registry, and that build fails. -Use a trusted named branch for the transition; do not bypass the trusted-file -checkout or mix the old target's script with a PR-owned registry. - -The obsolete `ci/scripts/` copies and old node-info path have been removed now -that main's Jenkinsfiles use the canonical layout. The live `task_build.py`, -source-mode test scripts and artifact interfaces remain available. Older target -branches must retain their matching legacy commands; they are not supplied by -this checkout. - -Before enabling the migration on production jobs, validate it with the existing -Jenkins plugin versions and sandbox settings on a trusted branch: run all five -entry points, suspend/resume across a node step, and exercise spot loss, a normal -build-command failure, superseded builds, and a failed on-demand retry. Check -trusted helper changes, untrusted PRs without helper changes, and refusal of -untrusted modified/deleted helpers, including a release-target PR. Verify image -overrides and disabled deployment gates before -checking enabled docs/Docker deployment in the normal trusted deployment context. -Local mocked execution checks pipeline control flow but cannot certify Jenkins -CPS serialization, sandbox approvals, SCM trust configuration, or real agent loss. -No job path or controller reconfiguration is part of this migration. - -# Infrastructure - -The community's infrastructure configuration and documentation live in -[tlc-pack/ci](https://github.com/tlc-pack/ci). Coordinate external callers when -changing any retained script interface. diff --git a/ci/jenkins/artifacts.py b/ci/jenkins/artifacts.py index 6092c4fcfb..39bf1798f2 100644 --- a/ci/jenkins/artifacts.py +++ b/ci/jenkins/artifacts.py @@ -15,20 +15,13 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. -"""Bundle registry for CI artifact stashing. +"""Named CI artifacts. Uploads expand globs; downloads copy the saved paths. -Single source of truth for the file lists uploaded to / downloaded from S3 by -``ci/jenkins/scripts/s3.py``. This module deliberately carries nothing else — -docker image tags live in ``ci/jenkins/docker-images.ini`` and pipeline -configuration lives in ``ci/jenkins/generated/common.groovy``. +Examples:: -CLI: ``python3 ci/jenkins/artifacts.py <bundle> [<bundle> ...]`` resolves bundle -names to their file paths (one per line; exit 1 on unknown name). Used by -``s3.py`` at Jenkins runtime and by any external caller that needs -data-driven artifact lists. Entries may be repository-relative literal paths or -glob patterns. Only s3.py upload expands patterns; this registry and its CLI -return the declarative entries unchanged. Downloads normally copy the whole -artifact prefix. + python3 ci/jenkins/artifacts.py tvm_lib cpptest + ./ci/jenkins/scripts/s3.py --action upload --bucket BUCKET --prefix PREFIX \ + --artifact tvm_lib --artifact cpptest """ import sys diff --git a/ci/jenkins/generated/arm_jenkinsfile.groovy b/ci/jenkins/generated/arm_jenkinsfile.groovy index c826e2d104..348fb57dd1 100644 --- a/ci/jenkins/generated/arm_jenkinsfile.groovy +++ b/ci/jenkins/generated/arm_jenkinsfile.groovy @@ -27,8 +27,21 @@ stage('Build') { script: "${docker_run} ${images.ci_arm} ./tests/scripts/task_config_build_arm.sh build", label: 'Create ARM CMake config', ) - ci.cmake_build(images.ci_arm, 'build') - ci.make_cpp_tests(images.ci_arm, 'build') - ci.upload_artifacts('arm', '--artifacts tvm_lib --artifacts cpptest') + sh ( + script: "${docker_run} --env CI_NUM_EXECUTORS ${images.ci_arm} ./tests/scripts/task_build.py --sccache-bucket tvm-sccache-prod --sccache-region us-west-2 --build-dir build", + label: 'Run CMake build', + ) + sh ( + script: """ + set -eux + ${docker_run} ${images.ci_arm} python3 ./tests/scripts/task_build.py \ + --sccache-bucket tvm-sccache-prod \ + --sccache-region us-west-2 \ + --cmake-target cpptest \ + --build-dir build + """, + label: 'Make C++ tests', + ) + ci.upload_artifacts('arm', '--artifact tvm_lib --artifact cpptest') } } diff --git a/ci/jenkins/generated/common.groovy b/ci/jenkins/generated/common.groovy index f7a87f3889..3bcf454f23 100644 --- a/ci/jenkins/generated/common.groovy +++ b/ci/jenkins/generated/common.groovy @@ -34,8 +34,8 @@ properties([parameters(image_platforms.collect { name, platform -> string(name: "${name}_param", defaultValue: '') })]) -// Internal helpers, with dependencies before their callers. -// Keep these callable for existing pipelines. +// Shared utilities; platform Jenkinsfiles own stages, parallel groups and commands. +// Keep existing helpers callable for older Jenkinsfiles. def per_exec_ws(folder) { return "workspace/exec_${env.EXECUTOR_NUMBER}/" + folder @@ -71,6 +71,9 @@ def checkout_trusted_files() { } // Trust contributors listed in the pinned target revision. + // That target must contain the matching scripts and artifacts.py registry. + // Before the coordinated rename reaches the target, use a trusted branch; + // never mix target scripts with untrusted PR artifact metadata. grep_code = sh( returnStatus: true, script: "git show '${upstream_revision}:CONTRIBUTORS.md' | grep '@${env.CHANGE_AUTHOR}'", @@ -286,8 +289,12 @@ def run_on_node(label, name, folder, image, variables, body) { // Public entry points. -// The caller supplies exact node labels; omitting retry_instance disables fallback. -// Docker setup stays in the body so each attempt uses the caller's chosen image. +// with_node(instance: 'CPU-SPOT', retry_instance: 'CPU', name: 'Build', +// folder: 'tvm/build', variables: ['PLATFORM=cpu']) { docker_init(image); ... } +// name marks skipped stages; folder isolates executor workspaces; variables is optional. +// Skip before allocation/setup. Each attempt checks out sources, then times the body. +// Keep docker_init explicit in the body. Omit retry_instance for a single attempt; +// otherwise failures retry only for the newest build, with cancellation propagated. def with_node(options, body) { if (skip_ci) { Utils.markStageSkippedForConditional(options.name) @@ -318,6 +325,10 @@ def with_node(options, body) { } } +// upload_artifacts('cpu', '--artifact tvm_lib --artifact cpptest') +// Or select paths: '--path "build/*.whl"'. Options pass unchanged to s3.py. +// Registry artifacts and explicit paths are mutually exclusive; repeat one-value flags. +// Upload globs are sorted/deduplicated; download(tag) copies the whole saved prefix. def upload_artifacts(tag, options) { upload(tag, options) } @@ -365,8 +376,8 @@ def start() { } } -// Legacy compatibility functions. Keep names, signatures and behavior until -// existing pipeline callers have migrated after merge. +// Legacy compatibility functions. Preserve signatures and behavior for older +// Jenkinsfiles; current platform files spell out build and test commands directly. def python_unittest(image) { sh ( diff --git a/ci/jenkins/generated/cpu_jenkinsfile.groovy b/ci/jenkins/generated/cpu_jenkinsfile.groovy index edadf43273..760ce31831 100644 --- a/ci/jenkins/generated/cpu_jenkinsfile.groovy +++ b/ci/jenkins/generated/cpu_jenkinsfile.groovy @@ -27,9 +27,22 @@ stage('Build') { script: "${docker_run} ${images.ci_cpu} ./tests/scripts/task_config_build_cpu.sh build", label: 'Create CPU CMake config', ) - ci.cmake_build(images.ci_cpu, 'build') - ci.make_cpp_tests(images.ci_cpu, 'build') - ci.upload_artifacts('cpu', '--artifacts tvm_lib --artifacts cpptest') + sh ( + script: "${docker_run} --env CI_NUM_EXECUTORS ${images.ci_cpu} ./tests/scripts/task_build.py --sccache-bucket tvm-sccache-prod --sccache-region us-west-2 --build-dir build", + label: 'Run CMake build', + ) + sh ( + script: """ + set -eux + ${docker_run} ${images.ci_cpu} python3 ./tests/scripts/task_build.py \ + --sccache-bucket tvm-sccache-prod \ + --sccache-region us-west-2 \ + --cmake-target cpptest \ + --build-dir build + """, + label: 'Make C++ tests', + ) + ci.upload_artifacts('cpu', '--artifact tvm_lib --artifact cpptest') } } @@ -40,8 +53,14 @@ stage('Test') { variables: ['PLATFORM=cpu', 'TEST_STEP_NAME=unittest: CPU']) { ci.docker_init(images.ci_cpu) ci.download('cpu') - ci.cpp_unittest(images.ci_cpu) - ci.python_unittest(images.ci_cpu) + sh ( + script: "${docker_run} --env CI_NUM_EXECUTORS ${images.ci_cpu} ./tests/scripts/task_cpp_unittest.sh", + label: 'Run C++ tests', + ) + sh ( + script: "${docker_run} ${images.ci_cpu} ./tests/scripts/task_python_unittest.sh", + label: 'Run Python unit tests', + ) } }) } diff --git a/ci/jenkins/generated/gpu_jenkinsfile.groovy b/ci/jenkins/generated/gpu_jenkinsfile.groovy index 668c0dadd2..2ed9a9e0b2 100644 --- a/ci/jenkins/generated/gpu_jenkinsfile.groovy +++ b/ci/jenkins/generated/gpu_jenkinsfile.groovy @@ -24,8 +24,11 @@ stage('Build') { name: 'BUILD: GPU', folder: 'tvm/build-gpu', variables: ['PLATFORM=gpu']) { ci.docker_init(images.ci_gpu) sh "${docker_run} --no-gpu ${images.ci_gpu} ./tests/scripts/task_config_build_gpu.sh build" - ci.cmake_build("${images.ci_gpu} --no-gpu", 'build') - ci.upload_artifacts('gpu', '--artifacts tvm_lib --artifacts tvm_lib_gpu_extra') + sh ( + script: "${docker_run} --env CI_NUM_EXECUTORS ${images.ci_gpu} --no-gpu ./tests/scripts/task_build.py --sccache-bucket tvm-sccache-prod --sccache-region us-west-2 --build-dir build", + label: 'Run CMake build', + ) + ci.upload_artifacts('gpu', '--artifact tvm_lib --artifact tvm_lib_gpu_extra') // Check the compiler-only GPU configuration after publishing the test build. sh 'rm -rf build' @@ -62,7 +65,7 @@ stage('Test') { script: "${docker_run} ${images.ci_gpu} ./tests/scripts/task_python_docs.sh", label: 'Build docs', ) - ci.upload_artifacts('docs', '--paths docs.tgz') + ci.upload_artifacts('docs', '--path docs.tgz') sh( script: "aws s3 cp --no-progress _docs s3://${s3_bucket}/${s3_prefix}/docs --recursive", label: 'Upload docs to S3', @@ -118,7 +121,7 @@ stage('Deploy') { ci.deploy_step('Deploy Docs', env.DOCS_DEPLOY_ENABLED == 'yes', 'tvm/deploy-docs') { ci.init_git() sh( - script: "./${jenkins_scripts_root}/s3.py --action download --bucket ${s3_bucket} --prefix ${s3_prefix}/docs --paths docs.tgz", + script: "./${jenkins_scripts_root}/s3.py --action download --bucket ${s3_bucket} --prefix ${s3_prefix}/docs --path docs.tgz", label: 'Download docs folder from S3', ) deploy_docs() diff --git a/ci/jenkins/generated/wasm_jenkinsfile.groovy b/ci/jenkins/generated/wasm_jenkinsfile.groovy index da38b1c237..8547cfff50 100644 --- a/ci/jenkins/generated/wasm_jenkinsfile.groovy +++ b/ci/jenkins/generated/wasm_jenkinsfile.groovy @@ -27,9 +27,25 @@ stage('Build') { script: "${docker_run} ${images.ci_wasm} ./tests/scripts/task_config_build_wasm.sh build", label: 'Create WASM CMake config', ) - ci.cmake_build(images.ci_wasm, 'build') - ci.make_cpp_tests(images.ci_wasm, 'build') - ci.cpp_unittest(images.ci_wasm) + sh ( + script: "${docker_run} --env CI_NUM_EXECUTORS ${images.ci_wasm} ./tests/scripts/task_build.py --sccache-bucket tvm-sccache-prod --sccache-region us-west-2 --build-dir build", + label: 'Run CMake build', + ) + sh ( + script: """ + set -eux + ${docker_run} ${images.ci_wasm} python3 ./tests/scripts/task_build.py \ + --sccache-bucket tvm-sccache-prod \ + --sccache-region us-west-2 \ + --cmake-target cpptest \ + --build-dir build + """, + label: 'Make C++ tests', + ) + sh ( + script: "${docker_run} --env CI_NUM_EXECUTORS ${images.ci_wasm} ./tests/scripts/task_cpp_unittest.sh", + label: 'Run C++ tests', + ) sh( script: "${docker_run} ${images.ci_wasm} ./tests/scripts/task_web_wasm.sh", label: 'Run WASM lint and tests', diff --git a/ci/jenkins/scripts/s3.py b/ci/jenkins/scripts/s3.py index f48986cb58..25ae3f24af 100755 --- a/ci/jenkins/scripts/s3.py +++ b/ci/jenkins/scripts/s3.py @@ -158,25 +158,27 @@ if __name__ == "__main__": parser.add_argument( "--prefix", help="s3 bucket + tag (e.g. s3://tvm-ci-prod/PR-1234/cpu", required=True ) - selection = parser.add_mutually_exclusive_group() - selection.add_argument( - "--paths", - "--items", + parser.add_argument( + "--path", dest="paths", - nargs="+", - help="repository-relative files, folders or upload glob patterns", + action="append", + help="repository-relative file, folder or upload glob pattern (repeatable)", + metavar="PATH", ) - selection.add_argument( - "--artifacts", + parser.add_argument( + "--artifact", "--bundle", dest="artifacts", - nargs="+", - action="extend", - help="artifact names from ci/jenkins/artifacts.py (repeatable)", + action="append", + help="artifact name from ci/jenkins/artifacts.py (repeatable)", metavar="NAME", ) + # Legacy --items accepts several paths and retains the last occurrence. + parser.add_argument("--items", nargs="+", help="files and folders to transfer") args = parser.parse_args() + if args.artifacts is not None and (args.paths is not None or args.items is not None): + parser.error("--artifact/--bundle and --path/--items are mutually exclusive") logging.info(args) @@ -198,11 +200,11 @@ if __name__ == "__main__": if args.artifacts is not None: items = resolve_artifacts(args.artifacts) - elif args.paths is not None: - items = args.paths + elif args.paths is not None or args.items is not None: + items = (args.paths or []) + (args.items or []) else: if args.action == "upload": - logging.error("Cannot upload without --paths or --artifacts") + logging.error("Cannot upload without --path or --artifact") exit(1) else: # Download the whole prefix
