LuciferYang commented on code in PR #13535: URL: https://github.com/apache/gluten/pull/13535#discussion_r4490653749
########## .github/workflows/update_fork_build_status.yml: ########## @@ -0,0 +1,182 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# A pull request from a fork runs velox_backend_x86.yml in the fork, on push. This +# workflow reports that run on the pull request as the check named below: it creates +# the check when it first finds the run, and updates it on later passes. It only reads +# run metadata, so it never runs pull request code and needs no pull_request_target. +# Ported from apache/spark's update_build_status.yml (SPARK-60578). +name: Update fork build status + +on: + schedule: + - cron: '*/15 * * * *' + workflow_dispatch: + +permissions: {} + +concurrency: + group: ${{ github.workflow }} + cancel-in-progress: false + +jobs: + update: + if: github.repository == 'apache/gluten' + runs-on: ubuntu-slim Review Comment: `ubuntu-slim` is a GitHub-hosted runner label. apache/spark's `update_build_status.yml` runs on it every 15 minutes, for example in https://github.com/apache/spark/actions/runs/42152405769. ########## .github/workflows/update_fork_build_status.yml: ########## @@ -0,0 +1,182 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# A pull request from a fork runs velox_backend_x86.yml in the fork, on push. This +# workflow reports that run on the pull request as the check named below: it creates +# the check when it first finds the run, and updates it on later passes. It only reads +# run metadata, so it never runs pull request code and needs no pull_request_target. +# Ported from apache/spark's update_build_status.yml (SPARK-60578). +name: Update fork build status + +on: + schedule: + - cron: '*/15 * * * *' + workflow_dispatch: + +permissions: {} + +concurrency: + group: ${{ github.workflow }} + cancel-in-progress: false + +jobs: + update: + if: github.repository == 'apache/gluten' + runs-on: ubuntu-slim + permissions: + actions: read + checks: write Review Comment: Listing the PRs and check runs of a public repository doesn't need `pull-requests: read`. apache/spark's `update_build_status.yml` runs with only `actions: read` and `checks: write` and walked 713 open PRs in https://github.com/apache/spark/actions/runs/42152405769. The job here now also has `contents: read`, for the checkout of the script file. ########## .github/actions/restore-upstream-ccache/action.yml: ########## @@ -0,0 +1,88 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +name: Restore ccache from apache/gluten main +description: >- + Download the ccache stash that apache/infrastructure-actions/stash/save stored + for `key` on apache/gluten main. stash/restore only searches the repository the + run belongs to, so a fork run uses this after missing its own stash. Without a + stash for `key`, the newest stash for the same key prefix is used, as + actions/cache restore-keys would. +inputs: + key: + description: The key the stash was saved with, ending in a hashFiles() hash. + required: true + path: + description: The directory to restore into. + required: true +outputs: + hit: + description: "'true' if a stash was downloaded." + value: ${{ steps.download.outcome == 'success' }} +runs: + using: composite + steps: + - name: Find the stash + id: find + uses: actions/github-script@v9 + env: + KEY: ${{ inputs.key }} + with: + script: | + const repo = { owner: 'apache', repo: 'gluten' } + // stash/save appends the branch name to the key. Only stashes saved by + // apache/gluten's own runs on main count. + const usable = (a) => !a.expired && a.name.endsWith('-main') + && a.workflow_run.head_branch == 'main' + && a.workflow_run.head_repository_id == a.workflow_run.repository_id + const exact = process.env.KEY + '-main' + const { data } = await github.rest.actions.listArtifactsForRepo({ + ...repo, name: exact, per_page: 10 + }) + let stash = data.artifacts.find(usable) Review Comment: The API returns the newest artifact first, and the newest usable stash is the one to restore. Expired stashes are the oldest ones, so a later page can't hold a usable stash that the first page lacks. ########## dev/ci-velox-buildstatic-centos-8.sh: ########## @@ -24,6 +24,10 @@ if [ "$(uname -m)" = "aarch64" ]; then export VCPKG_FORCE_SYSTEM_BINARIES=1; fi +# Rewrite absolute paths under the checkout to relative ones, so a ccache saved by +# a checkout at another path still hits. +export CCACHE_BASEDIR=$(cd "$(dirname "$BASH_SOURCE")/.."; pwd) Review Comment: Bash doesn't word-split the value of an assignment, `export NAME=...` included. With the checkout under a directory whose name contains spaces, the script sets `CCACHE_BASEDIR` to the full path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
