voonhous commented on code in PR #18837:
URL: https://github.com/apache/hudi/pull/18837#discussion_r3629272554
##########
.github/workflows/hudi_trino_ci.yml:
##########
@@ -0,0 +1,117 @@
+name: Hudi Trino Connector CI
+
+on:
+ push:
+ branches:
+ - master
+ - 'release-*'
+ paths:
+ - 'hudi-trino/**'
+ - '.github/workflows/hudi_trino_ci.yml'
+ # No `paths:` filter here on purpose. test-hudi-trino-plugin is a required
status check
+ # in .asf.yaml, and a path-filtered workflow is never instantiated on PRs
that miss the
+ # filter, leaving the required context permanently pending. Run on every PR
instead and
+ # skip the expensive steps via the detect-trino-changes job below.
+ pull_request:
+ branches:
+ - master
+ - 'release-*'
+ workflow_dispatch:
+
+concurrency:
+ group: hudi-trino-ci-${{ github.ref }}
+ cancel-in-progress: ${{ !contains(github.ref, 'master') &&
!contains(github.ref, 'release-') }}
+
+env:
+ MVN_ARGS: -e -ntp -B -V -Dgpg.skip -Djacoco.skip -Pwarn-log
+
+jobs:
+ changes:
+ name: detect-trino-changes
+ runs-on: ubuntu-latest
+ outputs:
+ trino: ${{ steps.filter.outputs.trino }}
+ steps:
+ - name: Detect hudi-trino changes
+ id: filter
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ REPO: ${{ github.repository }}
+ EVENT: ${{ github.event_name }}
+ PR_NUMBER: ${{ github.event.pull_request.number }}
+ BEFORE_SHA: ${{ github.event.before }}
+ AFTER_SHA: ${{ github.sha }}
+ run: |
+ set -euo pipefail
+ TRINO=false
+ if [ "$EVENT" = "pull_request" ]; then
+ FILES=$(gh api --paginate "repos/$REPO/pulls/$PR_NUMBER/files"
--jq '.[].filename')
+ elif [ "$EVENT" = "push" ]; then
+ FILES=$(gh api "repos/$REPO/compare/$BEFORE_SHA...$AFTER_SHA" --jq
'.files[].filename')
+ else
+ # workflow_dispatch and anything else: always run the full build.
+ FILES=""
+ TRINO=true
+ fi
+ echo "Changed files:"
+ printf '%s\n' "$FILES"
+ while IFS= read -r f; do
+ [ -z "$f" ] && continue
+ case "$f" in
+ hudi-trino/*) TRINO=true ;;
Review Comment:
Broadened in 42fd963: the PR filter and push `paths:` now cover the full
`-am` closure of the upstream install step (`hudi-common`, `hudi-io`,
`hudi-hadoop-common`, `hudi-hadoop-mr`, `hudi-timeline-service`,
`hudi-tests-common`, client/sync modules) plus the root and `hudi-client` poms.
On cost: the from-source Trino install sounds heavy but is ~4 min with
`-DskipTests -Dair.check.skip-all=true`; the latest full run was ~11.5 min end
to end (Hudi modules 1.5 min, Trino 4 min, connector build+tests 5.5 min).
Those PRs already wait hours on Azure CI, so gating on this is effectively free.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
