damccorm commented on code in PR #34476:
URL: https://github.com/apache/beam/pull/34476#discussion_r2020999393
##########
.github/workflows/run_rc_validation_go_wordcount.yml:
##########
@@ -0,0 +1,172 @@
+name: Validate Go SDK Release Candidate
+
+on:
+ workflow_dispatch:
+ inputs:
+ rc_tag:
+ description: 'Beam RC Tag (e.g., v2.59.0-RC1)'
+ required: true
+ type: string
+ container_tag:
+ description: 'Beam Go SDK Container Tag (e.g., 2.59.0rc1)'
+ required: true
+ type: string
+
+# This allows a subsequently queued workflow run to interrupt previous runs
+concurrency:
+ group: '${{ github.workflow }} @ ${{ github.event.inputs.rc_tag }}' # Group
by RC tag
+ cancel-in-progress: true
+
+# Setting explicit permissions (copied from Java Mobile Gaming workflow)
+permissions:
+ actions: write
+ pull-requests: write
+ checks: write
+ contents: read
+ deployments: read
+ id-token: write # Note: Likely not needed for self-hosted runners without WIF
+ issues: write
+ discussions: read
+ packages: read
+ pages: read
+ repository-projects: read
+ security-events: read
+ statuses: read
+
+env:
+ GCS_OUTPUT_PREFIX: gs://rc-validation-migration-tests/go-wordcount #
Constant output prefix
+ GCP_PROJECT_ID: apache-beam-testing
+ GCP_REGION: us-central1
+ GCS_TEMP_LOCATION: gs://rc-validation-migration-tests/temp/
+ GCS_INPUT_PATH: gs://apache-beam-samples/shakespeare/kinglear.txt
+
+jobs:
+ setup:
+ runs-on: self-hosted
+ outputs:
+ rc_tag: ${{ github.event.inputs.rc_tag }}
+ container_tag: ${{ github.event.inputs.container_tag }}
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+
+ - name: Set up environment
+ uses: ./.github/actions/setup-environment-action
+ with:
+ go-version: default, otherwise remove if default is 1.21
Review Comment:
Should this be a comment? Seems like this will definitely error
##########
.github/workflows/run_rc_validation_go_wordcount.yml:
##########
@@ -0,0 +1,172 @@
+name: Validate Go SDK Release Candidate
+
+on:
+ workflow_dispatch:
+ inputs:
+ rc_tag:
+ description: 'Beam RC Tag (e.g., v2.59.0-RC1)'
+ required: true
+ type: string
+ container_tag:
+ description: 'Beam Go SDK Container Tag (e.g., 2.59.0rc1)'
+ required: true
+ type: string
+
+# This allows a subsequently queued workflow run to interrupt previous runs
+concurrency:
+ group: '${{ github.workflow }} @ ${{ github.event.inputs.rc_tag }}' # Group
by RC tag
+ cancel-in-progress: true
+
+# Setting explicit permissions (copied from Java Mobile Gaming workflow)
+permissions:
+ actions: write
+ pull-requests: write
+ checks: write
+ contents: read
+ deployments: read
+ id-token: write # Note: Likely not needed for self-hosted runners without WIF
+ issues: write
+ discussions: read
+ packages: read
+ pages: read
+ repository-projects: read
+ security-events: read
+ statuses: read
+
+env:
+ GCS_OUTPUT_PREFIX: gs://rc-validation-migration-tests/go-wordcount #
Constant output prefix
+ GCP_PROJECT_ID: apache-beam-testing
+ GCP_REGION: us-central1
+ GCS_TEMP_LOCATION: gs://rc-validation-migration-tests/temp/
+ GCS_INPUT_PATH: gs://apache-beam-samples/shakespeare/kinglear.txt
+
+jobs:
+ setup:
+ runs-on: self-hosted
+ outputs:
+ rc_tag: ${{ github.event.inputs.rc_tag }}
+ container_tag: ${{ github.event.inputs.container_tag }}
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+
+ - name: Set up environment
+ uses: ./.github/actions/setup-environment-action
+ with:
+ go-version: default, otherwise remove if default is 1.21
+
+ - name: Fetch Go SDK RC and Tidy Modules
+ working-directory: ./sdks/go/examples/wordcount
+ run: |
+ go get -d github.com/apache/beam/sdks/v2@${{
github.event.inputs.rc_tag }}
+ go mod tidy
+
+ # Cache Go modules to speed up subsequent jobs
+ - name: Cache Go modules
+ uses: actions/cache@v4
+ with:
+ path: |
+ ~/.cache/go-build
+ ~/go/pkg/mod
+ key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
+ restore-keys: |
+ ${{ runner.os }}-go-
+
+ validate-go-rc-prism:
+ needs: setup
+ runs-on: self-hosted # Changed to self-hosted
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+
+ - name: Set up environment
+ uses: ./.github/actions/setup-environment-action
+ with:
+ go-version: default
+
+ # Restore cached Go modules
+ - name: Restore Go modules cache
+ uses: actions/cache@v4
+ with:
+ path: |
+ ~/.cache/go-build
+ ~/go/pkg/mod
+ key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
+ restore-keys: |
+ ${{ runner.os }}-go-
+
+ # Assuming gcloud/gsutil is available and authenticated on the
self-hosted runner
+ - name: Download Input File from GCS (Prism)
+ working-directory: ./sdks/go/examples/wordcount
+ run: gsutil cp ${{ env.GCS_INPUT_PATH }} ./kinglear.txt
+
+ - name: Run Go WordCount with PrismRunner
+ working-directory: ./sdks/go/examples/wordcount
+ run: |
+ go run wordcount.go \
+ --input ./kinglear.txt \
+ --output ./output_prism.txt \
+ --runner=PrismRunner \
+ --environment_type=DOCKER \
+ --environment_config=apache/beam_go_sdk:${{
needs.setup.outputs.container_tag }}
+
+ - name: Check output file
+ working-directory: ./sdks/go/examples/wordcount
+ run: |
+ echo "--- PrismRunner WordCount Output ---"
+ cat output_prism.txt* # Output might be sharded
Review Comment:
Is the idea we will still ask folks to take a look at this output as part of
the release instructions?
##########
.github/workflows/run_rc_validation_go_wordcount.yml:
##########
@@ -0,0 +1,172 @@
+name: Validate Go SDK Release Candidate
+
+on:
+ workflow_dispatch:
+ inputs:
+ rc_tag:
+ description: 'Beam RC Tag (e.g., v2.59.0-RC1)'
+ required: true
+ type: string
+ container_tag:
+ description: 'Beam Go SDK Container Tag (e.g., 2.59.0rc1)'
+ required: true
+ type: string
+
+# This allows a subsequently queued workflow run to interrupt previous runs
+concurrency:
+ group: '${{ github.workflow }} @ ${{ github.event.inputs.rc_tag }}' # Group
by RC tag
+ cancel-in-progress: true
+
+# Setting explicit permissions (copied from Java Mobile Gaming workflow)
+permissions:
+ actions: write
+ pull-requests: write
+ checks: write
+ contents: read
+ deployments: read
+ id-token: write # Note: Likely not needed for self-hosted runners without WIF
+ issues: write
+ discussions: read
+ packages: read
+ pages: read
+ repository-projects: read
+ security-events: read
+ statuses: read
+
+env:
+ GCS_OUTPUT_PREFIX: gs://rc-validation-migration-tests/go-wordcount #
Constant output prefix
+ GCP_PROJECT_ID: apache-beam-testing
+ GCP_REGION: us-central1
+ GCS_TEMP_LOCATION: gs://rc-validation-migration-tests/temp/
+ GCS_INPUT_PATH: gs://apache-beam-samples/shakespeare/kinglear.txt
+
+jobs:
+ setup:
+ runs-on: self-hosted
+ outputs:
+ rc_tag: ${{ github.event.inputs.rc_tag }}
+ container_tag: ${{ github.event.inputs.container_tag }}
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+
+ - name: Set up environment
+ uses: ./.github/actions/setup-environment-action
+ with:
+ go-version: default, otherwise remove if default is 1.21
+
+ - name: Fetch Go SDK RC and Tidy Modules
+ working-directory: ./sdks/go/examples/wordcount
+ run: |
+ go get -d github.com/apache/beam/sdks/v2@${{
github.event.inputs.rc_tag }}
+ go mod tidy
+
+ # Cache Go modules to speed up subsequent jobs
+ - name: Cache Go modules
+ uses: actions/cache@v4
+ with:
+ path: |
+ ~/.cache/go-build
Review Comment:
I don't think we've built anything, so I don't think this will do anything.
In general, I don't see any real value in this caching step; the go builds are
quite fast anyways.
##########
.github/workflows/run_rc_validation_go_wordcount.yml:
##########
@@ -0,0 +1,172 @@
+name: Validate Go SDK Release Candidate
+
+on:
+ workflow_dispatch:
+ inputs:
+ rc_tag:
+ description: 'Beam RC Tag (e.g., v2.59.0-RC1)'
+ required: true
+ type: string
+ container_tag:
+ description: 'Beam Go SDK Container Tag (e.g., 2.59.0rc1)'
+ required: true
+ type: string
+
+# This allows a subsequently queued workflow run to interrupt previous runs
+concurrency:
+ group: '${{ github.workflow }} @ ${{ github.event.inputs.rc_tag }}' # Group
by RC tag
+ cancel-in-progress: true
+
+# Setting explicit permissions (copied from Java Mobile Gaming workflow)
+permissions:
+ actions: write
+ pull-requests: write
+ checks: write
+ contents: read
+ deployments: read
+ id-token: write # Note: Likely not needed for self-hosted runners without WIF
+ issues: write
+ discussions: read
+ packages: read
+ pages: read
+ repository-projects: read
+ security-events: read
+ statuses: read
+
+env:
+ GCS_OUTPUT_PREFIX: gs://rc-validation-migration-tests/go-wordcount #
Constant output prefix
+ GCP_PROJECT_ID: apache-beam-testing
+ GCP_REGION: us-central1
+ GCS_TEMP_LOCATION: gs://rc-validation-migration-tests/temp/
+ GCS_INPUT_PATH: gs://apache-beam-samples/shakespeare/kinglear.txt
+
+jobs:
+ setup:
+ runs-on: self-hosted
+ outputs:
+ rc_tag: ${{ github.event.inputs.rc_tag }}
+ container_tag: ${{ github.event.inputs.container_tag }}
Review Comment:
We don't need this as an output - instead of referencing outputs in future
jobs, you can just directly reference `github.event.inputs.container_tag` -
this just adds a layer of indirection
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
