damccorm commented on code in PR #37467: URL: https://github.com/apache/beam/pull/37467#discussion_r2755489553
########## .agent/skills/ci-cd/SKILL.md: ########## @@ -0,0 +1,193 @@ +--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +name: ci-cd +description: Guides understanding and working with Apache Beam's CI/CD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration. +--- + +# CI/CD in Apache Beam + +## Overview +Apache Beam uses GitHub Actions for CI/CD. Workflows are located in `.github/workflows/`. + +## Workflow Types + +### PreCommit Workflows +- Run on PRs and merges +- Validate code changes before merge +- Naming: `beam_PreCommit_*.yml` + +### PostCommit Workflows +- Run after merge and on schedule +- More comprehensive testing +- Naming: `beam_PostCommit_*.yml` + +### Scheduled Workflows +- Run nightly on master +- Check for external dependency impacts +- Tag master with `nightly-master` + +## Key Workflows + +### PreCommit +| Workflow | Description | +|----------|-------------| +| `beam_PreCommit_Java.yml` | Java build and tests | +| `beam_PreCommit_Python.yml` | Python tests | +| `beam_PreCommit_Go.yml` | Go tests | +| `beam_PreCommit_RAT.yml` | License header checks | +| `beam_PreCommit_Spotless.yml` | Code formatting | + +### PostCommit - Java +| Workflow | Description | +|----------|-------------| +| `beam_PostCommit_Java.yml` | Full Java test suite | +| `beam_PostCommit_Java_ValidatesRunner_*.yml` | Runner validation tests | +| `beam_PostCommit_Java_Examples_*.yml` | Example pipeline tests | + +### PostCommit - Python +| Workflow | Description | +|----------|-------------| +| `beam_PostCommit_Python.yml` | Full Python test suite | +| `beam_PostCommit_Python_ValidatesRunner_*.yml` | Runner validation | +| `beam_PostCommit_Python_Examples_*.yml` | Examples | + +### Load & Performance Tests +| Workflow | Description | +|----------|-------------| +| `beam_LoadTests_*.yml` | Load testing | +| `beam_PerformanceTests_*.yml` | I/O performance | + +## Triggering Tests + +### Automatic +- PRs trigger PreCommit tests +- Merges trigger PostCommit tests + +### Manual via PR Comment Review Comment: This doesn't work anymore ########## .agent/skills/ci-cd/SKILL.md: ########## @@ -0,0 +1,193 @@ +--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +name: ci-cd +description: Guides understanding and working with Apache Beam's CI/CD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration. +--- + +# CI/CD in Apache Beam + +## Overview +Apache Beam uses GitHub Actions for CI/CD. Workflows are located in `.github/workflows/`. + +## Workflow Types + +### PreCommit Workflows +- Run on PRs and merges +- Validate code changes before merge +- Naming: `beam_PreCommit_*.yml` + +### PostCommit Workflows +- Run after merge and on schedule +- More comprehensive testing +- Naming: `beam_PostCommit_*.yml` + +### Scheduled Workflows +- Run nightly on master +- Check for external dependency impacts +- Tag master with `nightly-master` + +## Key Workflows + +### PreCommit +| Workflow | Description | +|----------|-------------| +| `beam_PreCommit_Java.yml` | Java build and tests | +| `beam_PreCommit_Python.yml` | Python tests | +| `beam_PreCommit_Go.yml` | Go tests | +| `beam_PreCommit_RAT.yml` | License header checks | +| `beam_PreCommit_Spotless.yml` | Code formatting | + +### PostCommit - Java +| Workflow | Description | +|----------|-------------| +| `beam_PostCommit_Java.yml` | Full Java test suite | +| `beam_PostCommit_Java_ValidatesRunner_*.yml` | Runner validation tests | +| `beam_PostCommit_Java_Examples_*.yml` | Example pipeline tests | + +### PostCommit - Python +| Workflow | Description | +|----------|-------------| +| `beam_PostCommit_Python.yml` | Full Python test suite | +| `beam_PostCommit_Python_ValidatesRunner_*.yml` | Runner validation | +| `beam_PostCommit_Python_Examples_*.yml` | Examples | + +### Load & Performance Tests +| Workflow | Description | +|----------|-------------| +| `beam_LoadTests_*.yml` | Load testing | +| `beam_PerformanceTests_*.yml` | I/O performance | + +## Triggering Tests + +### Automatic +- PRs trigger PreCommit tests +- Merges trigger PostCommit tests + +### Manual via PR Comment +``` +retest this please +``` + +### Specific Test Suites +Use trigger phrases from [catalog](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) Review Comment: We should recommend trigger files, not phrases - https://github.com/apache/beam/tree/112685de58d7b24b70b8d8eb7171f0e28b35a2f1/.github/workflows#running-workflows-manually ########## .agent/skills/README.md: ########## @@ -0,0 +1,62 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Apache Beam Skills + +This directory contains skills that help the agent perform specialized tasks in the Apache Beam codebase. Review Comment: Could you link to http://antigravity.google/docs/skills ? ########## .agent/skills/ci-cd/SKILL.md: ########## @@ -0,0 +1,193 @@ +--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +name: ci-cd +description: Guides understanding and working with Apache Beam's CI/CD system using GitHub Actions. Use when debugging CI failures, understanding test workflows, or modifying CI configuration. +--- + +# CI/CD in Apache Beam + +## Overview +Apache Beam uses GitHub Actions for CI/CD. Workflows are located in `.github/workflows/`. + +## Workflow Types + +### PreCommit Workflows +- Run on PRs and merges +- Validate code changes before merge +- Naming: `beam_PreCommit_*.yml` + +### PostCommit Workflows +- Run after merge and on schedule +- More comprehensive testing +- Naming: `beam_PostCommit_*.yml` + +### Scheduled Workflows +- Run nightly on master +- Check for external dependency impacts +- Tag master with `nightly-master` + +## Key Workflows + +### PreCommit +| Workflow | Description | +|----------|-------------| +| `beam_PreCommit_Java.yml` | Java build and tests | +| `beam_PreCommit_Python.yml` | Python tests | +| `beam_PreCommit_Go.yml` | Go tests | +| `beam_PreCommit_RAT.yml` | License header checks | +| `beam_PreCommit_Spotless.yml` | Code formatting | + +### PostCommit - Java +| Workflow | Description | +|----------|-------------| +| `beam_PostCommit_Java.yml` | Full Java test suite | +| `beam_PostCommit_Java_ValidatesRunner_*.yml` | Runner validation tests | +| `beam_PostCommit_Java_Examples_*.yml` | Example pipeline tests | + +### PostCommit - Python +| Workflow | Description | +|----------|-------------| +| `beam_PostCommit_Python.yml` | Full Python test suite | +| `beam_PostCommit_Python_ValidatesRunner_*.yml` | Runner validation | +| `beam_PostCommit_Python_Examples_*.yml` | Examples | + +### Load & Performance Tests +| Workflow | Description | +|----------|-------------| +| `beam_LoadTests_*.yml` | Load testing | +| `beam_PerformanceTests_*.yml` | I/O performance | + +## Triggering Tests + +### Automatic +- PRs trigger PreCommit tests +- Merges trigger PostCommit tests + +### Manual via PR Comment +``` +retest this please +``` + +### Specific Test Suites +Use trigger phrases from [catalog](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) + +### Workflow Dispatch +Most workflows support manual triggering via GitHub UI. + +## Understanding Test Results + +### Finding Logs +1. Go to PR → Checks tab +2. Click on failed workflow +3. Expand failed job +4. View step logs + +### Common Failure Patterns + +#### Flaky Tests +- Random failures unrelated to change +- Solution: Comment `retest this please` Review Comment: Same thing here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
