This is an automated email from the ASF dual-hosted git repository. sxnan pushed a commit to branch release-0.2 in repository https://gitbox.apache.org/repos/asf/flink-agents.git
commit aa9ae88fedfd257f861ab2d7f1d237008e542356 Author: xuannan.sxn <[email protected]> AuthorDate: Thu Mar 19 11:37:33 2026 +0800 [AONE-80306915] Add internal maintenance scripts Add helper scripts for upstream sync, patch review, release branch creation, publish branch creation, and coordinated version updates. Upstream-candidate: no --- tools/internal/README.md | 141 ++++++++++++++ tools/internal/create_release_branch.sh | 209 ++++++++++++++++++++ tools/internal/create_release_publish_branch.sh | 174 +++++++++++++++++ tools/internal/list_patches.sh | 128 ++++++++++++ tools/internal/sync_upstream.sh | 246 ++++++++++++++++++++++++ tools/internal/update_version.sh | 127 ++++++++++++ 6 files changed, 1025 insertions(+) diff --git a/tools/internal/README.md b/tools/internal/README.md new file mode 100644 index 00000000..dc08d161 --- /dev/null +++ b/tools/internal/README.md @@ -0,0 +1,141 @@ +# Internal Maintenance Scripts + +This directory contains helper scripts for maintaining the internal Flink Agents fork. + +For the full branch, version, and release policy, see [INTERNAL_DEVELOPMENT.md](../../INTERNAL_DEVELOPMENT.md). + +## When To Use These Scripts + +- Sync `main` or `release-x.y` maintenance branches with upstream. +- Review internal patches before upstream contribution or before cutting a release line. +- Create a `release-x.y` maintenance branch from `upstream/release-x.y`. +- Create a publish branch `release/x.y-vvr-a.b.c` from a `release-x.y` maintenance branch. +- Update Java, Python, and docs versions in one step. + +## Prerequisites + +- The repository should have both `origin` and `upstream` remotes configured. +- The worktree should be clean before running these scripts. +- `mvn` must be available for version updates. +- `main` should be aligned with `origin/main` before creating a `release-x.y` maintenance branch. +- `release-x.y` should be aligned with `origin/release-x.y` before creating a publish branch. + +## Script Overview + +| Script | Purpose | Typical scenario | +|---|---|---| +| `sync_upstream.sh` | Sync `main` or `release-x.y` with upstream | Weekly sync, pre-release sync, bugfix intake from upstream | +| `list_patches.sh` | List internal `[AONE-*]` patches and upstream candidates | Patch review, upstream contribution preparation | +| `create_release_branch.sh` | Create `release-x.y` maintenance branch and keep it on `SNAPSHOT` | Start maintaining a new upstream release line | +| `create_release_publish_branch.sh` | Create `release/x.y-vvr-a.b.c` publish branch and set exact release version | Internal release publication | +| `update_version.sh` | Update Maven, Python, and docs versions together | Manual version correction or controlled version switch | + +## Common Flows + +### Sync Main + +Use when `main` needs the latest upstream commits while keeping internal patches rebase-friendly. + +```bash +./tools/internal/sync_upstream.sh main +``` + +### Sync A Release Maintenance Branch + +Use when `release-x.y` needs upstream bugfixes from `upstream/release-x.y`. + +```bash +./tools/internal/sync_upstream.sh release-0.3 +``` + +`sync_upstream.sh` only supports `main` and `release-*` maintenance branches. It rejects published branches such as `release/0.3-vvr-11.6.0`. + +### Review Internal Patches + +List all internal patches that are still ahead of upstream: + +```bash +./tools/internal/list_patches.sh +``` + +Only show patches explicitly marked as upstream candidates: + +```bash +./tools/internal/list_patches.sh --upstream-candidate +``` + +Review a recent range relative to a commit: + +```bash +./tools/internal/list_patches.sh --since HEAD~10 +``` + +### Create A Release Maintenance Branch + +Use when upstream has created `release-x.y` and the internal fork needs its own long-lived maintenance branch. + +```bash +./tools/internal/create_release_branch.sh 0.3 +``` + +This script: + +- fetches `origin` and `upstream` +- creates `release-0.3` from `upstream/release-0.3` +- merges `origin/main` +- keeps the branch version on `0.3-vvr-SNAPSHOT` + +### Create A Publish Branch + +Use when releasing a concrete internal build from an existing `release-x.y` maintenance branch. + +```bash +./tools/internal/create_release_publish_branch.sh 0.3 11.6.0 +``` + +This script: + +- checks that `release-0.3` matches `origin/release-0.3` +- creates `release/0.3-vvr-11.6.0` from `release-0.3` +- updates versions to `0.3-vvr-11.6.0` and `0.3+vvr.11.6.0` +- commits the version change on the publish branch + +Do not apply hotfixes directly on `release/0.3-vvr-11.6.0`. Fix `release-0.3` first, then cut a new publish branch. + +### Update Version Manually + +Use when the version needs to be switched explicitly outside the branch-creation scripts. + +Set a maintenance branch back to `SNAPSHOT`: + +```bash +./tools/internal/update_version.sh 0.3-vvr-SNAPSHOT +``` + +Set a publish branch to an exact release version: + +```bash +./tools/internal/update_version.sh 0.3-vvr-11.6.0 +``` + +`update_version.sh` updates: + +- all Maven module versions +- `python/pyproject.toml` +- `docs/config.toml` + +It commits the change only if the target version actually changes files. + +## Recommended Order + +1. Review pending internal patches with `list_patches.sh`. +2. Sync `main` with `sync_upstream.sh main`. +3. Create `release-x.y` with `create_release_branch.sh <version>`. +4. Sync `release-x.y` over time with `sync_upstream.sh release-x.y`. +5. Cut `release/x.y-vvr-a.b.c` with `create_release_publish_branch.sh <version> <vvr_version>`. + +## Notes + +- These scripts assume the internal fork model documented in [INTERNAL_DEVELOPMENT.md](../../INTERNAL_DEVELOPMENT.md). +- If a script stops on merge or rebase conflicts, resolve conflicts manually before continuing. +- If you change the workflow, update both this file and [INTERNAL_DEVELOPMENT.md](../../INTERNAL_DEVELOPMENT.md) together. diff --git a/tools/internal/create_release_branch.sh b/tools/internal/create_release_branch.sh new file mode 100755 index 00000000..0f49fc9d --- /dev/null +++ b/tools/internal/create_release_branch.sh @@ -0,0 +1,209 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +####################################### +# Create internal release maintenance branch. +# +# This script creates an internal release maintenance branch following the workflow: +# 1. Ensure main branch has been merged with upstream/main (固化) +# 2. Create release maintenance branch based on upstream/release-x.y +# 3. Merge main branch into release maintenance branch +# 4. Reset the branch version to x.y-vvr-SNAPSHOT +# +# Prerequisites: +# - Main branch should have been synced and merged with upstream/main +# - Upstream release branch should exist +# +# Usage: +# ./create_release_branch.sh <version> +# +# Arguments: +# version - Apache version (e.g., "0.3") +# +# Examples: +# ./create_release_branch.sh 0.3 +# +# This will: +# - Create release-0.3 branch based on upstream/release-0.3 +# - Merge origin/main into release-0.3 +# - Set version to 0.3-vvr-SNAPSHOT +####################################### + +set -o errexit +set -o nounset + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) +PROJECT_ROOT=$(cd "${SCRIPT_DIR}/../.." && pwd) + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +print_info() { + echo -e "${GREEN}[INFO]${NC} $1" +} + +print_warn() { + echo -e "${YELLOW}[WARN]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +check_origin_main() { + if ! git remote | grep -q "^origin$"; then + print_error "Origin remote not found." + exit 1 + fi + + if ! git rev-parse --verify main >/dev/null 2>&1; then + print_error "Local branch main not found." + exit 1 + fi + + if ! git rev-parse --verify origin/main >/dev/null 2>&1; then + print_error "Remote-tracking branch origin/main not found." + exit 1 + fi + + local local_main_commit + local origin_main_commit + local_main_commit=$(git rev-parse main) + origin_main_commit=$(git rev-parse origin/main) + + if [[ "${local_main_commit}" != "${origin_main_commit}" ]]; then + print_error "Local main and origin/main are not aligned." + print_info "local main: $(git rev-parse --short main)" + print_info "origin/main: $(git rev-parse --short origin/main)" + print_info "Please sync and push main before creating a release maintenance branch." + exit 1 + fi +} + +if [ -z "${1:-}" ]; then + echo "Usage: $0 <version>" + echo "" + echo "Arguments:" + echo " version - Apache version (e.g., '0.3')" + echo "" + echo "Example:" + echo " $0 0.3" + exit 1 +fi + +VERSION="$1" +RELEASE_BRANCH="release-${VERSION}" +INTERNAL_VERSION="${VERSION}-vvr-SNAPSHOT" + +cd "${PROJECT_ROOT}" + +# Check if upstream remote exists +if ! git remote | grep -q "^upstream$"; then + print_error "Upstream remote not found. Please add it first:" + echo " git remote add upstream [email protected]:apache/flink-agents.git" + exit 1 +fi + +# Check for uncommitted changes +if ! git diff-index --quiet HEAD --; then + print_error "You have uncommitted changes. Please commit or stash them first." + git status --short + exit 1 +fi + +# Fetch remotes +print_info "Fetching upstream and origin..." +git fetch upstream +git fetch origin + +# Ensure the main branch we are about to merge is up to date +check_origin_main + +# Check if upstream release branch exists +if ! git rev-parse --verify "upstream/${RELEASE_BRANCH}" >/dev/null 2>&1; then + print_error "Upstream branch upstream/${RELEASE_BRANCH} does not exist." + print_info "Available upstream release branches:" + git branch -r | grep "upstream/release-" || true + exit 1 +fi + +# Check if local release branch already exists +if git rev-parse --verify "${RELEASE_BRANCH}" >/dev/null 2>&1; then + print_error "Local branch ${RELEASE_BRANCH} already exists." + print_info "If you want to recreate it, delete it first:" + echo " git branch -D ${RELEASE_BRANCH}" + exit 1 +fi + +# Check if remote release branch already exists +if git rev-parse --verify "origin/${RELEASE_BRANCH}" >/dev/null 2>&1; then + print_error "Remote-tracking branch origin/${RELEASE_BRANCH} already exists." + print_info "Please check out the existing branch instead of recreating it." + exit 1 +fi + +# Show current internal patches on main +print_info "Current internal patches on main branch:" +git log upstream/main..origin/main --oneline || true +echo "" + +# Confirm with user +print_warn "About to create release maintenance branch ${RELEASE_BRANCH} with version ${INTERNAL_VERSION}" +echo "" +echo "This will:" +echo " 1. Create ${RELEASE_BRANCH} based on upstream/${RELEASE_BRANCH}" +echo " 2. Merge origin/main into ${RELEASE_BRANCH}" +echo " 3. Set version to ${INTERNAL_VERSION}" +echo "" +read -p "Continue? (y/N) " -n 1 -r +echo +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Aborted." + exit 0 +fi + +# Step 1: Create release branch based on upstream release +print_info "Creating ${RELEASE_BRANCH} based on upstream/${RELEASE_BRANCH}..." +git checkout -b "${RELEASE_BRANCH}" "upstream/${RELEASE_BRANCH}" + +# Step 2: Merge main branch +print_info "Merging origin/main into ${RELEASE_BRANCH}..." +if ! git merge origin/main --no-ff -m "[internal] Merge internal patches into ${RELEASE_BRANCH}"; then + print_error "Merge failed. Conflicts detected in:" + git diff --name-only --diff-filter=U + print_warn "Please resolve conflicts manually and run:" + echo " git add ." + echo " git commit" + echo " ${SCRIPT_DIR}/update_version.sh ${INTERNAL_VERSION}" + exit 1 +fi + +# Step 3: Update version +print_info "Updating version to ${INTERNAL_VERSION}..." +"${SCRIPT_DIR}/update_version.sh" "${INTERNAL_VERSION}" + +print_info "Release maintenance branch created successfully!" +echo "" +echo "Next steps:" +echo " 1. Review and test the release maintenance branch" +echo " 2. Push to origin: git push origin ${RELEASE_BRANCH}" +echo " 3. Publish later via: ${SCRIPT_DIR}/create_release_publish_branch.sh ${VERSION} <vvr_version>" diff --git a/tools/internal/create_release_publish_branch.sh b/tools/internal/create_release_publish_branch.sh new file mode 100755 index 00000000..99cd4157 --- /dev/null +++ b/tools/internal/create_release_publish_branch.sh @@ -0,0 +1,174 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +####################################### +# Create an internal publish branch from a release maintenance branch. +# +# This script creates a publish branch following the workflow: +# 1. Ensure release-x.y is up to date and aligned with origin/release-x.y +# 2. Create release/x.y-vvr-a.b.c from release-x.y +# 3. Update version to the exact release version +# +# Usage: +# ./create_release_publish_branch.sh <version> <vvr_version> +# +# Arguments: +# version - Apache version (e.g., "0.3") +# vvr_version - VVR version (e.g., "11.6.0") +# +# Examples: +# ./create_release_publish_branch.sh 0.3 11.6.0 +# +# This will: +# - Create release/0.3-vvr-11.6.0 based on release-0.3 +# - Set version to 0.3-vvr-11.6.0 +# - Leave release-0.3 on 0.3-vvr-SNAPSHOT +####################################### + +set -o errexit +set -o nounset + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) +PROJECT_ROOT=$(cd "${SCRIPT_DIR}/../.." && pwd) + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +print_info() { + echo -e "${GREEN}[INFO]${NC} $1" +} + +print_warn() { + echo -e "${YELLOW}[WARN]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +check_origin_release_branch() { + local branch=$1 + + if ! git remote | grep -q "^origin$"; then + print_error "Origin remote not found." + exit 1 + fi + + if ! git rev-parse --verify "${branch}" >/dev/null 2>&1; then + print_error "Local branch ${branch} not found." + print_info "Please create or check out ${branch} first." + exit 1 + fi + + if ! git rev-parse --verify "origin/${branch}" >/dev/null 2>&1; then + print_error "Remote-tracking branch origin/${branch} not found." + print_info "Please push ${branch} before creating a publish branch." + exit 1 + fi + + local local_commit + local origin_commit + local_commit=$(git rev-parse "${branch}") + origin_commit=$(git rev-parse "origin/${branch}") + + if [[ "${local_commit}" != "${origin_commit}" ]]; then + print_error "Local ${branch} and origin/${branch} are not aligned." + print_info "local ${branch}: $(git rev-parse --short "${branch}")" + print_info "origin/${branch}: $(git rev-parse --short "origin/${branch}")" + print_info "Please sync and push ${branch} before creating a publish branch." + exit 1 + fi +} + +if [ -z "${1:-}" ] || [ -z "${2:-}" ]; then + echo "Usage: $0 <version> <vvr_version>" + echo "" + echo "Arguments:" + echo " version - Apache version (e.g., '0.3')" + echo " vvr_version - VVR version (e.g., '11.6.0')" + echo "" + echo "Example:" + echo " $0 0.3 11.6.0" + exit 1 +fi + +VERSION="$1" +VVR_VERSION="$2" +MAINTENANCE_BRANCH="release-${VERSION}" +PUBLISH_BRANCH="release/${VERSION}-vvr-${VVR_VERSION}" +INTERNAL_VERSION="${VERSION}-vvr-${VVR_VERSION}" + +cd "${PROJECT_ROOT}" + +if ! git diff-index --quiet HEAD --; then + print_error "You have uncommitted changes. Please commit or stash them first." + git status --short + exit 1 +fi + +print_info "Fetching origin..." +git fetch origin + +check_origin_release_branch "${MAINTENANCE_BRANCH}" + +if git rev-parse --verify "${PUBLISH_BRANCH}" >/dev/null 2>&1; then + print_error "Local branch ${PUBLISH_BRANCH} already exists." + exit 1 +fi + +if git rev-parse --verify "origin/${PUBLISH_BRANCH}" >/dev/null 2>&1; then + print_error "Remote-tracking branch origin/${PUBLISH_BRANCH} already exists." + print_info "Please use the existing publish branch instead of recreating it." + exit 1 +fi + +print_info "Latest commits on ${MAINTENANCE_BRANCH}:" +git log --oneline -n 10 "${MAINTENANCE_BRANCH}" || true +echo "" + +print_warn "About to create publish branch ${PUBLISH_BRANCH} with version ${INTERNAL_VERSION}" +echo "" +echo "This will:" +echo " 1. Create ${PUBLISH_BRANCH} from ${MAINTENANCE_BRANCH}" +echo " 2. Set version to ${INTERNAL_VERSION}" +echo " 3. Leave ${MAINTENANCE_BRANCH} on ${VERSION}-vvr-SNAPSHOT" +echo "" +read -p "Continue? (y/N) " -n 1 -r +echo +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Aborted." + exit 0 +fi + +print_info "Creating ${PUBLISH_BRANCH} from ${MAINTENANCE_BRANCH}..." +git checkout -b "${PUBLISH_BRANCH}" "${MAINTENANCE_BRANCH}" + +print_info "Updating version to ${INTERNAL_VERSION}..." +"${SCRIPT_DIR}/update_version.sh" "${INTERNAL_VERSION}" + +print_info "Publish branch created successfully!" +echo "" +echo "Next steps:" +echo " 1. Review and test the publish branch" +echo " 2. Push to origin: git push origin ${PUBLISH_BRANCH}" +echo " 3. Build and publish artifacts" +echo " 4. Do not commit hotfixes directly on ${PUBLISH_BRANCH}; fix ${MAINTENANCE_BRANCH} and cut a new publish branch instead" diff --git a/tools/internal/list_patches.sh b/tools/internal/list_patches.sh new file mode 100755 index 00000000..ea06054c --- /dev/null +++ b/tools/internal/list_patches.sh @@ -0,0 +1,128 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +####################################### +# List internal patches for review and upstream contribution. +# +# This script helps identify internal patches that: +# - Are not yet in upstream +# - Are candidates for contributing back to the community +# +# Usage: +# ./list_patches.sh [options] +# +# Options: +# --upstream-candidate Only show patches marked as upstream candidates +# --since <commit> Show patches since the given commit +# --all Show all internal patches (default) +# +# Examples: +# ./list_patches.sh # Show all internal patches +# ./list_patches.sh --upstream-candidate # Show upstream candidates only +# ./list_patches.sh --since HEAD~10 # Show last 10 patches +####################################### + +set -o errexit +set -o nounset + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) +PROJECT_ROOT=$(cd "${SCRIPT_DIR}/../.." && pwd) + +# Colors for output +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +cd "${PROJECT_ROOT}" + +# Parse arguments +UPSTREAM_CANDIDATE_ONLY=false +SINCE="" + +while [[ $# -gt 0 ]]; do + case $1 in + --upstream-candidate) + UPSTREAM_CANDIDATE_ONLY=true + shift + ;; + --since) + SINCE="$2" + shift 2 + ;; + --all) + shift + ;; + *) + echo "Unknown option: $1" + exit 1 + ;; + esac +done + +# Check if upstream remote exists +if ! git remote | grep -q "^upstream$"; then + echo -e "${YELLOW}Warning: Upstream remote not found. Some features may not work correctly.${NC}" + echo "To add upstream remote:" + echo " git remote add upstream [email protected]:apache/flink-agents.git" +fi + +echo -e "${BLUE}=== Internal Patches Summary ===${NC}" +echo "" + +# Get the base reference +if [ -n "${SINCE}" ]; then + BASE="${SINCE}" +elif git remote | grep -q "^upstream$"; then + BASE="upstream/main" +else + BASE="origin/main" +fi + +# Count internal patches (with [AONE-] prefix) +INTERNAL_COUNT=$(git log "${BASE}.." --oneline --grep='\[AONE-' 2>/dev/null | wc -l | tr -d ' ') +echo -e "Internal patches ([AONE-*]): ${GREEN}${INTERNAL_COUNT}${NC}" + +# Count upstream candidates +CANDIDATE_COUNT=$(git log "${BASE}.." --oneline --grep='Upstream-candidate: yes' 2>/dev/null | wc -l | tr -d ' ') +echo -e "Upstream candidates: ${GREEN}${CANDIDATE_COUNT}${NC}" + +echo "" +echo -e "${BLUE}=== Internal Patches ===${NC}" +echo "" + +if [ "${UPSTREAM_CANDIDATE_ONLY}" = true ]; then + echo "Patches marked as upstream candidates:" + echo "" + git log "${BASE}.." --oneline --grep='Upstream-candidate: yes' 2>/dev/null || echo "No upstream candidates found." +else + echo "All internal patches ([AONE-*] commits):" + echo "" + git log "${BASE}.." --oneline --grep='\[AONE-' 2>/dev/null || echo "No internal patches found." +fi + +echo "" + +# Show detailed info for upstream candidates +if [ "${UPSTREAM_CANDIDATE_ONLY}" = true ] && [ "${CANDIDATE_COUNT}" -gt 0 ]; then + echo -e "${BLUE}=== Detailed Information ===${NC}" + echo "" + + git log "${BASE}.." --grep='Upstream-candidate: yes' --format="%h %s%n%b%n---" 2>/dev/null +fi diff --git a/tools/internal/sync_upstream.sh b/tools/internal/sync_upstream.sh new file mode 100755 index 00000000..3a4f2669 --- /dev/null +++ b/tools/internal/sync_upstream.sh @@ -0,0 +1,246 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +set -o errexit +set -o nounset + +####################################### +# Sync internal fork with upstream Apache flink-agents repository. +# +# This script handles both main and release maintenance branches: +# - For main branch: uses rebase to keep internal patches independent +# - For release-* maintenance branches: uses merge to sync upstream bugfixes +# - Published branches under release/ are intentionally excluded +# +# Usage: +# ./sync_upstream.sh [branch] +# +# Arguments: +# branch - Optional. The branch to sync (default: current branch) +# +# Examples: +# ./sync_upstream.sh # Sync current branch +# ./sync_upstream.sh main # Sync main branch +# ./sync_upstream.sh release-0.3 # Sync release-0.3 maintenance branch +####################################### + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) +PROJECT_ROOT=$(cd "${SCRIPT_DIR}/../.." && pwd) +BUILD_SCRIPT="${PROJECT_ROOT}/tools/build.sh" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +print_info() { + echo -e "${GREEN}[INFO]${NC} $1" +} + +print_warn() { + echo -e "${YELLOW}[WARN]${NC} $1" +} + +print_error() { + echo -e "${RED}[ERROR]${NC} $1" +} + +# Check if upstream remote exists +check_upstream() { + if ! git remote | grep -q "^upstream$"; then + print_error "Upstream remote not found. Please add it first:" + echo " git remote add upstream [email protected]:apache/flink-agents.git" + exit 1 + fi +} + +# Get current branch name +get_current_branch() { + git rev-parse --abbrev-ref HEAD +} + +# Check if there are uncommitted changes +check_clean_worktree() { + if ! git diff-index --quiet HEAD --; then + print_error "You have uncommitted changes. Please commit or stash them first." + git status --short + exit 1 + fi +} + +# Create backup branch +create_backup() { + local branch=$1 + local timestamp=$(date +%Y%m%d_%H%M%S) + local backup_branch="${branch}-backup-${timestamp}" + print_info "Creating backup branch: ${backup_branch}" >&2 + git branch "${backup_branch}" + echo "${backup_branch}" +} + +# Sync main branch using rebase +sync_main() { + print_info "Syncing main branch with upstream/main (using rebase)..." + + # Fetch upstream + print_info "Fetching upstream..." + git fetch upstream + + # Check if upstream has new commits + local upstream_new=$(git rev-list HEAD..upstream/main --count 2>/dev/null || echo "unknown") + if [[ "${upstream_new}" == "0" ]]; then + print_info "Upstream/main is already up to date. Nothing to sync." + return 0 + fi + print_info "Upstream/main has ${upstream_new} new commits." + + # Create backup + local backup_branch=$(create_backup "main") + + # Show internal patches that will be rebased + print_info "Internal patches to rebase:" + git log upstream/main..HEAD --oneline || true + + # Perform rebase + print_info "Rebasing onto upstream/main..." + if ! git rebase upstream/main; then + print_error "Rebase failed. Conflicts detected in:" + git diff --name-only --diff-filter=U + print_warn "Aborting rebase and restoring from backup..." + git rebase --abort || true + git reset --hard "main" + print_error "Please resolve conflicts manually." + print_info "Your original work is preserved in branch: ${backup_branch}" + exit 1 + fi + + # Verify build + print_info "Verifying build..." + cd "${PROJECT_ROOT}" + if ! "${BUILD_SCRIPT}"; then + print_error "Build failed after sync. Restoring from backup..." + git reset --hard "${backup_branch}" + print_error "Please fix build issues and try again." + exit 1 + fi + + print_info "Sync completed successfully!" + print_info "Backup branch: ${backup_branch}" + echo "" + print_warn "IMPORTANT: This operation may have force-pushed your branch." + echo " Team members should run:" + echo " git fetch origin" + echo " git reset --hard origin/main" + echo "" + print_info "When ready, push with:" + echo " git push origin main --force-with-lease" +} + +# Sync release maintenance branch using merge +sync_release() { + local branch=$1 + local upstream_branch="upstream/${branch}" + + print_info "Syncing ${branch} with ${upstream_branch} (using merge)..." + + # Fetch upstream + print_info "Fetching upstream..." + git fetch upstream + + # Check if upstream branch exists + if ! git rev-parse --verify "${upstream_branch}" >/dev/null 2>&1; then + print_error "Upstream branch ${upstream_branch} does not exist." + exit 1 + fi + + # Check if upstream has new commits + local upstream_new=$(git rev-list HEAD..${upstream_branch} --count 2>/dev/null || echo "unknown") + if [[ "${upstream_new}" == "0" ]]; then + print_info "${upstream_branch} is already up to date. Nothing to sync." + return 0 + fi + print_info "${upstream_branch} has ${upstream_new} new commits." + + # Create backup + local backup_branch=$(create_backup "${branch}") + + # Perform merge + print_info "Merging ${upstream_branch}..." + if ! git merge "${upstream_branch}" -m "[internal] Merge upstream ${branch}"; then + print_error "Merge failed. Conflicts detected in:" + git diff --name-only --diff-filter=U + print_warn "Aborting merge and restoring from backup..." + git merge --abort || true + git reset --hard "${branch}" + print_error "Please resolve conflicts manually." + print_info "Your original work is preserved in branch: ${backup_branch}" + exit 1 + fi + + # Verify build + print_info "Verifying build..." + cd "${PROJECT_ROOT}" + if ! "${BUILD_SCRIPT}"; then + print_error "Build failed after sync. Restoring from backup..." + git reset --hard "${backup_branch}" + print_error "Please fix build issues and try again." + exit 1 + fi + + print_info "Sync completed successfully!" + print_info "Backup branch: ${backup_branch}" + echo "" + print_info "When ready, push with:" + echo " git push origin ${branch}" +} + +# Main +main() { + cd "${PROJECT_ROOT}" + + check_upstream + check_clean_worktree + + local target_branch="${1:-$(get_current_branch)}" + + # Switch to target branch if not already on it + if [[ "$(get_current_branch)" != "${target_branch}" ]]; then + print_info "Switching to branch: ${target_branch}" + git checkout "${target_branch}" + fi + + print_info "Target branch: ${target_branch}" + + if [[ "${target_branch}" == "main" ]]; then + sync_main + elif [[ "${target_branch}" == release-* ]]; then + sync_release "${target_branch}" + elif [[ "${target_branch}" == release/* ]]; then + print_error "Published release branch ${target_branch} must not be synced with upstream." + print_info "Sync the corresponding maintenance branch release-x.y instead." + exit 1 + else + print_error "Unknown branch type: ${target_branch}" + print_info "This script only supports 'main' and release maintenance branches named 'release-*'." + exit 1 + fi +} + +main "$@" diff --git a/tools/internal/update_version.sh b/tools/internal/update_version.sh new file mode 100755 index 00000000..850112f4 --- /dev/null +++ b/tools/internal/update_version.sh @@ -0,0 +1,127 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +####################################### +# Update version for internal maintenance or publish branches. +# +# This script updates version numbers in all required locations: +# - pom.xml files (Maven version) +# - python/pyproject.toml (Python version) +# - docs/config.toml (Documentation version) +# +# Internal version format: x.y-vvr-a.b.c +# - x.y: Apache flink-agents version +# - a.b.c: VVR version +# +# Usage: +# ./update_version.sh <version> +# +# Arguments: +# version - The internal version to set (e.g., "0.3-vvr-11.6.0") +# +# Examples: +# ./update_version.sh 0.3-vvr-11.6.0 # Release version +# ./update_version.sh 0.3-vvr-SNAPSHOT # Development version +####################################### + +set -o errexit +set -o nounset +set -o xtrace + +MVN=${MVN:-mvn} + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) +PROJECT_ROOT=$(cd "${SCRIPT_DIR}/../.." && pwd) + +if [ -z "${1:-}" ]; then + echo "Usage: $0 <version>" + echo "" + echo "Examples:" + echo " $0 0.3-vvr-11.6.0 # Release version for VVR 11.6.0" + echo " $0 0.3-vvr-SNAPSHOT # Development version" + exit 1 +fi + +NEW_VERSION="$1" + +# Validate version format +if ! echo "${NEW_VERSION}" | grep -qE "^[0-9]+\.[0-9]+-vvr-"; then + echo "Error: Invalid version format. Expected: x.y-vvr-a.b.c or x.y-vvr-SNAPSHOT" + echo "Got: ${NEW_VERSION}" + exit 1 +fi + +cd "${PROJECT_ROOT}" + +if ! git diff-index --quiet HEAD --; then + echo "Error: You have uncommitted changes. Please commit or stash them first." + git status --short + exit 1 +fi + +# Convert Java version to Python version format +# Java: 0.3-vvr-11.6.0 -> Python: 0.3+vvr.11.6.0 +# Java: 0.3-vvr-SNAPSHOT -> Python: 0.3+vvr.dev0 +PYTHON_VERSION=$(echo "${NEW_VERSION}" | sed 's/-vvr-/+vvr./' | sed 's/SNAPSHOT/dev0/') + +echo "Updating version to:" +echo " Java: ${NEW_VERSION}" +echo " Python: ${PYTHON_VERSION}" + +# Update Maven version in all pom files +echo "Updating Maven version..." +${MVN} org.codehaus.mojo:versions-maven-plugin:2.8.1:set \ + -DnewVersion="${NEW_VERSION}" \ + -DgenerateBackupPoms=false \ + --quiet + +# Update Python version +echo "Updating Python version..." +cd python +# Update version line in pyproject.toml +if [[ "${NEW_VERSION}" == *"-SNAPSHOT" ]]; then + # Development version: 0.3+vvr.dev0 + perl -pi -e "s#^version = \".*\"#version = \"${PYTHON_VERSION}\"#" pyproject.toml +else + # Release version: 0.3+vvr.11.6.0 + perl -pi -e "s#^version = \".*\"#version = \"${PYTHON_VERSION}\"#" pyproject.toml +fi +cd .. + +# Update docs version if docs/config.toml exists +if [ -f "docs/config.toml" ]; then + echo "Updating docs version..." + cd docs + perl -pi -e "s#^ Version = .*# Version = \"${NEW_VERSION}\"#" config.toml + perl -pi -e "s#^ VersionTitle = .*# VersionTitle = \"${NEW_VERSION}\"#" config.toml + cd .. +fi + +if git diff --quiet --; then + echo "Version is already ${NEW_VERSION}. No changes to commit." + exit 0 +fi + +# Commit the changes +echo "Committing version update..." +git commit -am "[internal] Update version to ${NEW_VERSION}" + +echo "" +echo "Version updated successfully!" +echo "Don't forget to push the change."
