This is an automated email from the ASF dual-hosted git repository.
raulcd pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 8595105ec2 GH-49743: [Release] Split the vote thread preparation into
its own shell script (#49770)
8595105ec2 is described below
commit 8595105ec26eea97182c285572ddd91cc1186a24
Author: Raúl Cumplido <[email protected]>
AuthorDate: Fri Apr 17 09:47:07 2026 +0200
GH-49743: [Release] Split the vote thread preparation into its own shell
script (#49770)
### Rationale for this change
Currently running the script is failing if default flags are used and also
failing if we run the source vote in separation as documented on the
release.rst page.
### What changes are included in this PR?
Splitting the script into two each one for it's own purpose will make it
easier to focus and have less "flag" incompatibilities which to be fair is
unnecessary for the purpose of these minor scripts.
### Are these changes tested?
Yes, I've tested the new script in isolation to generate a new vote email.
### Are there any user-facing changes?
No
* GitHub Issue: #49743
Lead-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
---
dev/release/02-source-test.rb | 83 ----------------------
dev/release/02-source.sh | 71 ------------------
.../{02-source-test.rb => 09-vote-email-test.rb} | 48 ++++---------
dev/release/{02-source.sh => 09-vote-email.sh} | 83 ++++------------------
docs/source/developers/release.rst | 2 +-
5 files changed, 29 insertions(+), 258 deletions(-)
diff --git a/dev/release/02-source-test.rb b/dev/release/02-source-test.rb
index fdd1db9c60..bc97c43ab2 100644
--- a/dev/release/02-source-test.rb
+++ b/dev/release/02-source-test.rb
@@ -46,8 +46,6 @@ class SourceTest < Test::Unit::TestCase
sh(env, @tarball_script, @release_version, "0")
FileUtils.mkdir_p("artifacts")
sh("mv", @archive_name, "artifacts/")
- File.write("artifacts/#{@archive_name}.sha512",
- sh(env, "shasum", "-a", "512", "artifacts/#{@archive_name}"))
output = sh(env, @script, @release_version, "0")
sh("tar", "xf", "artifacts/#{@archive_name}")
output
@@ -74,85 +72,4 @@ class SourceTest < Test::Unit::TestCase
Dir.glob("dist/pyarrow-*.tar.gz"))
end
end
-
- def test_vote
- github_token = File.read(@env)[/^GH_TOKEN=(.*)$/, 1]
- uri = URI.parse("https://api.github.com/graphql";)
- n_issues_query = {
- "query" => <<-QUERY,
- query {
- search(query: "repo:apache/arrow is:issue is:closed
milestone:#{@release_version}",
- type: ISSUE) {
- issueCount
- }
- }
- QUERY
- }
- response = Net::HTTP.post(uri,
- n_issues_query.to_json,
- "Content-Type" => "application/json",
- "Authorization" => "Bearer #{github_token}")
- n_resolved_issues =
JSON.parse(response.body)["data"]["search"]["issueCount"]
- github_repository = ENV["GITHUB_REPOSITORY"] || "apache/arrow"
- github_api_url = "https://api.github.com";
- verify_prs = URI("#{github_api_url}/repos/#{github_repository}/pulls" +
- "?state=open" +
- "&head=apache:release-#{@release_version}-rc0")
- verify_pr_url = nil
- headers = {
- "Accept" => "application/vnd.github+json",
- }
-
- if github_token
- headers["Authorization"] = "Bearer #{github_token}"
- end
- verify_prs.open(headers) do |response|
- verify_pr_url = (JSON.parse(response.read)[0] || {})["html_url"]
- end
- output = source("VOTE")
- tarball_hash = Digest::SHA512.file("artifacts/#{@archive_name}").to_s
- assert_equal(<<-VOTE.strip, output[/^-+$(.+?)^-+$/m, 1].strip)
-To: [email protected]
-Subject: [VOTE] Release Apache Arrow #{@release_version} - RC0
-
-Hi,
-
-I would like to propose the following release candidate (RC0) of Apache
-Arrow version #{@release_version}. This is a release consisting of
#{n_resolved_issues}
-resolved GitHub issues[1].
-
-This release candidate is based on commit:
-#{@current_commit} [2]
-
-The source release rc0 is hosted at [3].
-The binary artifacts are hosted at [4][5][6][7][8][9].
-The changelog is located at [10].
-
-Please download, verify checksums and signatures, run the unit tests,
-and vote on the release. See [11] for the SHA-512 checksum for this RC and [12]
-for how to validate a release candidate.
-
-See also a verification result on GitHub pull request [13].
-
-The vote will be open for at least 72 hours.
-
-[ ] +1 Release this as Apache Arrow #{@release_version}
-[ ] +0
-[ ] -1 Do not release this as Apache Arrow #{@release_version} because...
-
-[1]:
https://github.com/apache/arrow/issues?q=is%3Aissue+milestone%3A#{@release_version}+is%3Aclosed
-[2]: https://github.com/apache/arrow/tree/#{@current_commit}
-[3]:
https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-#{@release_version}-rc0
-[4]: https://packages.apache.org/artifactory/arrow/almalinux-rc/
-[5]: https://packages.apache.org/artifactory/arrow/amazon-linux-rc/
-[6]: https://packages.apache.org/artifactory/arrow/centos-rc/
-[7]: https://packages.apache.org/artifactory/arrow/debian-rc/
-[8]: https://packages.apache.org/artifactory/arrow/ubuntu-rc/
-[9]:
https://github.com/apache/arrow/releases/tag/apache-arrow-#{@release_version}-rc0
-[10]: https://github.com/apache/arrow/blob/#{@current_commit}/CHANGELOG.md
-[11]: #{tarball_hash}
-[12]: https://arrow.apache.org/docs/developers/release_verification.html
-[13]: #{verify_pr_url || "null"}
- VOTE
- end
end
diff --git a/dev/release/02-source.sh b/dev/release/02-source.sh
index a99e529065..1fe0adcac2 100755
--- a/dev/release/02-source.sh
+++ b/dev/release/02-source.sh
@@ -25,7 +25,6 @@ set -eu
: ${SOURCE_RAT:=${SOURCE_DEFAULT}}
: ${SOURCE_UPLOAD:=${SOURCE_DEFAULT}}
: ${SOURCE_PR:=${SOURCE_DEFAULT}}
-: ${SOURCE_VOTE:=${SOURCE_DEFAULT}}
SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
@@ -114,73 +113,3 @@ if [ ${SOURCE_PR} -gt 0 ]; then
--verify-source \
--version=${version}
fi
-
-if [ ${SOURCE_VOTE} -gt 0 ]; then
- curl_common_options=(--header "Authorization: Bearer ${GH_TOKEN}")
-
- curl_options=("${curl_common_options[@]}")
- curl_options+=(--data "{\"query\": \"query {search(query:
\\\"repo:apache/arrow is:issue is:closed milestone:${version}\\\", type:ISSUE)
{issueCount}}\"}")
- curl_options+=("https://api.github.com/graphql";)
- n_resolved_issues=$(curl "${curl_options[@]}" | jq ".data.search.issueCount")
-
- curl_options=("${curl_common_options[@]}")
- curl_options+=(--header "Accept: application/vnd.github+json")
- curl_options+=(--get)
- curl_options+=(--data "state=open")
- curl_options+=(--data "head=apache:${rc_branch}")
- curl_options+=(https://api.github.com/repos/${GITHUB_REPOSITORY}/pulls)
- verify_pr_url=$(curl "${curl_options[@]}" | jq -r ".[0].html_url")
- # Read the checksum so we can include it in the vote thread email.
- sha512_path="artifacts/${tarball}.sha512"
- [[ -f "${sha512_path}" ]] || { echo "Error: ${sha512_path} must exist"; exit
1; }
- tarball_hash=$(cat "${sha512_path}" | awk '{print $1}')
-
- echo "The following draft email has been created to send to the"
- echo "[email protected] mailing list"
- echo ""
- echo "---------------------------------------------------------"
- cat <<MAIL
-To: [email protected]
-Subject: [VOTE] Release Apache Arrow ${version} - RC${rc}
-
-Hi,
-
-I would like to propose the following release candidate (RC${rc}) of Apache
-Arrow version ${version}. This is a release consisting of ${n_resolved_issues}
-resolved GitHub issues[1].
-
-This release candidate is based on commit:
-${release_hash} [2]
-
-The source release rc${rc} is hosted at [3].
-The binary artifacts are hosted at [4][5][6][7][8][9].
-The changelog is located at [10].
-
-Please download, verify checksums and signatures, run the unit tests,
-and vote on the release. See [11] for the SHA-512 checksum for this RC and [12]
-for how to validate a release candidate.
-
-See also a verification result on GitHub pull request [13].
-
-The vote will be open for at least 72 hours.
-
-[ ] +1 Release this as Apache Arrow ${version}
-[ ] +0
-[ ] -1 Do not release this as Apache Arrow ${version} because...
-
-[1]:
https://github.com/apache/arrow/issues?q=is%3Aissue+milestone%3A${version}+is%3Aclosed
-[2]: https://github.com/apache/arrow/tree/${release_hash}
-[3]: ${rc_url}
-[4]: https://packages.apache.org/artifactory/arrow/almalinux-rc/
-[5]: https://packages.apache.org/artifactory/arrow/amazon-linux-rc/
-[6]: https://packages.apache.org/artifactory/arrow/centos-rc/
-[7]: https://packages.apache.org/artifactory/arrow/debian-rc/
-[8]: https://packages.apache.org/artifactory/arrow/ubuntu-rc/
-[9]:
https://github.com/apache/arrow/releases/tag/apache-arrow-${version}-rc${rc}
-[10]: https://github.com/apache/arrow/blob/${release_hash}/CHANGELOG.md
-[11]: ${tarball_hash}
-[12]: https://arrow.apache.org/docs/developers/release_verification.html
-[13]: ${verify_pr_url}
-MAIL
- echo "---------------------------------------------------------"
-fi
diff --git a/dev/release/02-source-test.rb b/dev/release/09-vote-email-test.rb
similarity index 79%
copy from dev/release/02-source-test.rb
copy to dev/release/09-vote-email-test.rb
index fdd1db9c60..bae314b245 100644
--- a/dev/release/02-source-test.rb
+++ b/dev/release/09-vote-email-test.rb
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
-class SourceTest < Test::Unit::TestCase
+class VoteEmailTest < Test::Unit::TestCase
include GitRunnable
include VersionDetectable
@@ -24,7 +24,7 @@ class SourceTest < Test::Unit::TestCase
detect_versions
@tag_name_no_rc = "apache-arrow-#{@release_version}"
@archive_name = "apache-arrow-#{@release_version}.tar.gz"
- @script = File.expand_path("dev/release/02-source.sh")
+ @script = File.expand_path("dev/release/09-vote-email.sh")
@tarball_script =
File.expand_path("dev/release/utils-create-release-tarball.sh")
@env = File.expand_path("dev/release/.env")
@@ -35,47 +35,25 @@ class SourceTest < Test::Unit::TestCase
end
end
- def source(*targets)
+ def vote_email(*targets)
env = {
- "SOURCE_DEFAULT" => "0",
+ "VOTE_DEFAULT" => "0",
"release_hash" => @current_commit,
}
targets.each do |target|
- env["SOURCE_#{target}"] = "1"
+ env["VOTE_#{target}"] = "1"
end
sh(env, @tarball_script, @release_version, "0")
FileUtils.mkdir_p("artifacts")
sh("mv", @archive_name, "artifacts/")
- File.write("artifacts/#{@archive_name}.sha512",
- sh(env, "shasum", "-a", "512", "artifacts/#{@archive_name}"))
- output = sh(env, @script, @release_version, "0")
- sh("tar", "xf", "artifacts/#{@archive_name}")
- output
- end
-
- def test_symbolic_links
- source
- Dir.chdir(@tag_name_no_rc) do
- assert_equal([],
- Find.find(".").find_all {|path| File.symlink?(path)})
- end
- end
-
- def test_python_version
- source
- Dir.chdir("#{@tag_name_no_rc}/python") do
- sh("python", "-m", "build", "--sdist")
- if on_release_branch?
- pyarrow_source_archive = "dist/pyarrow-#{@release_version}.tar.gz"
- else
- pyarrow_source_archive = "dist/pyarrow-#{@release_version}a0.tar.gz"
- end
- assert_equal([pyarrow_source_archive],
- Dir.glob("dist/pyarrow-*.tar.gz"))
+ Dir.chdir("artifacts") do
+ File.write("#{@archive_name}.sha512",
+ sh(env, "shasum", "-a", "512", @archive_name))
end
+ sh(env, @script, @release_version, "0")
end
- def test_vote
+ def test_template
github_token = File.read(@env)[/^GH_TOKEN=(.*)$/, 1]
uri = URI.parse("https://api.github.com/graphql";)
n_issues_query = {
@@ -109,9 +87,9 @@ class SourceTest < Test::Unit::TestCase
verify_prs.open(headers) do |response|
verify_pr_url = (JSON.parse(response.read)[0] || {})["html_url"]
end
- output = source("VOTE")
+ output = vote_email("TEMPLATE")
tarball_hash = Digest::SHA512.file("artifacts/#{@archive_name}").to_s
- assert_equal(<<-VOTE.strip, output[/^-+$(.+?)^-+$/m, 1].strip)
+ assert_equal(<<-TEMPLATE.strip, output[/^-+$(.+?)^-+$/m, 1].strip)
To: [email protected]
Subject: [VOTE] Release Apache Arrow #{@release_version} - RC0
@@ -153,6 +131,6 @@ The vote will be open for at least 72 hours.
[11]: #{tarball_hash}
[12]: https://arrow.apache.org/docs/developers/release_verification.html
[13]: #{verify_pr_url || "null"}
- VOTE
+ TEMPLATE
end
end
diff --git a/dev/release/02-source.sh b/dev/release/09-vote-email.sh
similarity index 69%
copy from dev/release/02-source.sh
copy to dev/release/09-vote-email.sh
index a99e529065..b5aebc1740 100755
--- a/dev/release/02-source.sh
+++ b/dev/release/09-vote-email.sh
@@ -20,12 +20,10 @@
set -eu
-: ${SOURCE_DEFAULT:=1}
-: ${SOURCE_DOWNLOAD:=${SOURCE_DEFAULT}}
-: ${SOURCE_RAT:=${SOURCE_DEFAULT}}
-: ${SOURCE_UPLOAD:=${SOURCE_DEFAULT}}
-: ${SOURCE_PR:=${SOURCE_DEFAULT}}
-: ${SOURCE_VOTE:=${SOURCE_DEFAULT}}
+: ${VOTE_DEFAULT:=1}
+: ${VOTE_CLEANUP:=${VOTE_DEFAULT}}
+: ${VOTE_DOWNLOAD:=${VOTE_DEFAULT}}
+: ${VOTE_TEMPLATE:=${VOTE_DEFAULT}}
SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
@@ -41,81 +39,26 @@ rc=$2
. "${SOURCE_DIR}/utils-env.sh"
tag=apache-arrow-${version}-rc${rc}
-maint_branch=maint-${version}
rc_branch="release-${version}-rc${rc}"
rc_url="https://dist.apache.org/repos/dist/dev/arrow/${tag}";
-echo "Preparing source for tag ${tag}"
-
: ${release_hash:=$(cd "${SOURCE_TOP_DIR}" && git rev-list --max-count=1
${tag})}
: ${GITHUB_REPOSITORY:=apache/arrow}
-if [ ${SOURCE_UPLOAD} -gt 0 ]; then
- if [ -z "$release_hash" ]; then
- echo "Cannot continue: unknown git tag: $tag"
- exit
- fi
-fi
-
echo "Using commit $release_hash"
-tarball=apache-arrow-${version}.tar.gz
+tarball_sha512=apache-arrow-${version}.tar.gz.sha512
-if [ ${SOURCE_DOWNLOAD} -gt 0 ]; then
- # Wait for the release candidate workflow to finish before attempting
- # to download the tarball from the GitHub Release.
- . $SOURCE_DIR/utils-watch-gh-workflow.sh ${tag} "release_candidate.yml"
+if [ ${VOTE_DOWNLOAD} -gt 0 ]; then
+ echo "Downloading tarball checksum for ${tag}"
rm -rf artifacts
gh release download ${tag} \
--dir artifacts \
- --repo "${GITHUB_REPOSITORY}"
-fi
-
-if [ ${SOURCE_RAT} -gt 0 ]; then
- "${SOURCE_DIR}/run-rat.sh" artifacts/${tarball}
-fi
-
-if [ ${SOURCE_UPLOAD} -gt 0 ]; then
- # check out the arrow RC folder
- svn co --depth=empty https://dist.apache.org/repos/dist/dev/arrow tmp
-
- # add the release candidate for the tag
- mkdir -p tmp/${tag}
-
- # copy the release candidate tarball and related files into the tmp dir
- cp artifacts/${tarball}* tmp/${tag}
-
- # commit to svn
- svn add tmp/${tag}
- svn ci -m "Apache Arrow ${version} RC${rc}" tmp/${tag}
-
- # clean up
- rm -rf artifacts
- rm -rf tmp
-
- echo "Success! The release candidate is available here:"
- echo " ${rc_url}"
- echo ""
- echo "Commit SHA1: ${release_hash}"
- echo ""
+ --repo "${GITHUB_REPOSITORY}" \
+ --pattern "${tarball_sha512}"
fi
-# Create Pull Request and Crossbow comment to run verify source tasks
-if [ ${SOURCE_PR} -gt 0 ]; then
- archery crossbow \
- verify-release-candidate \
- --base-branch=${maint_branch} \
- --create-pr \
- --head-branch=${rc_branch} \
- --pr-body="PR to verify Release Candidate" \
- --pr-title="WIP: [Release] Verify ${rc_branch}" \
- --remote=https://github.com/${GITHUB_REPOSITORY} \
- --rc=${rc} \
- --verify-source \
- --version=${version}
-fi
-
-if [ ${SOURCE_VOTE} -gt 0 ]; then
+if [ ${VOTE_TEMPLATE} -gt 0 ]; then
curl_common_options=(--header "Authorization: Bearer ${GH_TOKEN}")
curl_options=("${curl_common_options[@]}")
@@ -131,7 +74,7 @@ if [ ${SOURCE_VOTE} -gt 0 ]; then
curl_options+=(https://api.github.com/repos/${GITHUB_REPOSITORY}/pulls)
verify_pr_url=$(curl "${curl_options[@]}" | jq -r ".[0].html_url")
# Read the checksum so we can include it in the vote thread email.
- sha512_path="artifacts/${tarball}.sha512"
+ sha512_path="artifacts/${tarball_sha512}"
[[ -f "${sha512_path}" ]] || { echo "Error: ${sha512_path} must exist"; exit
1; }
tarball_hash=$(cat "${sha512_path}" | awk '{print $1}')
@@ -184,3 +127,7 @@ The vote will be open for at least 72 hours.
MAIL
echo "---------------------------------------------------------"
fi
+
+if [ ${VOTE_CLEANUP} -gt 0 ]; then
+ rm -rf artifacts
+fi
diff --git a/docs/source/developers/release.rst
b/docs/source/developers/release.rst
index 0ec81c1e6c..631019f446 100644
--- a/docs/source/developers/release.rst
+++ b/docs/source/developers/release.rst
@@ -280,7 +280,7 @@ Verify the Release
# Once the automatic verification has passed start the vote thread
# on [email protected]. To regenerate the email template use
- SOURCE_DEFAULT=0 SOURCE_VOTE=1 dev/release/02-source.sh <version>
<rc-number>
+ dev/release/09-vote-email.sh <version> <rc-number>
See :ref:`release_verification` for details.
