(solr-operator) branch main updated: Workaround dist.apache.org download restrictions (#752)

[gerlowskija]

This is an automated email from the ASF dual-hosted git repository.

gerlowskija pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/solr-operator.git


The following commit(s) were added to refs/heads/main by this push:
     new 0ffddb6  Workaround dist.apache.org download restrictions (#752)
0ffddb6 is described below

commit 0ffddb6a4be25ffb3fdad6cb3ffe759d89c5fcf2
Author: Jason Gerlowski <gerlowsk...@apache.org>
AuthorDate: Fri Jan 24 14:06:09 2025 -0500

    Workaround dist.apache.org download restrictions (#752)
    
    After a "passing" RC,the wizard has RMs download the CRDs and helm
    charts from a 'staging' area on dist.apache.org and then upload them to
    the final location.  We have scripts to do this, but these were broken
    recently when dist.apache.org changed its robots.txt to disallow unknown
    "crawlers".
    
    This commit gets our scripting working again by tweaking a `wget`
    invocation to not strictly obey the robots.txt for dist.apache.org,
    which likely isn't intended for restricting foundation-internal usecases
    such as ours.
---
 hack/release/upload/upload_crds.sh | 2 +-
 hack/release/upload/upload_helm.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/hack/release/upload/upload_crds.sh 
b/hack/release/upload/upload_crds.sh
index 44a46fa..5d524f5 100755
--- a/hack/release/upload/upload_crds.sh
+++ b/hack/release/upload/upload_crds.sh
@@ -95,7 +95,7 @@ echo "Pulling CRDs from the staged url and uploading to 
release location ${CRDS_
     cd "${TMP_DIR}"
 
     # Download CRD files from the staged location
-    wget -r -np -nH -nd --level=1 -A "*.yaml*" "${RELEASE_URL}/crds/"
+    wget -e robots=off -r -np -nH -nd --level=1 -A "*.yaml*" 
"${RELEASE_URL}/crds/"
 
     rm -f robots.txt*
 
diff --git a/hack/release/upload/upload_helm.sh 
b/hack/release/upload/upload_helm.sh
index 7e155ae..a0aa526 100755
--- a/hack/release/upload/upload_helm.sh
+++ b/hack/release/upload/upload_helm.sh
@@ -96,7 +96,7 @@ echo "Pulling Helm chart from the staged url and uploading to 
release Helm repo.
     cd "${TMP_DIR}"
 
     # Pull Helm charts from staged location
-    wget -r -np -nH -nd --level=1 -A "*.tgz*" "${RELEASE_URL}/helm-charts"
+    wget -e robots=off -r -np -nH -nd --level=1 -A "*.tgz*" 
"${RELEASE_URL}/helm-charts"
 
     rm -f robots.txt*