(airflow) branch fix-registry-incremental-provider-flag updated: Also exclude provider HTML pages during incremental S3 sync

[kaxilnaik]

This is an automated email from the ASF dual-hosted git repository.

kaxilnaik pushed a commit to branch fix-registry-incremental-provider-flag
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to 
refs/heads/fix-registry-incremental-provider-flag by this push:
     new a867767322f Also exclude provider HTML pages during incremental S3 sync
a867767322f is described below

commit a867767322f76134ff7abc120f257527d958077e
Author: Kaxil Naik <kaxiln...@apache.org>
AuthorDate: Tue Mar 17 18:34:59 2026 +0000

    Also exclude provider HTML pages during incremental S3 sync
    
    Non-target provider pages are rebuilt without connection/parameter data
    (the version-specific extraction files don't exist locally). Without
    this exclude, the incremental build overwrites complete HTML pages on
    S3 with versions missing the connection builder section.
---
 .github/workflows/registry-build.yml | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/.github/workflows/registry-build.yml 
b/.github/workflows/registry-build.yml
index 0a5d7bc954f..fcfeb65ab35 100644
--- a/.github/workflows/registry-build.yml
+++ b/.github/workflows/registry-build.yml
@@ -252,27 +252,32 @@ jobs:
           PROVIDER: ${{ inputs.provider }}
         run: |
           # Incremental builds only extract data for the target provider(s).
-          # Eleventy still generates API files for every provider, but
-          # non-target files contain incomplete/empty data (no version info,
-          # empty connections, etc.).  Exclude the entire per-provider API
-          # subtree from the main sync, then selectively upload only the
-          # target provider's files.
-          EXCLUDE_PROVIDER_API=()
+          # Eleventy rebuilds all pages, but non-target providers have
+          # incomplete data (no connections/parameters, wrong version info).
+          # Exclude both per-provider API JSON and HTML pages from the main
+          # sync, then selectively upload only the target provider's files.
+          EXCLUDE_PROVIDERS=()
           if [[ -n "${PROVIDER}" ]]; then
-            EXCLUDE_PROVIDER_API=(--exclude "api/providers/*")
+            EXCLUDE_PROVIDERS=(
+              --exclude "api/providers/*"
+              --exclude "providers/*"
+            )
           fi
 
           aws s3 sync registry/_site/ "${S3_BUCKET}" \
             --cache-control "${REGISTRY_CACHE_CONTROL}" \
             --exclude "pagefind/*" \
-            "${EXCLUDE_PROVIDER_API[@]}"
+            "${EXCLUDE_PROVIDERS[@]}"
 
-          # For incremental builds, sync only the target provider's API files.
+          # For incremental builds, sync only the target provider's files.
           if [[ -n "${PROVIDER}" ]]; then
             for pid in ${PROVIDER}; do
               aws s3 sync "registry/_site/api/providers/${pid}/" \
                 "${S3_BUCKET}api/providers/${pid}/" \
                 --cache-control "${REGISTRY_CACHE_CONTROL}"
+              aws s3 sync "registry/_site/providers/${pid}/" \
+                "${S3_BUCKET}providers/${pid}/" \
+                --cache-control "${REGISTRY_CACHE_CONTROL}"
             done
           fi