kou commented on code in PR #13453:
URL: https://github.com/apache/arrow/pull/13453#discussion_r909080320
##########
.github/workflows/r_nightly.yml:
##########
@@ -100,18 +149,37 @@ jobs:
# strip superfluous nested dirs
new_paths <- sub(art_path, ".", new_paths)
dirs <- dirname(new_paths)
- dir_result <- sapply(dirs, dir.create, recursive = TRUE)
+ sapply(dirs, dir.create, recursive = TRUE, showWarnings = FALSE)
- if (!all(dir_result)) {
- stop("There was an issue while creating the folders!")
- }
-
- copy_result <- file.copy(current_path, new_paths)
+ # overwrite allows us to "force push" a new version with the same
name
+ copy_result <- file.copy(current_path, new_paths, overwrite = TRUE)
if (!all(copy_result)) {
stop("There was an issue while copying the files!")
}
+ - name: Prune Repository
+ shell: bash
+ env:
+ KEEP: ${{ github.event.inputs.keep || 14 }}
+ run: |
+ prune() {
+ # list files | retain $KEEP newest files | delete everything else
+ ls -t $1/arrow* | tail -n +$((KEEP + 1)) | xargs --no-run-if-empty
rm
+ }
+
+ # find leaf sub dirs
+ repo_dirs=$(find repo -type d -links 2)
+ # We want to retain $keep (14) versions of each pkg/lib so we call
+ # prune on each leaf dir and not on repo/.
+ for dir in ${repo_dirs[@]}; do
+ prune $dir
+ done
Review Comment:
Can we use `find repo/* -mtime +$KEEP -type f -print0 | xargs -0 rm -rf`
here?
##########
.github/workflows/r_nightly.yml:
##########
@@ -77,10 +89,47 @@ jobs:
echo "No files found. Stopping upload."
exit 1
fi
+ - name: Cache Repo
+ uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+ - name: Sync from Remote
Review Comment:
I think that we don't need this if we use `actions/cache` for `repo/`.
If we use `actions/cache` for `repo/`, `repo/` has the same contents of
https://nightlies.apache.org/arrow/r/ eventually.
##########
.github/workflows/r_nightly.yml:
##########
@@ -59,6 +65,12 @@ jobs:
- name: Install Archery
shell: bash
run: pip install -e arrow/dev/archery[all]
+ - uses: actions/cache@v3
+ with:
+ path: repo
+ key: r-nightly-${{ github.run_id }}
+ restore-keys: r-nightly-
+
Review Comment:
It seems this is duplicated.
##########
.github/workflows/r_nightly.yml:
##########
@@ -124,14 +192,32 @@ jobs:
tools::write_PACKAGES(dir, type = ifelse(on_win, "win.binary",
"mac.binary"), verbose = TRUE )
}
- name: Show repo contents
- run: ls -R repo
- - name: Upload Files
- uses: burnett01/rsync-deployments@5.2
- with:
- switches: -avzr
- path: repo/*
- remote_path: ${{ secrets.NIGHTLIES_RSYNC_PATH }}/arrow/r
- remote_host: ${{ secrets.NIGHTLIES_RSYNC_HOST }}
- remote_port: ${{ secrets.NIGHTLIES_RSYNC_PORT }}
- remote_user: ${{ secrets.NIGHTLIES_RSYNC_USER }}
- remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}
+ run: tree repo
+ - name: Sync to Remote
Review Comment:
Why do we implement this instead of using `burnett01/rsync-deployments@5.2`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
