Raúl Cumplido created ARROW-18128:
-------------------------------------
Summary: [Java][CI] Java nightlies does not remove binaries
keeping the newest ones
Key: ARROW-18128
URL: https://issues.apache.org/jira/browse/ARROW-18128
Project: Apache Arrow
Issue Type: Bug
Components: Continuous Integration, Java
Reporter: Raúl Cumplido
Fix For: 11.0.0
After some investigation on why some of our jars did not have a correct
10.0.0-SNAPSHOT folder on the repository, i.e:
[https://nightlies.apache.org/arrow/java/org/apache/arrow/arrow-vector/]{color:#1d1c1d}
{color}
[https://nightlies.apache.org/arrow/java/org/apache/arrow/flight-core/]
It seems that when pruning the old artifacts we are not ordering based on newer
ones:
{code:java}
- name: Prune Repository
shell: bash
env:
KEEP: ${{ github.event.inputs.keep || 14 }}
run: |
for i in `ls -t repo/org/apache/arrow`; do
find repo/org/apache/arrow/$i -mindepth 1 -maxdepth 1 -type d
-print0 \
| xargs -0 ls -t -d \
| tail -n +$((KEEP + 1)) \
| xargs rm -rf
done {code}
that makes us delete based on the output from find and having things like:
{code:java}
[DIR] 2022-09-21/ 2022-09-21 15:43 -
[DIR] 2022-09-22/ 2022-09-22 20:53 -
[DIR] 2022-09-23/ 2022-09-23 15:14 -
[DIR] 2022-10-11/ 2022-10-11 15:18 -
[DIR] 2022-10-12/ 2022-10-12 18:04 -
[DIR] 2022-10-13/ 2022-10-13 20:35 -
[DIR] 2022-10-14/ 2022-10-14 17:28 -
[DIR] 2022-10-15/ 2022-10-15 14:10 -
[DIR] 2022-10-16/ 2022-10-16 14:13 -
[DIR] 2022-10-17/ 2022-10-17 14:21 -
[DIR] 2022-10-18/ 2022-10-18 16:24 -
[DIR] 2022-10-19/ 2022-10-19 14:31 -
[DIR] 2022-10-20/ 2022-10-20 17:09 - {code}
See artifacts for 21st-23rd September and then jumping to 11th October.
We should fix how we prune the older artifacts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
