[spark] branch master updated: [SPARK-44688][INFRA] Add a file existence check before executing `free_disk_space` and `free_disk_space_container`

[yangjie01]

This is an automated email from the ASF dual-hosted git repository.

yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new d264ee37f31 [SPARK-44688][INFRA] Add a file existence check before 
executing `free_disk_space` and `free_disk_space_container`
d264ee37f31 is described below

commit d264ee37f316b32bf37fff770b72b5841b84f7cc
Author: yangjie01 <yangji...@baidu.com>
AuthorDate: Sun Aug 6 17:41:11 2023 +0800

    [SPARK-44688][INFRA] Add a file existence check before executing 
`free_disk_space` and `free_disk_space_container`
    
    ### What changes were proposed in this pull request?
    This pr add a file existence check before executing `dev/free_disk_space` 
and `dev/free_disk_space_container`
    
    ### Why are the changes needed?
    We added `free_disk_space` and `free_disk_space_container` to clean up the 
disk, but because the daily tests of other branches and the master branch share 
the yml file, we should check if the file exists before execution, otherwise it 
will affect the daily tests of other branches.
    
    - branch-3.5: https://github.com/apache/spark/actions/runs/5761479443
    - branch-3.4: https://github.com/apache/spark/actions/runs/5760423900
    - branch-3.3: https://github.com/apache/spark/actions/runs/5759384052
    
    <img width="1073" alt="image" 
src="https://github.com/apache/spark/assets/1475305/6e46b34b-645a-4da5-b9c3-8a89bfacabcb";>
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    - Pass GitHub Action
    
    Closes #42359 from LuciferYang/test-free_disk_space-exist.
    
    Authored-by: yangjie01 <yangji...@baidu.com>
    Signed-off-by: yangjie01 <yangji...@baidu.com>
---
 .github/workflows/build_and_test.yml | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/build_and_test.yml 
b/.github/workflows/build_and_test.yml
index 04585481a9c..cd68c0904d9 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -241,7 +241,10 @@ jobs:
         restore-keys: |
           ${{ matrix.java }}-${{ matrix.hadoop }}-coursier-
     - name: Free up disk space
-      run: ./dev/free_disk_space
+      run: |
+        if [ -f ./dev/free_disk_space ]; then
+          ./dev/free_disk_space
+        fi
     - name: Install Java ${{ matrix.java }}
       uses: actions/setup-java@v3
       with:
@@ -419,7 +422,9 @@ jobs:
           # uninstall libraries dedicated for ML testing
           python3.9 -m pip uninstall -y torch torchvision torcheval torchtnt 
tensorboard mlflow
         fi
-        ./dev/free_disk_space_container
+        if [ -f ./dev/free_disk_space_container ]; then
+          ./dev/free_disk_space_container
+        fi
     - name: Install Java ${{ matrix.java }}
       uses: actions/setup-java@v3
       with:
@@ -519,7 +524,10 @@ jobs:
         restore-keys: |
           sparkr-coursier-
     - name: Free up disk space
-      run: ./dev/free_disk_space_container
+      run: |
+        if [ -f ./dev/free_disk_space_container ]; then
+          ./dev/free_disk_space_container
+        fi
     - name: Install Java ${{ inputs.java }}
       uses: actions/setup-java@v3
       with:
@@ -629,7 +637,10 @@ jobs:
         restore-keys: |
           docs-maven-
     - name: Free up disk space
-      run: ./dev/free_disk_space_container
+      run: |
+        if [ -f ./dev/free_disk_space_container ]; then
+          ./dev/free_disk_space_container
+        fi
     - name: Install Java 8
       uses: actions/setup-java@v3
       with:


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org