[spark] branch master updated: [SPARK-45651][BUILD] Log memory usage of publish snapshot workflow

[gurwls223]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new e57a16d7af1 [SPARK-45651][BUILD] Log memory usage of publish snapshot 
workflow
e57a16d7af1 is described below

commit e57a16d7af14d1ce0c14d01dd220c63acb98517d
Author: Enrico Minack <git...@enrico.minack.dev>
AuthorDate: Thu Oct 26 10:39:32 2023 +0900

    [SPARK-45651][BUILD] Log memory usage of publish snapshot workflow
    
    ### What changes were proposed in this pull request?
    This logs memory consumption while publishing snapshots. This is to 
investigate whether the suspected high memory usage is the root cause of 
`publish_snapshots` failures for master.
    
    Merging this after #43512 allows to run this manually.
    
    ### Why are the changes needed?
    The working assumption is that high memory usage is the root cause. This 
logging should provide proof or disproof for this assumption. This can be 
reverted once more is known or SPARK-45651 is fixed.
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    Locally
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No
    
    Closes #43513 from EnricoMi/publish-snapshot-log-memory.
    
    Authored-by: Enrico Minack <git...@enrico.minack.dev>
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
 .github/workflows/publish_snapshot.yml | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/publish_snapshot.yml 
b/.github/workflows/publish_snapshot.yml
index 6d55f1afed0..3354ab88a39 100644
--- a/.github/workflows/publish_snapshot.yml
+++ b/.github/workflows/publish_snapshot.yml
@@ -70,4 +70,12 @@ jobs:
         GPG_KEY: "not_used"
         GPG_PASSPHRASE: "not_used"
         GIT_REF: ${{ matrix.branch }}
-      run: ./dev/create-release/release-build.sh publish-snapshot
+      run: |
+        while true
+        do
+          date
+          top -b -n 1 -i
+          sleep 1
+          echo
+        done | sed "s/^/mem: /" &
+        ./dev/create-release/release-build.sh publish-snapshot


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org