[spark] branch branch-3.4 updated: [SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS` to override JVM memory setting

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.4 by this push:
     new d47974c5f12 [SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS` 
to override JVM memory setting
d47974c5f12 is described below

commit d47974c5f12322d576d9821b6c66d7db675938c2
Author: Dongjoon Hyun <dongj...@apache.org>
AuthorDate: Thu Jan 26 16:39:17 2023 -0800

    [SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS` to override JVM 
memory setting
    
    ### What changes were proposed in this pull request?
    
    This PR aims to fix a bug which `build/sbt` doesn't allow JVM memory 
setting via `SBT_OPTS`.
    
    ### Why are the changes needed?
    
    `SBT_OPTS` is supposed to be used in this way in the community.
    
https://github.com/apache/spark/blob/e30bb538e480940b1963eb14c3267662912d8584/appveyor.yml#L54
    
    However, `SBT_OPTS` memory setting like the following is ignored because ` 
-Xms4096m -Xmx4096m -XX:ReservedCodeCacheSize=512m` is injected by default 
after `SBT_OPTS`. We should switch the order.
    ```
    $ SBT_OPTS="-Xmx6g" build/sbt package
    ```
    
https://github.com/apache/spark/blob/e30bb538e480940b1963eb14c3267662912d8584/build/sbt-launch-lib.bash#L124
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. This is a dev-only change.
    
    ### How was this patch tested?
    
    Manually run the following.
    ```
    $ SBT_OPTS="-Xmx6g" build/sbt package
    ```
    
    While running the above command, check the JVM options.
    ```
    $ ps aux | grep java
    dongjoon         36683 434.3  3.1 418465456 1031888 s001  R+    1:11PM   
0:19.86 /Users/dongjoon/.jenv/versions/temurin17/bin/java -Xms4096m -Xmx4096m 
-XX:ReservedCodeCacheSize=512m -Xmx6g -jar build/sbt-launch-1.8.2.jar package
    ```
    
    Closes #39758 from dongjoon-hyun/SPARK-42201.
    
    Authored-by: Dongjoon Hyun <dongj...@apache.org>
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
    (cherry picked from commit 66ec1eb630a4682f5ad2ed2ee989ffcce9031608)
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
 build/sbt-launch-lib.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/build/sbt-launch-lib.bash b/build/sbt-launch-lib.bash
index 8fb6672bddc..01ba6b929f9 100755
--- a/build/sbt-launch-lib.bash
+++ b/build/sbt-launch-lib.bash
@@ -183,8 +183,8 @@ run() {
 
   # run sbt
   execRunner "$java_cmd" \
-    ${SBT_OPTS:-$default_sbt_opts} \
     $(get_mem_opts $sbt_mem) \
+    ${SBT_OPTS:-$default_sbt_opts} \
     ${java_opts} \
     ${java_args[@]} \
     -jar "$sbt_jar" \


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org