This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new cf30da241571 [SPARK-47110][INFRA] Reenble AmmoniteTest tests in Maven
builds
cf30da241571 is described below
commit cf30da241571af80d120cfc08c281b43c40d2847
Author: Kousuke Saruta <[email protected]>
AuthorDate: Mon Sep 22 22:54:21 2025 -0700
[SPARK-47110][INFRA] Reenble AmmoniteTest tests in Maven builds
### What changes were proposed in this pull request?
This PR aims to reenable `AmmoniteTest` tests in Maven builds using
`script` command.
The usage of the command is different from macOS and Linux so this change
consider the difference.
### Why are the changes needed?
For better test coverage.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
I created a branch on my repo based on this change which runs Maven tests
for macOS and Ubuntu, and I confirmed both tests passed.
* [Test on
Ubuntu](https://github.com/sarutak/spark/actions/runs/17779850500)
* [Test on macOS](https://github.com/sarutak/spark/actions/runs/17779850568)
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #52369 from sarutak/reenble-ammonite-tests.
Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/maven_test.yml | 22 ++++++++++++----------
1 file changed, 12 insertions(+), 10 deletions(-)
diff --git a/.github/workflows/maven_test.yml b/.github/workflows/maven_test.yml
index 5b1d3b47858d..dfd61cc46389 100644
--- a/.github/workflows/maven_test.yml
+++ b/.github/workflows/maven_test.yml
@@ -78,19 +78,13 @@ jobs:
connector#kafka-0-10,connector#kafka-0-10-sql,connector#kafka-0-10-token-provider,connector#spark-ganglia-lgpl,connector#protobuf,connector#avro,connector#kinesis-asl
- >-
sql#api,sql#catalyst,resource-managers#yarn,resource-managers#kubernetes#core
+ - >-
+ connect
# Here, we split Hive and SQL tests into some of slow ones and the
rest of them.
included-tags: [ "" ]
excluded-tags: [ "" ]
comment: [ "" ]
include:
- # Connect tests
- - modules: connect
- java: ${{ inputs.java }}
- hadoop: ${{ inputs.hadoop }}
- hive: hive2.3
- # TODO(SPARK-47110): Reenble AmmoniteTest tests in Maven builds
- excluded-tags: org.apache.spark.tags.AmmoniteTest
- comment: ""
# Hive tests
- modules: sql#hive
java: ${{ inputs.java }}
@@ -183,10 +177,18 @@ jobs:
run: |
python3.11 -m pip install 'numpy>=1.22' pyarrow pandas scipy
unittest-xml-reporting 'grpcio==1.67.0' 'grpcio-status==1.67.0'
'protobuf==5.29.1'
python3.11 -m pip list
- # Run the tests.
+ # Run the tests using script command.
+ # BSD's script command doesn't support -c option, and the usage is
different from Linux's one.
+ # The kind of script command is tested by `script -qec true`.
- name: Run tests
env: ${{ fromJSON(inputs.envs) }}
+ shell: |
+ bash -c "if script -qec true 2>/dev/null; then script -qec bash\
{0}; else script -qe /dev/null bash {0}; fi"
run: |
+ # Fix for TTY related issues when launching the Ammonite REPL in
tests.
+ export TERM=vt100
+ # `set -e` to make the exit status as expected due to use script
command to run the commands
+ set -e
export MAVEN_OPTS="-Xss64m -Xmx4g -Xms4g
-XX:ReservedCodeCacheSize=128m -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
export MAVEN_CLI_OPTS="--no-transfer-progress"
export JAVA_VERSION=${{ matrix.java }}
@@ -204,7 +206,7 @@ jobs:
if [[ "$INCLUDED_TAGS" != "" ]]; then
./build/mvn $MAVEN_CLI_OPTS -pl "$TEST_MODULES" -Pyarn
-Pkubernetes -Pvolcano -Phive -Phive-thriftserver -Phadoop-cloud -Pjvm-profiler
-Pspark-ganglia-lgpl -Pkinesis-asl -Djava.version=${JAVA_VERSION/-ea}
-Dtest.include.tags="$INCLUDED_TAGS" test -fae
elif [[ "$MODULES_TO_TEST" == "connect" ]]; then
- ./build/mvn $MAVEN_CLI_OPTS -Dtest.exclude.tags="$EXCLUDED_TAGS"
-Djava.version=${JAVA_VERSION/-ea} -pl
sql/connect/client/jvm,sql/connect/common,sql/connect/server test -fae
+ ./build/mvn $MAVEN_CLI_OPTS -Djava.version=${JAVA_VERSION/-ea} -pl
sql/connect/client/jvm,sql/connect/common,sql/connect/server test -fae
elif [[ "$EXCLUDED_TAGS" != "" ]]; then
./build/mvn $MAVEN_CLI_OPTS -pl "$TEST_MODULES" -Pyarn
-Pkubernetes -Pvolcano -Phive -Phive-thriftserver -Phadoop-cloud -Pjvm-profiler
-Pspark-ganglia-lgpl -Pkinesis-asl -Djava.version=${JAVA_VERSION/-ea}
-Dtest.exclude.tags="$EXCLUDED_TAGS" test -fae
elif [[ "$MODULES_TO_TEST" == *"sql#hive-thriftserver"* ]]; then
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
