zhenlineo opened a new pull request, #40274: URL: https://github.com/apache/spark/pull/40274
### What changes were proposed in this pull request? Make use of the new spark-connect script to make the Scala client test to not directly depends on any other modules. The dependency is still there but hidden by the spark-connect script. When calling the script to start the server, the script performs a `build/sbt package` to ensure all the jars are build and can be found in the correct path. After the change, we can use the following commands to run the Scala client tests: ``` build/mvn clean build/mvn compile -pl connector/connect/client/jvm build/mvn test -pl connector/connect/client/jvm ``` ``` build/sbt clean build/sbt "testOnly org.apache.spark.sql.ClientE2ETestSuite" ``` ``` build/sbt clean build/sbt "connect-client-jvm/test" ``` **Scala 2.13** ``` build/mvn clean build/mvn -Pscala-2.13 compile -pl connector/connect/client/jvm -am -DskipTests build/mvn -Pscala-2.13 test -pl connector/connect/client/jvm // These commands failed with errors to find catalyst ArrowWriter. The error seems unrelated to this change. ``` ``` build/sbt clean build/sbt "testOnly org.apache.spark.sql.ClientE2ETestSuite" -Pscala-2.13 ``` ``` build/sbt clean build/sbt "connect-client-jvm/test" -Pscala-2.13 ``` After the change, the waiting time to run the E2ESuite is ~3min for a clean build. Then ~1min for subsequent runs. The test is slower only because we moved the build time from many commands to this single command. There is no limitations to add more tests as the delay is only caused by the shared server start time. Once the server is started, the tests run fast. ### Why are the changes needed? A single command for maven users to mvn clean install to run all tests and build. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Build, Manual tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
