pan3793 commented on code in PR #4875:
URL: https://github.com/apache/zeppelin/pull/4875#discussion_r1804434188
##########
.github/workflows/core.yml:
##########
@@ -401,68 +391,16 @@ jobs:
rm -rf spark/interpreter/metastore_db
./mvnw verify -pl spark-submit,spark/interpreter -am
-Dtest=org/apache/zeppelin/spark/* -Pspark-3.4 -Pspark-scala-2.13 -Pintegration
-DfailIfNoTests=false ${MAVEN_ARGS}
- name: run spark-3.5 tests with scala-2.13 and python-${{ matrix.python
}}
- if: matrix.python >= '3.8'
run: |
rm -rf spark/interpreter/metastore_db
./mvnw verify -pl spark-submit,spark/interpreter -am
-Dtest=org/apache/zeppelin/spark/* -Pspark-3.5 -Pspark-scala-2.13 -Pintegration
-DfailIfNoTests=false ${MAVEN_ARGS}
- # The version combination is based on the facts:
- # 1. official Livy 0.8 binary tarball is built against Spark 2.4
- # 2. official Spark 2.4 binary tarball is built against Scala 2.11
- # 3. Spark 2.4 support Python 2.7, 3.4 to 3.7
- livy-0-8-with-spark-2-4-under-python37:
- runs-on: ubuntu-20.04
- steps:
- - name: Checkout
- uses: actions/checkout@v4
- - name: Tune Runner VM
- uses: ./.github/actions/tune-runner-vm
- - name: Set up JDK 8
- uses: actions/setup-java@v4
- with:
- distribution: 'temurin'
- java-version: 8
- - name: Cache local Maven repository
- uses: actions/cache@v4
- with:
- path: |
- ~/.m2/repository
- !~/.m2/repository/org/apache/zeppelin/
- ~/.spark-dist
- ~/.cache
- key: ${{ runner.os }}-zeppelin-${{ hashFiles('**/pom.xml') }}
- restore-keys: |
- ${{ runner.os }}-zeppelin-
- - name: install environment
- run: |
- ./mvnw install -DskipTests -pl livy -am ${MAVEN_ARGS}
- ./testing/downloadSpark.sh "2.4.8" "2.7"
- ./testing/downloadLivy.sh "0.8.0-incubating" "2.11"
Review Comment:
Livy 0.8 also provides 2.12 binary, which should work with Spark 3 and may
support Java 11, though I haven't tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
