[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r526417656 ## File path: .github/workflows/build_and_test.yml ## @@ -433,11 +407,32 @@ jobs: key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }} restore-keys: | scala-213-coursier- -- name: Install Java 11 +- name: Install Java 8 uses: actions/setup-java@v1 with: -java-version: 11 +java-version: 8 - name: Build with SBT run: | ./dev/change-scala-version.sh 2.13 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile + Review comment: Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r526037235 ## File path: .github/workflows/build_and_test.yml ## @@ -433,11 +407,32 @@ jobs: key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }} restore-keys: | scala-213-coursier- -- name: Install Java 11 +- name: Install Java 8 uses: actions/setup-java@v1 with: -java-version: 11 +java-version: 8 - name: Build with SBT run: | ./dev/change-scala-version.sh 2.13 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile + Review comment: @mridulm, oh but if there's any of them fails, it shows a clear X like ![Screen Shot 2020-11-18 at 9 07 02 PM](https://user-images.githubusercontent.com/6477701/99528700-077e5680-29e2-11eb-9769-f27b414657da.png) So it should be fine. I do believe reviewers check the X or test failures from Jenkins before merging it in :-). Actually, Java 11 and Scala 2.13 too. Jenkins PR builder does not run them as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r526037235 ## File path: .github/workflows/build_and_test.yml ## @@ -433,11 +407,32 @@ jobs: key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }} restore-keys: | scala-213-coursier- -- name: Install Java 11 +- name: Install Java 8 uses: actions/setup-java@v1 with: -java-version: 11 +java-version: 8 - name: Build with SBT run: | ./dev/change-scala-version.sh 2.13 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile + Review comment: @mridulm, oh but if there's any of them fails, it shows a clear X like ![Screen Shot 2020-11-18 at 9 07 02 PM](https://user-images.githubusercontent.com/6477701/99528700-077e5680-29e2-11eb-9769-f27b414657da.png) So it should be fine. I do believe reviewers check the X or test failures from Jenkins before merging it in :-). Actually, Java 11 and Scala 2.13 too. PR builder does not run them as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r526037235 ## File path: .github/workflows/build_and_test.yml ## @@ -433,11 +407,32 @@ jobs: key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }} restore-keys: | scala-213-coursier- -- name: Install Java 11 +- name: Install Java 8 uses: actions/setup-java@v1 with: -java-version: 11 +java-version: 8 - name: Build with SBT run: | ./dev/change-scala-version.sh 2.13 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile + Review comment: @mridulm, oh but if there's any of them fails, it shows a clear X like ![Screen Shot 2020-11-18 at 9 07 02 PM](https://user-images.githubusercontent.com/6477701/99528700-077e5680-29e2-11eb-9769-f27b414657da.png) So it should be fine. I do believe reviewers checks the X or test failures from Jenkins before merging it in :-). Actually, Java 11 and Scala 2.13 doo. PR builder does not run them as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524973278 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN" export MAVEN_CLI_OPTS="--no-transfer-progress" -mkdir -p ~/.m2 Review comment: Thankfully seems now working fine https://github.com/apache/spark/pull/30394/checks?check_run_id=1410917199. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524947360 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN" export MAVEN_CLI_OPTS="--no-transfer-progress" -mkdir -p ~/.m2 Review comment: Sure, I ran the build without cache at https://github.com/apache/spark/pull/30394. If it passes, I'll keep it. If it fails, I will recover it back with a comment :-). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524945013 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524943842 ## File path: .github/workflows/build_and_test.yml ## @@ -416,12 +390,11 @@ jobs: run: | export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN" export MAVEN_CLI_OPTS="--no-transfer-progress" -mkdir -p ~/.m2 Review comment: @dongjoon-hyun, I removed this though. Maven seems creating it automatically (whether the cache exists or not). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524942149 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: Oh, sure. Then it makes sense to keep `install`. +1 for keeping it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524918338 ## File path: .github/workflows/build_and_test.yml ## @@ -421,7 +395,7 @@ jobs: rm -rf ~/.m2/repository/org/apache/spark Review comment: That works to me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524883064 ## File path: .github/workflows/build_and_test.yml ## @@ -433,11 +407,32 @@ jobs: key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }} restore-keys: | scala-213-coursier- -- name: Install Java 11 +- name: Install Java 8 uses: actions/setup-java@v1 with: -java-version: 11 +java-version: 8 - name: Build with SBT run: | ./dev/change-scala-version.sh 2.13 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile + Review comment: @mridulm, would you mind moving it below here? It will show an X obviously if one of builds fails anyway. It seems like it can cause some conflicts easily when we backport something to other branches for GitHub Actions. Currently, we're running the builds in all active branches, and porting back things. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml
HyukjinKwon commented on a change in pull request #30391: URL: https://github.com/apache/spark/pull/30391#discussion_r524881430 ## File path: .github/workflows/build_and_test.yml ## @@ -433,11 +407,32 @@ jobs: key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }} restore-keys: | scala-213-coursier- -- name: Install Java 11 +- name: Install Java 8 uses: actions/setup-java@v1 with: -java-version: 11 +java-version: 8 - name: Build with SBT run: | ./dev/change-scala-version.sh 2.13 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile + Review comment: @dongjoon-hyun, I just move it back below. Seems like it affects when things are ported back (Hadoop 2 build is only in master branch). I think it will still obviously show the red X one of builds fails anyway .. so I think it might be fine to move below here together with Java 11 and Scala 2.13 builds. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org