[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-18 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r526417656



##
File path: .github/workflows/build_and_test.yml
##
@@ -433,11 +407,32 @@ jobs:
 key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') 
}}
 restore-keys: |
   scala-213-coursier-
-- name: Install Java 11
+- name: Install Java 8
   uses: actions/setup-java@v1
   with:
-java-version: 11
+java-version: 8
 - name: Build with SBT
   run: |
 ./dev/change-scala-version.sh 2.13
 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver 
-Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile
+

Review comment:
   Thanks!





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-18 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r526037235



##
File path: .github/workflows/build_and_test.yml
##
@@ -433,11 +407,32 @@ jobs:
 key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') 
}}
 restore-keys: |
   scala-213-coursier-
-- name: Install Java 11
+- name: Install Java 8
   uses: actions/setup-java@v1
   with:
-java-version: 11
+java-version: 8
 - name: Build with SBT
   run: |
 ./dev/change-scala-version.sh 2.13
 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver 
-Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile
+

Review comment:
   @mridulm, oh but if there's any of them fails, it shows a clear X like
   
   ![Screen Shot 2020-11-18 at 9 07 02 
PM](https://user-images.githubusercontent.com/6477701/99528700-077e5680-29e2-11eb-9769-f27b414657da.png)
   
   So it should be fine. I do believe reviewers check the X or test failures 
from Jenkins before merging it in :-). Actually, Java 11 and Scala 2.13 too. 
Jenkins PR builder does not run them as well.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-18 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r526037235



##
File path: .github/workflows/build_and_test.yml
##
@@ -433,11 +407,32 @@ jobs:
 key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') 
}}
 restore-keys: |
   scala-213-coursier-
-- name: Install Java 11
+- name: Install Java 8
   uses: actions/setup-java@v1
   with:
-java-version: 11
+java-version: 8
 - name: Build with SBT
   run: |
 ./dev/change-scala-version.sh 2.13
 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver 
-Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile
+

Review comment:
   @mridulm, oh but if there's any of them fails, it shows a clear X like
   
   ![Screen Shot 2020-11-18 at 9 07 02 
PM](https://user-images.githubusercontent.com/6477701/99528700-077e5680-29e2-11eb-9769-f27b414657da.png)
   
   So it should be fine. I do believe reviewers check the X or test failures 
from Jenkins before merging it in :-). Actually, Java 11 and Scala 2.13 too. PR 
builder does not run them as well.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-18 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r526037235



##
File path: .github/workflows/build_and_test.yml
##
@@ -433,11 +407,32 @@ jobs:
 key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') 
}}
 restore-keys: |
   scala-213-coursier-
-- name: Install Java 11
+- name: Install Java 8
   uses: actions/setup-java@v1
   with:
-java-version: 11
+java-version: 8
 - name: Build with SBT
   run: |
 ./dev/change-scala-version.sh 2.13
 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver 
-Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile
+

Review comment:
   @mridulm, oh but if there's any of them fails, it shows a clear X like
   
   ![Screen Shot 2020-11-18 at 9 07 02 
PM](https://user-images.githubusercontent.com/6477701/99528700-077e5680-29e2-11eb-9769-f27b414657da.png)
   
   So it should be fine. I do believe reviewers checks the X or test failures 
from Jenkins before merging it in :-). Actually, Java 11 and Scala 2.13 doo. PR 
builder does not run them as well.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-17 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524973278



##
File path: .github/workflows/build_and_test.yml
##
@@ -416,12 +390,11 @@ jobs:
   run: |
 export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g 
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
 export MAVEN_CLI_OPTS="--no-transfer-progress"
-mkdir -p ~/.m2

Review comment:
   Thankfully seems now working fine 
https://github.com/apache/spark/pull/30394/checks?check_run_id=1410917199.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-16 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524947360



##
File path: .github/workflows/build_and_test.yml
##
@@ -416,12 +390,11 @@ jobs:
   run: |
 export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g 
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
 export MAVEN_CLI_OPTS="--no-transfer-progress"
-mkdir -p ~/.m2

Review comment:
   Sure, I ran the build without cache at 
https://github.com/apache/spark/pull/30394. If it passes, I'll keep it. If it 
fails, I will recover it back with a comment :-).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-16 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524945013



##
File path: .github/workflows/build_and_test.yml
##
@@ -421,7 +395,7 @@ jobs:
 rm -rf ~/.m2/repository/org/apache/spark

Review comment:
   Done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-16 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524943842



##
File path: .github/workflows/build_and_test.yml
##
@@ -416,12 +390,11 @@ jobs:
   run: |
 export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g 
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
 export MAVEN_CLI_OPTS="--no-transfer-progress"
-mkdir -p ~/.m2

Review comment:
   @dongjoon-hyun, I removed this though. Maven seems creating it 
automatically (whether the cache exists or not).





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-16 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524942149



##
File path: .github/workflows/build_and_test.yml
##
@@ -421,7 +395,7 @@ jobs:
 rm -rf ~/.m2/repository/org/apache/spark

Review comment:
   Oh, sure. Then it makes sense to keep `install`. +1 for keeping it.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-16 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524918338



##
File path: .github/workflows/build_and_test.yml
##
@@ -421,7 +395,7 @@ jobs:
 rm -rf ~/.m2/repository/org/apache/spark

Review comment:
   That works to me.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-16 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524883064



##
File path: .github/workflows/build_and_test.yml
##
@@ -433,11 +407,32 @@ jobs:
 key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') 
}}
 restore-keys: |
   scala-213-coursier-
-- name: Install Java 11
+- name: Install Java 8
   uses: actions/setup-java@v1
   with:
-java-version: 11
+java-version: 8
 - name: Build with SBT
   run: |
 ./dev/change-scala-version.sh 2.13
 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver 
-Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile
+

Review comment:
   @mridulm, would you mind moving it below here? It will show an X 
obviously if one of builds fails anyway. It seems like it can cause some 
conflicts easily when we backport something to other branches for GitHub 
Actions. Currently, we're running the builds in all active branches, and 
porting back things.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30391: [SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure GitHub Actions yaml

2020-11-16 Thread GitBox 


HyukjinKwon commented on a change in pull request #30391:
URL: https://github.com/apache/spark/pull/30391#discussion_r524881430



##
File path: .github/workflows/build_and_test.yml
##
@@ -433,11 +407,32 @@ jobs:
 key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') 
}}
 restore-keys: |
   scala-213-coursier-
-- name: Install Java 11
+- name: Install Java 8
   uses: actions/setup-java@v1
   with:
-java-version: 11
+java-version: 8
 - name: Build with SBT
   run: |
 ./dev/change-scala-version.sh 2.13
 ./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver 
-Phadoop-cloud -Pkinesis-asl -Djava.version=11 -Pscala-2.13 compile test:compile
+

Review comment:
   @dongjoon-hyun, I just move it back below. Seems like it affects when 
things are ported back (Hadoop 2 build is only in master branch). I think it 
will still obviously show the red X one of builds fails anyway .. so I think it 
might be fine to move below here together with Java 11 and Scala 2.13 builds.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org