Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-19 Thread via GitHub
LuciferYang commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1862302511 I think we should make https://github.com/apache/spark/pull/44343 as TODO of this one, and explicitly add TODO in the code. On the other hand, please update the PR title and PR descri

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
panbingkun commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1430999273 ## .github/workflows/build_maven.yml: ## @@ -30,3 +30,8 @@ jobs: name: Run uses: ./.github/workflows/maven_test.yml if: github.repository == 'apache/s

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
LuciferYang commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1430997017 ## .github/workflows/build_maven.yml: ## @@ -30,3 +30,8 @@ jobs: name: Run uses: ./.github/workflows/maven_test.yml if: github.repository == 'apache/

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
LuciferYang commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1430986535 ## .github/workflows/maven_test.yml: ## @@ -115,6 +115,7 @@ jobs: HIVE_PROFILE: ${{ matrix.hive }} SPARK_LOCAL_IP: localhost GITHUB_PREV_SHA: $

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
LuciferYang commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1430984227 ## .github/workflows/maven_test.yml: ## @@ -40,7 +40,7 @@ on: description: Additional environment variables to set when running the tests. Should be in JS

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
LuciferYang commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1430984227 ## .github/workflows/maven_test.yml: ## @@ -40,7 +40,7 @@ on: description: Additional environment variables to set when running the tests. Should be in JS

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
LuciferYang commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1430982220 ## .github/workflows/maven_test.yml: ## @@ -40,7 +40,7 @@ on: description: Additional environment variables to set when running the tests. Should be in JS

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
panbingkun commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1862192217 > How about just add a filter at > > https://github.com/apache/spark/blob/8cd466136528e479514a1d8dfe8ecaba9d0f8cce/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalC

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-18 Thread via GitHub
LuciferYang commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1862067513 How about just add a filter at https://github.com/apache/spark/blob/8cd466136528e479514a1d8dfe8ecaba9d0f8cce/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatal

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-14 Thread via GitHub
panbingkun commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1857169015 In this PR, some version tests have been skipped through environment variables. After obtaining "Approve", I will remove some logic added for testing and observation. -- This is

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-14 Thread via GitHub
panbingkun commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1427454717 ## sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala: ## @@ -66,6 +66,13 @@ class HiveExternalCatalogVersionsSuite extends Sp

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-14 Thread via GitHub
panbingkun commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1427424297 ## sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala: ## @@ -212,45 +219,49 @@ class HiveExternalCatalogVersionsSuite extends

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-14 Thread via GitHub
panbingkun commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1855662944 Based on the above discussion, 1. For the testing of already released files, we will skip it. 2. In response to the issue with `the master`, we choose to use a mechanism of `r

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-12 Thread via GitHub
panbingkun commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1853325778 I don't think a four part approach may necessarily solve the above problem, because in our testing, we found another jar: `org.slf4j#slf4j-api#1.7.30")` with a similar problem that doe

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-11 Thread via GitHub
LuciferYang commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1849624745 https://github.com/apache/spark/pull/44208#issuecomment-1849595508 also cc @pan3793 @yaooqinn Do you have any suggestions on this issue? -- This is an automated message from t

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-11 Thread via GitHub
LuciferYang commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1849595508 In other words, if we run this test case in a completely clean environment, ``` mv ~/.m2 ~/m2 mv ~/.ivy2 ~/ivy2 build/mvn clean install -DskipTests -Phive build/mvn

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-11 Thread via GitHub
LuciferYang commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1849587079 Wait, I found that if I delete the local `.m2` and `.ivy2` folders, then I can reproduce this test failure locally. -- This is an automated message from the Apache Git Servi

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-10 Thread via GitHub
panbingkun commented on code in PR #44208: URL: https://github.com/apache/spark/pull/44208#discussion_r1422033773 ## .github/workflows/build_and_test.yml: ## @@ -,3 +,9 @@ jobs: cd ui-test npm install --save-dev node --experimental-vm-mod

Re: [PR] [SPARK-46302][BUILD] Fix maven daily testing [spark]

2023-12-10 Thread via GitHub
panbingkun commented on PR #44208: URL: https://github.com/apache/spark/pull/44208#issuecomment-1849472640 Friendly ping @HyukjinKwon @zhengruifeng @LuciferYang I have discussed with @LuciferYang at local , and although this approach seems a bit strange at the moment, I have not yet