EnricoMi commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1782614836
> We should fix
https://github.com/apache/spark/actions/workflows/publish_snapshot.yml
The issue has been fixed by #43538, thanks!
--
This is an automated message from the Apach
HyukjinKwon commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1761097309
Made a quick PR that potentially can fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
HyukjinKwon commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1761073296
We should fix
https://github.com/apache/spark/actions/workflows/publish_snapshot.yml
--
This is an automated message from the Apache Git Service.
To respond to the message, please l
LuciferYang commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1761069596
>
https://github.com/apache/spark/blob/0257b77528a3a0d0ba08df5363470e4bc5928b06/build/mvn#L39
>
> By default, the peak memory usage for a full compile with Maven and Java
17 is
LuciferYang commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1761064829
https://github.com/apache/spark/blob/0257b77528a3a0d0ba08df5363470e4bc5928b06/build/mvn#L39
By default, the peak memory usage for a full compile with Maven and Java 17
is about
EnricoMi commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1760981447
Looks like there is a GitHub workflow that publishes snapshots, and it is
failing since Sep, 14th, producing inconsistent snapshots:
https://github.com/apache/spark/actions/workflows/pub
cloud-fan commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1760962151
cc @dongjoon-hyun as well. I'm also curious about how these processes are
maintained. Do we have a bot to release the snapshot release?
--
This is an automated message from the Apache
EnricoMi commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1760908921
This is it, the spark-sql snapshot is outdated. I thought `sbt publishLocal`
in Spark master would have fixed that for me.
Who is best to reach out to fix the snapshots? cc @Hyukji
Hisoka-X commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1760655511
I believe the reason is code not match, the snapshot jar from
https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-sql_2.13/4.0.0-SNAPSHOT/
is be updated on 9/13
EnricoMi commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1759680505
This is my pom.xml, I can't see where I'd be introducing different spark
jars in build (compile) and testing:
```pom
http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.or
Hisoka-X commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1759593737
This will happend when your build depend code commit not same with test
depend code commit. Could you check is your build depend jar same as test
depend jar (special on `xmlExpressions.s
EnricoMi commented on PR #43102:
URL: https://github.com/apache/spark/pull/43102#issuecomment-1759541718
This is strange, calling into any SQL function in
org.apache.spark.sql.functions fails for latest 4.0.0-SNAPSHOT
```
sum($"id")
java.lang.NoClassDefFoundError:
org/apach
12 matches
Mail list logo
