Re: [ANNOUNCE] Apache Spark 3.2.3 released

Thanks, Chao! 发件人: Maxim Gekk 日期: 2022年11月30日 星期三 19:40 收件人: Jungtaek Lim 抄送: Wenchen Fan , Chao Sun , dev , user 主题: Re: [ANNOUNCE] Apache Spark 3.2.3 released Thank you, Chao! On Wed, Nov 30, 2022 at 12:42 PM Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> wrote: Thanks Chao for

Re: [ANNOUNCE] Apache Spark 3.3.1 released

Thanks Yuming and all developers ~ Yang Jie 发件人: Maxim Gekk 日期: 2022年10月26日 星期三 15:19 收件人: Hyukjin Kwon 抄送: "L. C. Hsieh" , Dongjoon Hyun , Yuming Wang , dev , User 主题: Re: [ANNOUNCE] Apache Spark 3.3.1 released Congratulations everyone with the new release, and thanks to Yuming for his

Re: [Java 17] --add-exports required?

: 2022年6月23日 星期四 14:11 收件人: "Yang,Jie(INF)" 抄送: "user@spark.apache.org" 主题: Re: [Java 17] --add-exports required? Hi. I am running on macOS 12.4, using an ‘Adoptium’ JDK from https://adoptium.net/download<https://mailshield.baidu.com/check?q=U8F1V2tHFnSLZMX%2fpIYOpCo62

Re: [Java 17] --add-exports required?

Hi, Greg "--add-exports java.base/sun.nio.ch=ALL-UNNAMED " does not need to be added when SPARK-33772 is completed, so in order to answer your question, I need more details for testing: 1. Where can I download Java 17 (Temurin-17+35)? 2. What test commands do you use? Yang Jie 在 2022/6/23

Re: Spark Parquet write OOM

This is a DirectByteBuffer OOM，so plan 2 may not work, we can increase the capacity of DirectByteBuffer size by configuring `-XX:MaxDirectMemorySize` and this is a Java opts. However, we'd better check the length of memory to be allocated, because `-XX:MaxDirectMemorySize` and `-Xmx` should

Re: Log4J 2 Support

It may be more feasible to replace the current slf4j + log4j with log4j2-api, some projects that spark relies on may also use log4j at the code level, such as EventCounter and ContainerLogAppender in Hadoop, directly removing the dependency on log4j may lead to some code dependencies loss.