Hi Matei, 3.2 will also include support for pushed based shuffle (spip SPARK-30602).
Regards, Mridul On Mon, Aug 9, 2021 at 9:26 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > > Are you referring to what version of Koala project? 1.8.1? > > Yes, the latest version 1.8.1. > > 2021년 8월 10일 (화) 오전 11:07, Igor Costa <igorco...@gmail.com>님이 작성: > >> Hi Matei, nice update >> >> >> Just one question, when you mention “ We are working on Spark 3.2.0 as >> our next release, with a release candidate likely to come soon. Spark 3.2 >> includes a new Pandas API for Apache Spark based on the Koalas project” >> >> >> Are you referring to what version of Koala project? 1.8.1? >> >> >> >> Cheers >> Igor >> >> On Tue, 10 Aug 2021 at 13:31, Matei Zaharia <matei.zaha...@gmail.com> >> wrote: >> >>> It’s time for our quarterly report to the ASF board, which we need to >>> send out this Wednesday. I wrote the draft below based on community >>> activity — let me know if you’d like to add or change anything: >>> >>> ====================================== >>> >>> Description: >>> >>> Apache Spark is a fast and general engine for large-scale data >>> processing. It offers high-level APIs in Java, Scala, Python, R and SQL as >>> well as a rich set of libraries including stream processing, machine >>> learning, and graph analytics. >>> >>> Issues for the board: >>> >>> - None >>> >>> Project status: >>> >>> - We made a number of maintenance releases in the past three months. We >>> released Apache Spark 3.1.2 and 3.0.3 in June as maintenance releases for >>> the 3.x branches. We also released Apache Spark 2.4.8 on May 17 as a bug >>> fix release for the Spark 2.x line. This may be the last release on 2.x >>> unless major new bugs are found. >>> >>> - We added three PMC members: Liang-Chi Hsieh, Kousuke Saruta and >>> Takeshi Yamamuro. >>> >>> - We are working on Spark 3.2.0 as our next release, with a release >>> candidate likely to come soon. Spark 3.2 includes a new Pandas API for >>> Apache Spark based on the Koalas project, a RocksDB state store for >>> Structured Streaming, native support for session windows, error message >>> standardization, and significant improvements to Spark SQL, such as the use >>> of adaptive query execution by default. >>> >>> Trademarks: >>> >>> - No changes since the last report. >>> >>> Latest releases: >>> >>> - Spark 3.1.2 was released on June 23rd, 2021. >>> - Spark 3.0.3 was released on June 1st, 2021. >>> - Spark 2.4.8 was released on May 17th, 2021. >>> >>> Committers and PMC: >>> >>> - The latest committers were added on March 11th, 2021 (Atilla Zsolt >>> Piros, Gabor Somogyi, Kent Yao, Maciej Szymkiewicz, Max Gekk, and Yi Wu). >>> - The latest PMC member was added on June 20th, 2021 (Kousuke Saruta). >>> >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >>> >>> -- >> Sent from Gmail Mobile >> >
