Re: [DISCUSS] SPIP: Spark Connect - A client and server interface for Apache Spark.

how would scala udfs be supported in this? On Fri, Jun 3, 2022 at 1:52 PM Martin Grund wrote: > Hi Everyone, > > We would like to start a discussion on the "Spark Connect" proposal. > Please find the links below: > > *JIRA* - https://issues.apache.org/jira/browse/SPARK-39375 > *SPIP Document* -

Re: [VOTE] Release Spark 3.3.0 (RC4)

It's fixed at https://github.com/apache/spark/pull/36762. On Fri, Jun 3, 2022 at 2:20 PM Sean Owen wrote: > > Ah yeah, I think it's this change from 15 hrs ago. That needs to be .toSeq: > >

Re: [VOTE] Release Spark 3.3.0 (RC4)

Ah yeah, I think it's this change from 15 hrs ago. That needs to be .toSeq: https://github.com/apache/spark/commit/4a0f0ff6c22b85cb0fc1eef842da8dbe4c90543a#diff-01813c3e2e933ed573e4a93750107f004a86e587330cba5e91b5052fa6ade2a5R146 On Fri, Jun 3, 2022 at 4:13 PM Sean Owen wrote: > In Scala 2.13,

Re: [VOTE] Release Spark 3.3.0 (RC4)

In Scala 2.13, I'm getting errors like this: analyzer should replace current_timestamp with literals *** FAILED *** java.lang.ClassCastException: class scala.collection.mutable.ArrayBuffer cannot be cast to class scala.collection.immutable.Seq (scala.collection.mutable.ArrayBuffer and

Re: The draft of the Spark 3.3.0 release notes

- Support lambda `column` parameter of `DataFrame.rename`(SPARK-38763 ) This did work before 18.jan.2022, see JIRA post for more information. I think we can remove this one from the list. fre. 3. jun. 2022 kl. 20:42 skrev Dongjoon

Re: The draft of the Spark 3.3.0 release notes

You are right. After SPARK-36837, we tried to ship Apache Spark 3.3.0 with Apache Kafka 3.1.1 via the following PR. https://github.com/apache/spark/pull/36135 [WIP][SPARK-38850][BUILD] Upgrade Kafka to 3.1.1 However, the final decision was to revert it from `branch-3.3` and move directly to

[DISCUSS] SPIP: Spark Connect - A client and server interface for Apache Spark.

Hi Everyone, We would like to start a discussion on the "Spark Connect" proposal. Please find the links below: *JIRA* - https://issues.apache.org/jira/browse/SPARK-39375 *SPIP Document* - https://docs.google.com/document/d/1Mnl6jmGszixLW4KcJU5j9IgpG9-UabS0dcM6PM2XGDc/edit#heading=h.wmsrrfealhrj

Re: The draft of the Spark 3.3.0 release notes

i thought SPARK-36837 didnt make it in? i see it in notes On Fri, Jun 3, 2022 at 4:31 AM Maxim Gekk wrote: > Hi All, > > I am preparing the release notes of Spark 3.3.0. Here is a draft document: > >

Re: The draft of the Spark 3.3.0 release notes

It's more of a missing feature than a bug, but I guess it's all subjective. Adam On Fri, Jun 3, 2022 at 10:24 AM Maxim Gekk wrote: > Hi Adam, > > The release notes focus on new features and user-facing improvements > mostly. SPARK-37618 seems

[VOTE] Release Spark 3.3.0 (RC4)

Please vote on releasing the following candidate as Apache Spark version 3.3.0. The vote is open until 11:59pm Pacific time June 7th and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.3.0 [ ] -1 Do not release this package

Re: The draft of the Spark 3.3.0 release notes

Hi Adam, The release notes focus on new features and user-facing improvements mostly. SPARK-37618 seems like a bug fix, that's why I didn't put it in the doc. Maxim Gekk Software Engineer Databricks, Inc. On Fri, Jun 3, 2022 at 2:20 PM Adam

Re: The draft of the Spark 3.3.0 release notes

I don't think I see https://issues.apache.org/jira/browse/SPARK-37618 which got merged post branch cut. Adam On Fri, Jun 3, 2022 at 4:25 AM Maxim Gekk wrote: > Hi All, > > I am preparing the release notes of Spark 3.3.0. Here is a draft document: > >

The draft of the Spark 3.3.0 release notes

Hi All, I am preparing the release notes of Spark 3.3.0. Here is a draft document: https://docs.google.com/document/d/1gGySrLGvIK8bajKdGjTI_mDqk0-YPvHmPN64YjoWfOQ/edit?usp=sharing Please take a look and let me know if I missed any major changes or something. Maxim Gekk Software Engineer