Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Thomas graves 
+1

Tom

On Mon, Jun 19, 2023 at 9:41 PM Dongjoon Hyun  wrote:

> Please vote on releasing the following candidate as Apache Spark version
> 3.4.1.
>
> The vote is open until June 23rd 1AM (PST) and passes if a majority +1 PMC
> votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.1
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see https://spark.apache.org/
>
> The tag to be voted on is v3.4.1-rc1 (commit
> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
> https://github.com/apache/spark/tree/v3.4.1-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1443/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
>
> The list of bug fixes going into 3.4.1 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12352874
>
> This release is using the release script of the tag v3.4.1-rc1.
>
> FAQ
>
> =
> How can I help test this release?
> =
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with a out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.1?
> ===
>
> The current list of open tickets targeted at 3.4.1 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.1
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>

Re: [VOTE][SPIP] PySpark Test Framework

2023-06-22 Thread L. C. Hsieh 
+1

On Thu, Jun 22, 2023 at 3:10 PM Xinrong Meng  wrote:
>
> +1
>
> Thanks for driving that!
>
> On Wed, Jun 21, 2023 at 10:25 PM Ruifeng Zheng  wrote:
>>
>> +1
>>
>> On Thu, Jun 22, 2023 at 1:11 PM Dongjoon Hyun  
>> wrote:
>>>
>>> +1
>>>
>>> Dongjoon
>>>
>>> On Wed, Jun 21, 2023 at 8:56 PM Hyukjin Kwon  wrote:

 +1

 On Thu, 22 Jun 2023 at 02:20, Jacek Laskowski  wrote:
>
> +0
>
> Pozdrawiam,
> Jacek Laskowski
> 
> "The Internals Of" Online Books
> Follow me on https://twitter.com/jaceklaskowski
>
>
>
> On Wed, Jun 21, 2023 at 5:11 PM Amanda Liu  
> wrote:
>>
>> Hi all,
>>
>> I'd like to start the vote for SPIP: PySpark Test Framework.
>>
>> The high-level summary for the SPIP is that it proposes an official test 
>> framework for PySpark. Currently, there are only disparate open-source 
>> repos and blog posts for PySpark testing resources. We can streamline 
>> and simplify the testing process by incorporating test features, such as 
>> a PySpark Test Base class (which allows tests to share Spark sessions) 
>> and test util functions (for example, asserting dataframe and schema 
>> equality).
>>
>> SPIP doc: 
>> https://docs.google.com/document/d/1OkyBn3JbEHkkQgSQ45Lq82esXjr9rm2Vj7Ih_4zycRc/edit#heading=h.f5f0u2riv07v
>>
>> JIRA ticket: https://issues.apache.org/jira/browse/SPARK-44042
>>
>> Discussion thread: 
>> https://lists.apache.org/thread/trwgbgn3ycoj8b8k8lkxko2hql23o41n
>>
>> Please vote on the SPIP for the next 72 hours:
>> [ ] +1: Accept the proposal as an official SPIP
>> [ ] +0
>> [ ] -1: I don’t think this is a good idea because __.
>>
>> Thank you!
>>
>> Best,
>> Amanda Liu

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE][SPIP] PySpark Test Framework

2023-06-22 Thread Xinrong Meng 
+1

Thanks for driving that!

On Wed, Jun 21, 2023 at 10:25 PM Ruifeng Zheng  wrote:

> +1
>
> On Thu, Jun 22, 2023 at 1:11 PM Dongjoon Hyun 
> wrote:
>
>> +1
>>
>> Dongjoon
>>
>> On Wed, Jun 21, 2023 at 8:56 PM Hyukjin Kwon 
>> wrote:
>>
>>> +1
>>>
>>> On Thu, 22 Jun 2023 at 02:20, Jacek Laskowski  wrote:
>>>
 +0

 Pozdrawiam,
 Jacek Laskowski
 
 "The Internals Of" Online Books 
 Follow me on https://twitter.com/jaceklaskowski

 


 On Wed, Jun 21, 2023 at 5:11 PM Amanda Liu <
 amandastephanie...@gmail.com> wrote:

> Hi all,
>
> I'd like to start the vote for SPIP: PySpark Test Framework.
>
> The high-level summary for the SPIP is that it proposes an official
> test framework for PySpark. Currently, there are only disparate 
> open-source
> repos and blog posts for PySpark testing resources. We can streamline and
> simplify the testing process by incorporating test features, such as a
> PySpark Test Base class (which allows tests to share Spark sessions) and
> test util functions (for example, asserting dataframe and schema 
> equality).
>
> *SPIP doc:*
> https://docs.google.com/document/d/1OkyBn3JbEHkkQgSQ45Lq82esXjr9rm2Vj7Ih_4zycRc/edit#heading=h.f5f0u2riv07v
>
> *JIRA ticket:* https://issues.apache.org/jira/browse/SPARK-44042
>
> *Discussion thread:*
> https://lists.apache.org/thread/trwgbgn3ycoj8b8k8lkxko2hql23o41n
>
> Please vote on the SPIP for the next 72 hours:
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because __.
>
> Thank you!
>
> Best,
> Amanda Liu
>

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Gengliang Wang 
+1

On Thu, Jun 22, 2023 at 11:14 AM Driesprong, Fokko 
wrote:

> Thank you for running the release Dongjoon
>
> +1
>
> Tested against Iceberg and it looks good.
>
>
> Op do 22 jun 2023 om 18:03 schreef yangjie01 :
>
>> +1
>>
>>
>>
>> *发件人**: *Dongjoon Hyun 
>> *日期**: *2023年6月22日 星期四 23:35
>> *收件人**: *Chao Sun 
>> *抄送**: *Yuming Wang , Jacek Laskowski <
>> ja...@japila.pl>, dev 
>> *主题**: *Re: [VOTE] Release Spark 3.4.1 (RC1)
>>
>>
>>
>> Thank you everyone for your participation.
>>
>> The vote is open until June 23rd 1AM (PST) and I'll conclude this vote
>> after that.
>>
>> Dongjoon.
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Jun 22, 2023 at 8:29 AM Chao Sun  wrote:
>>
>> +1
>>
>> On Thu, Jun 22, 2023 at 6:52 AM Yuming Wang  wrote:
>> >
>> > +1.
>> >
>> > On Thu, Jun 22, 2023 at 4:41 PM Jacek Laskowski 
>> wrote:
>> >>
>> >> +1
>> >>
>> >> Builds and runs fine on Java 17, macOS.
>> >>
>> >> $ ./dev/change-scala-version.sh 2.13
>> >> $ mvn \
>> >>
>> -Pkubernetes,hadoop-cloud,hive,hive-thriftserver,scala-2.13,volcano,connect
>> \
>> >> -DskipTests \
>> >> clean install
>> >>
>> >> $ python/run-tests --parallelism=1 --testnames 'pyspark.sql.session
>> SparkSession.sql'
>> >> ...
>> >> Tests passed in 28 second
>> >>
>> >> Pozdrawiam,
>> >> Jacek Laskowski
>> >> 
>> >> "The Internals Of" Online Books
>> >> Follow me on https://twitter.com/jaceklaskowski
>> 
>> >>
>> >>
>> >>
>> >> On Tue, Jun 20, 2023 at 4:41 AM Dongjoon Hyun 
>> wrote:
>> >>>
>> >>> Please vote on releasing the following candidate as Apache Spark
>> version 3.4.1.
>> >>>
>> >>> The vote is open until June 23rd 1AM (PST) and passes if a majority
>> +1 PMC votes are cast, with a minimum of 3 +1 votes.
>> >>>
>> >>> [ ] +1 Release this package as Apache Spark 3.4.1
>> >>> [ ] -1 Do not release this package because ...
>> >>>
>> >>> To learn more about Apache Spark, please see
>> https://spark.apache.org/
>> 
>> >>>
>> >>> The tag to be voted on is v3.4.1-rc1 (commit
>> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
>> >>> https://github.com/apache/spark/tree/v3.4.1-rc1
>> 
>> >>>
>> >>> The release files, including signatures, digests, etc. can be found
>> at:
>> >>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
>> 
>> >>>
>> >>> Signatures used for Spark RCs can be found in this file:
>> >>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>> 
>> >>>
>> >>> The staging repository for this release can be found at:
>> >>>
>> https://repository.apache.org/content/repositories/orgapachespark-1443/
>> 
>> >>>
>> >>> The documentation corresponding to this release can be found at:
>> >>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
>> 
>> >>>
>> >>> The list of bug fixes going into 3.4.1 can be found at the following
>> URL:
>> >>> https://issues.apache.org/jira/projects/SPARK/versions/12352874
>> 
>> >>>
>> >>> This release is using the release script of the tag v3.4.1-rc1.
>> >>>
>> >>> FAQ
>> >>>
>> >>> =
>> >>> How can I help test this release?
>> >>> =
>> >>>
>> >>> If you are a Spark user, you can help us test this release by taking
>> >>> an existing Spark workload and running on this release candidate, then
>> >>> reporting any regressions.
>> >>>
>> >>> If you're working in PySpark you can set up a virtual env and install
>> >>> the current RC and see if anything important breaks, in the Java/Scala
>> >>> you can add the staging repository to your projects resolvers and test
>> >>> with the RC (make sure to clean up the artifact cache before/after so
>> >>> you don't end up building with a out of date RC going forward).
>> >>>
>> >>> ===
>> >>> What should happen to JIRA tickets still targeting 3.4.1?
>> >>> ===
>> >>>
>> >>> The current list of open tickets targeted at 3.4.1 can be found at:
>> >>> https://issues.apache.org/jira/projects/SPARK
>>

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Driesprong, Fokko 
Thank you for running the release Dongjoon

+1

Tested against Iceberg and it looks good.


Op do 22 jun 2023 om 18:03 schreef yangjie01 :

> +1
>
>
>
> *发件人**: *Dongjoon Hyun 
> *日期**: *2023年6月22日 星期四 23:35
> *收件人**: *Chao Sun 
> *抄送**: *Yuming Wang , Jacek Laskowski ,
> dev 
> *主题**: *Re: [VOTE] Release Spark 3.4.1 (RC1)
>
>
>
> Thank you everyone for your participation.
>
> The vote is open until June 23rd 1AM (PST) and I'll conclude this vote
> after that.
>
> Dongjoon.
>
>
>
>
>
>
>
> On Thu, Jun 22, 2023 at 8:29 AM Chao Sun  wrote:
>
> +1
>
> On Thu, Jun 22, 2023 at 6:52 AM Yuming Wang  wrote:
> >
> > +1.
> >
> > On Thu, Jun 22, 2023 at 4:41 PM Jacek Laskowski  wrote:
> >>
> >> +1
> >>
> >> Builds and runs fine on Java 17, macOS.
> >>
> >> $ ./dev/change-scala-version.sh 2.13
> >> $ mvn \
> >>
> -Pkubernetes,hadoop-cloud,hive,hive-thriftserver,scala-2.13,volcano,connect
> \
> >> -DskipTests \
> >> clean install
> >>
> >> $ python/run-tests --parallelism=1 --testnames 'pyspark.sql.session
> SparkSession.sql'
> >> ...
> >> Tests passed in 28 second
> >>
> >> Pozdrawiam,
> >> Jacek Laskowski
> >> 
> >> "The Internals Of" Online Books
> >> Follow me on https://twitter.com/jaceklaskowski
> 
> >>
> >>
> >>
> >> On Tue, Jun 20, 2023 at 4:41 AM Dongjoon Hyun 
> wrote:
> >>>
> >>> Please vote on releasing the following candidate as Apache Spark
> version 3.4.1.
> >>>
> >>> The vote is open until June 23rd 1AM (PST) and passes if a majority +1
> PMC votes are cast, with a minimum of 3 +1 votes.
> >>>
> >>> [ ] +1 Release this package as Apache Spark 3.4.1
> >>> [ ] -1 Do not release this package because ...
> >>>
> >>> To learn more about Apache Spark, please see https://spark.apache.org/
> 
> >>>
> >>> The tag to be voted on is v3.4.1-rc1 (commit
> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
> >>> https://github.com/apache/spark/tree/v3.4.1-rc1
> 
> >>>
> >>> The release files, including signatures, digests, etc. can be found at:
> >>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
> 
> >>>
> >>> Signatures used for Spark RCs can be found in this file:
> >>> https://dist.apache.org/repos/dist/dev/spark/KEYS
> 
> >>>
> >>> The staging repository for this release can be found at:
> >>>
> https://repository.apache.org/content/repositories/orgapachespark-1443/
> 
> >>>
> >>> The documentation corresponding to this release can be found at:
> >>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
> 
> >>>
> >>> The list of bug fixes going into 3.4.1 can be found at the following
> URL:
> >>> https://issues.apache.org/jira/projects/SPARK/versions/12352874
> 
> >>>
> >>> This release is using the release script of the tag v3.4.1-rc1.
> >>>
> >>> FAQ
> >>>
> >>> =
> >>> How can I help test this release?
> >>> =
> >>>
> >>> If you are a Spark user, you can help us test this release by taking
> >>> an existing Spark workload and running on this release candidate, then
> >>> reporting any regressions.
> >>>
> >>> If you're working in PySpark you can set up a virtual env and install
> >>> the current RC and see if anything important breaks, in the Java/Scala
> >>> you can add the staging repository to your projects resolvers and test
> >>> with the RC (make sure to clean up the artifact cache before/after so
> >>> you don't end up building with a out of date RC going forward).
> >>>
> >>> ===
> >>> What should happen to JIRA tickets still targeting 3.4.1?
> >>> ===
> >>>
> >>> The current list of open tickets targeted at 3.4.1 can be found at:
> >>> https://issues.apache.org/jira/projects/SPARK
> 
> and search for "Target Version/s" = 3.4.1
> >>>
> >>> Committers should look at those and triage. Extremely important bug
> >>> fixes, documentation, and API tweaks that impact compatibi

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread yangjie01 
+1

发件人: Dongjoon Hyun 
日期: 2023年6月22日 星期四 23:35
收件人: Chao Sun 
抄送: Yuming Wang , Jacek Laskowski , dev 

主题: Re: [VOTE] Release Spark 3.4.1 (RC1)

Thank you everyone for your participation.

The vote is open until June 23rd 1AM (PST) and I'll conclude this vote after 
that.

Dongjoon.



On Thu, Jun 22, 2023 at 8:29 AM Chao Sun 
mailto:sunc...@apache.org>> wrote:
+1

On Thu, Jun 22, 2023 at 6:52 AM Yuming Wang 
mailto:yumw...@apache.org>> wrote:
>
> +1.
>
> On Thu, Jun 22, 2023 at 4:41 PM Jacek Laskowski 
> mailto:ja...@japila.pl>> wrote:
>>
>> +1
>>
>> Builds and runs fine on Java 17, macOS.
>>
>> $ ./dev/change-scala-version.sh 2.13
>> $ mvn \
>> -Pkubernetes,hadoop-cloud,hive,hive-thriftserver,scala-2.13,volcano,connect \
>> -DskipTests \
>> clean install
>>
>> $ python/run-tests --parallelism=1 --testnames 'pyspark.sql.session 
>> SparkSession.sql'
>> ...
>> Tests passed in 28 second
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> 
>> "The Internals Of" Online Books
>> Follow me on 
>> https://twitter.com/jaceklaskowski
>>
>>
>>
>> On Tue, Jun 20, 2023 at 4:41 AM Dongjoon Hyun 
>> mailto:dongj...@apache.org>> wrote:
>>>
>>> Please vote on releasing the following candidate as Apache Spark version 
>>> 3.4.1.
>>>
>>> The vote is open until June 23rd 1AM (PST) and passes if a majority +1 PMC 
>>> votes are cast, with a minimum of 3 +1 votes.
>>>
>>> [ ] +1 Release this package as Apache Spark 3.4.1
>>> [ ] -1 Do not release this package because ...
>>>
>>> To learn more about Apache Spark, please see 
>>> https://spark.apache.org/
>>>
>>> The tag to be voted on is v3.4.1-rc1 (commit 
>>> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
>>> https://github.com/apache/spark/tree/v3.4.1-rc1
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
>>>
>>> Signatures used for Spark RCs can be found in this file:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1443/
>>>
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
>>>
>>> The list of bug fixes going into 3.4.1 can be found at the following URL:
>>> https://issues.apache.org/jira/projects/SPARK/versions/12352874
>>>
>>> This release is using the release script of the tag v3.4.1-rc1.
>>>
>>> FAQ
>>>
>>> =
>>> How can I help test this release?
>>> =
>>>
>>> If you are a Spark user, you can help us test this release by taking
>>> an existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> If you're working in PySpark you can set up a virtual env and install
>>> the current RC and see if anything important breaks, in the Java/Scala
>>> you can add the staging repository to your projects resolvers and test
>>> with the RC (make sure to clean up the artifact cache before/after so
>>> you don't end up building with a out of date RC going forward).
>>>
>>> ===
>>> What should happen to JIRA tickets still targeting 3.4.1?
>>> ===
>>>
>>> The current list of open tickets targeted at 3.4.1 can be found at:
>>> https://issues.apache.org/jira/projects/SPARK
>>>  and search for "Target Version/s" = 3.4.1
>>>
>>> Committers should look at those and triage. Extremely important bug
>>> fixes, documentation, and API tweaks that impact compatibility should
>>> be worked on immediately. Everything else please retarget to an
>>> appropriate release.
>>>
>>> ==
>>> But my bug isn't fixed?
>>> ==
>>>
>>> In order to make timely releases, we will typically not hold the
>>> release unless th

Re: [VOTE] Apache Spark PMC asks Databricks to differentiate its Spark version string

2023-06-22 Thread Mich Talebzadeh 
Sorry I believe I opinionated on it but did not vote.

-1 for me

For reasons already brought up and discussed.

HTH

Mich Talebzadeh,
Solutions Architect/Engineering Lead
Palantir Technologies Limited
London
United Kingdom


   view my Linkedin profile



 https://en.everybodywiki.com/Mich_Talebzadeh



*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.




On Thu, 22 Jun 2023 at 16:39, Dongjoon Hyun  wrote:

> Thank you, Sean, Mitch, Hyukjin, and Maciej for your participation.
>
> The vote is open until June 23rd 1AM (PST) and I'll conclude this vote
> after that.
>
> Dongjoon.
>
> PS. Steve's email seems to arrive to this thread mistakenly. :)
>
>
>
> On Wed, Jun 21, 2023 at 3:12 AM Steve Loughran 
> wrote:
>
>> I'd say everyone should *and* http UA in all the clients who make calls
>> of object stores should, as it helps field issues there. s3a and abfs
>> clients do provide the ability to add params there -please set them in your
>> deployments
>>
>> On Fri, 16 Jun 2023 at 21:53, Dongjoon Hyun  wrote:
>>
>>> Please vote on the following statement. The vote is open until June 23th
>>> 1AM (PST) and passes if a majority +1 PMC votes are cast, with a minimum of
>>> 3 +1 votes.
>>>
>>> Apache Spark PMC asks Databricks to differentiate its Spark
>>> version string to avoid confusions because Apache Spark PMC
>>> is responsible for ensuring to follow ASF requirements[1] and
>>> respects ASF's legal advice [2, 3],
>>>
>>> [ ] +1 Yes
>>> [ ] -1 No because ...
>>>
>>> 
>>> 1. https://www.apache.org/foundation/governance/pmcs#organization
>>> 2. https://lists.apache.org/thread/mzhggd0rpz8t4d7vdsbhkp38mvd3lty4
>>> 3. https://www.apache.org/foundation/marks/downstream.html#source
>>>
>>

Re: [VOTE] Apache Spark PMC asks Databricks to differentiate its Spark version string

2023-06-22 Thread Dongjoon Hyun 
Thank you, Sean, Mitch, Hyukjin, and Maciej for your participation.

The vote is open until June 23rd 1AM (PST) and I'll conclude this vote
after that.

Dongjoon.

PS. Steve's email seems to arrive to this thread mistakenly. :)



On Wed, Jun 21, 2023 at 3:12 AM Steve Loughran 
wrote:

> I'd say everyone should *and* http UA in all the clients who make calls of
> object stores should, as it helps field issues there. s3a and abfs clients
> do provide the ability to add params there -please set them in your
> deployments
>
> On Fri, 16 Jun 2023 at 21:53, Dongjoon Hyun  wrote:
>
>> Please vote on the following statement. The vote is open until June 23th
>> 1AM (PST) and passes if a majority +1 PMC votes are cast, with a minimum of
>> 3 +1 votes.
>>
>> Apache Spark PMC asks Databricks to differentiate its Spark
>> version string to avoid confusions because Apache Spark PMC
>> is responsible for ensuring to follow ASF requirements[1] and
>> respects ASF's legal advice [2, 3],
>>
>> [ ] +1 Yes
>> [ ] -1 No because ...
>>
>> 
>> 1. https://www.apache.org/foundation/governance/pmcs#organization
>> 2. https://lists.apache.org/thread/mzhggd0rpz8t4d7vdsbhkp38mvd3lty4
>> 3. https://www.apache.org/foundation/marks/downstream.html#source
>>
>

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Dongjoon Hyun 
Thank you everyone for your participation.

The vote is open until June 23rd 1AM (PST) and I'll conclude this vote
after that.

Dongjoon.



On Thu, Jun 22, 2023 at 8:29 AM Chao Sun  wrote:

> +1
>
> On Thu, Jun 22, 2023 at 6:52 AM Yuming Wang  wrote:
> >
> > +1.
> >
> > On Thu, Jun 22, 2023 at 4:41 PM Jacek Laskowski  wrote:
> >>
> >> +1
> >>
> >> Builds and runs fine on Java 17, macOS.
> >>
> >> $ ./dev/change-scala-version.sh 2.13
> >> $ mvn \
> >>
> -Pkubernetes,hadoop-cloud,hive,hive-thriftserver,scala-2.13,volcano,connect
> \
> >> -DskipTests \
> >> clean install
> >>
> >> $ python/run-tests --parallelism=1 --testnames 'pyspark.sql.session
> SparkSession.sql'
> >> ...
> >> Tests passed in 28 second
> >>
> >> Pozdrawiam,
> >> Jacek Laskowski
> >> 
> >> "The Internals Of" Online Books
> >> Follow me on https://twitter.com/jaceklaskowski
> >>
> >>
> >>
> >> On Tue, Jun 20, 2023 at 4:41 AM Dongjoon Hyun 
> wrote:
> >>>
> >>> Please vote on releasing the following candidate as Apache Spark
> version 3.4.1.
> >>>
> >>> The vote is open until June 23rd 1AM (PST) and passes if a majority +1
> PMC votes are cast, with a minimum of 3 +1 votes.
> >>>
> >>> [ ] +1 Release this package as Apache Spark 3.4.1
> >>> [ ] -1 Do not release this package because ...
> >>>
> >>> To learn more about Apache Spark, please see https://spark.apache.org/
> >>>
> >>> The tag to be voted on is v3.4.1-rc1 (commit
> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
> >>> https://github.com/apache/spark/tree/v3.4.1-rc1
> >>>
> >>> The release files, including signatures, digests, etc. can be found at:
> >>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
> >>>
> >>> Signatures used for Spark RCs can be found in this file:
> >>> https://dist.apache.org/repos/dist/dev/spark/KEYS
> >>>
> >>> The staging repository for this release can be found at:
> >>>
> https://repository.apache.org/content/repositories/orgapachespark-1443/
> >>>
> >>> The documentation corresponding to this release can be found at:
> >>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
> >>>
> >>> The list of bug fixes going into 3.4.1 can be found at the following
> URL:
> >>> https://issues.apache.org/jira/projects/SPARK/versions/12352874
> >>>
> >>> This release is using the release script of the tag v3.4.1-rc1.
> >>>
> >>> FAQ
> >>>
> >>> =
> >>> How can I help test this release?
> >>> =
> >>>
> >>> If you are a Spark user, you can help us test this release by taking
> >>> an existing Spark workload and running on this release candidate, then
> >>> reporting any regressions.
> >>>
> >>> If you're working in PySpark you can set up a virtual env and install
> >>> the current RC and see if anything important breaks, in the Java/Scala
> >>> you can add the staging repository to your projects resolvers and test
> >>> with the RC (make sure to clean up the artifact cache before/after so
> >>> you don't end up building with a out of date RC going forward).
> >>>
> >>> ===
> >>> What should happen to JIRA tickets still targeting 3.4.1?
> >>> ===
> >>>
> >>> The current list of open tickets targeted at 3.4.1 can be found at:
> >>> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.1
> >>>
> >>> Committers should look at those and triage. Extremely important bug
> >>> fixes, documentation, and API tweaks that impact compatibility should
> >>> be worked on immediately. Everything else please retarget to an
> >>> appropriate release.
> >>>
> >>> ==
> >>> But my bug isn't fixed?
> >>> ==
> >>>
> >>> In order to make timely releases, we will typically not hold the
> >>> release unless the bug in question is a regression from the previous
> >>> release. That being said, if there is something which is a regression
> >>> that has not been correctly targeted please ping me or a committer to
> >>> help target the issue.
>

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Chao Sun 
+1

On Thu, Jun 22, 2023 at 6:52 AM Yuming Wang  wrote:
>
> +1.
>
> On Thu, Jun 22, 2023 at 4:41 PM Jacek Laskowski  wrote:
>>
>> +1
>>
>> Builds and runs fine on Java 17, macOS.
>>
>> $ ./dev/change-scala-version.sh 2.13
>> $ mvn \
>> -Pkubernetes,hadoop-cloud,hive,hive-thriftserver,scala-2.13,volcano,connect \
>> -DskipTests \
>> clean install
>>
>> $ python/run-tests --parallelism=1 --testnames 'pyspark.sql.session 
>> SparkSession.sql'
>> ...
>> Tests passed in 28 second
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> 
>> "The Internals Of" Online Books
>> Follow me on https://twitter.com/jaceklaskowski
>>
>>
>>
>> On Tue, Jun 20, 2023 at 4:41 AM Dongjoon Hyun  wrote:
>>>
>>> Please vote on releasing the following candidate as Apache Spark version 
>>> 3.4.1.
>>>
>>> The vote is open until June 23rd 1AM (PST) and passes if a majority +1 PMC 
>>> votes are cast, with a minimum of 3 +1 votes.
>>>
>>> [ ] +1 Release this package as Apache Spark 3.4.1
>>> [ ] -1 Do not release this package because ...
>>>
>>> To learn more about Apache Spark, please see https://spark.apache.org/
>>>
>>> The tag to be voted on is v3.4.1-rc1 (commit 
>>> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
>>> https://github.com/apache/spark/tree/v3.4.1-rc1
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
>>>
>>> Signatures used for Spark RCs can be found in this file:
>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1443/
>>>
>>> The documentation corresponding to this release can be found at:
>>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
>>>
>>> The list of bug fixes going into 3.4.1 can be found at the following URL:
>>> https://issues.apache.org/jira/projects/SPARK/versions/12352874
>>>
>>> This release is using the release script of the tag v3.4.1-rc1.
>>>
>>> FAQ
>>>
>>> =
>>> How can I help test this release?
>>> =
>>>
>>> If you are a Spark user, you can help us test this release by taking
>>> an existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> If you're working in PySpark you can set up a virtual env and install
>>> the current RC and see if anything important breaks, in the Java/Scala
>>> you can add the staging repository to your projects resolvers and test
>>> with the RC (make sure to clean up the artifact cache before/after so
>>> you don't end up building with a out of date RC going forward).
>>>
>>> ===
>>> What should happen to JIRA tickets still targeting 3.4.1?
>>> ===
>>>
>>> The current list of open tickets targeted at 3.4.1 can be found at:
>>> https://issues.apache.org/jira/projects/SPARK and search for "Target 
>>> Version/s" = 3.4.1
>>>
>>> Committers should look at those and triage. Extremely important bug
>>> fixes, documentation, and API tweaks that impact compatibility should
>>> be worked on immediately. Everything else please retarget to an
>>> appropriate release.
>>>
>>> ==
>>> But my bug isn't fixed?
>>> ==
>>>
>>> In order to make timely releases, we will typically not hold the
>>> release unless the bug in question is a regression from the previous
>>> release. That being said, if there is something which is a regression
>>> that has not been correctly targeted please ping me or a committer to
>>> help target the issue.

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Yuming Wang 
+1.

On Thu, Jun 22, 2023 at 4:41 PM Jacek Laskowski  wrote:

> +1
>
> Builds and runs fine on Java 17, macOS.
>
> $ ./dev/change-scala-version.sh 2.13
> $ mvn \
> -Pkubernetes,hadoop-cloud,hive,hive-thriftserver,scala-2.13,volcano,connect
> \
> -DskipTests \
> clean install
>
> $ python/run-tests --parallelism=1 --testnames 'pyspark.sql.session
> SparkSession.sql'
> ...
> Tests passed in 28 second
>
> Pozdrawiam,
> Jacek Laskowski
> 
> "The Internals Of" Online Books 
> Follow me on https://twitter.com/jaceklaskowski
>
> 
>
>
> On Tue, Jun 20, 2023 at 4:41 AM Dongjoon Hyun  wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 3.4.1.
>>
>> The vote is open until June 23rd 1AM (PST) and passes if a majority +1
>> PMC votes are cast, with a minimum of 3 +1 votes.
>>
>> [ ] +1 Release this package as Apache Spark 3.4.1
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see https://spark.apache.org/
>>
>> The tag to be voted on is v3.4.1-rc1 (commit
>> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
>> https://github.com/apache/spark/tree/v3.4.1-rc1
>>
>> The release files, including signatures, digests, etc. can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
>>
>> Signatures used for Spark RCs can be found in this file:
>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1443/
>>
>> The documentation corresponding to this release can be found at:
>> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
>>
>> The list of bug fixes going into 3.4.1 can be found at the following URL:
>> https://issues.apache.org/jira/projects/SPARK/versions/12352874
>>
>> This release is using the release script of the tag v3.4.1-rc1.
>>
>> FAQ
>>
>> =
>> How can I help test this release?
>> =
>>
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> If you're working in PySpark you can set up a virtual env and install
>> the current RC and see if anything important breaks, in the Java/Scala
>> you can add the staging repository to your projects resolvers and test
>> with the RC (make sure to clean up the artifact cache before/after so
>> you don't end up building with a out of date RC going forward).
>>
>> ===
>> What should happen to JIRA tickets still targeting 3.4.1?
>> ===
>>
>> The current list of open tickets targeted at 3.4.1 can be found at:
>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>> Version/s" = 3.4.1
>>
>> Committers should look at those and triage. Extremely important bug
>> fixes, documentation, and API tweaks that impact compatibility should
>> be worked on immediately. Everything else please retarget to an
>> appropriate release.
>>
>> ==
>> But my bug isn't fixed?
>> ==
>>
>> In order to make timely releases, we will typically not hold the
>> release unless the bug in question is a regression from the previous
>> release. That being said, if there is something which is a regression
>> that has not been correctly targeted please ping me or a committer to
>> help target the issue.
>>
>

Re: [VOTE] Release Spark 3.4.1 (RC1)

2023-06-22 Thread Jacek Laskowski 
+1

Builds and runs fine on Java 17, macOS.

$ ./dev/change-scala-version.sh 2.13
$ mvn \
-Pkubernetes,hadoop-cloud,hive,hive-thriftserver,scala-2.13,volcano,connect
\
-DskipTests \
clean install

$ python/run-tests --parallelism=1 --testnames 'pyspark.sql.session
SparkSession.sql'
...
Tests passed in 28 second

Pozdrawiam,
Jacek Laskowski

"The Internals Of" Online Books 
Follow me on https://twitter.com/jaceklaskowski




On Tue, Jun 20, 2023 at 4:41 AM Dongjoon Hyun  wrote:

> Please vote on releasing the following candidate as Apache Spark version
> 3.4.1.
>
> The vote is open until June 23rd 1AM (PST) and passes if a majority +1 PMC
> votes are cast, with a minimum of 3 +1 votes.
>
> [ ] +1 Release this package as Apache Spark 3.4.1
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see https://spark.apache.org/
>
> The tag to be voted on is v3.4.1-rc1 (commit
> 6b1ff22dde1ead51cbf370be6e48a802daae58b6)
> https://github.com/apache/spark/tree/v3.4.1-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-bin/
>
> Signatures used for Spark RCs can be found in this file:
> https://dist.apache.org/repos/dist/dev/spark/KEYS
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1443/
>
> The documentation corresponding to this release can be found at:
> https://dist.apache.org/repos/dist/dev/spark/v3.4.1-rc1-docs/
>
> The list of bug fixes going into 3.4.1 can be found at the following URL:
> https://issues.apache.org/jira/projects/SPARK/versions/12352874
>
> This release is using the release script of the tag v3.4.1-rc1.
>
> FAQ
>
> =
> How can I help test this release?
> =
>
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> If you're working in PySpark you can set up a virtual env and install
> the current RC and see if anything important breaks, in the Java/Scala
> you can add the staging repository to your projects resolvers and test
> with the RC (make sure to clean up the artifact cache before/after so
> you don't end up building with a out of date RC going forward).
>
> ===
> What should happen to JIRA tickets still targeting 3.4.1?
> ===
>
> The current list of open tickets targeted at 3.4.1 can be found at:
> https://issues.apache.org/jira/projects/SPARK and search for "Target
> Version/s" = 3.4.1
>
> Committers should look at those and triage. Extremely important bug
> fixes, documentation, and API tweaks that impact compatibility should
> be worked on immediately. Everything else please retarget to an
> appropriate release.
>
> ==
> But my bug isn't fixed?
> ==
>
> In order to make timely releases, we will typically not hold the
> release unless the bug in question is a regression from the previous
> release. That being said, if there is something which is a regression
> that has not been correctly targeted please ping me or a committer to
> help target the issue.
>