Re: [VOTE] SPARK 2.4.0 (RC5)

2018-11-01 Thread Xiangrui Meng
Just FYI, not to block the release. We found an issue with PySpark barrier execution mode: https://issues.apache.org/jira/browse/SPARK-25921. We should list it as a known issues in the release notes and get it fixed in 2.4.1. -Xiangrui On Thu, Nov 1, 2018 at 12:19 AM Wenchen Fan wrote: > This

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-11-01 Thread Wenchen Fan
This vote passes! I'll follow up with a formal release announcement soon. +1: Xiao Li (binding) Sean Owen (binding) Gengliang Wang Hyukjin Kwon Wenchen Fan (binding) Ryan Blue Bryan Cutler Marcelo Vanzin (binding) Reynold Xin (binding) Chitral Verma Dilip Biswal Denny Lee Felix Cheung (binding)

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-31 Thread Dongjoon Hyun
+1 Cheers, Dongjoon.

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-31 Thread Felix Cheung
+1 Checked R doc and all R API changes From: Denny Lee Sent: Wednesday, October 31, 2018 9:13 PM To: Chitral Verma Cc: Wenchen Fan; dev@spark.apache.org Subject: Re: [VOTE] SPARK 2.4.0 (RC5) +1 On Wed, Oct 31, 2018 at 12:54 PM Chitral Verma mailto:chitralve

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-31 Thread Denny Lee
+1 On Wed, Oct 31, 2018 at 12:54 PM Chitral Verma wrote: > +1 > > On Wed, 31 Oct 2018 at 11:56, Reynold Xin wrote: > >> +1 >> >> Look forward to the release! >> >> >> >> On Mon, Oct 29, 2018 at 3:22 AM Wenchen Fan wrote: >> >>> Please vote on releasing the following candidate as Apache Spark

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-31 Thread Reynold Xin
+1 Look forward to the release! On Mon, Oct 29, 2018 at 3:22 AM Wenchen Fan wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.4.0. > > The vote is open until November 1 PST and passes if a majority +1 PMC > votes are cast, with > a minimum of 3 +1 votes. >

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-31 Thread Marcelo Vanzin
+1 On Mon, Oct 29, 2018 at 3:22 AM Wenchen Fan wrote: > > Please vote on releasing the following candidate as Apache Spark version > 2.4.0. > > The vote is open until November 1 PST and passes if a majority +1 PMC votes > are cast, with > a minimum of 3 +1 votes. > > [ ] +1 Release this package

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-30 Thread Ryan Blue
+1 On Tue, Oct 30, 2018 at 4:42 AM Wenchen Fan wrote: > Thanks for reporting the bug! I'll list it as a known issue for 2.4.0 > > I'm adding my own +1, since all the known blockers are resolved. > > On Tue, Oct 30, 2018 at 2:56 PM Xiao Li wrote: > >> Yes, this is not a blocker. >>

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-30 Thread Wenchen Fan
Thanks for reporting the bug! I'll list it as a known issue for 2.4.0 I'm adding my own +1, since all the known blockers are resolved. On Tue, Oct 30, 2018 at 2:56 PM Xiao Li wrote: > Yes, this is not a blocker. > "spark.sql.optimizer.nestedSchemaPruning.enabled" is intentionally off by >

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-30 Thread Xiao Li
Yes, this is not a blocker. "spark.sql.optimizer.nestedSchemaPruning.enabled" is intentionally off by default. As DB Tsai said, column pruning of nested schema for Parquet tables is experimental. In this release, we encourage the whole community to try this new feature but it might have bugs like

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-30 Thread DB Tsai
+0 I understand that schema pruning is an experimental feature in Spark 2.4, and this can help a lot in read performance as people are trying to keep the hierarchical data in nested format. We just found a serious bug---it could fail parquet reader if a nested field and top level field are

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-29 Thread Hyukjin Kwon
+1 2018년 10월 30일 (화) 오전 11:03, Gengliang Wang 님이 작성: > +1 > > > 在 2018年10月30日,上午10:41,Sean Owen 写道: > > > > +1 > > > > Same result as in RC4 from me, and the issues I know of that were > > raised with RC4 are resolved. I tested vs Scala 2.12 and 2.11. > > > > These items are still targeted to

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-29 Thread Gengliang Wang
+1 > 在 2018年10月30日,上午10:41,Sean Owen 写道: > > +1 > > Same result as in RC4 from me, and the issues I know of that were > raised with RC4 are resolved. I tested vs Scala 2.12 and 2.11. > > These items are still targeted to 2.4.0; Xiangrui I assume these > should just be untargeted now, or

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-29 Thread Sean Owen
+1 Same result as in RC4 from me, and the issues I know of that were raised with RC4 are resolved. I tested vs Scala 2.12 and 2.11. These items are still targeted to 2.4.0; Xiangrui I assume these should just be untargeted now, or resolved? SPARK-25584 Document libsvm data source in doc site

Re: [VOTE] SPARK 2.4.0 (RC5)

2018-10-29 Thread Xiao Li
+1 On Mon, Oct 29, 2018 at 3:22 AM Wenchen Fan wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.4.0. > > The vote is open until November 1 PST and passes if a majority +1 PMC > votes are cast, with > a minimum of 3 +1 votes. > > [ ] +1 Release this package

[VOTE] SPARK 2.4.0 (RC5)

2018-10-29 Thread Wenchen Fan
Please vote on releasing the following candidate as Apache Spark version 2.4.0. The vote is open until November 1 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.4.0 [ ] -1 Do not release this package because ... To