Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread Felix Cheung
Reposting for shane here [SPARK-27178] https://github.com/apache/spark/commit/342e91fdfa4e6ce5cc3a0da085d1fe723184021b Is problematic too and it’s not in the rc8 cut https://github.com/apache/spark/commits/branch-2.4 (Personally I don’t want to delay 2.4.1 either..)

Re: [VOTE] [SPARK-24615] SPIP: Accelerator-aware Scheduling

2019-03-20 Thread Xiangrui Meng
Steve, the initial work would focus on GPUs, but we will keep the interfaces general to support other accelerators in the future. This was mentioned in the SPIP and draft design. Imran, you should have comment permission now. Thanks for making a pass! I don't think the proposed 3.0 features

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread Sean Owen
+1 for this RC. The tag is correct, licenses and sigs check out, tests of the source with most profiles enabled works for me. On Tue, Mar 19, 2019 at 5:28 PM DB Tsai wrote: > > Please vote on releasing the following candidate as Apache Spark version > 2.4.1. > > The vote is open until March 23

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread DB Tsai
Unfortunately, for this 2.4.1 RC cuts, we ran into couple critical bug fixes unexpectedly just right after each RC was cut, and some of the bugs were even found after the RC cut which is hard to know there is still a blocker beforehand. How about we start to test out the RC8 now given the

Re: [VOTE] [SPARK-24615] SPIP: Accelerator-aware Scheduling

2019-03-20 Thread Imran Rashid
Thanks for sending the updated docs. Can you please give everyone the ability to comment? I have some comments, but overall I think this is a good proposal and addresses my prior concerns. My only real concern is that I notice some mention of "must dos" for spark 3.0. I don't want to make any

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread Imran Rashid
Even if only PMC are able to veto a release, I believe all community members are encouraged to vote, even a -1, to express their opinions, right? I am -0.5 on the release because of SPARK-27112. It is not a regression, so in that sense I don't think it must hold the release. But it is fixing a

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread dhruve ashar
I agree with Imran on this. Since we are already on rc8, we don't want to indefinitely hold the release for one more fix, but it this one is a severe deadlock. Note to myself and other community members: May be we can be more proactive in checking and reporting inprogress PR reviews/JIRAs for any

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread Sean Owen
Is it a regression from 2.4.0? that's not the only criteria but part of it. The version link is https://issues.apache.org/jira/projects/SPARK/versions/12344117 On Wed, Mar 20, 2019 at 10:15 AM dhruve ashar wrote: > A deadlock bug was recently fixed and backported to 2.4, but the rc was > cut

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread dhruve ashar
A deadlock bug was recently fixed and backported to 2.4, but the rc was cut before that. I think we should include a critical bug like that in the current rc. issue: https://issues.apache.org/jira/browse/SPARK-27112 commit:

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread Stavros Kontopoulos
+1 (non-binding) On Wed, Mar 20, 2019 at 8:33 AM Sean Owen wrote: > (Only the PMC can veto a release) > That doesn't look like a regression. I get that it's important, but I > don't see that it should block this release. > > On Tue, Mar 19, 2019 at 11:00 PM Darcy Shen > wrote: > > > > -1 > >

Re: Introduce FORMAT clause to CAST with SQL:2016 datetime patterns

2019-03-20 Thread Maciej Szymkiewicz
One concern here is introduction of second formatting convention. This can not only cause confusion among users, but also result in some hard to spot bugs, when wrong format, with different meaning, is used. This is already a problem for Python and R users, with week year and months / minutes

Introduce FORMAT clause to CAST with SQL:2016 datetime patterns

2019-03-20 Thread Gabor Kaszab
Hey Hive and Spark communities, [dev@impala in cc] I'm working on an Impala improvement to introduce the FORMAT clause within CAST() operator and to implement ISO SQL:2016 datetime pattern support for this new FORMAT clause: https://issues.apache.org/jira/browse/IMPALA-4018 One example of the

Unsubscribe

2019-03-20 Thread AADIL AHMAD
Unsubscribe

Re: [VOTE] Release Apache Spark 2.4.1 (RC8)

2019-03-20 Thread Sean Owen
(Only the PMC can veto a release) That doesn't look like a regression. I get that it's important, but I don't see that it should block this release. On Tue, Mar 19, 2019 at 11:00 PM Darcy Shen wrote: > > -1 > > please backpoart SPARK-27160, a correctness issue about ORC native reader. > > see