Re: Spark 3.0 branch cut and code freeze on Jan 31?

[Hyukjin Kwon]

Sounds fine.
I am trying to get pandas UDF redesign done (SPARK-28264
) on time. Hope I can
make it.

2019년 12월 24일 (화) 오후 4:17, Wenchen Fan 님이 작성:

> Sounds good!
>
> On Tue, Dec 24, 2019 at 7:48 AM Reynold Xin  wrote:
>
>> We've pushed out 3.0 multiple times. The latest release window documented
>> on the website  says
>> we'd code freeze and cut branch-3.0 early Dec. It looks like we are
>> suffering a bit from the tragedy of the commons, that nobody is pushing for
>> getting the release out. I understand the natural tendency for each
>> individual is to finish or extend the feature/bug that the person has been
>> working on. At some point we need to say "this is it" and get the release
>> out. I'm happy to help drive this process.
>>
>> To be realistic, I don't think we should just code freeze *today*.
>> Although we have updated the website, contributors have all been operating
>> under the assumption that all active developments are still going on. I
>> propose we *cut the branch on **Jan 31**, and code freeze and switch
>> over to bug squashing mode, and try to get the 3.0 official release out in
>> Q1*. That is, by default no new features can go into the branch starting Jan
>> 31.
>>
>> What do you think?
>>
>> And happy holidays everybody.
>>
>>
>>
>>

Re: Spark 3.0 branch cut and code freeze on Jan 31?

[Wenchen Fan]

Sounds good!

On Tue, Dec 24, 2019 at 7:48 AM Reynold Xin  wrote:

> We've pushed out 3.0 multiple times. The latest release window documented
> on the website  says we'd
> code freeze and cut branch-3.0 early Dec. It looks like we are suffering a
> bit from the tragedy of the commons, that nobody is pushing for getting the
> release out. I understand the natural tendency for each individual is to
> finish or extend the feature/bug that the person has been working on. At
> some point we need to say "this is it" and get the release out. I'm happy
> to help drive this process.
>
> To be realistic, I don't think we should just code freeze *today*.
> Although we have updated the website, contributors have all been operating
> under the assumption that all active developments are still going on. I
> propose we *cut the branch on **Jan 31**, and code freeze and switch over
> to bug squashing mode, and try to get the 3.0 official release out in Q1*.
> That is, by default no new features can go into the branch starting Jan 31
> .
>
> What do you think?
>
> And happy holidays everybody.
>
>
>
>

Spark 3.0 branch cut and code freeze on Jan 31?

[Reynold Xin]

We've pushed out 3.0 multiple times. The latest release window documented on 
the website ( http://spark.apache.org/versioning-policy.html ) says we'd code 
freeze and cut branch-3.0 early Dec. It looks like we are suffering a bit from 
the tragedy of the commons, that nobody is pushing for getting the release out. 
I understand the natural tendency for each individual is to finish or extend 
the feature/bug that the person has been working on. At some point we need to 
say "this is it" and get the release out. I'm happy to help drive this process.

To be realistic, I don't think we should just code freeze * today *. Although 
we have updated the website, contributors have all been operating under the 
assumption that all active developments are still going on. I propose we *cut 
the branch on* *Jan 31* *, and code freeze and switch over to bug squashing 
mode, and try to get the 3.0 official release out in Q1*. That is, by default 
no new features can go into the branch starting Jan 31.

What do you think?

And happy holidays everybody.

Re: Jenkins looks hosed

[Marcelo Vanzin]

Enjoy your break Shane. And at least from my part, don't even bother
checking in - I'm sure we can survive a few days if Jenkins
misbehaves.

On Mon, Dec 23, 2019 at 2:01 PM Shane Knapp  wrote:
>
> i'll be out of the country and on vacation until early january, but
> i'll make a point to check in every couple of days to ensure that
> jenkins is happy.
>
> On Mon, Dec 23, 2019 at 12:25 PM Shane Knapp  wrote:
> >
> > yep, it was most definitely wedged.  restarted the service and it's back up!
> >
> > On Mon, Dec 23, 2019 at 12:23 PM Shane Knapp  wrote:
> > >
> > > checking it now.
> > >
> > > On Mon, Dec 23, 2019 at 11:27 AM Marcelo Vanzin
> > >  wrote:
> > > >
> > > > Just in the off-chance that someone with admin access to the Jenkins
> > > > servers is around this week... they seem to be in a pretty unhappy
> > > > state, I can't even load the UI.
> > > >
> > > > FYI in case you're waiting for your PR tests to finish (or even start 
> > > > running).
> > > >
> > > > --
> > > > Marcelo
> > > >
> > > > -
> > > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> > > >
> > >
> > >
> > > --
> > > Shane Knapp
> > > Computer Guy / Voice of Reason
> > > UC Berkeley EECS Research / RISELab Staff Technical Lead
> > > https://rise.cs.berkeley.edu
> >
> >
> >
> > --
> > Shane Knapp
> > Computer Guy / Voice of Reason
> > UC Berkeley EECS Research / RISELab Staff Technical Lead
> > https://rise.cs.berkeley.edu
>
>
>
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu



-- 
Marcelo

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Jenkins looks hosed

[Shane Knapp]

i'll be out of the country and on vacation until early january, but
i'll make a point to check in every couple of days to ensure that
jenkins is happy.

On Mon, Dec 23, 2019 at 12:25 PM Shane Knapp  wrote:
>
> yep, it was most definitely wedged.  restarted the service and it's back up!
>
> On Mon, Dec 23, 2019 at 12:23 PM Shane Knapp  wrote:
> >
> > checking it now.
> >
> > On Mon, Dec 23, 2019 at 11:27 AM Marcelo Vanzin
> >  wrote:
> > >
> > > Just in the off-chance that someone with admin access to the Jenkins
> > > servers is around this week... they seem to be in a pretty unhappy
> > > state, I can't even load the UI.
> > >
> > > FYI in case you're waiting for your PR tests to finish (or even start 
> > > running).
> > >
> > > --
> > > Marcelo
> > >
> > > -
> > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> > >
> >
> >
> > --
> > Shane Knapp
> > Computer Guy / Voice of Reason
> > UC Berkeley EECS Research / RISELab Staff Technical Lead
> > https://rise.cs.berkeley.edu
>
>
>
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu



-- 
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Jenkins looks hosed

[Shane Knapp]

yep, it was most definitely wedged.  restarted the service and it's back up!

On Mon, Dec 23, 2019 at 12:23 PM Shane Knapp  wrote:
>
> checking it now.
>
> On Mon, Dec 23, 2019 at 11:27 AM Marcelo Vanzin
>  wrote:
> >
> > Just in the off-chance that someone with admin access to the Jenkins
> > servers is around this week... they seem to be in a pretty unhappy
> > state, I can't even load the UI.
> >
> > FYI in case you're waiting for your PR tests to finish (or even start 
> > running).
> >
> > --
> > Marcelo
> >
> > -
> > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
> >
>
>
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu



-- 
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Jenkins looks hosed

[Shane Knapp]

checking it now.

On Mon, Dec 23, 2019 at 11:27 AM Marcelo Vanzin
 wrote:
>
> Just in the off-chance that someone with admin access to the Jenkins
> servers is around this week... they seem to be in a pretty unhappy
> state, I can't even load the UI.
>
> FYI in case you're waiting for your PR tests to finish (or even start 
> running).
>
> --
> Marcelo
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>


-- 
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Jenkins looks hosed

[Marcelo Vanzin]

Just in the off-chance that someone with admin access to the Jenkins
servers is around this week... they seem to be in a pretty unhappy
state, I can't even load the UI.

FYI in case you're waiting for your PR tests to finish (or even start running).

-- 
Marcelo

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Fw:Re: [VOTE][SPARK-29018][SPIP]:Build spark thrift server based on protocol v11

[Yuming Wang]

I'm  +1 for this SPIP for these two reasons:

1. The current thriftserver has some issues that are not easy to solve,
such as: SPARK-28636 .
2. The difference between the version of ORC we are using and the built-in
Hive is using is getting bigger and bigger. We can't ensure that there will
be no compatibility issues in the future. If thriftserver does not depend
on Hive, it will be much easier to upgrade the built-in Hive in the future.

On Sat, Dec 21, 2019 at 9:28 PM angers.zhu  wrote:

> Hi all,
>
> I have complete a Design doc about how to use and config this new thrift
> server, and some design detail about change and impersonation.
>
> Hope for your suggestions and ideas.
>
> SPIP DOC :
> https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit#heading=h.x97c6tj78zo0
> Design DOC :
> https://docs.google.com/document/d/1UKE9QTtHqSZBq0V_vEn54PlWaWPiRAKf_JvcT0skaSo/edit#heading=h.q1ed5q1ldh14
> Thrift server about configurations
> https://docs.google.com/document/d/1uI35qJmQO4FKE6pr0h3zetZqww-uI8QsQjxaYY_qb1s/edit?usp=drive_web&ouid=110963191229426834922
>
> Best Regards
> angers.zhu
> angers@gmail.com
>
> 
> 签名由 网易邮箱大师  定制
>
> - Forwarded Message -
> From： angers.zhu  
> Date： 12/18/2019 22:29
> To： dev-ow...@spark.apache.org 
> 
> Subject： Re: [VOTE][SPARK-29018][SPIP]:Build spark thrift server based on
> protocol v11
>
> Add spark-dev group access privilege to google.
> angers.zhu
> angers@gmail.com
>
> 
> 签名由 网易邮箱大师  定制
>
> On 12/18/2019 22:02，Sandeep Katta
>  wrote：
>
> I couldn't access the doc, please give permission to the spark-dev group
>
> On Wed, 18 Dec 2019 at 18:05, angers.zhu  wrote:
>
>> With the development of Spark and Hive，in current sql/hive-thriftserver
>> module,
>>
>> we need to do a lot of work to solve code conflicts for different
>> built-in hive versions.
>>
>> It's an annoying and unending work in current ways. And these issues have
>> limited
>>
>> our ability and convenience to develop new features for Spark’s thrift
>> server.
>>
>> We suppose to implement a new thrift server and JDBC driver based on 
>> Hive’s
>> latest v11
>>
>> TCLService.thrift thrift protocol. Finally, the new thrift server have
>> below feature:
>>
>>1.
>>
>>Build new module spark-service as spark’s thrift server
>>2.
>>
>>Don't need as much reflection and inherited code as `hive-thriftser`
>>modules
>>3.
>>
>>Support all functions current `sql/hive-thriftserver` support
>>4.
>>
>>Use all code maintained by spark itself, won’t depend on Hive
>>5.
>>
>>Support origin functions use spark’s own way, won't limited by Hive's
>>code
>>6.
>>
>>Support running without hive metastore or with hive metastore
>>7.
>>
>>Support user impersonation by Multi-tenant splited hive
>>authentication and DFS authentication
>>8.
>>
>>Support session hook for with spark’s own code
>>9.
>>
>>Add a new jdbc driver spark-jdbc, with spark’s own connection url
>>“jdbc:spark::/”
>>10.
>>
>>Support both hive-jdbc and spark-jdbc client, then we can support
>>most clients and BI platform
>>
>>
>>
>> https://issues.apache.org/jira/browse/SPARK-29018
>>
>> Google Doc:
>> https://docs.google.com/document/d/1ug4K5e2okF5Q2Pzi3qJiUILwwqkn0fVQaQ-Q95HEcJQ/edit#
>>
>> [ ] +1: Accept the proposal as an official SPIP
>> [ ] +0
>> [ ] -1: I don't think this is a good idea because ...
>>
>> I'll start with my +1
>> angers.zhu
>> angers@gmail.com
>>
>> 
>> 签名由 网易邮箱大师  定制
>>
>>