Re: Livy metrics

[Yiheng Wang]

+1

On Fri, Jan 10, 2020 at 10:00 AM 安辉  wrote:

> In my company, We are maintaining a metrc version of livy, I am willing to
> submit this patch, if you do not mind.
>
> -邮件原件-
> 发件人: Saisai Shao [mailto:sai.sai.s...@gmail.com]
> 发送时间: 2020年1月10日 9:39
> 收件人: dev@livy.incubator.apache.org
> 主题: Re: Livy metrics
>
> Currently we don't have concrete plan on this, it would be great if you
> could contribute on this.
>
> Thanks
> Saisai
>
> Shanyu Zhao  于2020年1月10日周五 上午8:44写道：
>
> > Hi,
> >
> > Are there ongoing effort to add Livy metrics? I saw that dropwizard is
> > added as dependency in pom.xml but didn't see it get used anywhere.
> >
> > Thanks,
> > Shanyu
> >
> 
> OPPO
>
>
> 本电子邮件及其附件含有OPPO公司的保密信息，仅限于邮件指明的收件人使用（包含个人及群组）。禁止任何人在未经授权的情况下以任何形式使用。如果您错收了本邮件，请立即以电子邮件通知发件人并删除本邮件及其附件。
>
> This e-mail and its attachments contain confidential information from
> OPPO, which is intended only for the person or entity whose address is
> listed above. Any use of the information contained herein in any way
> (including, but not limited to, total or partial disclosure, reproduction,
> or dissemination) by persons other than the intended recipient(s) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by phone or email immediately and delete it!
>

Re: [VOTE] Release Livy 0.7.0 based on RC4

[Yiheng Wang]

Check the release files. Looks good.

+1

On Tue, Jan 7, 2020 at 9:25 PM Saisai Shao  wrote:

> This vote is for releasing Livy 0.7.0 based on RC4.
>
> This RC mainly fixes the license issue.
>
> The vote will be open until Sunday  Jan 12, 23:59 UTC and
> will pass with a minimum of 3 +1 binding votes and a majority of positive
> votes.
>
> [+1] This release is ready to send to the Incubator PMC for approval
>
> [-1] This release is not ready because...
>
> This vote is held according to Apache Incubator release policy:
> https://incubator.apache.org/policy/incubation.html#releases
>
> The RC is based on tag v0.7.0-incubating-rc4:
>
> https://github.com/apache/incubator-livy/commit/664503355d8bae763989ebac087122e306239d54
>
> The release files can be found here:
> https://dist.apache.org/repos/dist/dev/incubator/livy/0.7.0-incubating-rc4/
>
> The staged maven artifacts can be found here:
> https://repository.apache.org/content/repositories/orgapachelivy-1013
>
> The list of resolved JIRAs in this release can be found here:
> https://issues.apache.org/jira/projects/LIVY/versions/12345179
>
> Thanks
> Saisai
>

Re: [VOTE] Release Livy 0.7.0 based on RC3

[Yiheng Wang]

Check the release files. Looks good.

+1

On Wed, Dec 18, 2019 at 5:25 PM runzhiwang  wrote:

> 
>   +1
>  
>  ** release files*
> I verify the apache-livy-0.7.0-incubating-bin.zip by creating session,
> statement, and everything is ok.
>  I also verify the sha512 and asc file. Both look good.
>
>   ** staged maven artifacts **
>  It's ok.
>  
>
>  ** resolved JIRAs **
>  It's ok. I check all the jiras in the commits.
>  
>
> Thanks
> RunzhiWang
>
>
>
>  --原始邮件--
>   发件人:"Saisai Shao"  发送时间:2019年12月18日(星期三) 中午1:58
>  收件人:"dev"
>  主题:[VOTE] Release Livy 0.7.0 based on RC3
>
>
>
> This vote is for releasing Livy 0.7.0 based on RC3.
>
> The vote will be open until Friday Dec 23, 23:59 UTC and
> will pass with a minimum of 3 +1 binding votes and a majority of positive
> votes.
>
> [+1] This release is ready to send to the Incubator PMC for approval
>
> [-1] This release is not ready because...
>
> This vote is held according to Apache Incubator release policy:
> https://incubator.apache.org/policy/incubation.html#releases
>
> The RC is based on tag v0.7.0-incubating-rc3:
>
> https://github.com/apache/incubator-livy/commit/0cc431a7e9236e313f9904ef121cad5df444d4da
>
> The release files can be found here:
> https://dist.apache.org/repos/dist/dev/incubator/livy/0.7.0-incubating-rc3/
>
> The staged maven artifacts can be found here:
> https://repository.apache.org/content/repositories/orgapachelivy-1012
>
> The list of resolved JIRAs in this release can be found here:
> https://issues.apache.org/jira/projects/LIVY/versions/12345179
>
> Thanks
> Saisai

[Vote][LIVY-718] Support multi-active high availability in Livy

[Yiheng Wang]

Dear Community

I'd like to call a vote on LIVY-718
 "Support
multi-active high availability in Livy".

Currently, Livy only supports single node recovery. This is not sufficient
in some production environments. In our scenario, the Livy server serves
many notebook and JDBC services. We want to make Livy service more
fault-tolerant and scalable.

There're already some proposals in the community for high availability. But
they're not so complete or just for active-standby high availability. So we
propose a multi-active high availability design to achieve the following
goals:

   - One or more servers will serve the client requests at the same time.
   - Sessions are allocated among different servers.
   - When one node crashes, the affected sessions will be moved to other
   active services.

Please find the design doc here
,
which has been reviewed for several weeks.

This vote is open until next Wednesday (Dec. 18).

[] +1: Accept the proposal
[] +0
[] -1: I don't think this is a good idea because ...

Thank you

Yiheng

Re: Support multi-active high availability in Livy

[Yiheng Wang]

Hi Community

We have received some feedback for this design. Here're the open questions
for discussion
1. Session allocation algorithm
We agree to make it pluggable. There're two allocation algorithms:
server-session mapping and consistent hashing. We haven't agreed on which
one is the default.
2. The semantic change of getAllSessions
We agree to not change its semantics. However, we haven't achieved
an agreement on the implementation.

We think the open questions are not a blocker. So can we have this design
doc approved?

Thanks
Yiheng

On Thu, Nov 28, 2019 at 6:37 PM Yiheng Wang  wrote:

> Hi Community
>
> We propose a multi-active high availability design for Livy. We think this
> feature is valuable for many users. Here're the JIRA and the design doc. If
> you're interested, please take a look and comment :-)
>
> JIRA:
> https://issues.apache.org/jira/projects/LIVY/issues/LIVY-718
>
> Design doc:
>
> https://docs.google.com/document/d/1bD3qYZpw14_NuCcSGUOfqQ0pqvSbCQsOLFuZp26Ohjc/edit?usp=sharing
>
> Best regards
> Yiheng
>