Re: Next release

Hi, I just merged the Spark 3 support yesterday, I would wait for most tests to stabilize this feature. In the meanwhile, you can build the master branch yourself to try this latest feature. Thanks Saisai Aliaksandr Sasnouskikh 于2020年7月2日周四 下午10:50写道： > Hi, thanks for notifying about the

Re: Review request

Sorry for the delay, I will take a review. Thanks Saisai Shingo Furuyama 于2020年3月24日周二 下午12:45写道： > Hello, > > I am Shingo Furuyama, working at Yahoo! JAPAN. > > We are trying to use livy 0.7.0-incubating and spark 2.4.5 with YARN > HDP2.6.4 and have sent a few tiny patches related to the

Re: About building and releasing livy java/scala doc

Just figured out there's a doc about it. Thanks! Best regards, Saisai Saisai Shao 于2020年2月2日周日 下午7:15写道： > Hi Team, > > Do you know how to build and release java/scala doc for new Livy (0.7.0) > release. I think document thing is the only remaining thing for 0.7.0 > relea

About building and releasing livy java/scala doc

Hi Team, Do you know how to build and release java/scala doc for new Livy (0.7.0) release. I think document thing is the only remaining thing for 0.7.0 release. Please help to guide, would be appreciated if there's doc about it. Thanks Saisai

Re: python-api CI failure

Thanks for the fix. Would you please create a separate PR about this issue? It should be enough to create a "minor" GH PR directly without JIRA issue. Best regards, Saisai Wing Yew Poon 于2020年1月31日周五 上午6:33写道： > I updated python-api/setup.py to include > > 'mock~=3.0.5', > > in

Re: [VOTE] Release Livy 0.7.0 based on RC4

> and > > looks good. > > > > +1 > > > > On Wed, Jan 8, 2020 at 8:07 PM Yiheng Wang wrote: > > > > > Check the release files. Looks good. > > > > > > +1 > > > > > > On Tue, Jan 7, 2020 at 9:25 PM Saisai Shao &

Re: .NET for Apache Spark interactive interpreter support

nInterpreter, SparkRInterpreter, and > SQLInterpreter classes get instantiated and used. > > > Regards, > -Steve > > On Sun, Jan 12, 2020 at 5:45 PM Saisai Shao > wrote: > > > Is this just a wrapper of Livy REST API, or it could support Livy Job > API? > >

Re: .NET for Apache Spark interactive interpreter support

Is this just a wrapper of Livy REST API, or it could support Livy Job API? >From my opinion, if it is just a wrapper of REST API, then it would be better to maintain out of Livy, since REST API is language independent, if we're going to have all the languages support in Livy, then it is hard to

Re: Livy metrics

Currently we don't have concrete plan on this, it would be great if you could contribute on this. Thanks Saisai Shanyu Zhao 于2020年1月10日周五 上午8:44写道： > Hi, > > Are there ongoing effort to add Livy metrics? I saw that dropwizard is > added as dependency in pom.xml but didn't see it get used

Re: [VOTE] Release Livy 0.7.0 based on RC4

My +1 of course. Saisai Shao 于2020年1月7日周二 下午9:25写道： > This vote is for releasing Livy 0.7.0 based on RC4. > > This RC mainly fixes the license issue. > > The vote will be open until Sunday Jan 12, 23:59 UTC and > will pass with a minimum of 3 +1 binding votes and a majority o

[VOTE] Release Livy 0.7.0 based on RC4

This vote is for releasing Livy 0.7.0 based on RC4. This RC mainly fixes the license issue. The vote will be open until Sunday Jan 12, 23:59 UTC and will pass with a minimum of 3 +1 binding votes and a majority of positive votes. [+1] This release is ready to send to the Incubator PMC for

Re: Fwd: [VOTE] Release Apache Livy 0.7.0 (incubating) based on RC3

nologies > -- > *E-mail:* *ajboz...@us.ibm.com* > *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > [image: Inactive hide details for Saisai Shao ---01

Fwd: [VOTE] Release Apache Livy 0.7.0 (incubating) based on RC3

Does anyone knows how to handle license issue for fonts like below? Thanks Saisai -- Forwarded message - 发件人： Justin Mclean Date: 2020年1月3日周五 上午9:59 Subject: Re: [VOTE] Release Apache Livy 0.7.0 (incubating) based on RC3 To: Hi, +1 (binding). There’s a license issue that

Re: Podling Livy Report Reminder - January 2020

I will work on this. Thanks Saisai Justin Mclean 于2020年1月1日周三 下午4:06写道： > Hi, > Just a friendly reminder the report is due today is anyone working on it? > Thanks, > Justin >

Re: [VOTE] Release Livy 0.7.0 based on RC3

19 11:58 PM > To: dev@livy.incubator.apache.org > Subject: Re: [VOTE] Release Livy 0.7.0 based on RC3 > > +1 > > вт, 24 дек. 2019 г. в 8:11, Saisai Shao : > > > The date is overdue, but we still didn't receive enough votes, let me > > extend the due date to Dec 29, 23:59 UTC.

Re: [VOTE] Release Livy 0.7.0 based on RC3

The date is overdue, but we still didn't receive enough votes, let me extend the due date to Dec 29, 23:59 UTC. Please help to vote. Best regards, Saisai Saisai Shao 于2019年12月20日周五 下午4:02写道： > +1 myself. > > Yiheng Wang 于2019年12月19日周四 上午11:41写道： > >> Check the release

Re: [VOTE] Release Livy 0.7.0 based on RC3

all the jiras in the commits. > > > > > > Thanks > > RunzhiWang > > > > > > > > --原始邮件-- > > 发件人:"Saisai Shao" > 发送时间:2019年12月18日(星期三) 中午1:58 > > 收件人:"dev" > > > 主题:[VOTE] Rele

Re: [VOTE] Release Livy 0.7.0 based on RC2

> livy-thriftserver/ > livy-thriftserver-session/ > minicluster-dependencies-parent/ > minicluster-dependencies_2.10/ > minicluster-dependencies_2.11/ > > ** release files* > I verify the sha512 and asc file. Both look good. > > Thanks > Yiheng > > On

Re: [Vote][LIVY-718] Support multi-active high availability in Livy

+1 rei sivan 于2019年12月12日周四 下午2:09写道： > +1 > > On Thu, Dec 12, 2019, 05:30 Yiheng Wang wrote: > > > Dear Community > > > > I'd like to call a vote on LIVY-718 > > "Support > > multi-active high availability in Livy". > > > >

Re: Support multi-active high availability in Livy

giorno mer 11 dic 2019 alle ore 14:50 Saisai Shao < > sai.sai.s...@gmail.com> > ha scritto: > > > I think it is feasible to move to next step. Let's create subtasks to > track > > all the works. > > > > Thanks > > Saisai > > > > Yiheng Wang

Re: Support multi-active high availability in Livy

I think it is feasible to move to next step. Let's create subtasks to track all the works. Thanks Saisai Yiheng Wang 于2019年12月10日周二 下午3:18写道： > Hi Community > > We have received some feedback for this design. Here're the open questions > for discussion > 1. Session allocation algorithm > We

Re: Plan to cut new branch and prepare to release 0.7

w ha scritto: > Regarding the thrift server GA, > should we consider supporting recovery in > thrift server? I open a JIRA > for it > https://issues.apache.org/jira/browse/LIVY-696 > > > welcome for > discussion. > > > 原始邮件 > 发件人: Saisai Shao > 收件人

Re: Plan to cut new branch and prepare to release 0.7

, > Marco > > Il giorno gio 10 ott 2019 alle ore 06:08 Jeff Zhang ha > scritto: > > > Make sense to me > > > > Saisai Shao 于2019年10月10日周四 下午12:04写道： > > > > > Hi all, > > > > > > I'm planning to cut a new branch and prepare for

Plan to cut new branch and prepare to release 0.7

Hi all, I'm planning to cut a new branch and prepare for the 0.7 release, which did lots of improvements to thrift module, I think we could make thrift module GA as for this release. Currently I'm waiting for this JIRAs to be merged before cutting the new branch. [LIVY-356][SERVER]Add LDAP

Re: Possible missing mentor(s)

nd busy -- life > > happens, I get it. May I ask what are you folks doing to scale PR review > > and merging? Are you adding new committers? Do you feel that 2-year old > > open PRs is where you wish to be and is the right way to grow a > community? > > > > -s

Re: [DISCUSS] Getting rid of old stuff

+1， To support J8, Spark 2.2+ might be the only option. I'm not sure if those vendors will continue to support Spark 2.2-, but IMHO for the community release, I think moving forward would be a better choice. Thanks Saisai Jeff Zhang 于2018年9月14日周五 上午9:39写道： > +1 > > > Alex Bozarth

Re: Podling report this month

Updated. Please review and sign off. Thanks Saisai Shao 于2018年7月9日周一 上午8:27写道： > Sorry I forgot about this, will draft this today. > > Thanks > Saisai > > Jean-Baptiste Onofré 于2018年7月8日周日 下午1:17写道： > >> Hi guys, >> >> we have to report this month:

Re: Podling report this month

Sorry I forgot about this, will draft this today. Thanks Saisai Jean-Baptiste Onofré 于2018年7月8日周日 下午1:17写道： > Hi guys, > > we have to report this month: > > https://wiki.apache.org/incubator/July2018 > > Did someone already start a report ? I can bootstrap one if you want. > > Thanks, >

Re: A new link request to my project and one question

, but it > would be better to add finer grained permissions. > > On Mon, Jun 25, 2018 at 6:30 PM, Saisai Shao > wrote: > > Yes, has a configuration "livy.superusers". Here in this case, the sql > > server user should be added as a superuser, who can impersonate other >

Re: A new link request to my project and one question

Yes, has a configuration "livy.superusers". Here in this case, the sql server user should be added as a superuser, who can impersonate other different users. Marcelo Vanzin 于2018年6月26日周二 上午9:12写道： > You're talking about another service between the user and the application. > > In that case a

Re: A new link request to my project and one question

Sure, I will merge the website code, thanks! For proxyUser thing, I think there's no particular reason not adding it, maybe we just forgot to add the proxyUser support. It would be better if you could create a JIRA to track this issue. If you're familiar with Livy code, you can also submit a PR

About new API set to create interactive and batch session

Hi Team, In our current API design to create interactive / batch session, we assume end user could upload jars, pyFiles and related dependencies to HDFS before creating the session, and we use one POST request to create session. But usually end user may not have the permission to access the HDFS

Re: Query on creating multiple livy sessions in parallel

This might be a BUG. If possible can you please create a JIRA to track this issue. Thanks! Best, Jerry 2018-03-19 20:18 GMT+08:00 Rao, Abhishek (Nokia - IN/Bangalore) < abhishek@nokia.com>: > Hi, > > We're trying to create multiple livy sessions in parallel and then using > them. But when

Fwd: [apache/incubator-livy] One of your dependencies may have a security vulnerability

Hi Alex, Would you please check again? Thanks! Best regards -- Forwarded message -- From: Apache Security Team <secur...@apache.org> Date: 2018-03-06 16:39 GMT+08:00 Subject: Re: [apache/incubator-livy] One of your dependencies may have a security vulnerability To: Saisa

Re: Podling Report Reminder - February 2018

Great, thanks Alex! Best regards, Jerry 2018-02-06 7:56 GMT+08:00 Alex Bozarth : > I've updated the report if a mentor could take a look and sign off (or > give feedback). > > > *Alex Bozarth* > Software Engineer > Spark Technology Center > -- >

Re: [VOTE] Livy 0.5.0-incubating (RC2)

asc, sha512 and md5 LGTM. Also checked the package, seems everything is landed. Alex, did you also publish Livy artifacts to staging repo? 2018-01-26 8:03 GMT+08:00 Alex Bozarth : > That works for me, voting will be open an extra day for a total of 96 > hours ending on

Fwd: [apache/incubator-livy] One of your dependencies may have a security vulnerability

Hi Alex, Is it due to your recent changes to add ruby file? Thanks Jerry -- Forwarded message -- From: Greg Stein Date: 2018-01-24 6:14 GMT+08:00 Subject: Fwd: [apache/incubator-livy] One of your dependencies may have a security vulnerability To:

Re: [DRAFT] Incubator PMC Board Report - January 2018

Sorry I was on paternity leave last two weeks, didn't have time to check it. Maybe you can also help on this next time. Thanks Jerry 2018-01-09 4:32 GMT+08:00 Alex Bozarth : > Saw that we didn't report this month, any reason why, or was everyone just > on holiday? > > >

Re: How to get the test logs from travis

Zeppelin does, they kind cat a lot of the logs and > related files which then get appended to the build results. > > https://github.com/apache/zeppelin/blob/master/.travis.yml > > On Wed, Sep 20, 2017 at 8:22 PM, Saisai Shao <sai.sai.s...@gmail.com> > wrote: > > &

How to get the test logs from travis

Hi Team, Currently it is quite painful to figure out the cause of failure on travis test. Do you know how to get test logs from travis, is there a way supported by travis to either upload environment to some places, or log on to travis to dig the files. Previously we uploaded logs to azure when

Re: user defined sessionId / URI for Livy sessions

I see. So based on this, we should manage a data structure in Livy Server to keep all the live sessions' name. Also regarding to session recovery, we should persist this structure to the reliable storage and recover after restart. I'm not pretty sure if it is a good feature or not. First because

Re: user defined sessionId / URI for Livy sessions

If we're using session name, how do we guarantee the uniqueness of this name? Thanks Jerry On Tue, Sep 12, 2017 at 4:51 AM, Alex Bozarth wrote: > I would agree with Marcelo's comment the JIRA that this isn't a good > feature for livy, but I'll take a look at your impl if

[ANNOUNCE] Apache Livy 0.4.0-incubating released

The Apache Livy team is proud to announce Apache Livy version 0.4.0-incubating. This is the first Livy release after entering the Apache Incubator. Livy is web service that exposes a REST interface for managing long running Apache Spark contexts in your cluster. With Livy, new applications can

Re: The process of Livy 0.4.0-incubating release

-- > *E-mail:* *ajboz...@us.ibm.com* <ajboz...@us.ibm.com> > *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > [image: Inactive hide details for Saisai Shao ---09/0

Re: The process of Livy 0.4.0-incubating release

ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > [image: Inactive hide details for Jeff Zhang ---08/30/2017 05:53:15 PM---I > think we'd better to announce after the artifacts are publis]Jeff Zhang > ---08/30/2017 05:5

Re: The process of Livy 0.4.0-incubating release

.ibm.com* <ajboz...@us.ibm.com> > *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > [image: Inactive hide details for Saisai Shao ---08/30/2017 12:05:24 > AM---

Re: The process of Livy 0.4.0-incubating release

will write a doc as well as release script for the next release. Sorry about it. Best regards, Jerry On Wed, Aug 30, 2017 at 2:10 PM, Saisai Shao <sai.sai.s...@gmail.com> wrote: > Hi Luciano, > > I'm trying to push maven artifacts to maven staging repository at > repository.apache.

Re: The process of Livy 0.4.0-incubating release

is profile for Livy? Also how can I create this profile? Thanks Jerry On Mon, Aug 28, 2017 at 3:39 PM, Luciano Resende <luckbr1...@gmail.com> wrote: > On Sun, Aug 27, 2017 at 11:53 PM, Saisai Shao <sai.sai.s...@gmail.com> > wrote: > > > Hi mentors, > > > > Wou

Re: Update ASF GitHub Bot Settings?

b.com/ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > [image: Inactive hide details for Saisai Shao ---08/28/2017 06:53:28 > PM---Hi Alex, Do you want to achieve what Spark did in the JIRA? A]Saisai > Shao ---08/28/2017 06:53:28

Re: Update ASF GitHub Bot Settings?

Hi Alex, Do you want to achieve what Spark did in the JIRA? AFAIK Spark uses a script to sync between github and jira, it doesn't enable gihub robot to mirror everything from github. Thanks Jerry On Tue, Aug 29, 2017 at 5:00 AM, Alex Bozarth wrote: > Opened an INFRA JIRA

The process of Livy 0.4.0-incubating release

Hi mentors, Would you please guide us on how to release the Livy 0.4.0-incubating, including artifact publish. Also regarding push artifacts to repo, are we inclining to change to Maven central repo, or we still use Cloudera repo, any suggestion? Besides, for Python package release, do we need

Please vote for Apache Livy 0.4.0-incubating release

Hi teams, Would you please help to vote for the Apache Livy release in general incubator mail list. Thanks a lot. http://mail-archives.apache.org/mod_mbox/incubator-general/201708.mbox/%3CCANvfmP8sAoofvRcs9AT5S-VW4LfiWeGfRXP2rhvEP4zyTag%2BYQ%40mail.gmail.com%3E Thanks

Re: resolve the scalability problem caused by app monitoring in livy with an actor-based design

Nan, I think Meisam already had a PR about this this, maybe you can discuss with him on the github based on the proposed code. Sorry I didn't follow the long discussion thread, but I think Paypal's solution sounds simpler. On Wed, Aug 23, 2017 at 12:07 AM, Nan Zhu wrote:

Re: [VOTE] Release Livy 0.4.0-incubating based on Livy 0.4.0 RC2

OK, sure, I will remove RC1 from the directory. Thanks Jerry On Tue, Aug 22, 2017 at 7:24 PM, John D. Ament wrote: > Hi, > > Looking at your release, it's confusing what we are voting on. If RC2 is > under vote, please remove RC1 from this directory. > > John > > On

Help to verify Apache Livy 0.4.0-incubating release

Hi all, We're under progress to make a first Apache release of Livy (0.4.0-incubating), we really hope you could verify the RC2[1] release (binary and source) locally and return us the feedbacks. We will call for an incubation vote next week if everything is fine. Thanks a lot for your help.

Re: To release a first Apache version Livy

cense file (see below as an example) > > http://svn.apache.org/repos/asf/tuscany/sca-java-2.x/ > trunk/distribution/all/src/main/release/bin/LICENSE > > > > But note that, the source release artifact, which should not include any > > dependency jars, should have th

Re: To release a first Apache version Livy

what is in the > root of git today. > > > > > > On Mon, Aug 7, 2017 at 2:16 AM, Saisai Shao <sai.sai.s...@gmail.com> > wrote: > > > Hi Team, > > > > Today I cut a RC1 release based on branch-0.4, here is the link ( > > https://github.c

Re: To release a first Apache version Livy

an Apache release to transition to. > > > Given that its still a new project, a shorter cadence would help make bug > fix releases available. > > > Btw, does anyone know whats holding up the Apache jira process? If not, I > can follow up on that. > > > Bikas >

Re: To release a first Apache version Livy

eleases come out when major > new functionality has added. My preference is a time based release cadence > because it provides regular bug and security related fixes available in a > released form for end users. > > > Thanks! > > Bikas > > > >

Re: To release a first Apache version Livy

Hi Team, We're planning to release a first Apache version of Livy. Would you please guide us the process of Apache incubating release, is there any doc to follow. Thanks a lot! Best regards Saisai On Tue, Aug 1, 2017 at 11:34 AM, Saisai Shao <sai.sai.s...@gmail.com> wrote: > OK, than

Re: To release a first Apache version Livy

.@us.ibm.com* <ajboz...@us.ibm.com> > *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > [image: Inactive hide details for Saisai Shao ---07/31/2017 07:12:11 > PM---Hi Ale

Re: To release a first Apache version Livy

;ajboz...@us.ibm.com> > *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > Jeff Zhang ---07/28/2017 01:12:24 AM---+1 for making the firs

Re: Need input: Location of Livy Documentation

I'm in favor of store the docs in main repo, while put seldom changed frameworks in website repo. I think you already mentioned the reason here (as the Documentation grows and changes from version to version it would be better to store it in the more regularly updated repo). What I saw is that

Re: When Livy JIRA system will be ready?

gt; Spark Technology Center > -- > *E-mail:* *ajboz...@us.ibm.com* <ajboz...@us.ibm.com> > *GitHub: **github.com/ajbozarth* <https://github.com/ajbozarth> > > > 505 Howard Street > San Francisco, CA 94105 > United States > > > > [image: Inactive hide details

Re: Commit style best practices, was Re: incubator-livy-website git commit: Fix bug in merge_livy_pr.py because incubator-livy-website repo has no branch it's name started with "branch-"

Sorry about it. Basically because I don't know whether JIRA is necessary for incubator-livy-website repo, also where to create JIRA. On Thu, Jul 20, 2017 at 1:20 PM, Alex Bozarth wrote: > +1. I think we've been using close to this format, but with "LIVY-XXX." > instead of