Makes sense. Thanks Jeff!
On Wed, Apr 5, 2017 at 11:18 PM Jeff Zhang <zjf...@gmail.com> wrote: > > HI Ruslan, > > ZEPPELIN-1595 is trying to make ZeppelinContext extensible and available > to other interpreters (Currently ZeppelinContext is only available in spark > interpreter) . As you said you want to make {var1} as z.get(var1), that > means you want to use ZeppelinContext in shell interpreter which require > ZEPPELIN-1595. > > > > Ruslan Dautkhanov <dautkha...@gmail.com>于2017年4月6日周四 上午5:46写道: > > Hi Jeff, > > I looked at the PR for ZEPPELIN-1595 > <https://issues.apache.org/jira/browse/ZEPPELIN-1595> > > It does not look it covers %sh interpreter. > > %sh and %sql interpters are somewhat unique as they don't have access to > Zeppelin API (please correct me if I'm wrong) > > So what https://issues.apache.org/jira/browse/ZEPPELIN-1967 is > suggesting, is to introduce syntax > that is used in Jupyter notebooks, i.e. {*var1*} will be implied as > z.get('var1'), for example: > > %sh > /path/to/script --param8={*var1*} --param9={*var2*} > > where var1 and var2 would be implied to be fetched as z.get('var1') > and z.get('var2') respectively. > > > Or similarly for %sql : > > %sql > create table dwh.table_{*year*} stores as parquet > as > select * from spark_df1 where year = {*year*} > > We miss a lot global variables for %sql and %sh so that a Zeppelin note > can be used as a single parametrized > orchestration for a whole workflow. > > > Thank you, > Ruslan Dautkhanov > > On Wed, Apr 5, 2017 at 12:01 AM, Jeff Zhang <zjf...@gmail.com> wrote: > > > Hi Ruslan, > > Regarding 'make zeppelinContext available in shell interpreter', you may > want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595 > > > Ruslan Dautkhanov <dautkha...@gmail.com>于2017年4月3日周一 下午12:05写道: > > That's exciting to see plans for 0.8.0 on the horizon. > > Here's my top list for 0.8 : > > - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle > timeout" > This is a most-wanted feature by our Zeppelin admins. It was mentioned at > least once on this email chain. > > - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z > variables to Shell Interpreter" > We had several of our users asking about this functionality. %sh and some > other interpreters can't be > parametrized by global variables. ZEPPELIN-1967 is one way of how this > can be solved. > > - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory > references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected" > Less of a critical compared to the above two, but it could complement > the multi-tenancy feature very well. > > > Best regards, > Ruslan Dautkhanov > > On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <felixcheun...@hotmail.com> > wrote: > > +1 with latest/stable. > > > > > ------------------------------ > *From:* moon soo Lee <m...@apache.org> > *Sent:* Tuesday, March 21, 2017 8:41:58 AM > *To:* users@zeppelin.apache.org > *Cc:* d...@zeppelin.apache.org > > *Subject:* Re: Roadmap for 0.8.0 > > > And if i suggest simplest way for us to set quality expectation to user, > which will be labeling release in download page. > > Currently releases are divided into 2 categories in download page. 'Latest > release' and 'Old releases'. I think we can treat 'Latest' as unstable and > add one more category 'Stable release'. > > For example, once 0.8.0 is released, > > Latest release : 0.8.0 > Stable release : 0.7.1 > Old release : 0.6.2, 0.6.1 .... > > Once we feel confident about the stability of latest release, we can just > change label from latest to stable in the download page. (and previous > stable goes to old releases) > We can even include formal vote for moving release from 'latest' to > 'stable' in our release process, if it is necessary. > > Thanks, > moon > > On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org> wrote: > > Yes, having longer RC period will help. > > But if i recall 0.7.0 release, although 21 people participated verifying > through 4 RC for 15days, it wasn't enough to catch all critical problems > during the release process. After the release, we've got much more number > of bug reports, in next few days. > > Basically, verifying RC is limited to people who subscribe mailing list + > willing to contribute time to verify RC, which is much smaller number of > people who download release from download page. So having longer RC period > will definitely help and i think we should do, but I think it's still not > enough to make sure the quality, considering past history. > > AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. > ASF release process defines how to release source code, but it does not > really restrict what kind of 'version' the project should have releases. > For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. > > Thanks, > moon > > [1] http://spark.apache.org/news/spark-2.0.0-preview.html > > On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote: > > I agree that it will help prolong RC period and use it actually. And also > we need code freeze for the new features and spend time to stabilize RC. > > On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com> > wrote: > > +1 on quality and stabilization. > > I'm not sure if releasing as preview or calling it unstable fits with the > ASF release process though. > > Other projects have code freeze, RC (and longer RC iteration time) etc. - > do we think those will help improve quality when the release is finally cut? > > > _____________________________ > From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> > Sent: Monday, March 20, 2017 6:13 PM > Subject: Re: Roadmap for 0.8.0 > > To: <users@zeppelin.apache.org>, dev <d...@zeppelin.apache.org> > > > > > > Strongly +1 for adding system test for different interpreter modes and > focus on bug fixing than new features. I do heard from some users complain > about the bugs of zeppelin major release. A stabilized release is very > necessary for community. > > > > > Best Regard, > Jeff Zhang > > > From: moon soo Lee <m...@apache.org<mailto:m...@apache.org > <m...@apache.org>>> > > Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org > <users@zeppelin.apache.org>>" <users@zeppelin.apache.org< > mailto:users@zeppelin.apache.org <users@zeppelin.apache.org>>> > > > Date: Tuesday, March 21, 2017 at 4:10 AM > > To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org > <users@zeppelin.apache.org>>" <users@zeppelin.apache.org< > mailto:users@zeppelin.apache.org <users@zeppelin.apache.org>>>, dev < > d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org > <d...@zeppelin.apache.org>>> > > > Subject: Re: Roadmap for 0.8.0 > > Great to see discussion for 0.8.0. > List of features for 0.8.0 looks really good. > > Interpreter factory refactoring > Interpreter layer supports various behavior depends on combination of > PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for > each combination as a first step. > Otherwise, any pullrequest will silently break one of behavior at any time > no matter we refactor or not. And fixing and testing this behavior is so > hard. > Once we have complete test cases, not only guarantee the behavior but also > make refactoring much easier. > > > 0.8.0 release > I'd like to suggest improvements on how we release a new version. > > In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 > months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) > > I think the same thing will happen again with 0.8.0, while we're going to > make lots of changes and add many new features. > After we released 0.8.0, while 'Stabilizing' the new release, user who > tried the new release may get wrong impression of the quality. Which is > very bad and we already repeated the mistake in 0.6.0 and 0.7.0. > > So from 0.8.0 release, I'd suggest we improve way we release new version > to give user proper expectation. I think there're several ways of doing it. > > 1. Release 0.8.0-preview officially and then release 0.8.0. > 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a > 'stable' release in the download page. Once 0.8.x release becomes stable > enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases. > > > After 0.8.0, > Since Zeppelin projects starts, project went through some major milestone, > like > > - project gets first users and first contributor > - project went into Apache Incubator > - project became TLP. > > And I think it's time to think about hitting another major milestone. > > Considering features we already have, features we're planning on 0.8, wide > adoption of Zeppelin in the industry, I think it's time to focus on make > project more mature and make a 1.0 release. Which i think big milestone for > the project. > > After 0.8.0 release, I suggest we more focus on bug fixes, stability > improvement, optimizing user experience than adding new features. And with > subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about > the quality, release it as a 1.0.0 instead of 0.8.x. > > Once we have 1.0.0 released, then I think we can make larger, experimental > changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x > branch. > > > Thanks, > moon > > On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheun...@hotmail.com< > mailto:felixcheun...@hotmail.com <felixcheun...@hotmail.com>>> wrote: > There are several pending visualization improvements/PRs that would be > very good to get them in as well. > > > ________________________________ > From: Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com > <jongy...@gmail.com>>> > Sent: Sunday, March 19, 2017 9:03:24 PM > To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org > <users@zeppelin.apache.org>> > Subject: Roadmap for 0.8.0 > > Hi dev & users, > > Recently, community submits very new features for Apache Zeppelin. I think > it's very positive signals to improve Apache Zeppelin and its community. > But in another aspect, we should focus on what the next release includes. I > think we need to summarize and prioritize them. Here is what I know: > > * Cluster management > * Admin feature > * Replace some context to separate users > * Helium online > > Feel free to talk if you want to add more things. I think we need to > choose which features will be included in 0.8.0, too. > > Regards, > Jongyoul Lee > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net > > >