Hi Jeff, I looked at the PR for ZEPPELIN-1595 <https://issues.apache.org/jira/browse/ZEPPELIN-1595>
It does not look it covers %sh interpreter. %sh and %sql interpters are somewhat unique as they don't have access to Zeppelin API (please correct me if I'm wrong) So what https://issues.apache.org/jira/browse/ZEPPELIN-1967 is suggesting, is to introduce syntax that is used in Jupyter notebooks, i.e. {*var1*} will be implied as z.get('var1'), for example: %sh /path/to/script --param8={*var1*} --param9={*var2*} where var1 and var2 would be implied to be fetched as z.get('var1') and z.get('var2') respectively. Or similarly for %sql : %sql create table dwh.table_{*year*} stores as parquet as select * from spark_df1 where year = {*year*} We miss a lot global variables for %sql and %sh so that a Zeppelin note can be used as a single parametrized orchestration for a whole workflow. Thank you, Ruslan Dautkhanov On Wed, Apr 5, 2017 at 12:01 AM, Jeff Zhang <zjf...@gmail.com> wrote: > > Hi Ruslan, > > Regarding 'make zeppelinContext available in shell interpreter', you may > want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595 > > > Ruslan Dautkhanov <dautkha...@gmail.com>于2017年4月3日周一 下午12:05写道: > >> That's exciting to see plans for 0.8.0 on the horizon. >> >> Here's my top list for 0.8 : >> >> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle >> timeout" >> This is a most-wanted feature by our Zeppelin admins. It was mentioned >> at least once on this email chain. >> >> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z >> variables to Shell Interpreter" >> We had several of our users asking about this functionality. %sh and >> some other interpreters can't be >> parametrized by global variables. ZEPPELIN-1967 is one way of how this >> can be solved. >> >> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory >> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected" >> Less of a critical compared to the above two, but it could complement >> the multi-tenancy feature very well. >> >> >> Best regards, >> Ruslan Dautkhanov >> >> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <felixcheun...@hotmail.com >> > wrote: >> >> +1 with latest/stable. >> >> >> >> >> ------------------------------ >> *From:* moon soo Lee <m...@apache.org> >> *Sent:* Tuesday, March 21, 2017 8:41:58 AM >> *To:* users@zeppelin.apache.org >> *Cc:* d...@zeppelin.apache.org >> >> *Subject:* Re: Roadmap for 0.8.0 >> >> >> And if i suggest simplest way for us to set quality expectation to user, >> which will be labeling release in download page. >> >> Currently releases are divided into 2 categories in download page. >> 'Latest release' and 'Old releases'. I think we can treat 'Latest' as >> unstable and add one more category 'Stable release'. >> >> For example, once 0.8.0 is released, >> >> Latest release : 0.8.0 >> Stable release : 0.7.1 >> Old release : 0.6.2, 0.6.1 .... >> >> Once we feel confident about the stability of latest release, we can just >> change label from latest to stable in the download page. (and previous >> stable goes to old releases) >> We can even include formal vote for moving release from 'latest' to >> 'stable' in our release process, if it is necessary. >> >> Thanks, >> moon >> >> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org> wrote: >> >> Yes, having longer RC period will help. >> >> But if i recall 0.7.0 release, although 21 people participated verifying >> through 4 RC for 15days, it wasn't enough to catch all critical problems >> during the release process. After the release, we've got much more number >> of bug reports, in next few days. >> >> Basically, verifying RC is limited to people who subscribe mailing list + >> willing to contribute time to verify RC, which is much smaller number of >> people who download release from download page. So having longer RC period >> will definitely help and i think we should do, but I think it's still not >> enough to make sure the quality, considering past history. >> >> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. >> ASF release process defines how to release source code, but it does not >> really restrict what kind of 'version' the project should have releases. >> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. >> >> Thanks, >> moon >> >> [1] http://spark.apache.org/news/spark-2.0.0-preview.html >> >> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote: >> >> I agree that it will help prolong RC period and use it actually. And also >> we need code freeze for the new features and spend time to stabilize RC. >> >> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com> >> wrote: >> >> +1 on quality and stabilization. >> >> I'm not sure if releasing as preview or calling it unstable fits with the >> ASF release process though. >> >> Other projects have code freeze, RC (and longer RC iteration time) etc. - >> do we think those will help improve quality when the release is finally cut? >> >> >> _____________________________ >> From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> >> Sent: Monday, March 20, 2017 6:13 PM >> Subject: Re: Roadmap for 0.8.0 >> >> To: <users@zeppelin.apache.org>, dev <d...@zeppelin.apache.org> >> >> >> >> >> >> Strongly +1 for adding system test for different interpreter modes and >> focus on bug fixing than new features. I do heard from some users complain >> about the bugs of zeppelin major release. A stabilized release is very >> necessary for community. >> >> >> >> >> Best Regard, >> Jeff Zhang >> >> >> From: moon soo Lee <m...@apache.org<mailto:m...@apache.org >> <m...@apache.org>>> >> >> Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org >> <users@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai >> lto:users@zeppelin.apache.org <users@zeppelin.apache.org>>> >> >> >> Date: Tuesday, March 21, 2017 at 4:10 AM >> >> To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org >> <users@zeppelin.apache.org>>" <users@zeppelin.apache.org<mai >> lto:users@zeppelin.apache.org <users@zeppelin.apache.org>>>, dev < >> d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org >> <d...@zeppelin.apache.org>>> >> >> >> Subject: Re: Roadmap for 0.8.0 >> >> Great to see discussion for 0.8.0. >> List of features for 0.8.0 looks really good. >> >> Interpreter factory refactoring >> Interpreter layer supports various behavior depends on combination of >> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for >> each combination as a first step. >> Otherwise, any pullrequest will silently break one of behavior at any >> time no matter we refactor or not. And fixing and testing this behavior is >> so hard. >> Once we have complete test cases, not only guarantee the behavior but >> also make refactoring much easier. >> >> >> 0.8.0 release >> I'd like to suggest improvements on how we release a new version. >> >> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 >> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) >> >> I think the same thing will happen again with 0.8.0, while we're going to >> make lots of changes and add many new features. >> After we released 0.8.0, while 'Stabilizing' the new release, user who >> tried the new release may get wrong impression of the quality. Which is >> very bad and we already repeated the mistake in 0.6.0 and 0.7.0. >> >> So from 0.8.0 release, I'd suggest we improve way we release new version >> to give user proper expectation. I think there're several ways of doing it. >> >> 1. Release 0.8.0-preview officially and then release 0.8.0. >> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a >> 'stable' release in the download page. Once 0.8.x release becomes stable >> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases. >> >> >> After 0.8.0, >> Since Zeppelin projects starts, project went through some major >> milestone, like >> >> - project gets first users and first contributor >> - project went into Apache Incubator >> - project became TLP. >> >> And I think it's time to think about hitting another major milestone. >> >> Considering features we already have, features we're planning on 0.8, >> wide adoption of Zeppelin in the industry, I think it's time to focus on >> make project more mature and make a 1.0 release. Which i think big >> milestone for the project. >> >> After 0.8.0 release, I suggest we more focus on bug fixes, stability >> improvement, optimizing user experience than adding new features. And with >> subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about >> the quality, release it as a 1.0.0 instead of 0.8.x. >> >> Once we have 1.0.0 released, then I think we can make larger, >> experimental changes on 2.0.0 branch aggressively, while we keep >> maintaining 1.0.x branch. >> >> >> Thanks, >> moon >> >> On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheun...@hotmail.com< >> mailto:felixcheun...@hotmail.com <felixcheun...@hotmail.com>>> wrote: >> There are several pending visualization improvements/PRs that would be >> very good to get them in as well. >> >> >> ________________________________ >> From: Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com >> <jongy...@gmail.com>>> >> Sent: Sunday, March 19, 2017 9:03:24 PM >> To: dev; users@zeppelin.apache.org<mailto:users@zeppelin.apache.org >> <users@zeppelin.apache.org>> >> Subject: Roadmap for 0.8.0 >> >> Hi dev & users, >> >> Recently, community submits very new features for Apache Zeppelin. I >> think it's very positive signals to improve Apache Zeppelin and its >> community. But in another aspect, we should focus on what the next release >> includes. I think we need to summarize and prioritize them. Here is what I >> know: >> >> * Cluster management >> * Admin feature >> * Replace some context to separate users >> * Helium online >> >> Feel free to talk if you want to add more things. I think we need to >> choose which features will be included in 0.8.0, too. >> >> Regards, >> Jongyoul Lee >> >> -- >> 이종열, Jongyoul Lee, 李宗烈 >> http://madeng.net >> >> >> >> >> >> -- >> 이종열, Jongyoul Lee, 李宗烈 >> http://madeng.net >> >>