Moon, having netinst package for the sake of simplicity and flexibility totally makes sense to me. If there is no strong objection, I would like to follow your approach for 0.6.0 release.
On Mon, Jun 20, 2016 at 6:23 PM, moon soo Lee <m...@apache.org> wrote: > "zeppelin-bin-min" package, I worried about lack of written policy which > goes in which does not. > Without policy, yes, we can always vote for list of interpreters. But > then, we'll need vote everytime we add/remove interpreter, and this > sounds not good. > > Even if it is true that majority of user uses 'spark', > other users may ask "zeppelin-bin-cassandra-min", "zeppelin-bin-flink-min" > and so on. > Once we have 'zeppelin-bin-min' package with 'spark', then there will be > no good excuse of not having other *min packages. > And we can end up with a lot of binary packages in each release. which is > not really optimal. > > For this reasons, I believe we'll need a written policy based on community > consensus to make 'zeppelin-bin-min'. > But I think making netinst script will be a lot easier and give more > flexibility to users than making written policy for 'zeppelin-bin-min'. > > Anyway, it's really great to hear volunteer some time to help. > Thanks Mohit Jaggi. > Whether we have multiple packages or not, we'll need a lot of help on > improving release script [1] and verification of release candidates. > > Regarding maintainers/contributors of each interpreter(s), > Spark community recently removed 'maintainer' role from review process [2] > for some reasons. > DuyHai Doan, could you give little more details about how your idea > different from maintainers of Spark? > > Thanks, > moon > > [1] https://github.com/apache/zeppelin/blob/master/dev/create_release.sh > [2] > https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=34835307&originalVersion=61&revisedVersion=66 > > > On Mon, Jun 20, 2016 at 3:10 AM DuyHai Doan <doanduy...@gmail.com> wrote: > >> +1 for zeppelin-bin-min release package >> >> What I would suggest is that for a specific package of Zeppelin with XXX >> interpreter(s) built-in is that the maintainers/contributors of each >> interpreter(s) can help releasing those "custom" builds for the community. >> Any thought on this idea ? >> >> On Mon, Jun 20, 2016 at 10:30 AM, Partridge, Lucas (GE Aviation) < >> lucas.partri...@ge.com> wrote: >> >>> I like the 'zeppelin-bin-netinst’ idea too. Hopefully it would be easy >>> to configure it to work with a proxy for users behind a corporate firewall. >>> >>> Thanks, Lucas. >>> >>> >>> >>> *From:* Mohit Jaggi [mailto:mohitja...@gmail.com] >>> *Sent:* 17 June 2016 18:06 >>> *To:* users@zeppelin.apache.org >>> *Subject:* EXT: Re: Ask opinion regarding 0.6.0 release package >>> >>> >>> >>> sure…that is possible. one can also make a build from source and >>> customize as needed. but not having to do that makes things easier. i do >>> believe that for the vast majority of cases a minimal build with spark (and >>> possibly other small items like shell, jdbc, python) will be quite >>> valuable, imho. >>> >>> is there a lot of overhead involved in having multiple binaries >>> available? i am happy to volunteer some time to help with this if needed. >>> >>> >>> >>> On Jun 17, 2016, at 9:45 PM, moon soo Lee <m...@apache.org> wrote: >>> >>> >>> >>> In case of no internet access, how about >>> >>> >>> >>> a. download 'zeppelin-bin-netinst' and run 'bin/install-interpreter.sh', >>> and then copy the package to production env. >>> >>> b. download 'zeppelin-bin-all' and copy the package to production env. >>> >>> >>> >>> ? >>> >>> >>> >>> Thanks, >>> >>> moon >>> >>> >>> >>> >>> >>> On Fri, Jun 17, 2016 at 9:10 AM Mohit Jaggi <mohitja...@gmail.com> >>> wrote: >>> >>> Many production environments have no internet access. A script like >>> this can be useful to some but it should not replace the proposed min >>> binary. >>> >>> Sent from my iPhone >>> >>> >>> On Jun 17, 2016, at 9:20 PM, moon soo Lee <m...@apache.org> wrote: >>> >>> Hi, >>> >>> >>> >>> Thanks for bringing this discussion. >>> >>> it's great idea minimize binary package size. >>> >>> >>> >>> Can we set a policy to decide which interpreter goes to >>> 'zeppelin-bin-min', which is not? >>> >>> >>> >>> One alternative is, instead of making 'zeppelin-bin-min', we can make >>> 'zeppelin-bin-netinst'. >>> >>> We can provide a shell script such as, 'bin/install-interpreter.sh' and >>> the script will download interpreters and their dependencies from maven >>> repository and store under /interpreter dir. By >>> leveraging DependencyResolver[1], i think we can make this feature in >>> couple of hours. >>> >>> >>> >>> Only spark interpreter can not be installed in simple way, while it >>> requires some python and R packages under /interpreter dir and they're not >>> available on maven repository, so it'll need special treatment, but all >>> other interpreters can be installed in the simple way. >>> >>> >>> >>> Then, 'zeppelin-bin-netinst' version can have minimal package size, and >>> still gives easy way to install all the interpreters. >>> >>> Also 'bin/install-interpreter.sh' will still useful even if we have >>> dynamic interpreter loading feature [2], to build offline package. >>> >>> >>> >>> what do you think? >>> >>> >>> >>> [1] >>> https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/DependencyResolver.java >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_zeppelin_blob_master_zeppelin-2Dinterpreter_src_main_java_org_apache_zeppelin_dep_DependencyResolver.java&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=b48EeMu0glDkXuGn72ZTy8ZteEiVBzmpbTqELmhgsRc&e=> >>> >>> [2] https://issues.apache.org/jira/browse/ZEPPELIN-598 >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_ZEPPELIN-2D598&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=MK9lpcZjSIlgFO0CVk6kMWB1bCPqpWK_0qhSjOQ5FzA&e=> >>> >>> >>> >>> >>> >>> On Fri, Jun 17, 2016 at 1:02 AM mina lee <mina...@apache.org> wrote: >>> >>> Hi all! >>> >>> >>> >>> Zeppelin just started release process. Prior to creating release >>> candidate I want to ask users' opinion about how you want it to be packaged. >>> >>> >>> >>> For the last release(0.5.6), we have released one binary package which >>> includes all interpreters. >>> >>> The concern with providing one type of binary package is that package >>> size will be quite big(~600MB). >>> >>> So I am planning to provide two binary packages: >>> >>> - zeppelin-0.6.0-bin-all.tgz (includes all interpreters) >>> >>> - zeppelin-0.6.0-bin-min.tgz (includes only most used interpreters) >>> >>> >>> >>> I am thinking about putting *spark(pyspark, sparkr, sql), python, jdbc, >>> shell, markdown, angular* in minimized package. >>> >>> Could you give your opinion on whether these sets are enough, or some of >>> them are ok to be excluded? >>> >>> >>> >>> Community's opinion will be helpful to make decision not only for 0.6.0 >>> but also for 0.7.0 release since we are planning to provide only minimized >>> package from 0.7.0 release. From the 0.7.0 version, interpreters those are >>> not included in binary package will be able to use dynamic interpreter >>> feature [1] which is in progress under [2]. >>> >>> >>> >>> Thanks, >>> >>> Mina >>> >>> >>> >>> [1] >>> http://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/manual/dynamicinterpreterload.html >>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__zeppelin.apache.org_docs_0.6.0-2DSNAPSHOT_manual_dynamicinterpreterload.html&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=4zHncvKGMfOlq-dTmD3m23Rv0jjkaqwWEnowkaJSHks&e=> >>> >>> [2] https://github.com/apache/zeppelin/pull/908 >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_zeppelin_pull_908&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=TB3EaiWKtliKYXmXWHJLyZK4Kti6Ev97GVBJFfhCcVw&e=> >>> >>> >>> >> >>
