Re: Ask opinion regarding 0.6.0 release package

2016-06-20 Thread mina lee 
Moon, having netinst package for the sake of simplicity and flexibility
totally makes sense to me.
If there is no strong objection, I would like to follow your approach for
0.6.0 release.

On Mon, Jun 20, 2016 at 6:23 PM, moon soo Lee  wrote:

> "zeppelin-bin-min" package, I worried about lack of written policy which
> goes in which does not.
> Without policy, yes, we can always vote for list of interpreters. But
> then, we'll need vote everytime we add/remove interpreter, and this
> sounds not good.
>
> Even if it is true that majority of user uses 'spark',
> other users may ask "zeppelin-bin-cassandra-min", "zeppelin-bin-flink-min"
> and so on.
> Once we have 'zeppelin-bin-min' package with 'spark', then there will be
> no good excuse of not having other *min packages.
> And we can end up with a lot of binary packages in each release. which is
> not really optimal.
>
> For this reasons, I believe we'll need a written policy based on community
> consensus to make 'zeppelin-bin-min'.
> But I think making netinst script will be a lot easier and give more
> flexibility to users than making written policy for 'zeppelin-bin-min'.
>
> Anyway, it's really great to hear volunteer some time to help.
> Thanks  Mohit Jaggi.
> Whether we have multiple packages or not, we'll need a lot of help on
> improving release script [1] and verification of release candidates.
>
> Regarding maintainers/contributors of each interpreter(s),
> Spark community recently removed 'maintainer' role from review process [2]
> for some reasons.
> DuyHai Doan, could you give little more details about how your idea
> different from maintainers of Spark?
>
> Thanks,
> moon
>
> [1] https://github.com/apache/zeppelin/blob/master/dev/create_release.sh
> [2]
> https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=34835307&originalVersion=61&revisedVersion=66
>
>
> On Mon, Jun 20, 2016 at 3:10 AM DuyHai Doan  wrote:
>
>> +1 for zeppelin-bin-min release package
>>
>> What I would suggest is that for a specific package of Zeppelin with XXX
>> interpreter(s) built-in is that the maintainers/contributors of each
>> interpreter(s) can help releasing those "custom" builds for the community.
>> Any thought on this idea ?
>>
>> On Mon, Jun 20, 2016 at 10:30 AM, Partridge, Lucas (GE Aviation) <
>> lucas.partri...@ge.com> wrote:
>>
>>> I like the 'zeppelin-bin-netinst’ idea too. Hopefully it would be easy
>>> to configure it to work with a proxy for users behind a corporate firewall.
>>>
>>> Thanks, Lucas.
>>>
>>>
>>>
>>> *From:* Mohit Jaggi [mailto:mohitja...@gmail.com]
>>> *Sent:* 17 June 2016 18:06
>>> *To:* users@zeppelin.apache.org
>>> *Subject:* EXT: Re: Ask opinion regarding 0.6.0 release package
>>>
>>>
>>>
>>> sure…that is possible. one can also make a build from source and
>>> customize as needed. but not having to do that makes things easier. i do
>>> believe that for the vast majority of cases a minimal build with spark (and
>>> possibly other small items like shell, jdbc, python) will be quite
>>> valuable, imho.
>>>
>>> is there a lot of overhead involved in having multiple binaries
>>> available? i am happy to volunteer some time to help with this if needed.
>>>
>>>
>>>
>>> On Jun 17, 2016, at 9:45 PM, moon soo Lee  wrote:
>>>
>>>
>>>
>>> In case of no internet access, how about
>>>
>>>
>>>
>>> a. download 'zeppelin-bin-netinst' and run 'bin/install-interpreter.sh',
>>> and then copy the package to production env.
>>>
>>> b. download 'zeppelin-bin-all' and copy the package to production env.
>>>
>>>
>>>
>>> ?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> moon
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jun 17, 2016 at 9:10 AM Mohit Jaggi 
>>> wrote:
>>>
>>> Many production environments have no internet access. A script like
>>>  this can be useful to some but it should not replace the proposed min
>>> binary.
>>>
>>> Sent from my iPhone
>>>
>>>
>>> On Jun 17, 2016, at 9:20 PM, moon soo Lee  wrote:
>>>
>>> Hi,
>>>
>>>
>>>
>>> Thanks for bringing this discussion.
>>>
>>> it's great idea minimize binary package size.
>>>
>>>
>>>
>>> Can we set a policy to decide which interpreter goes to
>>> 'zeppelin-bin-min', which is not?
>>>
>>>
>>>
>>> One alternative is, instead of making 'zeppelin-bin-min', we can make
>>> 'zeppelin-bin-netinst'.
>>>
>>> We can provide a shell script such as, 'bin/install-interpreter.sh' and
>>> the script will download interpreters and their dependencies from maven
>>> repository and store under /interpreter dir. By
>>> leveraging DependencyResolver[1], i think we can make this feature in
>>> couple of hours.
>>>
>>>
>>>
>>> Only spark interpreter can not be installed in simple way, while it
>>> requires some python and R packages under /interpreter dir and they're not
>>> available on maven repository, so it'll need special treatment, but all
>>> other interpreters can be installed in the simple way.
>>>
>>>
>>>
>>> Then, 'zeppelin-bin-netinst' version can have minimal package size, and
>>> still gives easy way to install al

Re: Ask opinion regarding 0.6.0 release package

2016-06-20 Thread moon soo Lee 
"zeppelin-bin-min" package, I worried about lack of written policy which
goes in which does not.
Without policy, yes, we can always vote for list of interpreters. But
then, we'll
need vote everytime we add/remove interpreter, and this sounds not good.

Even if it is true that majority of user uses 'spark',
other users may ask "zeppelin-bin-cassandra-min", "zeppelin-bin-flink-min"
and so on.
Once we have 'zeppelin-bin-min' package with 'spark', then there will be no
good excuse of not having other *min packages.
And we can end up with a lot of binary packages in each release. which is
not really optimal.

For this reasons, I believe we'll need a written policy based on community
consensus to make 'zeppelin-bin-min'.
But I think making netinst script will be a lot easier and give more
flexibility to users than making written policy for 'zeppelin-bin-min'.

Anyway, it's really great to hear volunteer some time to help.
Thanks  Mohit Jaggi.
Whether we have multiple packages or not, we'll need a lot of help on
improving release script [1] and verification of release candidates.

Regarding maintainers/contributors of each interpreter(s),
Spark community recently removed 'maintainer' role from review process [2]
for some reasons.
DuyHai Doan, could you give little more details about how your idea
different from maintainers of Spark?

Thanks,
moon

[1] https://github.com/apache/zeppelin/blob/master/dev/create_release.sh
[2]
https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=34835307&originalVersion=61&revisedVersion=66


On Mon, Jun 20, 2016 at 3:10 AM DuyHai Doan  wrote:

> +1 for zeppelin-bin-min release package
>
> What I would suggest is that for a specific package of Zeppelin with XXX
> interpreter(s) built-in is that the maintainers/contributors of each
> interpreter(s) can help releasing those "custom" builds for the community.
> Any thought on this idea ?
>
> On Mon, Jun 20, 2016 at 10:30 AM, Partridge, Lucas (GE Aviation) <
> lucas.partri...@ge.com> wrote:
>
>> I like the 'zeppelin-bin-netinst’ idea too. Hopefully it would be easy
>> to configure it to work with a proxy for users behind a corporate firewall.
>>
>> Thanks, Lucas.
>>
>>
>>
>> *From:* Mohit Jaggi [mailto:mohitja...@gmail.com]
>> *Sent:* 17 June 2016 18:06
>> *To:* users@zeppelin.apache.org
>> *Subject:* EXT: Re: Ask opinion regarding 0.6.0 release package
>>
>>
>>
>> sure…that is possible. one can also make a build from source and
>> customize as needed. but not having to do that makes things easier. i do
>> believe that for the vast majority of cases a minimal build with spark (and
>> possibly other small items like shell, jdbc, python) will be quite
>> valuable, imho.
>>
>> is there a lot of overhead involved in having multiple binaries
>> available? i am happy to volunteer some time to help with this if needed.
>>
>>
>>
>> On Jun 17, 2016, at 9:45 PM, moon soo Lee  wrote:
>>
>>
>>
>> In case of no internet access, how about
>>
>>
>>
>> a. download 'zeppelin-bin-netinst' and run 'bin/install-interpreter.sh',
>> and then copy the package to production env.
>>
>> b. download 'zeppelin-bin-all' and copy the package to production env.
>>
>>
>>
>> ?
>>
>>
>>
>> Thanks,
>>
>> moon
>>
>>
>>
>>
>>
>> On Fri, Jun 17, 2016 at 9:10 AM Mohit Jaggi  wrote:
>>
>> Many production environments have no internet access. A script like  this
>> can be useful to some but it should not replace the proposed min binary.
>>
>> Sent from my iPhone
>>
>>
>> On Jun 17, 2016, at 9:20 PM, moon soo Lee  wrote:
>>
>> Hi,
>>
>>
>>
>> Thanks for bringing this discussion.
>>
>> it's great idea minimize binary package size.
>>
>>
>>
>> Can we set a policy to decide which interpreter goes to
>> 'zeppelin-bin-min', which is not?
>>
>>
>>
>> One alternative is, instead of making 'zeppelin-bin-min', we can make
>> 'zeppelin-bin-netinst'.
>>
>> We can provide a shell script such as, 'bin/install-interpreter.sh' and
>> the script will download interpreters and their dependencies from maven
>> repository and store under /interpreter dir. By
>> leveraging DependencyResolver[1], i think we can make this feature in
>> couple of hours.
>>
>>
>>
>> Only spark interpreter can not be installed in simple way, while it
>> requires some python and R packages under /interpreter dir and they're not
>> available on maven repository, so it'll need special treatment, but all
>> other interpreters can be installed in the simple way.
>>
>>
>>
>> Then, 'zeppelin-bin-netinst' version can have minimal package size, and
>> still gives easy way to install all the interpreters.
>>
>> Also 'bin/install-interpreter.sh' will still useful even if we have
>> dynamic interpreter loading feature [2], to build offline package.
>>
>>
>>
>> what do you think?
>>
>>
>>
>> [1]
>> https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/DependencyResolver.java
>>

Improving web performance by reducing number of broadcasts

2016-06-20 Thread Johnny W. 
Hi zeppelin-users,

This is my first email to the top-level mailing list. Congratulations for
graduation!

We are hitting some performance issues when multiple users are connected to
the Zeppelin server. From the stack trace, many of the connections are
blocked on a HashMap, which is locked by
org.apache.zeppelin.socket.NotebookServer.broadcastNote.

Our largest notebook is around 800K, and there are around 10 - 20
connections to the Zeppelin server. I think it should be we are
broadcasting some large amount of data to multiple users, and some slow
connections hang the whole web interface.

Is there anyway to reduce the number of broadcasts to improve the web
performance? It is fine for us to refresh and get updates. I've attached
the full stack trace of this issue as well.

Thanks!

Johnny

Blocking Thread:
--
"qtp1874598090-2478" prio=10 tid=0x7f2fb0003800 nid=0x3373 waiting on
condition [0x7f329ebe9000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x000704e15db0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at
org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:219)
at
org.eclipse.jetty.websocket.common.BlockingWriteCallback$WriteBlocker.block(BlockingWriteCallback.java:83)
at
org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.blockingWrite(WebSocketRemoteEndpoint.java:107)
at
org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.sendString(WebSocketRemoteEndpoint.java:387)
at
org.apache.zeppelin.socket.NotebookSocket.send(NotebookSocket.java:69)
at
org.apache.zeppelin.socket.NotebookServer.broadcast(NotebookServer.java:304)
- locked <0x0007006b6100> (a java.util.HashMap)
at
org.apache.zeppelin.socket.NotebookServer.broadcastNote(NotebookServer.java:384)
at
org.apache.zeppelin.socket.NotebookServer.updateNote(NotebookServer.java:492)
at
org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:181)
at
org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:56)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128)
at
org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
at
org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:122)
at
org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:161)
at
org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:309)
at
org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:214)
at
org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220)
at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:258)
at
org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632)
at
org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480)
at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)

Blocked Thread:
--
"qtp1874598090-2498" prio=10 tid=0x7f306000f800 nid=0x4075 waiting for
monitor entry [0x7f329eae9000]
   java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.zeppelin.socket.NotebookServer.addConnectionToNote(NotebookServer.java:229)
- waiting to lock <0x0007006b6100> (a java.util.HashMap)
at
org.apache.zeppelin.socket.NotebookServer.sendNote(NotebookServer.java:432)
at
org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:145)
at
org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:56)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128)
at
org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
at
org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
at
org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onText

Re: [VIRTUAL] TOMORROW - Meetup for Data Scientists Tuesday 6/21 - 9AM Pacific : Zeppelin meets MADlib & HAWQ

2016-06-20 Thread Greg Chase 
This is a reminder about the virtual meeting tomorrow at 9AM Pacific
discussing running Apache Zeppelin with Apache MADlib and Apache HAWQ.
Join at this URL: 
https://pivotalcommunity.adobeconnect.com/madlib/

On Wed, Jun 15, 2016 at 12:20 PM, Greg Chase  wrote:

> Dear members of the Apache Zeppelin, Apache MADlib, and Apache HAWQ
> communities,
>
> We are hosting a cross-community virtual meeting for data science users
> this next Tuesday, June 21, 9AM Pacific.  No sign up is necessary, just
> join the event here .
> This meeting will also be recorded and posted.
>
> We'll be introducing users of Zeppelin to the capabilities of the MADlib
> SQL machine learning library.  We'll be showing users of MADlib how they
> can visualize their investigations and publish their work using the
> Zeppelin notebook.
>
> Agenda:
> * What is Apache Zeppelin?
> * What is Apache HAWQ (incubating) and Apache MADlib (incubating)? - 5
> min/Frank
> * Demo of Zeppelin and MADlib for data science running on HAWQ
>
> More about the technologies we'll be discussing:
>
> *Apache Zeppelin*  is a web-based notebook
> that enables interactive data analytics. It lets you make beautiful
> data-driven, interactive and collaborative documents with SQL, Scala and
> more.  Apache Zeppelin has plugins to many database engines including
> PostgreSQL, and massively parallel processing engines based off PostgreSQL
> including Apache HAWQ and the Greenplum Database.
>
> *Apache MADlib (incubating) * is a
> big data machine learning library in SQL for data scientists. It operates
> on data locally in PostgreSQL-compatible database engines. MADlib is
> optimized for parallel processing platforms such as Apache HAWQ and the
> Greenplum Database.
>
> *Apache HAWQ (incubating) * is an
> Apache Hadoop-Native SQL query engine that operates directly on data in a
> Hadoop cluster. It provides the highest degree of SQL completeness of any
> SQL on Hadoop engine. It scales elastically, is parallel processing, and
> integrates with MADlib and Zeppelin.
>
> Speakers will be:
>
> *Moon soo Lee -  CTO, NFLabs*
> LeeMoonSoo is a creator for Apache Zeppelin (incubating) and a Co-Founder,
> CTO at NFLabs.
>
> *Frank McQuillan - Director of Product Management, Pivotal Software*
> Frank McQuillan focuses on analytics and machine learning for large data
> sets.
>
> *Rahul Iyer - R&D Manager, Pivotal Software*
> Rahul Iyer leads the development team at Pivotal Software that contributes
> to the Apache MADlib project.
>
> See you this next Tuesday, June 21, 9AM Pacific.  No sign up is necessary,
> just join the event here
> .
>
>
>

Re: Ask opinion regarding 0.6.0 release package

2016-06-20 Thread DuyHai Doan 
+1 for zeppelin-bin-min release package

What I would suggest is that for a specific package of Zeppelin with XXX
interpreter(s) built-in is that the maintainers/contributors of each
interpreter(s) can help releasing those "custom" builds for the community.
Any thought on this idea ?

On Mon, Jun 20, 2016 at 10:30 AM, Partridge, Lucas (GE Aviation) <
lucas.partri...@ge.com> wrote:

> I like the 'zeppelin-bin-netinst’ idea too. Hopefully it would be easy to
> configure it to work with a proxy for users behind a corporate firewall.
>
> Thanks, Lucas.
>
>
>
> *From:* Mohit Jaggi [mailto:mohitja...@gmail.com]
> *Sent:* 17 June 2016 18:06
> *To:* users@zeppelin.apache.org
> *Subject:* EXT: Re: Ask opinion regarding 0.6.0 release package
>
>
>
> sure…that is possible. one can also make a build from source and customize
> as needed. but not having to do that makes things easier. i do believe that
> for the vast majority of cases a minimal build with spark (and possibly
> other small items like shell, jdbc, python) will be quite valuable, imho.
>
> is there a lot of overhead involved in having multiple binaries available?
> i am happy to volunteer some time to help with this if needed.
>
>
>
> On Jun 17, 2016, at 9:45 PM, moon soo Lee  wrote:
>
>
>
> In case of no internet access, how about
>
>
>
> a. download 'zeppelin-bin-netinst' and run 'bin/install-interpreter.sh',
> and then copy the package to production env.
>
> b. download 'zeppelin-bin-all' and copy the package to production env.
>
>
>
> ?
>
>
>
> Thanks,
>
> moon
>
>
>
>
>
> On Fri, Jun 17, 2016 at 9:10 AM Mohit Jaggi  wrote:
>
> Many production environments have no internet access. A script like  this
> can be useful to some but it should not replace the proposed min binary.
>
> Sent from my iPhone
>
>
> On Jun 17, 2016, at 9:20 PM, moon soo Lee  wrote:
>
> Hi,
>
>
>
> Thanks for bringing this discussion.
>
> it's great idea minimize binary package size.
>
>
>
> Can we set a policy to decide which interpreter goes to
> 'zeppelin-bin-min', which is not?
>
>
>
> One alternative is, instead of making 'zeppelin-bin-min', we can make
> 'zeppelin-bin-netinst'.
>
> We can provide a shell script such as, 'bin/install-interpreter.sh' and
> the script will download interpreters and their dependencies from maven
> repository and store under /interpreter dir. By
> leveraging DependencyResolver[1], i think we can make this feature in
> couple of hours.
>
>
>
> Only spark interpreter can not be installed in simple way, while it
> requires some python and R packages under /interpreter dir and they're not
> available on maven repository, so it'll need special treatment, but all
> other interpreters can be installed in the simple way.
>
>
>
> Then, 'zeppelin-bin-netinst' version can have minimal package size, and
> still gives easy way to install all the interpreters.
>
> Also 'bin/install-interpreter.sh' will still useful even if we have
> dynamic interpreter loading feature [2], to build offline package.
>
>
>
> what do you think?
>
>
>
> [1]
> https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/DependencyResolver.java
> 
>
> [2] https://issues.apache.org/jira/browse/ZEPPELIN-598
> 
>
>
>
>
>
> On Fri, Jun 17, 2016 at 1:02 AM mina lee  wrote:
>
> Hi all!
>
>
>
> Zeppelin just started release process. Prior to creating release candidate
> I want to ask users' opinion about how you want it to be packaged.
>
>
>
> For the last release(0.5.6), we have released one binary package which
> includes all interpreters.
>
> The concern with providing one type of binary package is that package size
> will be quite big(~600MB).
>
> So I am planning to provide two binary packages:
>
>   - zeppelin-0.6.0-bin-all.tgz (includes all interpreters)
>
>   - zeppelin-0.6.0-bin-min.tgz (includes only most used interpreters)
>
>
>
> I am thinking about putting *spark(pyspark, sparkr, sql), python, jdbc,
> shell, markdown, angular* in minimized package.
>
> Could you give your opinion on whether these sets are enough, or some of
> them are ok to be excluded?
>
>
>
> Community's opinion will be helpful to make decision not only for 0.6.0
> but also for 0.7.0 release since we are planning to provide only minimized
> package from 0.7.0 release. From the 0.7.0 version, interpreters thos

Ask opinion regarding 0.6.0 release package

2016-06-20 Thread Partridge, Lucas (GE Aviation) 
I like the 'zeppelin-bin-netinst’ idea too. Hopefully it would be easy to 
configure it to work with a proxy for users behind a corporate firewall.
Thanks, Lucas.

From: Mohit Jaggi [mailto:mohitja...@gmail.com]
Sent: 17 June 2016 18:06
To: users@zeppelin.apache.org
Subject: EXT: Re: Ask opinion regarding 0.6.0 release package

sure…that is possible. one can also make a build from source and customize as 
needed. but not having to do that makes things easier. i do believe that for 
the vast majority of cases a minimal build with spark (and possibly other small 
items like shell, jdbc, python) will be quite valuable, imho.
is there a lot of overhead involved in having multiple binaries available? i am 
happy to volunteer some time to help with this if needed.

On Jun 17, 2016, at 9:45 PM, moon soo Lee 
mailto:m...@apache.org>> wrote:

In case of no internet access, how about

a. download 'zeppelin-bin-netinst' and run 'bin/install-interpreter.sh', and 
then copy the package to production env.
b. download 'zeppelin-bin-all' and copy the package to production env.

?

Thanks,
moon


On Fri, Jun 17, 2016 at 9:10 AM Mohit Jaggi 
mailto:mohitja...@gmail.com>> wrote:
Many production environments have no internet access. A script like  this can 
be useful to some but it should not replace the proposed min binary.

Sent from my iPhone

On Jun 17, 2016, at 9:20 PM, moon soo Lee 
mailto:m...@apache.org>> wrote:
Hi,

Thanks for bringing this discussion.
it's great idea minimize binary package size.

Can we set a policy to decide which interpreter goes to 'zeppelin-bin-min', 
which is not?

One alternative is, instead of making 'zeppelin-bin-min', we can make 
'zeppelin-bin-netinst'.
We can provide a shell script such as, 'bin/install-interpreter.sh' and the 
script will download interpreters and their dependencies from maven repository 
and store under /interpreter dir. By leveraging DependencyResolver[1], i think 
we can make this feature in couple of hours.

Only spark interpreter can not be installed in simple way, while it requires 
some python and R packages under /interpreter dir and they're not available on 
maven repository, so it'll need special treatment, but all other interpreters 
can be installed in the simple way.

Then, 'zeppelin-bin-netinst' version can have minimal package size, and still 
gives easy way to install all the interpreters.
Also 'bin/install-interpreter.sh' will still useful even if we have dynamic 
interpreter loading feature [2], to build offline package.

what do you think?

[1] 
https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/DependencyResolver.java
[2] 
https://issues.apache.org/jira/browse/ZEPPELIN-598


On Fri, Jun 17, 2016 at 1:02 AM mina lee 
mailto:mina...@apache.org>> wrote:
Hi all!

Zeppelin just started release process. Prior to creating release candidate I 
want to ask users' opinion about how you want it to be packaged.

For the last release(0.5.6), we have released one binary package which includes 
all interpreters.
The concern with providing one type of binary package is that package size will 
be quite big(~600MB).
So I am planning to provide two binary packages:
  - zeppelin-0.6.0-bin-all.tgz (includes all interpreters)
  - zeppelin-0.6.0-bin-min.tgz (includes only most used interpreters)

I am thinking about putting spark(pyspark, sparkr, sql), python, jdbc, shell, 
markdown, angular in minimized package.
Could you give your opinion on whether these sets are enough, or some of them 
are ok to be excluded?

Community's opinion will be helpful to make decision not only for 0.6.0 but 
also for 0.7.0 release since we are planning to provide only minimized package 
from 0.7.0 release. From the 0.7.0 version, interpreters those are not included 
in binary package will be able to use dynamic interpreter feature [1] which is 
in progress under [2].

Thanks,
Mina

[1] 
http://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/manual/dynamicinterpreterload.html

Re: How to remove uglify operation from zeppelin-web

2016-06-20 Thread Corneau Damien 
If you are adding UI libraries through angular interpreter, you won't have
more logs without the uglify.
If you are trying to add libraries in the source code, I would advise you
to use the dev mode that refresh from source code and doesn't have uglify.
If you really want to build without uglify, you can remove uglify, usemin
and htmlmin here:
https://github.com/apache/zeppelin/blob/master/zeppelin-web/Gruntfile.js#L452
However since the files would already be concat, I'm not sure it would
solve your problem.

On Mon, Jun 20, 2016 at 4:19 PM, Vikash Kumar 
wrote:

> Hi,
>
> Becuase when we are adding our own UI libraries, it is giving some
> errors.Which we are not able to trace. I initialized uglify option as
> false.But still its compressing. We need to remove uglify for debug
> precess.
>
>
>
>
> *Thanks & Regards*
>
> *Vikash Kumar*
>
> --
> *From:* Corneau Damien 
> *Sent:* 18 June 2016 07:27:46
> *To:* users@zeppelin.apache.org
> *Subject:* Re: How to remove uglify operation from zeppelin-web
>
>
> This is happening in the grunt.js
>
> But why would you want to remove it? This is a basic production build rule.
> On Jun 17, 2016 22:27, "Vikash Kumar"  wrote:
>
>> Hi all,
>>
>> How to remove uglify operation from zeppelin-web that I can remove the
>> functionality of compression.
>>
>>
>>
>> *Thanks & Regards*
>>
>> *Vikash Kumar*
>>
>

Re: How to remove uglify operation from zeppelin-web

2016-06-20 Thread Vikash Kumar 
Hi,

Becuase when we are adding our own UI libraries, it is giving some errors.Which 
we are not able to trace. I initialized uglify option as false.But still its 
compressing. We need to remove uglify for debug precess.


Thanks & Regards
Vikash Kumar


From: Corneau Damien 
Sent: 18 June 2016 07:27:46
To: users@zeppelin.apache.org
Subject: Re: How to remove uglify operation from zeppelin-web


This is happening in the grunt.js

But why would you want to remove it? This is a basic production build rule.

On Jun 17, 2016 22:27, "Vikash Kumar" 
mailto:vikash.ku...@resilinc.com>> wrote:
Hi all,
How to remove uglify operation from zeppelin-web that I can remove the 
functionality of compression.

Thanks & Regards
Vikash Kumar