Re: IDE

[Alexander Bezzubov]

Hi,

right, maven integration in eclipse works, or you can also use old-school
way (with some optimization flags to save time) of generating eclipse
projects:

mvn eclipse:eclipse -Dcheckstyle.skip=true -Drat.skip=true
-Denforcer.skip=true -DskipTests -Pr -pl \!zeppelin-web

and then just "Import existing projects" from IDE.

--
Alex

On Wed, May 4, 2016 at 1:34 PM, Paul Bustios Belizario 
wrote:

> Hello Vijay,
>
> I use Eclipse and works well.
> I import the project with the option: File > Import > Maven > Existing
> Maven projects.
>
> Regards,
> Paul
>
> On Tue, May 3, 2016 at 2:44 AM Vijay Srinivasaraghavan <
> vijikar...@yahoo.com> wrote:
>
>> I think the compile error occurs only if we select "Search for projects
>> recursively" option in the "Import Project" screen (IntelliJ).
>>
>> Regards
>> Vijay
>>
>>
>> On Monday, May 2, 2016 9:19 PM, anish singh  wrote:
>>
>>
>> Hello Vijay,
>>
>> You are free to use the IDE of your choice to build/develop zeppelin, as
>> mentioned on the contributions guide page [0].
>>
>> [0].
>> https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md
>>
>> Thanks,
>> Anish.
>>
>> On Tue, May 3, 2016 at 9:42 AM, Vijay Srinivasaraghavan <
>> vijikar...@yahoo.com> wrote:
>>
>> Hello,
>>
>>
>>
>> What is the supported IDE to develop/build Zeppelin?
>>
>> I have tried IntelliJ (on Windows) but I am seeing lot of compile errors
>> though the maven projects are imported properly.
>>
>> I am able to build the project on the command line using "mvn" build
>> flags.
>>
>> Regards
>> Vijay
>>
>>
>>
>>
>>

Re: Zeppelin server hangs due to websocket deadlock bug in Jetty 8

[Alexander Bezzubov]

aiting to lock <0x0006c3b1fba8> (a java.util.HashMap)
> at
> org.apache.zeppelin.notebook.NoteInterpreterLoader.getInterpreterSettings(NoteInterpreterLoader.java:60)
> at
> org.apache.zeppelin.notebook.NoteInterpreterLoader.get(NoteInterpreterLoader.java:77)
> at org.apache.zeppelin.notebook.Note.runAll(Note.java:409)
> - locked <0x0006c34a12c0> (a java.util.ArrayList)
> at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebook.java:419)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
> - locked <0x0006c3abd630> (a java.lang.Object)
>
> Found 1 deadlock.
>
> On Thu, Apr 7, 2016 at 6:46 PM, Alexander Bezzubov <b...@apache.org> wrote:
>
>> Hi,
>>
>> thank you Eric, upgrading Jetty sounds like a great idea!
>>
>> Prasad, I think braodcastAll and synchronization of
>> note<->client_connection is used by default to achieve the ability to
>> collaborate over analysis with multiple people at same Note in realtime -
>> to notify all other clients who have this Note open about the changes that
>> you did in your browser tab (like in 2 different tabs you can see).
>>
>> I believe it might be possible to replace a map with concurrent
>> implementation to avoid excessive synchronization though, as we did in [1]
>> before. If same behaviour persist after upgrading to Jetty 9, could you
>> pelase create an separate issue for that and I will be happy help and look
>> more into it.
>>
>> Thanks!
>>
>>  1. https://issues.apache.org/jira/browse/ZEPPELIN-312
>>
>> --
>> Alex
>>
>>
>> On Fri, Apr 8, 2016 at 1:28 AM, Prasad Wagle <prasadwa...@gmail.com>
>> wrote:
>>
>>> Thanks Eric! I created
>>> https://issues.apache.org/jira/browse/ZEPPELIN-798 - Migrate to Jetty
>>> version 9 that has fix for websocket deadlock bug causing Zeppelin server
>>> hangs. This is pretty important for us so please let me know how I can help.
>>>
>>> For now, I have made some changes to reduce websocket communications and
>>> probability of hangs:
>>>
>>>- For the LIST_NOTES operation, I use broadcastNoteList(conn) that
>>>sends note list to the current connection instead of using broadcastAll.
>>>What is the reason for using broadcastAll?
>>>- I removed synchronized (noteSocketMap) from broadcast so that one
>>>bad socket does not hang the server. Do you think this can cause serious
>>>problems?
>>>
>>>
>>> On Thu, Apr 7, 2016 at 3:06 AM, Eric Charles <e...@apache.org> wrote:
>>>
>>>> On 07/04/16 07:18, Prasad Wagle wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We experienced three Zeppelin server hangs today. I have included one
>>>>> of
>>>>> the stack traces below. It is similar to the stack trace in a websocket
>>>>> deadlock bug in Jetty 8. From the bug report
>>>>> <https://bugs.eclipse.org/bugs/show_bug.cgi?id=389645>:
>>>>>
>>>>> However, Jetty 9 has already refactored the low level read/write on
>>>>> a socket heavily to compensate for websocket, spdy, and http/2
>>>>> Marking this as WONTFIX for Jetty 7/8
>>>>> Use Jetty 9
>>>>>
>>>>>
>>>>> Is there a workaround? Has anyone tried using Jetty 9 in Zeppelin? What
>>>>> is the effort involved?
>>>>>
>>>>
>>>>
>>>> I have upgraded the source code to Jetty 9 which implies a few
>>>> different constructs.
>>>>
>>>> Could you open a JIRA? I will then submit a PRo
>>>>
>>>>
>>>>> Thanks,
>>>>> Prasad
>>>>>
>>>>>
>>>>> *Stack trace*
>>>>>
>>>>>
>>>>> "pool-1-thread-10" #141 prio=5 os_prio=0 tid=0x01513000
>>>>> nid=0x6749 in Object.wait() [0x7fdab6ff4000]
>>>>> java.lang.Thread.State: TIMED_WAITING (on object monitor)
>>>>>  at java.lang.Object.wait(Native Method)
>>>>>  at
>>>>>
>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.blockWritable(SelectChannelEndPoint.java:494)
>>>>>  - locked <0x0006c50d9b48> (a
>>>>> org.eclipse.jetty.io.nio.SelectChannel

Re: Zeppelin server hangs due to websocket deadlock bug in Jetty 8

[Alexander Bezzubov]

Hi,

thank you Eric, upgrading Jetty sounds like a great idea!

Prasad, I think braodcastAll and synchronization of
note<->client_connection is used by default to achieve the ability to
collaborate over analysis with multiple people at same Note in realtime -
to notify all other clients who have this Note open about the changes that
you did in your browser tab (like in 2 different tabs you can see).

I believe it might be possible to replace a map with concurrent
implementation to avoid excessive synchronization though, as we did in [1]
before. If same behaviour persist after upgrading to Jetty 9, could you
pelase create an separate issue for that and I will be happy help and look
more into it.

Thanks!

 1. https://issues.apache.org/jira/browse/ZEPPELIN-312

--
Alex


On Fri, Apr 8, 2016 at 1:28 AM, Prasad Wagle  wrote:

> Thanks Eric! I created https://issues.apache.org/jira/browse/ZEPPELIN-798
> - Migrate to Jetty version 9 that has fix for websocket deadlock bug
> causing Zeppelin server hangs. This is pretty important for us so please
> let me know how I can help.
>
> For now, I have made some changes to reduce websocket communications and
> probability of hangs:
>
>- For the LIST_NOTES operation, I use broadcastNoteList(conn) that
>sends note list to the current connection instead of using broadcastAll.
>What is the reason for using broadcastAll?
>- I removed synchronized (noteSocketMap) from broadcast so that one
>bad socket does not hang the server. Do you think this can cause serious
>problems?
>
>
> On Thu, Apr 7, 2016 at 3:06 AM, Eric Charles  wrote:
>
>> On 07/04/16 07:18, Prasad Wagle wrote:
>>
>>> Hi,
>>>
>>> We experienced three Zeppelin server hangs today. I have included one of
>>> the stack traces below. It is similar to the stack trace in a websocket
>>> deadlock bug in Jetty 8. From the bug report
>>> :
>>>
>>> However, Jetty 9 has already refactored the low level read/write on
>>> a socket heavily to compensate for websocket, spdy, and http/2
>>> Marking this as WONTFIX for Jetty 7/8
>>> Use Jetty 9
>>>
>>>
>>> Is there a workaround? Has anyone tried using Jetty 9 in Zeppelin? What
>>> is the effort involved?
>>>
>>
>>
>> I have upgraded the source code to Jetty 9 which implies a few different
>> constructs.
>>
>> Could you open a JIRA? I will then submit a PRo
>>
>>
>>> Thanks,
>>> Prasad
>>>
>>>
>>> *Stack trace*
>>>
>>>
>>> "pool-1-thread-10" #141 prio=5 os_prio=0 tid=0x01513000
>>> nid=0x6749 in Object.wait() [0x7fdab6ff4000]
>>> java.lang.Thread.State: TIMED_WAITING (on object monitor)
>>>  at java.lang.Object.wait(Native Method)
>>>  at
>>>
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.blockWritable(SelectChannelEndPoint.java:494)
>>>  - locked <0x0006c50d9b48> (a
>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint)
>>>  at
>>>
>>> org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.blockWritable(SslConnection.java:723)
>>>  at
>>>
>>> org.eclipse.jetty.websocket.WebSocketGeneratorRFC6455.flush(WebSocketGeneratorRFC6455.java:248)
>>>  at
>>>
>>> org.eclipse.jetty.websocket.WebSocketGeneratorRFC6455.addFrame(WebSocketGeneratorRFC6455.java:114)
>>>  at
>>>
>>> org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameConnection.sendMessage(WebSocketConnectionRFC6455.java:439)
>>>  at
>>> org.apache.zeppelin.socket.NotebookSocket.send(NotebookSocket.java:89)
>>>  at
>>>
>>> org.apache.zeppelin.socket.NotebookServer.broadcast(NotebookServer.java:286)
>>>  - locked <0x0006c3a1cd08> (a java.util.HashMap)
>>>  at
>>>
>>> org.apache.zeppelin.socket.NotebookServer.broadcastNote(NotebookServer.java:370)
>>>  at
>>>
>>> org.apache.zeppelin.socket.NotebookServer$ParagraphJobListener.afterStatusChange(NotebookServer.java:945)
>>>  at org.apache.zeppelin.scheduler.Job.setStatus(Job.java:143)
>>>  at
>>>
>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.afterStatusChange(RemoteScheduler.java:379)
>>>  at
>>>
>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:261)
>>>  - locked <0x0006c5885178> (a
>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller)
>>>  at
>>>
>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:335)
>>>  at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>  at
>>>
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>  at
>>>
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>  at
>>>
>>> 

Re: Unable to start Zeppelin

[Alexander Bezzubov]

Hi Ankur,

Zeppelin has pluggable Notebook storage implementation
,
configured i.e though `conf/zeppelin-env.sh` using 'export
ZEPPELIN_NOTEBOOK_STORAGE=
"org.apache.zeppelin.notebook.repo.VFSNotebookRepo"' (this is a default, so
you should see somthing liek `Empty ZEPPELIN_NOTEBOOK_STORAGE conf
parameter, using default ` in the logs).

In your case it looks like this somehow is not configured properly, so
could you please check those files and see if that is the case? You can
always try adding a default one.

Hope this helps!

--
Alex

On Sat, Feb 20, 2016 at 7:20 PM, Ankur Jain  wrote:

> Hello Team,
>
>
>
> I am trying to start Zeppelin, but getting below error….
>
> Can you guide me how to resolve it?
>
>
>
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
>
> at
> org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:113)
>
> *Caused by: java.io.IOException: Requested storage index 0 isn't
> initialized, repository count is 0*
>
> at
> org.apache.zeppelin.notebook.repo.NotebookRepoSync.getRepo(NotebookRepoSync.java:228)
>
> at
> org.apache.zeppelin.notebook.repo.NotebookRepoSync.list(NotebookRepoSync.java:118)
>
> at
> org.apache.zeppelin.notebook.Notebook.loadAllNotes(Notebook.java:391)
>
> at
> org.apache.zeppelin.notebook.Notebook.(Notebook.java:108)
>
> at
> org.apache.zeppelin.server.ZeppelinServer.(ZeppelinServer.java:87)
>
>
>
>
>
> Thanks
>
> Ankur
> Information transmitted by this e-mail is proprietary to YASH Technologies
> and/ or its Customers and is intended for use only by the individual or
> entity to which it is addressed, and may contain information that is
> privileged, confidential or exempt from disclosure under applicable law. If
> you are not the intended recipient or it appears that this mail has been
> forwarded to you without proper authority, you are notified that any use or
> dissemination of this information in any manner is strictly prohibited. In
> such cases, please notify us immediately at i...@yash.com and delete this
> mail from your records.
>

Re: zeppelin multi user mode?

[Alexander Bezzubov]

Benjamin,
z-manager consists of 2 independant applications - installer and
multitenancy.

You can use only the second one that Hyung Sung pointed out with any
spark/zeppelin version.

If you have further questions, please do not hesitate to ask at
z-mana...@googlegroups.com https://groups.google.com/forum/#!forum/z-manager

On Thu, Feb 4, 2016, 15:13 Benjamin Kim  wrote:

> I forgot to mention that I don’t see Spark 1.6 in the list of versions
> when installing z-manager.
>
>
> On Feb 3, 2016, at 10:08 PM, Corneau Damien  wrote:
>
> @Benjamin,
> We do support version 1.6 of Spark, see:
> https://github.com/apache/incubator-zeppelin#spark-interpreter
>
> On Wed, Feb 3, 2016 at 9:47 PM, Benjamin Kim  wrote:
>
>> I see that the latest version of Spark supported is 1.4.1. When will the
>> latest versions of Spark be supported?
>>
>> Thanks,
>> Ben
>>
>>
>> On Feb 3, 2016, at 7:54 PM, Hyung Sung Shim  wrote:
>>
>> Hello yunfeng.
>>
>> You can also refer to
>> https://github.com/NFLabs/z-manager/tree/master/multitenancy.
>>
>> Thanks.
>>
>> 2016-02-04 3:56 GMT+09:00 Christopher Matta :
>>
>>> I have had luck with a single Zepplin installation and  config
>>> directories in each user home directory. That way each user gets their own
>>> instance and will not interfere with each other.
>>>
>>> You can start the Zepplin server with a config flag pointing to the
>>> config directory. Simply copy the config dir that comes with Zepplin to
>>> ~/.zeppelin and edit the zeppelin-site.xml to change default port for each
>>> user. Start like this:
>>> ./zeppelin.sh --config ~/.zeppelin start
>>>
>>>
>>> On Wednesday, February 3, 2016, Lin, Yunfeng 
>>> wrote:
>>>
 Hi guys,



 We are planning to use zeppelin for PROD for data scientists. One
 feature we desperately need is multi user mode.



 Currently, zeppelin is great for single user use. However, since
 zeppelin spark context are shared among all users in one zeppelin server,
 it is not very suitable when there are multiple users on the same zeppelin
 server since they are going to interfere with each other in one spark
 context.



 How do you guys address this need? Thanks.



>>>
>>>
>>> --
>>> Chris Matta
>>> cma...@mapr.com
>>> 215-701-3146
>>>
>>>
>>
>>
>
>

Re: R Interpreter

[Alexander Bezzubov]

Hi Ankur,

community is working on accepting https://github.com/elbamos/Zeppelin-With-R
contribution as a part of Zeppelin codebase before the next release.

--
Alex

On Tue, Feb 9, 2016, 06:48 DuyHai Doan  wrote:

> Look at the discussion on the second PR (#702) about this subject of
> having 2 PR for R
>
> On Mon, Feb 8, 2016 at 11:19 AM, Ankur Jain  wrote:
>
>> Hello Team,
>>
>>
>>
>> I see 2 R interpreter available. Are both same or different? Which one to
>> prefer?
>>
>>
>>
>> https://github.com/elbamos/Zeppelin-With-R/tree/rinterpreter
>>
>> https://github.com/apache/incubator-zeppelin/pull/208
>>
>>
>>
>>
>>
>> https://github.com/datalayer/zeppelin-R
>>
>> https://github.com/apache/incubator-zeppelin/pull/702
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Ankur
>> Information transmitted by this e-mail is proprietary to YASH
>> Technologies and/ or its Customers and is intended for use only by the
>> individual or entity to which it is addressed, and may contain information
>> that is privileged, confidential or exempt from disclosure under applicable
>> law. If you are not the intended recipient or it appears that this mail has
>> been forwarded to you without proper authority, you are notified that any
>> use or dissemination of this information in any manner is strictly
>> prohibited. In such cases, please notify us immediately at i...@yash.com
>> and delete this mail from your records.
>>
>
>

Re: Incoming conference for Zeppelin

[Alexander Bezzubov]

Great stuff, thank you for sharing!

On Thu, Feb 11, 2016 at 7:58 AM, DuyHai Doan  wrote:

> Update done, I have submitted the talk to other conferences too (Devoxx
> UK, Devoxx France, GeeCon Poland ...) I'll update and notify as soon as I
> receive confirmation that the talk is accepted
>
> On Wed, Feb 10, 2016 at 11:18 PM, moon soo Lee  wrote:
>
>> Done. you can edit anytime without asking.
>> I think it's always good idea to send mailing list an email about new
>> talks, so everyone in the mailing list can know about them.
>>
>> Thanks,
>> moon
>>
>>
>> On Thu, Feb 11, 2016 at 6:57 AM DuyHai Doan  wrote:
>>
>>> Yes can you please add me edit permission so I can add new incoming
>>> talks without asking again every time ?
>>>
>>>  Thank you
>>>
>>> Regards
>>>
>>> On Wed, Feb 10, 2016 at 10:52 PM, moon soo Lee  wrote:
>>>
 Thanks DuyHai for sharing information. and they look awesome.

 Looks like you are participating talk continuously, do you mind add you
 (cwiki id : doanduyhai) edit permission and let you update?

 Thanks,
 moon

 On Wed, Feb 10, 2016 at 11:34 PM DuyHai Doan 
 wrote:

> Hello
>
>  I'll give a couple of talks about Zeppelin in the next months:
>
> - Voxxed Day Vienna February 29 2016 :
> http://voxxeddaysvienna2016.sched.org/event/5nPx/apache-zeppelin-the-missing-gui-for-your-big-data-back-end
>
> - Riga Dev Days March 3rd 2016 (Day 2): http://rigadevday.lv/#/index
>
> - jDays March 8th 2016 : http://www.jdays.se/#schedule
>
> - Voxxed Days Bucharest March 11th (Cloud & Big Data track):
> https://voxxeddays.com/bucharest/#schedule
>
>
> Can anyone update the Community WIKI page here ?
> https://cwiki.apache.org/confluence/display/ZEPPELIN/Community
>
> Regards
>
> Duy Hai DOAN
>
>
>
>
>>>
>

[ANNOUNCE] Apache Zeppelin 0.5.6-incubating released

[Alexander Bezzubov]

The Apache Zeppelin (incubating) community is pleased to announce the
availability of the 0.5.6-incubating release.

The community puts significant effort into improving Apache Zeppelin since
the last release, focusing on having better API, new backends, improvements
on stability and project documentation.

More than 38 contributors provided new features, improvements and verifying
release. More than 110 issues has been resolved.

We encourage download the latest release at
http://zeppelin.incubator.apache.org/download.html

Detailed release note is available [1]

We welcome your help and feedback. For more information on the project
and how to get involved, visit our website at [2]

Thanks to all users and contributors who have helped to improve Apache
Zeppelin.

Regards,
The Apache Zeppelin community

1.
http://zeppelin.incubator.apache.org/releases/zeppelin-release-0.5.6-incubating.html
2. http://zeppelin.incubator.apache.org

Disclaimer:
Apache Zeppelin is an effort undergoing incubation at the Apache Software
Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is
required of all newly accepted projects until a further review indicates
that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects. While
incubation status is not necessarily a reflection of the completeness or
stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.

2015: year in review

[Alexander Bezzubov]

Dear fellow Zeppelin developers and useres,

the year 2015 is about to end so I wanted to say thank you to everybody
here - its been a great year for the project indeed!

Wish you a happy new year and want to share a small review of Zeppelin in
2015 I did
https://medium.com/@bzz_/apache-zeppelin-incubating-2015-year-in-review-a938d978a309

Hope you enjoy it and please, do not hesitate to share yours.

It was a pleasure to work with you guys, looking forward next year!

--
Alex

Re: Too many open files exception

[Alexander Bezzubov]

Hi,

wellcome to the Zeppelin community!

It looks like you are doing everything right but have some
platform-specific issue, that Spark is hitting the limit of open files on
your OS.

This should not happen, so could you please check what is the current open
file limit on your environment/OS and (just in case) cross-check
spark-secific mailing list, in case that is some kind of known issues.

--
Alex

On Wed, Dec 23, 2015, 10:07 Amirhossein Aleyasin 
wrote:

> Hello,
> I am new to zeppelin, I just installed it and tried to run the tutorial
> example.
> The "load data into Table" part works perfect, but when I wanted to submit
> the sample queries, it throws the following exception:
>
>
> java.io.FileNotFoundException:
> /tmp/blockmgr-5d2c5999-5593-4f83-9d6d-3c290523ce29/3f/temp_shuffle_102ac16f-b5c6-4cc4-9c8e-b6bc66f17eb5
> (Too many open files) at java.io.FileOutputStream.open(Native Method) at
> java.io.FileOutputStream.(FileOutputStream.java:221) at
> org.apache.spark.storage.DiskBlockObjectWriter.open(DiskBlockObjectWriter.scala:88)
> at
> org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.insertAll(BypassMergeSortShuffleWriter.java:110)
> at
> org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:73)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
> at org.apache.spark.scheduler.Task.run(Task.scala:88) at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
> This is the load table code:
>
> import org.apache.commons.io.IOUtils
> import java.net.URL
> import java.nio.charset.Charset
>
> // Zeppelin creates and injects sc (SparkContext) and sqlContext
> (HiveContext or SqlContext)
> // So you don't need create them manually
>
> // load bank data
> val bankText = sc.parallelize(
> IOUtils.toString(
> new URL("
> https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv;),
> Charset.forName("utf8")).split("\n"))
>
> case class Bank(age: Integer, job: String, marital: String, education:
> String, balance: Integer)
>
> val bank = bankText.map(s => s.split(";")).filter(s => s(0) !=
> "\"age\"").map(
> s => Bank(s(0).toInt,
> s(1).replaceAll("\"", ""),
> s(2).replaceAll("\"", ""),
> s(3).replaceAll("\"", ""),
> s(5).replaceAll("\"", "").toInt
> )
> ).toDF()
> bank.registerTempTable("bank")
>
>
> and this is the query:
>
> %sql
> select age, count(1) value
> from bank
> where age < 30
> group by age
> order by age
>
>
> Any help appreciated.
>
> Thanks
>
>
>

Re: Keep track notebooks

[Alexander Bezzubov]

Hi,

recently a new, Git notebooks storage [1] was introduced that keeps all the
notebooks in the single repository under version control.

 [1]
http://zeppelin.incubator.apache.org/docs/0.6.0-incubating-SNAPSHOT/storage/storage.html#Git


Please let me know if that helps!


On Tue, Dec 1, 2015 at 6:42 PM, Pablo Torre  wrote:

> Thanks Dano! I will take a look at this link.
>
> I appreciate a lot your help!
>
> 2015-12-01 3:28 GMT+01:00 dan young :
>
>> Maybe look into versioning the S3 bucket???
>>
>> http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html
>>
>> Regards,
>>
>> Dano
>>
>>
>> On Mon, Nov 30, 2015 at 2:25 PM Pablo Torre 
>> wrote:
>>
>>> But what happen if  I modify a notebook? Is it gonna save the changes?
>>> So I can have some kind of history
>>> El 30/11/2015 21:34, "Pablo Torre"  escribió:
>>>
 I have configured it and is working.
 El 30/11/2015 21:08, "dan young"  escribió:

> So you have configured Zeppelin accordingly; to store the Notebooks in
> S3, or you need to do that?
>
> On Mon, Nov 30, 2015 at 1:05 PM Pablo Torre 
> wrote:
>
>> Yes. Exactly, I am doing that.
>> El 30/11/2015 20:56, "dan young"  escribió:
>>
>>> Are you using AWS?  store them in s3???
>>>
>>> On Mon, Nov 30, 2015, 11:08 AM Pablo Torre 
>>> wrote:
>>>
 Hi guys,

 I would like to know if there is any way to keep track of the
 different notebooks.


 Thanks,
 Best.


 --
 Pablo Torre.
 Freelance software engineer and Ruby on Rails developer.
 Oleiros (Coruña)
 *Personal site *
 My blog 

>>>
>
>
> --
> Pablo Torre.
> Freelance software engineer and Ruby on Rails developer.
> Oleiros (Coruña)
> *Personal site *
> My blog 
>



-- 
--
Kind regards,
Alexander.

Re: Keep track notebooks

[Alexander Bezzubov]

Hi Pablo,

sorry for a confusion, right now it just creates a local Git repo out of
your /notebooks dir and keeps track of the history there.
It does not create any remotes (its only a local repo) so there is no such
option yet.

You can manually add the remotes you want, pointing to any hosting ( ie for
a Github, one can do "git remote add origin https://github.com/user/repo.git;)
and then make a "git push" if you want it to be synced to the remote
location.

May be the doc should be updated to reflect this.

Please let me know if that works for you - feedback is very appreciated.

--
Alex

On Fri, Dec 11, 2015, 01:45 Pablo Torre <pablotorr...@gmail.com> wrote:

> One question Alexander, how this option works? Because in the
> documentation, I can not see how to specify a repository.
>
> 
>   zeppelin.notebook.storage
>   org.apache.zeppelin.notebook.repo.GitNotebookRepo
>   notebook persistence layer implementation
> 
>
>
> 2015-12-10 14:48 GMT+01:00 Pablo Torre <pablotorr...@gmail.com>:
>
>> Thanks Alexander! I will take a look. Appreciate your help!
>>
>> 2015-12-10 12:52 GMT+01:00 Alexander Bezzubov <abezzu...@nflabs.com>:
>>
>>> Hi,
>>>
>>> recently a new, Git notebooks storage [1] was introduced that keeps all
>>> the notebooks in the single repository under version control.
>>>
>>>  [1]
>>> http://zeppelin.incubator.apache.org/docs/0.6.0-incubating-SNAPSHOT/storage/storage.html#Git
>>>
>>>
>>> Please let me know if that helps!
>>>
>>>
>>> On Tue, Dec 1, 2015 at 6:42 PM, Pablo Torre <pablotorr...@gmail.com>
>>> wrote:
>>>
>>>> Thanks Dano! I will take a look at this link.
>>>>
>>>> I appreciate a lot your help!
>>>>
>>>> 2015-12-01 3:28 GMT+01:00 dan young <danoyo...@gmail.com>:
>>>>
>>>>> Maybe look into versioning the S3 bucket???
>>>>>
>>>>> http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html
>>>>>
>>>>> Regards,
>>>>>
>>>>> Dano
>>>>>
>>>>>
>>>>> On Mon, Nov 30, 2015 at 2:25 PM Pablo Torre <pablotorr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> But what happen if  I modify a notebook? Is it gonna save the
>>>>>> changes? So I can have some kind of history
>>>>>> El 30/11/2015 21:34, "Pablo Torre" <pablotorr...@gmail.com> escribió:
>>>>>>
>>>>>>> I have configured it and is working.
>>>>>>> El 30/11/2015 21:08, "dan young" <danoyo...@gmail.com> escribió:
>>>>>>>
>>>>>>>> So you have configured Zeppelin accordingly; to store the Notebooks
>>>>>>>> in S3, or you need to do that?
>>>>>>>>
>>>>>>>> On Mon, Nov 30, 2015 at 1:05 PM Pablo Torre <pablotorr...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yes. Exactly, I am doing that.
>>>>>>>>> El 30/11/2015 20:56, "dan young" <danoyo...@gmail.com> escribió:
>>>>>>>>>
>>>>>>>>>> Are you using AWS?  store them in s3???
>>>>>>>>>>
>>>>>>>>>> On Mon, Nov 30, 2015, 11:08 AM Pablo Torre <
>>>>>>>>>> pablotorr...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi guys,
>>>>>>>>>>>
>>>>>>>>>>> I would like to know if there is any way to keep track of the
>>>>>>>>>>> different notebooks.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Best.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Pablo Torre.
>>>>>>>>>>> Freelance software engineer and Ruby on Rails developer.
>>>>>>>>>>> Oleiros (Coruña)
>>>>>>>>>>> *Personal site <http://www.pablotorrerodriguez.com>*
>>>>>>>>>>> My blog <http://www.aboutechnologies.com>
>>>>>>>>>>>
>>>>>>>>>>
>>>>
>>>>
>>>> --
>>>> Pablo Torre.
>>>> Freelance software engineer and Ruby on Rails developer.
>>>> Oleiros (Coruña)
>>>> *Personal site <http://www.pablotorrerodriguez.com>*
>>>> My blog <http://www.aboutechnologies.com>
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Kind regards,
>>> Alexander.
>>>
>>>
>>
>>
>> --
>> Pablo Torre.
>> Freelance software engineer and Ruby on Rails developer.
>> Oleiros (Coruña)
>> *Personal site <http://www.pablotorrerodriguez.com>*
>> My blog <http://www.aboutechnologies.com>
>>
>
>
>
> --
> Pablo Torre.
> Freelance software engineer and Ruby on Rails developer.
> Oleiros (Coruña)
> *Personal site <http://www.pablotorrerodriguez.com>*
> My blog <http://www.aboutechnologies.com>
>

Re: Zeppelin notes version control, scheduler and external deps

[Alexander Bezzubov]

Hi, sorry for the late reply,

answering only your first question:


>1. How do we version control Zeppelin notes?

Now, as https://github.com/apache/incubator-zeppelin/pull/497 has landed to
the master - you can enable versioning of the notebooks using Git.

Need to build latest Zeppelin though, or wait for a new release wich I
hope, will come this year.

On Sat, Dec 5, 2015, 16:18 armen donigian  wrote:

> Hi all,
> Installed Zeppelin on Amazon EMR and it's running swell. Had a few
> questions...
>
> 1. How do we version control Zeppelin notes?
>
> 2. How do you check for status of a long running Zeppelin task? Is there a
> web UI for this or do you simply check the Resource Manager UI
> @master-node:8088 (in case of AWS)?
>
> 3. Are there any known issues/limitations of running Zeppelin note
> scheduler in production for batch ETL jobs? Trying to assess it vs Amazon
> Data Pipelines.
>
> 4. When trying to add an external jar, I'm getting this error.
> %dep
> z.reset()
> z.load("com.databricks:spark-redshift_2.10:0.5.2")
> Must be used before SparkInterpreter (%spark) initialized
>
> Thanks
>

Re: Notebook feature - "Link this paragraph"

[Alexander Bezzubov]

Thank you Damien for pointing it out.

Indeed, it has been taken care of recently, and should be a part of the
next release.
Please let me know if that still does not work somehow - would be glad to
help.

--
Alex

On Wed, Dec 2, 2015 at 9:14 AM, Corneau Damien  wrote:

> Hi,
>
> This bug has been resolved recently in the master branch.
> You can find the resolved issue on Jira here:
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/ZEPPELIN-413
>
> If you build Zeppelin from the source code, you will need to pull new
> changes, if you use a release, you will need to wait next one. (0.5.6
> should be a small release)
> On Dec 2, 2015 08:59, "Jose Rivera-Rubio" 
> wrote:
>
>> I also recognised this bug. The feature that @Mark Luk used to exist when
>> clicking on "Link paragraph". It seems one of the latest commits might have
>> clobbered that option.
>>
>> *Jose Rivera*
>> p.s. We're Hiring !
>>
>> *Data Engineer Lead  **Intern Avenue *
>> Watch Intern Avenue on: BBC Dragons’ Den
>> 
>> e: jose.riv...@internavenue.com  |
>> www.internavenue.com | Great Interns Online
>>
>> On 1 December 2015 at 23:53, Mark Luk  wrote:
>>
>>> I understand one of the notebook features is "Link this paragraph". If
>>> my notebook has multiple paragraphs, and I want to publish a particular
>>> paragraph to some separate webpage, I can click on the option "link this
>>> paragraph", then get an URL that would only render the said paragraph.
>>> Please correct me if my understanding is wrong.
>>>
>>>
>>> However, the link I get seems to always redirect to the entire page. For
>>> example, say I'm using the Zeppelin Tutorial notebook - and I use "link
>>> this paragraph" on one of the paragraphs on the bottom. I put the URL in
>>> another browser window
>>>
>>> http://
>>> :8080/#/notebook/2A94M5J1Z/paragraph/20150213-230428_1231780373?asIframe
>>>
>>> and it immediately redirects to the main page
>>>
>>>
>>> http://:8080/#/notebook/2A94M5J1Z
>>>
>>> As another experiment, I put the URL in an iframe on a dummy HTML file.
>>> The HTML is very basic
>>>
>>> 
>>>
>>> 
>>> HELLO
>>>
>>> >> src="http://:8080/#/notebook/2A94M5J1Z/paragraph/20150213-230428_1231780373?asIframe"
>>> width=500px>
>>> 
>>> 
>>> 
>>>
>>> I load the HTML page with a browser, and I do see an iframe with the
>>> content of my zeppelin notebook. However, the content shown is the
>>> beginning of the page. I was expecting to just see the paragraph that I
>>> want.
>>>
>>> How do I only render a particular paragraph?
>>>
>>> thanks
>>> mark
>>>
>>>
>>>
>>

Re: [ANNOUNCE] Apache Zeppelin 0.5.5-incubating released

[Alexander Bezzubov]

Great job, thanks to everybody envolved!

Looking forward futher time-based releases of Zeppelin.

On Fri, Nov 20, 2015, 09:00 Pablo Torre  wrote:

> Congrats!!
>
> Good job guys, as user of Zeppelin I appreciate a lot this effort.
>
> Best,
>
> 2015-11-20 0:46 GMT+01:00 임정택 :
>
>> Congrats!
>>
>> As an user of Zeppelin, thanks for all the contributors to make Zeppelin
>> better!
>>
>> Best,
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2015-11-20 0:35 GMT+09:00 Hyung Sung Shim :
>>
>>> Great!
>>> Congratulations.
>>>
>>> 2015-11-19 22:33 GMT+09:00 moon soo Lee :
>>>
 The Apache Zeppelin (incubating) community is pleased to announce the
 availability of the 0.5.5-incubating release. The community puts
 significant effort into improving Apache Zeppelin since the last release,
 focusing on having new backend support, improvements on stability and
 simplifying the configuration. More than 60 contributors provided new
 features, improvements and verifying release. More than 90 issues has been
 resolved.

 We encourage download the latest release from
 http://zeppelin.incubator.apache.org/download.html

 Release note is available at
 http://zeppelin.incubator.apache.org/releases/zeppelin-release-0.5.5-incubating.html

 We welcome your help and feedback. For more information on the project
 and how to get involved, visit our website at
 http://zeppelin.incubator.apache.org/

 Thanks to all users and contributors who have helped to improve
 Apache Zeppelin.

 Regards,
 The Apache Zeppelin community


 Disclaimer:
 Apache Zeppelin is an effort undergoing incubation at the Apache
 Software
 Foundation (ASF), sponsored by the Apache Incubator PMC.
 Incubation is required of all newly accepted projects until a further
 review indicates that the infrastructure, communications, and decision
 making process have stabilized in a manner consistent with other
 successful ASF projects.
 While incubation status is not necessarily a reflection of the
 completeness or stability of the code, it does indicate that the
 project has yet to be fully endorsed by the ASF.

>>>
>>>
>>>
>>> --
>>>
>>> [image: 본문 이미지 1]
>>>
>>> (주)엔에프랩  |  콘텐츠서비스팀 |  팀장 심형성
>>>
>>> *E. hsshim*@nflabs.com 
>>>
>>> *T.* 02-3458-9650 *M. *010-4282-1230
>>>
>>> *A.* 서울특별시 강남구 논현동 216-2 하림빌딩 2층 NFLABS
>>>
>>
>>
>>
>> --
>> Name : 임 정택
>> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
>> Twitter : http://twitter.com/heartsavior
>> LinkedIn : http://www.linkedin.com/in/heartsavior
>>
>
>
>
> --
> Pablo Torre.
> Freelance software engineer and Ruby on Rails developer.
> Oleiros (Coruña)
> *Personal site *
> My blog 
>

Re: Get data from Oracle Database using Apache Zeppelin

[Alexander Bezzubov]

Great post, thank you fro sharing Alexander!

On Fri, Oct 30, 2015, 19:07 Alexander Kardapolov 
wrote:

> Hi everyone!
>
> Get data from Oracle Database using Apache Zeppelin #Russia #habrahabr
> #apachezeppelin @ZeppelinHub @Jetinfosystems
>
> http://habrahabr.ru/company/jetinfosystems/blog/269769/
>
> https://en.wikipedia.org/wiki/Habrahabr
>
> --
> BR, Alex Kardapolov
>

Re: Meet Moon soo Lee

[Alexander Bezzubov]

Well done, Moon!

And thanks a lot for sharing the video ;)

On Sat, Oct 31, 2015 at 6:30 PM, Dongjoon Hyun  wrote:

> Thank you for sharing.
>
> Cheers,
> Dongjoon.
>
> On Sat, Oct 31, 2015 at 6:05 PM, tog  wrote:
>
>> Hi there,
>>
>> In case you were not in AMS, here is the video ;-)
>> https://m.youtube.com/watch?v=J6Ei1RMG5Xo
>>
>> Cheers
>> Guillaume
>> --
>> PGP KeyID: 2048R/EA31CFC9  subkeys.pgp.net
>>
>
>

Re: Problem Installing Zeppelin

[Alexander Bezzubov]

Hi,

this is the maven plugin doing a Grunt build of the frontend webapplication
failing somehow.

For us to investigate the reason, could you post the OS you are using
(`uname -a`, `cat /etc/issue`), as well as results of running the same mvn
command but with `-e` switch please?

Another thing to try is to run grunt build inside zeppelin-web dir
manually, like

```
cd zeppelin-web
./npm install
./grunt build
```

Please let us know if that help!

--
Alex

On Sat, Oct 17, 2015 at 4:21 PM, Pablo Torre  wrote:

> I tried maven 3.2.5 and 3.3.3. I removed the .m2 folder before doing a new
> test, but nothing.
>
> El 17/10/2015 05:01, "Corneau Damien"  escribió:
>
>> Can you tell us your maven version?
>>
>> Also, you can try to run with -X for more debugging, or run mvn package
>> in zeppelin-web only (would be faster)
>>
>>
>>
>> On Fri, Oct 16, 2015 at 7:18 PM, Pablo Torre 
>> wrote:
>>
>>> Hi guys,
>>>
>>> I am trying to install zeppelin using the repository in github:
>>> https://github.com/apache/incubator-zeppelin executing next command:
>>>
>>> mvn clean package -DskipTests
>>>
>>>
>>>
>>> But I am getting this error:
>>>
>>>[INFO] Zeppelin ..
>>> SUCCESS [12:24.531s]
>>>
>>>[INFO] Zeppelin: Interpreter .
>>> SUCCESS [41.705s]
>>>
>>>[INFO] Zeppelin: Zengine .
>>> SUCCESS [18.383s]
>>>
>>>[INFO] Zeppelin: Spark dependencies ..
>>> SUCCESS [1:39.317s]
>>>
>>>[INFO] Zeppelin: Spark ...
>>> SUCCESS [1:19.381s]
>>>
>>>[INFO] Zeppelin: Markdown interpreter 
>>> SUCCESS [9.290s]
>>>
>>>[INFO] Zeppelin: Angular interpreter .
>>> SUCCESS [2.248s]
>>>
>>>[INFO] Zeppelin: Shell interpreter ...
>>> SUCCESS [2.128s]
>>>
>>>[INFO] Zeppelin: Hive interpreter 
>>> SUCCESS [15.261s]
>>>
>>>[INFO] Zeppelin: Apache Phoenix Interpreter ..
>>> SUCCESS [24.200s]
>>>
>>>[INFO] Zeppelin: Apache Geode interpreter 
>>> SUCCESS [34.011s]
>>>
>>>[INFO] Zeppelin: PostgreSQL interpreter ..
>>> SUCCESS [6.286s]
>>>
>>>[INFO] Zeppelin: Tajo interpreter 
>>> SUCCESS [9.434s]
>>>
>>>[INFO] Zeppelin: Flink ...
>>> SUCCESS [57.458s]
>>>
>>>[INFO] Zeppelin: Apache Ignite interpreter ...
>>> SUCCESS [8.344s]
>>>
>>>[INFO] Zeppelin: Kylin interpreter ...
>>> SUCCESS [4.417s]
>>>
>>>[INFO] Zeppelin: Lens interpreter 
>>> SUCCESS [20.959s]
>>>
>>>[INFO] Zeppelin: Cassandra ...
>>> SUCCESS [4:46.080s]
>>>
>>>[INFO] Zeppelin: web Application .
>>> FAILURE [4:15.302s]
>>>
>>>[INFO] Zeppelin: Server .. SKIPPED
>>>
>>>[INFO] Zeppelin: Packaging distribution .. SKIPPED
>>>
>>>[INFO]
>>> 
>>>
>>>[INFO] BUILD FAILURE
>>>
>>>[INFO]
>>> 
>>>
>>>[INFO] Total time: 28:40.600s
>>>
>>>[INFO] Finished at: Fri Oct 16 09:19:54 UTC 2015
>>>
>>>[INFO] Final Memory: 135M/483M
>>>
>>>[INFO]
>>> 
>>>
>>>[ERROR] Failed to execute goal
>>> com.github.eirslett:frontend-maven-plugin:0.0.23:grunt (grunt build) on
>>> project zeppelin-web: Failed to run task: 'grunt --no-color' failed. (error
>>> code 137) -> [Help 1]
>>>
>>>[ERROR]
>>>
>>>[ERROR] To see the full stack trace of the errors, re-run Maven
>>> with the -e switch.
>>>
>>>[ERROR] Re-run Maven using the -X switch to enable full debug
>>> logging.
>>>
>>>[ERROR]
>>>
>>>[ERROR] For more information about the errors and possible
>>> solutions, please read the following articles:
>>>
>>>[ERROR] [Help 1]
>>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>>>
>>>[ERROR]
>>>
>>>[ERROR] After correcting the problems, you can resume the build
>>> with the command
>>>
>>>[ERROR]   mvn  -rf :zeppelin-web
>>>
>>> Can anyone help me with this issue?
>>>
>>> Thanks
>>>
>>> --
>>> Pablo Torre.
>>> Freelance software engineer and Ruby on Rails developer.
>>> Oleiros (Coruña)
>>> *Personal site *
>>> My blog 
>>>
>>
>>


-- 
--
Kind regards,
Alexander.

Re: Help with loading a CSV using Spark-SQL & Spark-CSV

[Alexander Bezzubov]

Hi,

thank you for your interested in Zeppelin!

Couple of things I noticed: as you probably already know , %dep and %spark
parts should always be in separate paragraphs.

%spark already exposes sql context though `sqlc` variable, so you better
use sqlc.load("...") instead.

And of course to be able to use %spark interpreter in the notebook, you
need to make sure you have it binded (cog button, on the top right)

Hope this helps!

--
Kind regards,
Alex


On Mon, Sep 28, 2015 at 4:29 PM, Ryan  wrote:

> Hi,
>
> In a Zeppelin notebook, I am trying to load a csv using the spark-csv
> package by databricks. I am using the Hortonworks sandbox to run Zeppelin
> on. Unfortunately, the methods I have been trying have not been working.
>
> My latest attempt is:
> %dep
> z.load("com.databricks:spark-csv_2.10:1.2.0")
> %spark
> val crimeData = "hdfs://
> sandbox.hortonworks.com:8020/user/root/data/crime_incidents_2013_CSV.csv"
> sqlContext.load("hdfs://
> sandbox.hortonworks.com:8020/user/root/data/crime_incidents_2013_CSV.csv",
> Map("path" -> crimeData, "header" -> "true")).registerTempTable("crimes")
>
> This is the error I receive:
> :16: error: not found: value sqlContext sqlContext.load("hdfs://
> sandbox.hortonworks.com:8020/user/root/data/crime_incidents_2013_CSV.csv",
> Map("path" -> crimeData, "header" -> "true")).registerTempTable("crimes") ^
> :12: error: not found: value % %spark ^
> Thank you for any help in advance,
> Ryan
>

Re: Zeppelin server can get very slow when logs/ folder is too big

[Alexander Bezzubov]

Hi,

thank you for bringing it to our attention!
What was the rough size of the logs when it started to give you troubles?
Indeed muting or explicitly configuring log verbosity makes a lot of sense.
May be you could also create an issue in jira to track the progress?

Thank you!

--
BR,
Alex

On Sun, Sep 27, 2015 at 3:11 PM, David Salinas 
wrote:

> Hi,
>
> I recently remarked that Zeppelin becomes prohibitively slow when the size
> of log is too big : the UI becomes then severely unresponsive. Deleting
> logs fix it. Maybe they should be some strategy to clean this folder when
> the daemon is active...
>
> Best,
>
> David
>

Re: Build Zeppelin for Spark 1.5 on OSX

[Alexander Bezzubov]

Hi,

thank you for interest in Zeppelin!

It looks like frontend webapp build failed on executing the grunt task (a
build system we use in web app). Can you try running it manually (instead
of using maven)

```
cd zepelin-web
./grunt
```

and let me know if that works?

It suppose to print smth like this at the end

```
Done, without errors.

Execution Time (2015-09-18 03:12:36 UTC)
jshint:all616ms  ▇▇▇ 2%
wiredep:app   334ms  ▇▇ 1%
concurrent:test1.3s  ▇▇ 4%
karma:unit 3.3s   11%
concurrent:dist1.6s   6%
ngAnnotate:dist3.7s  ▇ 13%
uglify:generated  17.1s
 ▇▇▇
59%
Total 29s
```

--
Alex.

On Fri, Sep 18, 2015 at 6:07 AM, Mariano Kamp 
wrote:

> Hi.
>
> I am trying to install Zeppelin and so far this didn't work for me.
>
> I am sure this is just my own stupidity, but still could you please point
> me in the right direction?
>
> I tried a couple of different things, but to make things simple I ran a
> scenario again that I suspect should be fairly common: Starting from zero
> and installing the current version of Zeppelin, Spark and Hadoop.
>
>
> snow:~ mkamp$ git clone https://github.com/apache/incubator-zeppelin.git
> && cd incubator-zeppelin/ && mvn clean package -X -Pspark-1.5
> -Dhadoop.version=2.6.0 -Phadoop-2.6 -DskipTests
> ...
> I get this result:
>
> [INFO] Reactor Summary:
>
> [INFO]
>
> [INFO] Zeppelin .. SUCCESS [7.798s]
>
> [INFO] Zeppelin: Interpreter . SUCCESS
> [15.299s]
>
> [INFO] Zeppelin: Zengine . SUCCESS [4.090s]
>
> [INFO] Zeppelin: Spark dependencies .. SUCCESS
> [42.389s]
>
> [INFO] Zeppelin: Spark ... SUCCESS
> [38.540s]
>
> [INFO] Zeppelin: Markdown interpreter  SUCCESS [0.912s]
>
> [INFO] Zeppelin: Angular interpreter . SUCCESS [0.644s]
>
> [INFO] Zeppelin: Shell interpreter ... SUCCESS [0.583s]
>
> [INFO] Zeppelin: Hive interpreter  SUCCESS [4.171s]
>
> [INFO] Zeppelin: Apache Phoenix Interpreter .. SUCCESS [7.083s]
>
> [INFO] Zeppelin: Apache Geode interpreter  SUCCESS [4.097s]
>
> [INFO] Zeppelin: PostgreSQL interpreter .. SUCCESS [0.843s]
>
> [INFO] Zeppelin: Tajo interpreter  SUCCESS [1.055s]
>
> [INFO] Zeppelin: Flink ... SUCCESS
> [10.731s]
>
> [INFO] Zeppelin: Apache Ignite interpreter ... SUCCESS [1.338s]
>
> [INFO] Zeppelin: Kylin interpreter ... SUCCESS [0.763s]
>
> [INFO] Zeppelin: Lens interpreter  SUCCESS [3.754s]
>
> [INFO] Zeppelin: Cassandra ... SUCCESS
> [1:02.923s]
>
> [INFO] Zeppelin: web Application . FAILURE
> [4:43.080s]
>
> [INFO] Zeppelin: Server .. SKIPPED
>
> [INFO] Zeppelin: Packaging distribution .. SKIPPED
>
> [INFO]
> 
>
> [INFO] BUILD FAILURE
>
> [INFO]
> 
>
> [INFO] Total time: 8:11.183s
>
> [INFO] Finished at: Thu Sep 17 23:02:29 CEST 2015
>
> [INFO] Final Memory: 162M/1712M
>
> [INFO]
> 
>
> [ERROR] Failed to execute goal
> com.github.eirslett:frontend-maven-plugin:0.0.23:grunt (grunt build) on
> project zeppelin-web: Failed to run task: 'grunt --no-color' failed. (error
> code 3) -> [Help 1]
>
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal com.github.eirslett:frontend-maven-plugin:0.0.23:grunt (grunt build)
> on project zeppelin-web: Failed to run task
>
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>
> at
>
> [...]
>
> ... 19 more
>
> Caused by:
> com.github.eirslett.maven.plugins.frontend.lib.TaskRunnerException: 'grunt
> --no-color' failed. (error code 3)
>
> at
> com.github.eirslett.maven.plugins.frontend.lib.NodeTaskExecutor.execute(NodeTaskExecutor.java:38)
>
> at
> com.github.eirslett.maven.plugins.frontend.mojo.GruntMojo.execute(GruntMojo.java:69)
>
> ... 21 more
>
> [ERROR]
>
> [ERROR]
>
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
>
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>
> [ERROR]
>
> [ERROR] After correcting the problems, you can resume the build with the
> command
>
> [ERROR]   mvn  -rf :zeppelin-web
> Many thanks.
>
> Mariano
>
>
>


-- 
--
Kind regards,
Alexander.

Re: spark worker log stderr

[Alexander Bezzubov]

Hi,

I think at this point it might be worth creating an issue to track this and
bring it to attention dev@

Thanks!

On Wed, Sep 2, 2015 at 12:14 PM, zhao selery  wrote:

> I run scala code from zeppelin notebook, when I look at spark log from
> spark/work/app-20150901193944-0015/0/stderr
> Log file looks like:
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/home/zhaoxq/Downloads/zeppelin2/incubator-zeppelin/interpreter/spark/zeppelin-spark-0.6.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/zhaoxq/Downloads/zeppelin2/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/zhaoxq/spark/assembly/target/scala-2.10/spark-assembly-1.5.0-SNAPSHOT-hadoop2.2.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
>
> But if I run same code from spark-shell, the log file containts correct
> spark log info:
> 15/09/01 19:31:58 INFO CoarseGrainedExecutorBackend: Registered signal
> handlers for [TERM, HUP, INT]
> 15/09/01 19:31:58 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 15/09/01 19:31:59 INFO SecurityManager: Changing view acls to: zhaoxq
> 15/09/01 19:31:59 INFO SecurityManager: Changing modify acls to: zhaoxq
> 15/09/01 19:31:59 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(zhaoxq); users
> with modify permissions: Set(zhaoxq)
> 15/09/01 19:31:59 INFO Slf4jLogger: Slf4jLogger started
> .
> 
>
>
> Also, if I rebuild zeppelin with remote false mode, which is:
> change  setting.getOption().setRemote(false); in
>  org.apache.zeppelin.interpreter.InterpreterFactory,
> spark worker log is same as running from spark-shell, which is what I want.
>
>
> So, my question is how can I get useful spark worker log with zeppelin
> remote mode?  Thanks.
>



-- 
--
Kind regards,
Alexander.

Re: sql interpreter not found

[Alexander Bezzubov]

Hi,

that is a bit wired, as there is no separate sql interpreter (it is spark
one) - you check the interpreter bindings using wrench icon on the right
and make sure spark-local is activated and is the first (default) one.

Can you also send spark interpret logs, they should be in a separate file,
something like 'logs/zeppelin-interpreter-spark-root-myhost.log' ?

--
Kind regards,
Alex


On Thu, Sep 3, 2015 at 3:15 AM, Bill Kehoe  wrote:

> Hoping someone can help.
>
> I've successfully built zeppelin (on CentOS 6.6) and can execute the Scala
> %spark cell titled "Load Data Into Table" of the Tutorial notebook.
>
> But when I try to execute any of the 3 %sql cells that query the "bank"
> table, I get "sql interpreter not found" errors.
>
> I've included the relevant snippet of my logs/zeppelin-root-myhost.log
> file below.
>
> Is there some configuration setting I need to set to get Spark SQL
> contexts to work?
>
> I'm running in local mode but my host also has a spark 1.4 install (not
> sure if that has anything to do with the issue).
>
> Stack trace from logs/zeppelin-root-myhost.log:
>
> ERROR [2015-09-02 12:09:07,606] ({qtp1526310589-53}
> NotebookServer.java[runParagraph]:630) - Exception from run
> org.apache.zeppelin.interpreter.InterpreterException: sql interpreter not
> found
> at
> org.apache.zeppelin.notebook.NoteInterpreterLoader.get(NoteInterpreterLoader.java:148)
> at org.apache.zeppelin.notebook.Note.run(Note.java:278)
> at
> org.apache.zeppelin.socket.NotebookServer.runParagraph(NotebookServer.java:628)
> at
> org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:126)
> at
> org.apache.zeppelin.socket.NotebookSocket.onMessage(NotebookSocket.java:56)
> at
> org.eclipse.jetty.websocket.WebSocketConnectionRFC6455$WSFrameHandler.onFrame(WebSocketConnectionRFC6455.java:835)
> at
> org.eclipse.jetty.websocket.WebSocketParserRFC6455.parseNext(WebSocketParserRFC6455.java:349)
> at
> org.eclipse.jetty.websocket.WebSocketConnectionRFC6455.handle(WebSocketConnectionRFC6455.java:225)
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
> at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:745)
>



-- 
--
Kind regards,
Alexander.

Re: zeppelin tar file size

[Alexander Bezzubov]

Hi,

thank you for you interest in Zeppelin project!

Yes, by default a build that you did includes a lot of different
interpreters like Spark, Flink, Lens, etc so that is why the size is quite
substantial.

In case you are about to use existing Spark\Hadoop - as of
https://issues.apache.org/jira/browse/ZEPPELIN-160 now, there is an option
now to build Zeppelin with those dependencies in a provided scope (so they
are not included in the final archive).
Then you just need to set SPARK_HOME and HADOOP_HOME to be able to use
existing Spark\Hadoop.

Please, let me know if that helps!

On Thu, Sep 3, 2015 at 12:38 AM, MrAsanjar .  wrote:

> I build zeppelin with following options as it was documented:
> *mvn clean package  -Pspark-1.4 -Dspark.version=1.4.1
> -Dhadoop.version=2.4.0 -Phadoop-2.4 -Pyarn -DskipTests -P build-distr*
>
> However the generated tarfile in
> zeppelin-distribution/target/zeppelin-0.6.0-incubating-SNAPSHOT.tar.gz is *414
> Meg*, is that correct?
> I also noticed it does include spark, hadoop, and other tarfiles, do I
> need them if I am using existing hadoop & spark client configured and
> functioning?
>

Re: Anybody has Zeppelin working against AWS EMR 4.0?

[Alexander Bezzubov]

HI Francis,

thank you for your interest in using Zeppelin with AWS EMR, and please keep
sharing it.

The reason you have disconnected state is that before
https://github.com/apache/incubator-zeppelin/pull/170 as it is mentioned
here https://github.com/apache/incubator-zeppelin#run Zeppelin neededs 2
open ports: 8080 for REST API and 8081 for websocket communication (both
are configurable, but those are defaults) - and 'disconnected' usually
means that wesocket one is not available.

Can you make sure you both are accessible on your setup (i.e using nc)? I.e
the link you pointed have instructions to setup both port forwarded though
SSH and that is essential.

Another option would be to build Zeppelin \w #140 patch and see if that
solves the issue.

Please, let me know if that helps!

--
BR,
Alexander.

On Wed, Jul 29, 2015 at 8:00 AM, Francis Lau francis@smartsheet.com
wrote:

 Anyone has Zeppelin working against AWS EMR 4.0 with Spark?

 The 4.0 version of EMR was just released last week:
 http://aws.amazon.com/about-aws/whats-new/2015/07/amazon-emr-release-4-0-0-with-new-versions-of-apache-hadoop-hive-and-spark-now-available/

 I found this bootstrap and I got a new cluster up and running without
 errors: https://gist.github.com/andershammar/224e1077021d0ea376dd#comments

 But the Zepp UI shows the disconnected red label and I also cannot
 create a new notebook.

 I am very new to Zeppelin so it may be a rookie issue :) i.e. configs or
 connections.

 Help?

 --
 *Francis *

Re: how to configure yarn client mode with Zeppelin

[Alexander Bezzubov]

Indeed!

I think there also should be something like

`export HADOOP_CONF_DIR=$HADOOP_CONF_DIR`

pointing to YARN-configured hadoop setup, to pick a ResourceManager address.

Hope this helps!

--
BR,
Alexander.

On Wed, Jul 29, 2015 at 10:48 AM, Jongyoul Lee jongy...@gmail.com wrote:
 Hi,

 1. Copy your configuration file *-site.xml or more.
 2. Add 'export MATER=yarn-client' in your zeppelin-env.sh.
 3. run zeppelin-daemon.sh start
 4. Enjoy!

 JL.



 On Tue, Jul 28, 2015 at 6:26 PM, manya cancerian manyacancer...@gmail.com
 wrote:

 Can someone help me configure Zeppelin with yarn client mode? Quick steps
 will do !

 Regards




 --
 이종열, Jongyoul Lee, 李宗烈
 http://madeng.net



-- 
--
Kind regards,
Alexander.

Re: reload spark context

[Alexander Bezzubov]

Hi,

I'm not sure if that exactly 'reload spark context' is, but if you go
to 'Interpreters' and hit 'restart' button for the spark interpreter
that will restart Spark and create a new context.

Other 2 question look like a Spar-specific behaviour, so It might be
better to ask Spark community directly.

Hope this helps.

--
BR,
Alexander


On Mon, Jul 27, 2015 at 7:15 PM, Schmirr Wurst schmirrwu...@gmail.com wrote:
 Hi

 - Is there a way to reload the spark context in the gui ?
 - I've also realised that sc.hadoopConfiguration.set() is only working
 once, after the first time, even if I modify the params with the same
 function again, they don't seems to change...
 - By the way, is there a way to print the parameter I've set ?... I
 tried sc.hadoopConfiguration.get(myparam), but doesn't work...

 Best,
 Sw



-- 
--
Kind regards,
Alexander.

Re: Using Trello to Show Mid to High Level features in Apache Zeppelin

[Alexander Bezzubov]

Guys,

thank you for great suggestions!

Am I right that you suggest using Trello not instead of ASF hosted
JIRA, but together with it, and are volunteering to support it as a
tool for prioritizing user's feedback?

Also, how do you think, should we then move further discussion to the
d...@zeppelin.incubator.apache.org as, I assume, you want project
developers to use it?

Personally, am not aware of anything that JIRA with the plugins can
not do, that trello can. But I see your point of having a simpler and
more user-friendly tool for the end user's feedback.

Although question about whether the benefits at the end worth
supporting two systems is still is still open, I would be in favor of
making an experiment and giving it a try, in case somebody volunteers
to manage second one.

What do you think?



On Fri, Jul 24, 2015 at 8:13 AM, Marko Galesic
marko_gale...@progressive.com wrote:
 Hello A B!



 I’m really glad that you like the idea! I made sure that the board’s voting
 is public. However, you *do* need to be a Trello member in order to vote.
 You can use your Google account to sign in or create an account through
 Trello.



 I found more projects that use Trello as a Roadmapping tool:
 http://blog.trello.com/going-public-roadmapping-with-a-public-trello-board/



 Marko



 From: A B [mailto:netzbewoh...@gmail.com]
 Sent: Thursday, July 23, 2015 5:23 PM
 To: users@zeppelin.incubator.apache.org


 Subject: Re: Using Trello to Show Mid to High Level features in Apache
 Zeppelin



 Hi guys!

 I find the suggestion to vote via trello totally cool and would support it.
 So if everyone is OK with this, let's do this.



 I was looking for such a possibility to have a community process to
 prioritize something for quite some time (have also played with various JIRA
 workarounds) - but this just blows my mind. Wish I had known it before :)



 Marko, pls check if you set rights correctly - i cant vote.



 On Fri, Jul 24, 2015 at 12:12 AM, Marko Galesic
 marko_gale...@progressive.com wrote:

 Hi moon,



 I see your point that there would be overhead in managing two systems.
 However, I don’t believe that working within JIRA will achieve what I’m
 thinking of. I’m impressed there are people who use JIRA and seem to be end
 users; however, I speculate that these are advanced users – edging on
 developers rather than purely data scientists. There needs to be a
 separation between what the users want and backend implementation. An artist
 doesn’t necessarily tell the rendering engineer how to program a
 photo-realistic renderer; he just says “I want it to be easier to do X and
 be able to better control Y”. I’ll keep maintaining the board. You are at
 least one person that is aware of it, and there may be others. I’ve talked
 with co-workers, and they like the idea.



 There are two big things I see preventing me from posting\editing stuff, if
 I did:

 1.   I don’t have access to edit JIRA

 2.   Others may not necessarily agree with my interpretation of the
 issues (I edit the titles and prune to what I think is relevant, which is a
 guess, at best, right now).



 The real thought behind all of this is that the community would use the
 votes on specific cards as direction (or at least give an indication of what
 people are excited about); however, those cards are curated by me : /. I’m
 biased. This is a relatively esoteric project, so there is some inherent
 protection against trolls.



  All I’d ask is that votes could be reflected from this board to JIRA;
 it doesn’t seem like people vote on things, anyway ---



 I do believe that if Zeppelin gets more traction it will become the de facto
 tool for data science within the Hadoop ecosystem.



 Those are my thoughts,

 Marko



 From: moon soo Lee [mailto:m...@apache.org]
 Sent: Thursday, June 18, 2015 1:21 AM
 To: users@zeppelin.incubator.apache.org
 Cc: Brian G Durkin; Krishnachaitanya C Potluri; James J Boesger
 Subject: Re: Using Trello to Show Mid to High Level features in Apache
 Zeppelin



 Hi Marko Galesic,



 Thanks for interest to Zeppelin. Also really appreciate for asking
 involvement.



 About the trello you suggested, I checked and looks like you did nice job.



 In my understanding, beside of JIRA, you'd like to use Trello board to get
 users(who is not familiar with JIRA) requests and feedbacks. right?



 Personally, i think the idea make sense. There're definitely people who
 feels less comfortable of using JIRA.



 However, instead of maintaining separate issue tracking system for different
 target user groups, how about contributing to Zeppelin directly to solve the
 problem. So improvement can be done with Apache community.

 It can be documentation of how to create jira issue, it can be discussion of
 way of managing and organizing issues, it can be anything, we'll figure out.



 What do you think?



 Thanks,

 moon



 On Thu, Jun 11, 2015 at 3:02 PM Marko Galesic
 marko_gale...@progressive.com wrote:

 Hi 

Re: Spark memory configuration

[Alexander Bezzubov]

Hi,

thank you for your interest in Zeppelin!

You just have to set the 'Spark' interpreter properties in 'Interpreters' menu:

CPU
  spark.cores.max: 24
Mem
  spark.executor.memory 22g

You actually can use any of the
http://spark.apache.org/docs/latest/configuration.html#application-properties
there.

Hope this helps!

On Thu, Jul 23, 2015 at 10:32 PM, PHELIPOT, REMY remy.pheli...@atos.net wrote:
 Hello !

 I am trying to launch some very greedy processes on a Spark 1.4 Cluster
 using Zeppelin, and I don't understand how to configure Spark memory
 properly. I’ve tried to set SPARK_MASTER_MEMORY, SPARK_WORKER_MEMORY and
 SPARK_EXECUTOR_MEMORY environment variables on the spark cluster nodes, with
 no effect. I’ve also tried to add the following line in zeppelin-env.sh:
 export ZEPPELIN_JAVA_OPTS=-Dspark.home=/usr/spark
 -Dspark.spark.executor.memory=15g -Dspark.spark.worker.memory=15g
 But it didn’t work..

 How to configure properly Spark? It seems the configuration files are not
 used and the memory available is limited to 512m.

 Kind regards,

 Rémy

 Ce message et toutes les pièces jointes (ci-après le message) sont établis
 à l’intention exclusive des destinataires désignés. Il contient des
 informations confidentielles et pouvant être protégé par le secret
 professionnel. Si vous recevez ce message par erreur, merci d'en avertir
 immédiatement l'expéditeur et de détruire le message. Toute utilisation de
 ce message non conforme à sa destination, toute diffusion ou toute
 publication, totale ou partielle, est interdite, sauf autorisation expresse
 de l’émetteur. L'internet ne garantissant pas l'intégrité de ce message lors
 de son acheminement, Atos (et ses filiales) décline(nt) toute responsabilité
 au titre de son contenu. Bien que ce message ait fait l’objet d’un
 traitement anti-virus lors de son envoi, l’émetteur ne peut garantir
 l’absence totale de logiciels malveillants dans son contenu et ne pourrait
 être tenu pour responsable des dommages engendrés par la transmission de
 l’un d’eux.

 This message and any attachments (the message) are intended solely for the
 addressee(s). It contains confidential information, that may be privileged.
 If you receive this message in error, please notify the sender immediately
 and delete the message. Any use of the message in violation of its purpose,
 any dissemination or disclosure, either wholly or partially is strictly
 prohibited, unless it has been explicitly authorized by the sender. As its
 integrity cannot be secured on the internet, Atos and its subsidiaries
 decline any liability for the content of this message. Although the sender
 endeavors to maintain a computer virus-free network, the sender does not
 warrant that this transmission is virus-free and will not be liable for any
 damages resulting from any virus transmitted.



-- 
--
Kind regards,
Alexander.

Re: a small question about zeppelin-manager

[Alexander Bezzubov]

Hi,

those are very valid steps indeed and you do not need to build custom
spark version.

Hope this helps!

On Tue, Jul 21, 2015 at 7:46 PM, 江之源 jiangzhiy...@liulishuo.com wrote:
 hi
 To install the zeppelin with z-manager is a helpless method, because i
 install the zeppelin in manually way and failed.I have tried many many
 times.
 My cluster is spark1.3.0 hadoop 2.0.0-cdh4.5.0,and the model is standalone.
 I will install the zeppelin manually right now so i wanna you check my
 steps:

 1. git clone the repository from the github.
 2. mvn clean package
 3. mvn install -DskipTests -Dspark.version=1.3.0
 -Dhadoop.version=2.0.0-cdh4.5.0
 (did zeppelin support cdh4.5.0)
 Should i have to do the custom built spark
 like(-Dspark.version=1.1.0-Custom)
 4.modify my master spark://...:7077
 Is it completed? or i lost something please tell me.
 thanks
 jzy

 在 2015年7月21日，下午5:48，Alexander Bezzubov abezzu...@nflabs.com 写道：

 Hi,

 thank you for your interest in the project!

 It seems like the best way to get Zeppelin up and running in your case
 would be to build it manually with relevant Spark\Hadoop options as
 described here
 http://zeppelin.incubator.apache.org/docs/install/install.html

 Please, let me know if that helps.

 --
 BR,
 Alex

 On Tue, Jul 21, 2015 at 11:35 AM, 江之源 jiangzhiy...@liulishuo.com wrote:

 hi
 i installed zeppelin some time before, but it always failed in my server
 cluster. i found the z-management Occasionally. I installed and success in
 my server. But when i wanna to read in my HDFS file like:

 sc.textFile(hdfs://llscluster/tmp/jzyresult/part-04093).count()


 it throw the errors in my cluster:Job aborted due to stage failure: Task 15
 in stage 6.0 failed 4 times, most recent failure: Lost task 15.3 in stage
 6.0 (TID 386, lls7): java.io.EOFException

 when i modify it to the local model, it could read HDFS file successfully.
 My cluster is Spark1.3.0 Hadoop2.0.0-CDH4.5.0. but the install options just
 have Spark1.3.0 and Hadoop2.0.0-CDH-4.7.0. Is this the cause to read HDFS
 file failed?
 Look forward to your reply!
 THANK YOU!
 JZY




 --
 --
 Kind regards,
 Alexander.





-- 
--
Kind regards,
Alexander.

Re: a small question about zeppelin-manager

[Alexander Bezzubov]

Hi,

thank you for your interest in the project!

It seems like the best way to get Zeppelin up and running in your case
would be to build it manually with relevant Spark\Hadoop options as
described here http://zeppelin.incubator.apache.org/docs/install/install.html

Please, let me know if that helps.

--
BR,
Alex

On Tue, Jul 21, 2015 at 11:35 AM, 江之源 jiangzhiy...@liulishuo.com wrote:
 hi
 i installed zeppelin some time before, but it always failed in my server
 cluster. i found the z-management Occasionally. I installed and success in
 my server. But when i wanna to read in my HDFS file like:

 sc.textFile(hdfs://llscluster/tmp/jzyresult/part-04093).count()


 it throw the errors in my cluster:Job aborted due to stage failure: Task 15
 in stage 6.0 failed 4 times, most recent failure: Lost task 15.3 in stage
 6.0 (TID 386, lls7): java.io.EOFException

 when i modify it to the local model, it could read HDFS file successfully.
 My cluster is Spark1.3.0 Hadoop2.0.0-CDH4.5.0. but the install options just
 have Spark1.3.0 and Hadoop2.0.0-CDH-4.7.0. Is this the cause to read HDFS
 file failed?
 Look forward to your reply!
 THANK YOU!
 JZY



-- 
--
Kind regards,
Alexander.

Re: Does zeppelin work on windows?

[Alexander Bezzubov]

Hi Puneet,

thank you for your interest in Zeppelin.

Right now we officially do not support windows platform, but there are
at least two ways to get Zeppelin running there:
 - build using cygwin
 - or follow this procedure described by one of the community members
http://madhukaudantha.blogspot.kr/2015/04/building-zeppelin-in-windows-8.html

Hope this helps.

--
Alex

On Thu, Jul 16, 2015 at 1:25 PM, Puneet Khatod puneet.kha...@tavant.com wrote:
 Hi,



 Am new to zeppelin and have recently started exploring. I am planning to
 setup spark and zeppelin on windows instead of unix box.

 Does zeppelin support windows env?



 Thanks,

 Puneet

 
 Any comments or statements made in this email are not necessarily those of
 Tavant Technologies. The information transmitted is intended only for the
 person or entity to which it is addressed and may contain confidential
 and/or privileged material. If you have received this in error, please
 contact the sender and delete the material from any computer. All emails
 sent from or to Tavant Technologies may be subject to our monitoring
 procedures.



-- 
--
Kind regards,
Alexander.

Re: google charts for zeppelin

[Alexander Bezzubov]

Hi Axel,

I think the repo you are pointing out is outdated and is not useful
for the current Apache Zeppelin (incubating) codebase.

That being said, there is an engineer here on the list, Madhuka (in
CC) who is working on adding more charts to Zeppelin i.e
https://issues.apache.org/jira/browse/ZEPPELIN-157

AFIAK recently he has been experimenting with GCharts, so I wonder if
he can chime in and help you here.

Hope this helps!

--
Kind regards,
Alex

On Thu, Jul 16, 2015 at 12:47 PM, Axel Dahl a...@whisperstream.com wrote:
 Hi,

 I was trying to integrate google charts
 (https://developers.google.com/chart/) into zeppelin.

 From %pyspark I was hoping I could just do:

 print %html

!--Load the AJAX API--
 script type=text/javascript
 src=https://www.google.com/jsapi;/script
 script type=text/javascript

   // Load the Visualization API and the piechart package.
   google.load('visualization', '1.0', {'packages':['corechart']});

   // Set a callback to run when the Google Visualization API is loaded.
   google.setOnLoadCallback(drawChart);

 .

 .

 .

 

 But when I do that and run the paragraph, zeppelin just gives me a blank
 white screen and appears to lose the web socket connection.  I also found
 moon's chart repo here: https://github.com/NFLabs/zeppelin-gchart.  But not
 sure how it integrates into the existing code base.  Has anyone else
 achieved a gcharts integration or does anyone else know why I get the blank
 screen and how to avoid it?


 Thanks,


 -Axel



-- 
--
Kind regards,
Alexander.

Re: Provide binaries

[Alexander Bezzubov]

Hi,

thank you for asking!

We definitely looking into providing a convenience binaries for the
release, as soon as it is out.

Stay tuned!

On Fri, Jul 10, 2015 at 7:30 AM, Illu Ying illuying2...@gmail.com wrote:
 +1

 On Thu, Jul 9, 2015 at 3:28 PM, Abhi Basu 9000r...@gmail.com wrote:


 Can you please provide binaries for Zeppelin? People are having lots of
 trouble building with all the dependencies,  as I am reading in this forum.
 Maybe at least for the prominent distros like Horton and cloudera.

 Thanks

 Abhi


 Sent via the Samsung Galaxy Note®, an ATT 4G LTE smartphone





-- 
--
Kind regards,
Alexander.

Re: Not able to see registered table records and Pyspark not working

[Alexander Bezzubov]

Hi,

please mind that flag -Pyspark will download full spark binary
distribution, so it might take a while for the first time you do it.

That was the main reason behind hiding it under the separate profile.



On Thu, Jul 9, 2015 at 11:53 PM, moon soo Lee m...@apache.org wrote:
 You can still manually configure all the environment variables and
 properties for pyspark, but it is suggested to build with -Ppyspark from
 now.

 Thanks,
 moon


 On Wed, Jul 8, 2015 at 10:59 PM IT CTO goi@gmail.com wrote:

 Does this means that everyone who wants pySpark to work should use this
 option in the build from now on or is that going to be the default like
 spark 1.4 ?
 Eran

 On Thu, Jul 9, 2015 at 12:14 AM moon soo Lee m...@apache.org wrote:

 If your source code is older than 3 days? Because of -Ppyspark is merged
 about 3 days ago.

 Thanks,
 moon


 On Wed, Jul 8, 2015 at 1:58 PM Vadla, Karthik karthik.va...@intel.com
 wrote:

 I’m using this .zip https://github.com/apache/incubator-zeppelin



 Thanks

 Karthik



 From: moon soo Lee [mailto:m...@apache.org]
 Sent: Wednesday, July 8, 2015 1:37 PM
 To: users@zeppelin.incubator.apache.org
 Subject: Re: Not able to see registered table records and Pyspark not
 working



 Are you building on latest master?

 On Wed, Jul 8, 2015 at 1:34 PM Vadla, Karthik karthik.va...@intel.com
 wrote:

 Hi Moon,



 Yeah I tried below command. The build was successful, but at the end I
 got warning message as below

 [WARNING] The requested profile pyspark could not be activated because
 it does not exist.





 Pyspark exists on machine. Do I need to anything further.



 Thanks

 Karthik

 From: moon soo Lee [mailto:m...@apache.org]
 Sent: Wednesday, July 8, 2015 10:58 AM


 To: users@zeppelin.incubator.apache.org
 Subject: Re: Not able to see registered table records and Pyspark not
 working



 Hi



 I was meaning adding -Ppyspark profile, like



 mvn clean package -Pspark-1.3 -Ppyspark -Dhadoop.version=2.6.0-cdh5.4.0
 -Phadoop-2.6 –DskipTests

 Thanks,

 moon

 On Wed, Jul 8, 2015 at 10:43 AM Vadla, Karthik karthik.va...@intel.com
 wrote:

 Hi Moon,



 You mean to say I need to build something like this.

 mvn clean package -Ppyspark-1.3 -Dhadoop.version=2.6.0-cdh5.4.0
 -Phadoop-2.6 –DskipTests



 I have built my zeppelin with below command previously

 mvn clean package -Pspark-1.3 -Dhadoop.version=2.6.0-cdh5.4.0
 -Phadoop-2.6 –DskipTests





 Thanks

 Karthik

 From: moon soo Lee [mailto:m...@apache.org]
 Sent: Wednesday, July 8, 2015 10:20 AM
 To: users@zeppelin.incubator.apache.org
 Subject: Re: Not able to see registered table records and Pyspark not
 working



 Hi,



 If you build latest master branch with -Ppyspark maven profile, it'll
 help pyspark work without setting those environment variables.

 Hope this helps.



 Best,

 moon



 On Tue, Jul 7, 2015 at 3:47 PM Vadla, Karthik karthik.va...@intel.com
 wrote:

 Hi All,



 This part is commented in zeppelin-env.sh in my conf folder.



 # Pyspark (supported with Spark 1.2.1 and above)

 # To configure pyspark, you need to set spark distribution's path to
 'spark.home' property in Interpreter setting screen in Zeppelin GUI

 # export PYSPARK_PYTHON  # path to the python command. must be
 the same path on the driver(Zeppelin) and all workers.

 # export PYTHONPATH  # extra PYTHONPATH.



 Can you anyone help how to setup those.



 Appreciate your help.



 Thanks

 Karthik



 From: Vadla, Karthik [mailto:karthik.va...@intel.com]
 Sent: Tuesday, July 7, 2015 3:29 PM
 To: users@zeppelin.incubator.apache.org
 Subject: RE: Not able to see registered table records and Pyspark not
 working



 Hi Moon,



 Thanks for that.
 The problem is with my parsing. I resolved it.



 I have another question to ask.

 I’m just trying to run print command using pyspark interpreter.
 It is not responding .



 When I look at the log, I don’t have information except this



 INFO [2015-07-07 15:19:17,702] ({pool-1-thread-41}
 SchedulerFactory.java[jobStarted]:132) - Job
 paragraph_1436305204170_601291630 started by scheduler
 remoteinterpreter_267235421

 INFO [2015-07-07 15:19:17,702] ({pool-1-thread-41}
 Paragraph.java[jobRun]:194) - run paragraph 20150707-144004_475199059 using
 pyspark org.apache.zeppelin.interpreter.LazyOpenInterpreter@33a625a7

 INFO [2015-07-07 15:19:17,702] ({pool-1-thread-41}
 Paragraph.java[jobRun]:211) - RUN : list=range(1,10)

 print(list)

 INFO [2015-07-07 15:19:18,060] ({Thread-255}
 NotebookServer.java[broadcast]:251) - SEND  PROGRESS

 INFO [2015-07-07 15:19:18,678] ({Thread-255}
 NotebookServer.java[broadcast]:251) - SEND  PROGRESS

 INFO [2015-07-07 15:19:19,278] ({Thread-255}
 NotebookServer.java[broadcast]:251) - SEND  PROGRESS

 INFO [2015-07-07 15:19:19,879] ({Thread-255}
 NotebookServer.java[broadcast]:251) - SEND  PROGRESS





 Do I need to do any config settings in zeppelin-env.sh or
 zeppelin-site.xml???





 Thanks

 Karthik







 From: moon soo Lee 

Re: Zeppelin Interpreter end

[Alexander Bezzubov]

Hi,

right now, I'm not aware of such configuration in Zeppelin (please,
feel free to open the issue\submit a patch).

AFAIK dynamic YARN resource allocation is up to the user and is not
configured by default right now, which looks like one possible
solution to the problem you describe (at least cpu-wise)

As a workaround for your usecase, you can manually re-start Spark
interpreter (Interpreter menu - restart), which, because of
lazy-loading, will not occupy any resources until somebody actually
runs it.

Hope this helps!


On Tue, Jun 30, 2015 at 8:23 AM, Litt, Shaun sl...@conversantmedia.com wrote:
 Hi,  I am new to zeppelin and just got it configured to run in my YARN
 cluster, but I was wondering if there is a configuration or even a hard
 setting that shuts down interpreters after in-activity.  It seems like the
 interpreter (and it’s yarn consumption) hang out indefinitely, ideally there
 would be a clean way (like logout or a shutdown button within the notebook)
 to shutdown these interpreters, but additionally there should be a way for
 an admin of zeppelin to impose and idle timeout.  As a note to the scope of
 this, is dynamic yarn resource allocation configured (such that once a
 paragraph finishes it can release vcores)?



 Thanks,

 Shaun



 This email and any files included with it may contain privileged,
 proprietary and/or confidential information that is for the sole use
 of the intended recipient(s).  Any disclosure, copying, distribution,
 posting, or use of the information contained in or attached to this
 email is prohibited unless permitted by the sender.  If you have
 received this email in error, please immediately notify the sender
 via return email, telephone, or fax and destroy this original transmission
 and its included files without reading or saving it in any manner.
 Thank you.



-- 
--
Kind regards,
Alexander.

Re: Zeppelin in multi user environment

[Alexander Bezzubov]

Hi,

thank you for asking,

indeed, as Moon mentioned, we are working on making a standalone tool
available that is a reverse proxy, capable of launching separate
docker container per-user for the chosen spark\hadoop version,
implementing the A. architecture from above.


--
Alex

On Mon, Jun 29, 2015 at 3:37 PM, IT CTO goi@gmail.com wrote:
 Thanks!
 That's clarify the issue...
 Can you share what NFLabs doing in open source?
 Eran

 On Sun, Jun 28, 2015 at 10:10 PM moon soo Lee m...@apache.org wrote:

 Hi,

 Here's something i know about multi-tenancy for Zeppelin.

 A. Reverse Proxy + Zeppelin on docker.

 Setup a reverse proxy, who is doing authentication and redirect user to
 proper Zeppelin instance running on docker container.
 I saw many companies are already using Zeppelin in this way.

 My company (NFLabs) also uses this way for one of internal cluster. And
 now preparing open source the tools that helps set up and use this type of
 environment.

 As far as i know, NFLabs has no plan to make commercial package of
 Zeppelin which has more features(such as security enabled zeppelin) than
 Apache version. One commercial service NFLabs doing is collaboration/sharing
 service for Zeppelin notebook with access control (like github for git).


 B. Shiro security. PullRequest-53

 Which enables dedicated notebook space for each user.
 I like the approach and really make sense.

 There're couple of issues i can think.
   - compiler context are shared among users
   - user can still read other users's notebook directly from filesystem
   - user is not distinguished in interpreter level.
   - restarting Zeppelin is required for many cases. That'll impact all
 connected user.

 Therefore, it can be used for basic authentication, but need more work for
 multi-tenant environment.

 So, i'd like to say, A is more like what's possible now, B is more like
 future work.

 Thanks,
 moon


 On Sun, Jun 28, 2015 at 3:09 AM Eric Charles e...@apache.org wrote:

 There is also https://github.com/apache/incubator-zeppelin/pull/53 which
 proposes to add shiro security (user authentication on the web part). This
 does not address what Ophir mentions (separated environment for e.g. spark
 interpreters to avoid variables shared across simultaneous authenticated
 users).

 My company (Datalayer) has also developed a multiuser extension to
 Zeppelin that addresses both web and interpreter user environment
 separation.

 To achieve this, we had to change the interpreter API to propagate the
 authenticated user to the interpreters.

 On 2015-06-28 11:54, IT CTO wrote:

 Thanks Ophir!
 That means that I have to wrap zeppelin with my own site which launch a
 zeppelin server on behalf of every requesting user. this is an option but I
 want avoiding it.
 pls, share wherever you come across during this journey
 Eran

 On Sun, Jun 28, 2015 at 12:09 PM Ophir Cohen oph...@gmail.com wrote:

 Actually it a bit more than that:
 Even the variables shared across notebooks!

 I think that NFLabs has a commercial version that supports groups and
 users.
 In my organisation we are looking on few solutions for that.
 One of them is using different instances - maybe even on the same
 machine.
 I'm going to test it soon - but you are right, currently it's a problem.

 BTW
 Running different Zeppelin instances isn't such a bad idea as you get
 the efficiency from the yarn resource manager that can be the same cluster
 (assuming you using yarn)

 On Sun, Jun 28, 2015 at 10:00 AM, IT CTO goi@gmail.com wrote:

 Hi,
 we are in the process of testing Zeppelin as our investigation platform
 inside the organization.
 One of the first question raised was with regard to multi user
 environment - currently, as I see it, all users run against the same
 zeppelin server and have access and availability to all notebooks.

 What are other people do with regard to that?
 Does the road-map have a multi-tenant solution for zeppelin? security?

 Eran






-- 
--
Kind regards,
Alexander.

Re: Importing HBase data to Zeppelin notebook

[Alexander Bezzubov]

Hey Ignacia,

this error message my be a bit cryptic indeed, but what this means is that
this paragraph should be run first, after the interpreter start.

Easiest way to do that - go to Interpreters menu on top and there manually
restart particular interpreter the you are using at that notebook. Then get
back to the notebook and run that paragraph first.
Or just re-start zeppelin and run it first, either should work.

Please let me know if that helps!

On Fri, Jun 12, 2015 at 2:38 AM, Ignacio Alvarez ignacioalv...@gmail.com
wrote:

 Thanks moon!

 I haven't figured out how to initialize the Spark interpreter after the
 dependencies. What I get from the documentation is that I just need to do
 this:

 %dep
 z.reset()
 z.addRepo(cloudera).url(
 https://repository.cloudera.com/artifactory/cloudera-repos/;)
 z.load(org.apache.hbase:hbase:1.0.0-cdh5.4.)
 z.load(org.apache.hbase:hbase-client:1.0.0-cdh5.4.)
 z.load(org.apache.hbase:hbase-common:1.0.0-cdh5.4.)
 z.load(org.apache.hbase:hbase-server:1.0.0-cdh5.4.)


 on the first paragraph of the notebook
 But I get a response: Must be used before SparkInterpreter (%spark)
 initialized

 Can you help me out with further instructions?


 On Tue, Jun 9, 2015 at 8:11 AM, moon soo Lee m...@apache.org wrote:

 Hi,

 For dependency library loading, please check

 http://zeppelin.incubator.apache.org/docs/interpreter/spark.html#dependencyloading

 And there're nice hbase shell interpreter implementation
 https://github.com/apache/incubator-zeppelin/pull/55

 Thanks,
 moon


 On Tue, Jun 9, 2015 at 6:30 AM Ignacio Alvarez ignacioalv...@gmail.com
 wrote:

 hello all,

 is there a way to configure the zepellin spark shell to get access to
 HBase data?

 Adding this line to spark-defaults.conf --:

 spark.executor.extraClassPath
 /opt/cloudera/parcels/CDH/lib/hive/lib/hive-hbase-handler.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-server.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-protocol.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-hadoop2-compat.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-client.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-common.jar:/opt/cloudera/parcels/CDH/lib/hbase/lib/htrace-core.jar


 And then adding the following driver class while submitting the spark
 job would work on the spark shell:

 --driver-class-path

 /opt/cloudera/parcels/CDH/lib/hbase/hbase-server.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-protocol.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-hadoop2-compat.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-client.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-common.jar:/opt/cloudera/parcels/CDH/lib/hbase/lib/htrace-core.jar


 Is there a way to add this configuration to the zepelling notebook?


 Thanks,

 Ignacio




 --
 Ignacio Alvarez, PhD

 Research Scientist, Intel Corporation, Hillsboro, OR




-- 
--
Kind regards,
Alexander.

Re: Question

[Alexander Bezzubov]

Indeed, also the focus of Zeppelin is mostly cluster computing system
backends.

But answering your question about unique features - collaborative analysis
is the one from the top of my head: try opening multiple tabs of single
Zeppelin instance and start making changes (adding new paragraphs and
running them)

Rich GUI is using angular AKA ZEPPELIN-25
https://issues.apache.org/jira/browse/ZEPPELIN-25 is another one very
powerful, not properly document yet.

--
Kind regards,
Alexander

On 05 Jun 2015, at 00:03, IT CTO goi@gmail.com wrote:

Zeppelin is the client for running code against different back-ends using
different languages (scala, SQL, md, etc) this is similar to things you can
get with IPython and IScala for example. the nice thing about Zeppelin is
it's easy of use for an analyst working with with different back-ends and
running analysis.
HTH
Eran

On Thu, Jun 4, 2015 at 5:16 PM Raj Sharma ink...@gmail.com wrote:

 Hi,

 What is difference between IPython and Zeppelin?

 What can Zeppelin can do that IPython cannot do?

 Thanks,

 Raj

Re: Zeppelin on remote machine

[Alexander Bezzubov]

Hi Harsh,

can you please post the spark interpreter (I assume that's what you are
using) logs here too?
You can find them under ./log/zeppelin-interpreter-spark-*.log

It looks like it ZeppelinServer has troubles communicating to the
Interpreter process somehow.

--
Best regards,
Alexander

Re: Using Zeppelin with EC2 spark cluster

[Alexander Bezzubov]

Hi Jihun,

thanks for your interest in Zeppelin!
It looks like zeppelin server having troubles connecting to spark
interpreter process.

Could you please post logs/zeppelin-* server and interpreter logs?

Thanks.

On Mon, May 25, 2015 at 2:23 PM, Jihun choi jihun.c...@kongstudios.net
wrote:

 Hi,
 I'm a noob user with Zeppelin.

 In order to do distributed computing, I've installed Spark cluster on EC2
 (with master  5 slaves) and make Zeppelin server to refer the Spark
 master.
 I've tried to set master url at 'interpreter' menu in Zeppelin as
 following:
 spark://ec2-52-24-213-147.us-west-2.compute.amazonaws.com:7077 (Spark
 master url which is running on AWS with 5 slaves)
 And I also set export MASTER= in 'zeppelin-env.sh' as same.

 However, when I run script on the notebook, the following error is occured:
 org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:221)
 org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
 org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:212)
 org.apache.zeppelin.scheduler.Job.run(Job.java:170)
 org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:296)
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 java.util.concurrent.FutureTask.run(FutureTask.java:262)
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 java.lang.Thread.run(Thread.java:745)

 Could you give me some hints please?

 Best,




-- 
--
Kind regards,
Alexander.

Re: Zeppelin packaging for Spark 1.3 gives errors

[Alexander Bezzubov]

Hi Roopa,

which command did you use to build Zeppelin?
Please try 'mvn package ' instead and let us know if that works.


On Tue, May 26, 2015 at 8:45 AM, Corneau Damien cornead...@gmail.com
wrote:

 Hi,

 Tests tends to fail when people compile zeppelin (works fine on CI).
 Can you try compiling with -DskipTests?
  On May 25, 2015 6:02 PM, Roopa Prabhu rooparu...@gmail.com wrote:

 Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test
 (test) on project zeppelin-server: There are test failures - [Help 1]

 [ERROR]

 [ERROR] To see the full stack trace of the errors, re-run Maven with the
 -e switch.

 [ERROR] Re-run Maven using the -X switch to enable full debug logging.

 [ERROR]

 [ERROR] For more information about the errors and possible solutions,
 please read the following articles:

 [ERROR] [Help 1]
 http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

 [ERROR]

 [ERROR] After correcting the problems, you can resume the build with the
 command

 [ERROR]   mvn goals -rf :zeppelin-server





-- 
--
Kind regards,
Alexander.

Re: adding custom jars to zeppelin

[Alexander Bezzubov]

Hi Roopa,

thanks for your interest!

This is definitely possible and the simplest way would be to use a
dependency loading mechanism that is embedded in Zeppelin using %dep in
the first paragraph as described here
http://zeppelin.incubator.apache.org/docs/interpreter/spark.html#dependencyloading
to get mongo jars in a classpath.

--
Alexander

On Thu, May 7, 2015 at 12:49 PM, Roopa Prabhu rooparu...@gmail.com wrote:

 Hi,

 I would like to use mongodb as my data source and therefore need to
 access/use spark-mongodb api's in zeppelin UI, to generate my temporary
 tables.
 Is it possible to add these jars ? Can I add the maven/sbt dependencies
 for the same? If so, what are the steps to be followed to access these
 api's in the zeppelin UI

 RP




-- 
--
Kind regards,
Alexander.

Re: Problems with web proxy

[Alexander Bezzubov]

Hi Frank,

mvn build in zeppelin-web calls 'npm install'  'grunt build' underneath,
 could you try doing those manually (with 'rm -rf node_modules') and
posting npm-debug.log\output in case it is still failing?

Just to clarify your setup: you need to build Zeppelin on a machine with
internet available only behind proxy?


On Thu, Feb 26, 2015 at 5:23 AM, frank.schil...@thomsonreuters.com wrote:


  Sorry, I forgot to change the subject line. Hopefully this one is more
 descriptive.

  Frank



   Hi,

  I’m having problems building Zeppelin because of the web proxy. It fails
 while building the web Application:

Failed to execute goal
 com.github.eirslett:frontend-maven-plugin:0.0.20:npm (npm install) on
 project zeppelin-web: Failed to run task: 'npm install --color=false
 --proxy=http://NA:NA@webproxy:80
 http://NA:n...@webproxy.int.westgroup.com:80'


  It seems like I need to set user name and password for the web proxy,
 but those are actually not required.


  I tried out setting the web proxy (without the user name and password)
 with npm, but no success.


  Any help would be highly appreciated,


  Thanks,

 Frank

Re: Problems with web proxy

[Alexander Bezzubov]

Ok, there is one more step between 'npm install' and 'grunt build' - it is
a 'bower --allow-root install' to get third-party dependencies for a webapp.

Please try ruining that first and then 'grunt build --force' again.

On Thu, Feb 26, 2015 at 12:59 PM, frank.schil...@thomsonreuters.com wrote:

   After running ‘sudo npm install’ and ‘grunt build’  I got the following
 warning/error message:


  Running wiredep:app (wiredep) task

 Warning: Error: Cannot find where you keep your Bower packages. Use
 --force to continue.


  Aborted due to warnings.




From: Alexander Bezzubov b...@apache.org
 Reply-To: users@zeppelin.incubator.apache.org 
 users@zeppelin.incubator.apache.org
 Date: Wednesday, February 25, 2015 at 7:24 PM
 To: users@zeppelin.incubator.apache.org 
 users@zeppelin.incubator.apache.org
 Subject: Re: Problems with web proxy

   Hi Frank,

  mvn build in zeppelin-web calls 'npm install'  'grunt build'
 underneath,  could you try doing those manually (with 'rm -rf
 node_modules') and posting npm-debug.log\output in case it is still failing?

  Just to clarify your setup: you need to build Zeppelin on a machine with
 internet available only behind proxy?


 On Thu, Feb 26, 2015 at 5:23 AM, frank.schil...@thomsonreuters.com
 wrote:


  Sorry, I forgot to change the subject line. Hopefully this one is more
 descriptive.

  Frank



   Hi,

  I’m having problems building Zeppelin because of the web proxy. It
 fails while building the web Application:

   Failed to execute goal
 com.github.eirslett:frontend-maven-plugin:0.0.20:npm (npm install) on
 project zeppelin-web: Failed to run task: 'npm install --color=false
 --proxy=http://NA:NA@webproxy:80
 http://NA:n...@webproxy.int.westgroup.com:80'


  It seems like I need to set user name and password for the web proxy,
 but those are actually not required.


  I tried out setting the web proxy (without the user name and password)
 with npm, but no success.


  Any help would be highly appreciated,


  Thanks,

 Frank