Re: building zeppelin from source - web not working
Hello,
I followed your command and I ran Zeppelin with the latest master and
successfully accessed the web.
Could you please share more information?
Best regards,
Jongyoul Lee
2024년 5월 16일 (목) 오후 6:15, Michal Vince 님이 작성:
> hi
>
> I`m trying to build zeppelin
> <https://zeppelin.apache.org/docs/latest/setup/basics/how_to_build.html>
> from master with
>
>
>
> ./mvnw -B package -DskipTests -Pbuild-distr -Pspark-3.5 -Pinclude-hadoop
> -Phadoop3 -Pspark-scala-2.12 -Pweb-dist -pl
> '!groovy,!submarine,!livy,!hbase,!file,!flink'
>
> but after starting zeppelin with
>
>
>
>
>
> ./bin/zeppelin-daemon.sh start
>
> I`m getting 404 when trying to open zeppelin in web browser
>
>
>
> in log i can see
>
> WARN [2024-05-16 10:55:14,656] ({main} WebAppContext.java[doStart]:533) -
> Failed startup of context
> o.e.j.w.WebAppContext@2e48362c{/,jar:file:///home/mvince/remote_projects/github/zeppelin/zeppelin-distribution/target/zeppelin-0.12
> .0-SNAPSHOT/zeppelin-0.12.0-SNAPSHOT/zeppelin-web-0.12.0-SNAPSHOT.war!/,STOPPED}{/home/mvince/remote_projects/github/zeppelin/zeppelin-distribution/target/zeppelin-0.12.0-SNAPSHOT/zeppelin-0.12.0-SNAPSHOT/zeppelin-web-0.12.0-SNAPSHO
> T.war}
> java.io.FileNotFoundException: JAR entry
> WEB-INF/lib/hadoop-client-api-3.3.6.jar!/ not found in
> /home/mvince/remote_projects/github/zeppelin/zeppelin-distribution/target/zeppelin-0.12.0-SNAPSHOT/zeppelin-0.12.0-SNAPSHOT/zeppelin-web
> -0.12.0-SNAPSHOT.war
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:146)
> at
> java.base/sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:92)
> at
> org.eclipse.jetty.webapp.MetaInfConfiguration.getTlds(MetaInfConfiguration.java:445)
> at
> org.eclipse.jetty.webapp.MetaInfConfiguration.scanForTlds(MetaInfConfiguration.java:361)
> at
> org.eclipse.jetty.webapp.MetaInfConfiguration.scanJars(MetaInfConfiguration.java:172)
> at
> org.eclipse.jetty.webapp.MetaInfConfiguration.preConfigure(MetaInfConfiguration.java:106)
> at
> org.eclipse.jetty.webapp.WebAppContext.preConfigure(WebAppContext.java:488)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:523)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
> at
> io.micrometer.core.instrument.binder.jetty.TimedHandler.doStart(TimedHandler.java:162)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at org.eclipse.jetty.server.Server.start(Server.java:423)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
> at org.eclipse.jetty.server.Server.doStart(Server.java:387)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:262)
>
>
> is there some config i keep missing?
> thanks
>
>
--
Best regards,
Jongyoul Lee
[RESULT] [VOTE] Revert Shell Interpreter
Hello, The vote for keeping Shell Interpreter for the next release was passed with 9 for +1 (2 binding votes) 1 for 0 1 for -1 (Objection was resolved) The voting thread. - https://lists.apache.org/thread/z0ygr40dm5r2syhgo68m6zx0qxkj1o0l Thank you for your contribution and positive feedback. I'll make a new minor release including Shell Interpreter again soon. Best regards, Jongyoul Lee
Re: [VOTE] Revert Shell Interpreter
Hello, The vote for keeping Shell Interpreter for the next release was passed with 9 for +1 (2 binding votes) 1 for 0 1 for -1 (Objection was resolved) The voting thread. - https://lists.apache.org/thread/z0ygr40dm5r2syhgo68m6zx0qxkj1o0l Thank you for your contribution and positive feedback. I'll make a new minor release including Shell Interpreter again soon. Best regards, Jongyoul Lee 2024년 5월 12일 (일) 오후 9:44, Jongyoul Lee 님이 작성: > +1 > > > 2024년 4월 30일 (화) 오후 11:45, Jongyoul Lee 님이 작성: > >> Hello Nathan, >> >> Thank you for the reply. We already talked about what you mentioned in >> the discussion thread I attached in the first email. Moreover, Zeppelin >> already provides a way to disable the Shell interpreter so you can disable >> it even though releases include the Shell interpreter. >> >> Best regards, >> Jongyoul Lee >> >> 2024년 4월 30일 (화) 오후 10:52, Rutland, Nathan (CTR) < >> nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: >> >>> My reason for removing shell interpreter is that one can easily do: >>> >>> >>> >>> %sh >>> >>> cat /etc/zeppelin/conf/shiro.ini >>> >>> >>> >>> This would expose all the passwords in a normal system. I have been >>> trying to disable sh interpreter ahead of this feature without great >>> success. >>> >>> >>> >>> *From:* Jongyoul Lee >>> *Sent:* Monday, April 29, 2024 7:30 PM >>> *To:* Rutland, Nathan (CTR) >> > >>> *Cc:* dev ; users >>> *Subject:* Re: [VOTE] Revert Shell Interpreter >>> >>> >>> >>> *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click >>> links or open attachments unless you recognize and/or trust the sender. >>> Contact your component SOC with questions or concerns. >>> >>> >>> >>> Hello, >>> >>> >>> >>> I planed to complete this vote today following general rules for Apache >>> Zeppelin community history but I prolong this vote to one day more as we >>> have one veto so that we wait for the reason. >>> >>> >>> >>> Best regards, >>> >>> Jongyoul Lee >>> >>> >>> >>> 2024년 4월 30일 (화) 오전 9:24, Jongyoul Lee 님이 작성: >>> >>> Hello Nathan, >>> >>> >>> >>> Thank you for interests for this issue. >>> >>> >>> >>> By the way, I should have explained but all voter who gave -1 need to >>> leave some comments about why they disagree this vote. It's the voting >>> policy for all apache projects.[1] >>> >>> >>> >>> Best regards, >>> >>> Jongyoul Lee >>> >>> >>> >>> [1] https://www.apache.org/foundation/voting.html#Veto >>> <https://urldefense.us/v3/__https:/www.apache.org/foundation/voting.html*Veto__;Iw!!BClRuOV5cvtbuNI!ADJCW-K2BWvFLcKduDDrZuDLcY_SnFpwZ1uA4DyNLIW8eXGx07SMJu23ydDmlrhAAX1t6bRpjnuThKFYhfoL1-aEtyirt5sktEur$> >>> >>> >>> >>> 2024년 4월 30일 (화) 오전 2:36, Rutland, Nathan (CTR) < >>> nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: >>> >>> -1 >>> >>> >>> >>> *From:* Jongyoul Lee >>> *Sent:* Saturday, April 27, 2024 7:21 AM >>> *To:* users ; dev >>> *Subject:* [VOTE] Revert Shell Interpreter >>> >>> >>> >>> *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click >>> links or open attachments unless you recognize and/or trust the sender. >>> Contact your component SOC with questions or concerns. >>> >>> >>> >>> Hello community, >>> >>> >>> >>> I propose the vote for reverting Shell interpreter. >>> >>> >>> >>> Please check the discussion thread[1] before you participate in this >>> vote. >>> >>> >>> >>> [] +1 Including Shell interpreter again in the next Zeppelin release >>> >>> [] 0 no opinion >>> >>> [] -1 Removing Shell interpreter in the next Zeppelin release as it is >>> >>> >>> >>> Best regards, >>> >>> Jongyoul Lee >>> >>> >>> >>> [1] https://lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn >>> <https://urldefense.us/v3/__https:/lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn__;!!BClRuOV5cvtbuNI!CtFUaJ9Py5GaMHmEN0yaMqbSrwRW7L2PzpnNIa2VEqCR9uhwd_ZtcD22qIG35Lls05f8The9HpIROCcowJlxkv_SASaxvJF9qXI-$> >>> >>> >> >> -- >> Best regards, >> Jongyoul Lee >> > > > -- > Best regards, > Jongyoul Lee > -- Best regards, Jongyoul Lee
Re: [VOTE] Revert Shell Interpreter
+1 2024년 4월 30일 (화) 오후 11:45, Jongyoul Lee 님이 작성: > Hello Nathan, > > Thank you for the reply. We already talked about what you mentioned in the > discussion thread I attached in the first email. Moreover, Zeppelin already > provides a way to disable the Shell interpreter so you can disable it even > though releases include the Shell interpreter. > > Best regards, > Jongyoul Lee > > 2024년 4월 30일 (화) 오후 10:52, Rutland, Nathan (CTR) < > nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: > >> My reason for removing shell interpreter is that one can easily do: >> >> >> >> %sh >> >> cat /etc/zeppelin/conf/shiro.ini >> >> >> >> This would expose all the passwords in a normal system. I have been >> trying to disable sh interpreter ahead of this feature without great >> success. >> >> >> >> *From:* Jongyoul Lee >> *Sent:* Monday, April 29, 2024 7:30 PM >> *To:* Rutland, Nathan (CTR) >> *Cc:* dev ; users >> *Subject:* Re: [VOTE] Revert Shell Interpreter >> >> >> >> *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click >> links or open attachments unless you recognize and/or trust the sender. >> Contact your component SOC with questions or concerns. >> >> >> >> Hello, >> >> >> >> I planed to complete this vote today following general rules for Apache >> Zeppelin community history but I prolong this vote to one day more as we >> have one veto so that we wait for the reason. >> >> >> >> Best regards, >> >> Jongyoul Lee >> >> >> >> 2024년 4월 30일 (화) 오전 9:24, Jongyoul Lee 님이 작성: >> >> Hello Nathan, >> >> >> >> Thank you for interests for this issue. >> >> >> >> By the way, I should have explained but all voter who gave -1 need to >> leave some comments about why they disagree this vote. It's the voting >> policy for all apache projects.[1] >> >> >> >> Best regards, >> >> Jongyoul Lee >> >> >> >> [1] https://www.apache.org/foundation/voting.html#Veto >> <https://urldefense.us/v3/__https:/www.apache.org/foundation/voting.html*Veto__;Iw!!BClRuOV5cvtbuNI!ADJCW-K2BWvFLcKduDDrZuDLcY_SnFpwZ1uA4DyNLIW8eXGx07SMJu23ydDmlrhAAX1t6bRpjnuThKFYhfoL1-aEtyirt5sktEur$> >> >> >> >> 2024년 4월 30일 (화) 오전 2:36, Rutland, Nathan (CTR) < >> nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: >> >> -1 >> >> >> >> *From:* Jongyoul Lee >> *Sent:* Saturday, April 27, 2024 7:21 AM >> *To:* users ; dev >> *Subject:* [VOTE] Revert Shell Interpreter >> >> >> >> *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click >> links or open attachments unless you recognize and/or trust the sender. >> Contact your component SOC with questions or concerns. >> >> >> >> Hello community, >> >> >> >> I propose the vote for reverting Shell interpreter. >> >> >> >> Please check the discussion thread[1] before you participate in this vote. >> >> >> >> [] +1 Including Shell interpreter again in the next Zeppelin release >> >> [] 0 no opinion >> >> [] -1 Removing Shell interpreter in the next Zeppelin release as it is >> >> >> >> Best regards, >> >> Jongyoul Lee >> >> >> >> [1] https://lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn >> <https://urldefense.us/v3/__https:/lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn__;!!BClRuOV5cvtbuNI!CtFUaJ9Py5GaMHmEN0yaMqbSrwRW7L2PzpnNIa2VEqCR9uhwd_ZtcD22qIG35Lls05f8The9HpIROCcowJlxkv_SASaxvJF9qXI-$> >> >> > > -- > Best regards, > Jongyoul Lee > -- Best regards, Jongyoul Lee
Re: [VOTE] Revert Shell Interpreter
Hello Nathan, Thank you for the reply. We already talked about what you mentioned in the discussion thread I attached in the first email. Moreover, Zeppelin already provides a way to disable the Shell interpreter so you can disable it even though releases include the Shell interpreter. Best regards, Jongyoul Lee 2024년 4월 30일 (화) 오후 10:52, Rutland, Nathan (CTR) < nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: > My reason for removing shell interpreter is that one can easily do: > > > > %sh > > cat /etc/zeppelin/conf/shiro.ini > > > > This would expose all the passwords in a normal system. I have been > trying to disable sh interpreter ahead of this feature without great > success. > > > > *From:* Jongyoul Lee > *Sent:* Monday, April 29, 2024 7:30 PM > *To:* Rutland, Nathan (CTR) > *Cc:* dev ; users > *Subject:* Re: [VOTE] Revert Shell Interpreter > > > > *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click > links or open attachments unless you recognize and/or trust the sender. > Contact your component SOC with questions or concerns. > > > > Hello, > > > > I planed to complete this vote today following general rules for Apache > Zeppelin community history but I prolong this vote to one day more as we > have one veto so that we wait for the reason. > > > > Best regards, > > Jongyoul Lee > > > > 2024년 4월 30일 (화) 오전 9:24, Jongyoul Lee 님이 작성: > > Hello Nathan, > > > > Thank you for interests for this issue. > > > > By the way, I should have explained but all voter who gave -1 need to > leave some comments about why they disagree this vote. It's the voting > policy for all apache projects.[1] > > > > Best regards, > > Jongyoul Lee > > > > [1] https://www.apache.org/foundation/voting.html#Veto > <https://urldefense.us/v3/__https:/www.apache.org/foundation/voting.html*Veto__;Iw!!BClRuOV5cvtbuNI!ADJCW-K2BWvFLcKduDDrZuDLcY_SnFpwZ1uA4DyNLIW8eXGx07SMJu23ydDmlrhAAX1t6bRpjnuThKFYhfoL1-aEtyirt5sktEur$> > > > > 2024년 4월 30일 (화) 오전 2:36, Rutland, Nathan (CTR) < > nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: > > -1 > > > > *From:* Jongyoul Lee > *Sent:* Saturday, April 27, 2024 7:21 AM > *To:* users ; dev > *Subject:* [VOTE] Revert Shell Interpreter > > > > *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click > links or open attachments unless you recognize and/or trust the sender. > Contact your component SOC with questions or concerns. > > > > Hello community, > > > > I propose the vote for reverting Shell interpreter. > > > > Please check the discussion thread[1] before you participate in this vote. > > > > [] +1 Including Shell interpreter again in the next Zeppelin release > > [] 0 no opinion > > [] -1 Removing Shell interpreter in the next Zeppelin release as it is > > > > Best regards, > > Jongyoul Lee > > > > [1] https://lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn > <https://urldefense.us/v3/__https:/lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn__;!!BClRuOV5cvtbuNI!CtFUaJ9Py5GaMHmEN0yaMqbSrwRW7L2PzpnNIa2VEqCR9uhwd_ZtcD22qIG35Lls05f8The9HpIROCcowJlxkv_SASaxvJF9qXI-$> > > -- Best regards, Jongyoul Lee
Re: [VOTE] Revert Shell Interpreter
For your information, I used wrong recipients. The previous email is for users@ and dev@. Best regards, Jongyoul Lee 2024년 4월 30일 (화) 오전 9:30, Jongyoul Lee 님이 작성: > Hello, > > I planed to complete this vote today following general rules for Apache > Zeppelin community history but I prolong this vote to one day more as we > have one veto so that we wait for the reason. > > Best regards, > Jongyoul Lee > > 2024년 4월 30일 (화) 오전 9:24, Jongyoul Lee 님이 작성: > >> Hello Nathan, >> >> Thank you for interests for this issue. >> >> By the way, I should have explained but all voter who gave -1 need to >> leave some comments about why they disagree this vote. It's the voting >> policy for all apache projects.[1] >> >> Best regards, >> Jongyoul Lee >> >> [1] https://www.apache.org/foundation/voting.html#Veto >> >> 2024년 4월 30일 (화) 오전 2:36, Rutland, Nathan (CTR) < >> nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: >> >>> -1 >>> >>> >>> >>> *From:* Jongyoul Lee >>> *Sent:* Saturday, April 27, 2024 7:21 AM >>> *To:* users ; dev >>> *Subject:* [VOTE] Revert Shell Interpreter >>> >>> >>> >>> *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click >>> links or open attachments unless you recognize and/or trust the sender. >>> Contact your component SOC with questions or concerns. >>> >>> >>> >>> Hello community, >>> >>> >>> >>> I propose the vote for reverting Shell interpreter. >>> >>> >>> >>> Please check the discussion thread[1] before you participate in this >>> vote. >>> >>> >>> >>> [] +1 Including Shell interpreter again in the next Zeppelin release >>> >>> [] 0 no opinion >>> >>> [] -1 Removing Shell interpreter in the next Zeppelin release as it is >>> >>> >>> >>> Best regards, >>> >>> Jongyoul Lee >>> >>> >>> >>> [1] https://lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn >>> <https://urldefense.us/v3/__https:/lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn__;!!BClRuOV5cvtbuNI!CtFUaJ9Py5GaMHmEN0yaMqbSrwRW7L2PzpnNIa2VEqCR9uhwd_ZtcD22qIG35Lls05f8The9HpIROCcowJlxkv_SASaxvJF9qXI-$> >>> >>
Re: [VOTE] Revert Shell Interpreter
Hello, I planed to complete this vote today following general rules for Apache Zeppelin community history but I prolong this vote to one day more as we have one veto so that we wait for the reason. Best regards, Jongyoul Lee 2024년 4월 30일 (화) 오전 9:24, Jongyoul Lee 님이 작성: > Hello Nathan, > > Thank you for interests for this issue. > > By the way, I should have explained but all voter who gave -1 need to > leave some comments about why they disagree this vote. It's the voting > policy for all apache projects.[1] > > Best regards, > Jongyoul Lee > > [1] https://www.apache.org/foundation/voting.html#Veto > > 2024년 4월 30일 (화) 오전 2:36, Rutland, Nathan (CTR) < > nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: > >> -1 >> >> >> >> *From:* Jongyoul Lee >> *Sent:* Saturday, April 27, 2024 7:21 AM >> *To:* users ; dev >> *Subject:* [VOTE] Revert Shell Interpreter >> >> >> >> *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click >> links or open attachments unless you recognize and/or trust the sender. >> Contact your component SOC with questions or concerns. >> >> >> >> Hello community, >> >> >> >> I propose the vote for reverting Shell interpreter. >> >> >> >> Please check the discussion thread[1] before you participate in this vote. >> >> >> >> [] +1 Including Shell interpreter again in the next Zeppelin release >> >> [] 0 no opinion >> >> [] -1 Removing Shell interpreter in the next Zeppelin release as it is >> >> >> >> Best regards, >> >> Jongyoul Lee >> >> >> >> [1] https://lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn >> <https://urldefense.us/v3/__https:/lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn__;!!BClRuOV5cvtbuNI!CtFUaJ9Py5GaMHmEN0yaMqbSrwRW7L2PzpnNIa2VEqCR9uhwd_ZtcD22qIG35Lls05f8The9HpIROCcowJlxkv_SASaxvJF9qXI-$> >> >
Re: [VOTE] Revert Shell Interpreter
Hello Nathan, Thank you for interests for this issue. By the way, I should have explained but all voter who gave -1 need to leave some comments about why they disagree this vote. It's the voting policy for all apache projects.[1] Best regards, Jongyoul Lee [1] https://www.apache.org/foundation/voting.html#Veto 2024년 4월 30일 (화) 오전 2:36, Rutland, Nathan (CTR) < nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성: > -1 > > > > *From:* Jongyoul Lee > *Sent:* Saturday, April 27, 2024 7:21 AM > *To:* users ; dev > *Subject:* [VOTE] Revert Shell Interpreter > > > > *CAUTION: *This email originated from outside of CISA/DHS. DO NOT click > links or open attachments unless you recognize and/or trust the sender. > Contact your component SOC with questions or concerns. > > > > Hello community, > > > > I propose the vote for reverting Shell interpreter. > > > > Please check the discussion thread[1] before you participate in this vote. > > > > [] +1 Including Shell interpreter again in the next Zeppelin release > > [] 0 no opinion > > [] -1 Removing Shell interpreter in the next Zeppelin release as it is > > > > Best regards, > > Jongyoul Lee > > > > [1] https://lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn > <https://urldefense.us/v3/__https:/lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn__;!!BClRuOV5cvtbuNI!CtFUaJ9Py5GaMHmEN0yaMqbSrwRW7L2PzpnNIa2VEqCR9uhwd_ZtcD22qIG35Lls05f8The9HpIROCcowJlxkv_SASaxvJF9qXI-$> >
[VOTE] Revert Shell Interpreter
Hello community, I propose the vote for reverting Shell interpreter. Please check the discussion thread[1] before you participate in this vote. [] +1 Including Shell interpreter again in the next Zeppelin release [] 0 no opinion [] -1 Removing Shell interpreter in the next Zeppelin release as it is Best regards, Jongyoul Lee [1] https://lists.apache.org/thread/opf3h6b0wc4pnsvqsy0b50vzmozd9qbn
Re: Zeppelin/Helium packages - not showing up!
Hello, It sounds good to use Hulium. By the way, I don't plan to integrate Apache Echarts by myself. However, it's a good candidate so that you can start to contribute to Zeppelin. By the way, I add users@z.a.o to cc to share your idea. Best regards, Jongyoul Lee 2024년 4월 23일 (화) 오전 11:42, Nagesh R 님이 작성: > Hey Lee, > > On another note, did you try integrating apache echarts into zeppelin? If > so, please provide any information you can. > > Thanks- Nagesh > > On Mon, Apr 22, 2024, 10:36 AM Nagesh R wrote: > >> Thank you for reaching out to me, Lee... >> >> I resolved the issue...the problem was my NIC card on my VM was not >> accessing the internet - I fixed... >> >> Now I can see all the Helium packages... >> >> Stay well my friend! >> >> Thank you - Nagesh! >> >> >> On Sun, Apr 21, 2024 at 8:52 AM Jongyoul Lee wrote: >> >>> Hello, >>> >>> Thank you for asking. >>> >>> By the way, which version of Zeppelin are you using? >>> >>> Moreover, for the first time, it takes time to parse and cache >>> helium.json. Could you please tell me or add screenshots? >>> >>> Best regards, >>> Jongyoul Lee >>> >>> 2024년 4월 19일 (금) 오전 12:13, Nagesh R 님이 작성: >>> >>>> Hello Jongyoul, >>>> >>>> How are you? >>>> >>>> I have installed zeppelin on ubuntu and was trying to get the helium >>>> packages up and running... >>>> >>>> I saw your post on github to change the zeppelin-site.xml file from s3 >>>> to 'helium,https://zeppelin.apache.org/helium.json'. >>>> >>>> I did that and I still dont see - the packages in the zeppelin UI. >>>> >>>> Can you pl. let me know as to what I am doing wrong? >>>> >>>> Thank You Sir! >>>> - Nagesh >>>> >>> >>> >>> -- >>> Best regards, >>> Jongyoul Lee >>> >> -- Best regards, Jongyoul Lee
Re: Zeppelin/Helium packages - not showing up!
Hello, Thank you for asking. By the way, which version of Zeppelin are you using? Moreover, for the first time, it takes time to parse and cache helium.json. Could you please tell me or add screenshots? Best regards, Jongyoul Lee 2024년 4월 19일 (금) 오전 12:13, Nagesh R 님이 작성: > Hello Jongyoul, > > How are you? > > I have installed zeppelin on ubuntu and was trying to get the helium > packages up and running... > > I saw your post on github to change the zeppelin-site.xml file from s3 to > 'helium,https://zeppelin.apache.org/helium.json'. > > I did that and I still dont see - the packages in the zeppelin UI. > > Can you pl. let me know as to what I am doing wrong? > > Thank You Sir! > - Nagesh > -- Best regards, Jongyoul Lee
[DISCUSS] Shell interpreter
Hello, I want to discuss Shell interpreter issue with you. For your information, we had a security report using Shell interpreter to execute malicious code with a system account. As you know, it's a kind of characteristic of Apache Zeppelin but some contributors including me thought it was too risky even if it's a feature. Moreover, I thought that we had some workarounds to do similar executions. However, after releasing it, there were many questions via several channels about the deprecation of Shell interpreter. I would like to follow the community's decision. For one more piece of information, we already have a security page to warn the code execution feature so we can keep the Shell interpreter without any further treatment. Could you please give me your opinion on this? If we conclude keeping it, I'll release a new release of 0.11.2 including Shell interpreter again. Best regards, Jongyoul Lee
CVE-2024-31861: Apache Zeppelin: Code injection by Shell interpreter
Severity: important
Affected versions:
- Apache Zeppelin 0.10.1 before 0.11.1
Description:
Improper Control of Generation of Code ('Code Injection') vulnerability in
Apache Zeppelin.
The attackers can use Shell interpreter as a code generation gateway, and
execute the generated code as a normal way.
This issue affects Apache Zeppelin: from 0.10.1 before 0.11.1.
Users are recommended to upgrade to version 0.11.1, which doesn't have Shell
interpreter by default.
Credit:
Esa Hiltunen (finder)
https://teragrep.com (finder)
References:
https://github.com/apache/zeppelin/pull/4708
https://zeppelin.apache.org/
https://www.cve.org/CVERecord?id=CVE-2024-31861
CVE-2024-31867: Apache Zeppelin: LDAP search filter query Injection Vulnerability
Severity: moderate Affected versions: - Apache Zeppelin 0.8.2 before 0.11.1 Description: Improper Input Validation vulnerability in Apache Zeppelin. The attackers can execute malicious queries by setting improper configuration properties to LDAP search filter. This issue affects Apache Zeppelin: from 0.8.2 before 0.11.1. Users are recommended to upgrade to version 0.11.1, which fixes the issue. Credit: Qing Xu (finder) References: https://github.com/apache/zeppelin/pull/4714 https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2024-31867
Re: Shell interpreter in v0.11.1 is not installed
Hello Nils, Thank you for sharing your concerns and suggestions. I agree with you that we missed some information or instructions. I know it's an excuse but we were challenged with some security issues recently and we had to handle it with priority. I also hope it can be improved well. For the usage, thank you for letting me know your use cases. Shell interpreter looks like a mandatory component for you. We are discussing the Shell interpreter issue with the security team so I'll share how we can use it sooner than later. Best regards, Jongyoul Lee 2024년 4월 9일 (화) 오후 4:40, Nils Glueck 님이 작성: > Hello Jongyoul, > > thanks for your effort. > > The only thing that was a bit irritating to me is the fact that docs and > online pages do not really mention the removal as of now. Such as docs > for version 0.11.1.[1] Again, maybe missing something here. > > About the usage. We degraded Zeppelin from an interactive development > tool to a task runner that would run notebooks using Shell interpreter > when triggered from an external scheduler. So Zeppelin itself seems a > bit over-engineered for our purposes. > > Considering switching to good old Jenkins instead. > > Will keep an eye on any further releases from Zeppelin though. Good that > there seems to be still basic maintenance. :) Cheers > > Nils > > [1] > https://zeppelin.apache.org/docs/0.11.1/usage/interpreter/installation.html > -- Best regards, Jongyoul Lee
CVE-2024-31868: Apache Zeppelin: XSS vulnerability in the helium module
Severity: moderate Affected versions: - Apache Zeppelin 0.8.2 before 0.11.1 Description: Improper Encoding or Escaping of Output vulnerability in Apache Zeppelin. The attackers can modify helium.json and exposure XSS attacks to normal users. This issue affects Apache Zeppelin: from 0.8.2 before 0.11.1. Users are recommended to upgrade to version 0.11.1, which fixes the issue. Credit: H Ming (finder) References: https://github.com/apache/zeppelin/pull/4728 https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2024-31868
CVE-2024-31866: Apache Zeppelin: Interpreter download command does not escape malicious code injection
Severity: moderate Affected versions: - Apache Zeppelin 0.8.2 before 0.11.1 Description: Improper Encoding or Escaping of Output vulnerability in Apache Zeppelin. The attackers can execute shell scripts or malicious code by overriding configuration like ZEPPELIN_INTP_CLASSPATH_OVERRIDES. This issue affects Apache Zeppelin: from 0.8.2 before 0.11.1. Users are recommended to upgrade to version 0.11.1, which fixes the issue. Credit: Esa Hiltunen (finder) https://teragrep.com (finder) References: https://github.com/apache/zeppelin/pull/4715 https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2024-31866
CVE-2024-31865: Apache Zeppelin: Cron arbitrary user impersonation with improper privileges
Severity: moderate Affected versions: - Apache Zeppelin 0.8.2 before 0.11.1 Description: Improper Input Validation vulnerability in Apache Zeppelin. The attackers can call updating cron API with invalid or improper privileges so that the notebook can run with the privileges. This issue affects Apache Zeppelin: from 0.8.2 before 0.11.1. Users are recommended to upgrade to version 0.11.1, which fixes the issue. Credit: Esa Hiltunen (finder) https://teragrep.com (finder) References: https://github.com/apache/zeppelin/pull/4631 https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2024-31865
CVE-2024-31864: Apache Zeppelin: Remote code execution by adding malicious JDBC connection string
Severity: moderate
Affected versions:
- Apache Zeppelin before 0.11.1
Description:
Improper Control of Generation of Code ('Code Injection') vulnerability in
Apache Zeppelin.
The attacker can inject sensitive configuration or malicious code when
connecting MySQL database via JDBC driver.
This issue affects Apache Zeppelin: before 0.11.1.
Users are recommended to upgrade to version 0.11.1, which fixes the issue.
This issue is being tracked as ZEPPELIN-5990
Credit:
rg (finder)
References:
https://github.com/apache/zeppelin/pull/4709
https://www.cve.org/CVERecord?id=CVE-2020-11974
https://zeppelin.apache.org/
https://www.cve.org/CVERecord?id=CVE-2024-31864
https://issues.apache.org/jira/browse/ZEPPELIN-5990
CVE-2024-31863: Apache Zeppelin: Replacing other users notebook, bypassing any permissions
Severity: moderate Affected versions: - Apache Zeppelin 0.10.1 before 0.11.0 Description: Authentication Bypass by Spoofing vulnerability by replacing to exsiting notes in Apache Zeppelin.This issue affects Apache Zeppelin: from 0.10.1 before 0.11.0. Users are recommended to upgrade to version 0.11.0, which fixes the issue. Credit: Esa Hiltunen (finder) https://teragrep.com (finder) References: https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2024-31863
CVE-2024-31862: Apache Zeppelin: Denial of service with invalid notebook name
Severity: moderate Affected versions: - Apache Zeppelin 0.10.1 before 0.11.0 Description: Improper Input Validation vulnerability in Apache Zeppelin when creating a new note from Zeppelin's UI.This issue affects Apache Zeppelin: from 0.10.1 before 0.11.0. Users are recommended to upgrade to version 0.11.0, which fixes the issue. Credit: Esa Hiltunen (finder) https://teragrep.com (finder) References: https://github.com/apache/zeppelin/pull/4632 https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2024-31862
CVE-2022-47894: Apache Zeppelin SAP: connecting to a malicious SAP server allowed it to perform XXE
Severity: moderate Affected versions: - Apache Zeppelin SAP 0.8.0 before 0.11.0 Description: Improper Input Validation vulnerability in Apache Zeppelin SAP.This issue affects Apache Zeppelin SAP: from 0.8.0 before 0.11.0. As this project is retired, we do not plan to release a version that fixes this issue. Users are recommended to find an alternative or restrict access to the instance to trusted users. For more information, the fix already was merged in the source code but Zeppelin decided to retire the SAP component NOTE: This vulnerability only affects products that are no longer supported by the maintainer. This issue is being tracked as ZEPPELIN-5665 Credit: kuiplatain@knownsec 404 Team (finder) References: https://github.com/apache/zeppelin/pull/4302 https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2022-47894 https://issues.apache.org/jira/browse/ZEPPELIN-5665
CVE-2021-28656: Apache Zeppelin: CSRF vulnerability in the Credentials page
Severity: low Affected versions: - Apache Zeppelin through 0.9.0 Description: Cross-Site Request Forgery (CSRF) vulnerability in Credential page of Apache Zeppelin allows an attacker to submit malicious request. This issue affects Apache Zeppelin Apache Zeppelin version 0.9.0 and prior versions. Credit: Jiang Qingzhi (finder) References: https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2021-28656
CVE-2024-31860: Apache Zeppelin: Path traversal vulnerability
Severity: low Affected versions: - Apache Zeppelin 0.9.0 before 0.11.0 Description: Improper Input Validation vulnerability in Apache Zeppelin. By adding relative path indicators(E.g ..), attackers can see the contents for any files in the filesystem that the server account can access. This issue affects Apache Zeppelin: from 0.9.0 before 0.11.0. Users are recommended to upgrade to version 0.11.0, which fixes the issue. Credit: Kai Zhao (finder) References: https://github.com/apache/zeppelin/pull/4632 https://zeppelin.apache.org/ https://www.cve.org/CVERecord?id=CVE-2024-31860
Re: Shell interpreter in v0.11.1 is not installed
Hello, Thank you for checking it. For 0.11.1, it was obliterated including deploying the 0.11.1 binary to the maven repository. Thus, installing it via install-interpreter.sh is impossible as well. Unfortunately, we don't plan to include it on official deployments until all shell interpreter security issues are resolved. For the commit, you are right it's the only commit in the master repository. However, we have one more commit as a hotfix in the branch-0.11. - https://github.com/apache/zeppelin/commit/9a51785a3a86d93888a0d959815b097ba41b35d5 I'm also preparing how to include it when you build it by yourself. Thank you for your interest. Moreover, I appreciate it if you shared your experience of how you are using shell interpreter. We can also consider for providing the feature natively, not using shell interpreter. Hope this helps you. Best regards, Jongyoul Lee 2024년 4월 9일 (화) 오전 7:20, Nils Glueck 님이 작성: > Hello, > > after switching from Zeppelin 0.11.0 to version 0.11.1, I noticed that > the Shell interpreter had disappeared from the config, while my Docker > file remained the same (using netinst binary). Also, I noted at least > one commit related to somewhat removal of Shell interpreter.[1] > > 1) Am I missing something, or has the Shell interpreter been removed > from the regular netinst binary build? > 2) I am using netinst Zeppelin 0.11.1 installation, followed by "RUN > ./bin/install-interpreter.sh --name shell", which confirms that Shell > interpreter has been installed. However, it does not show up when > checking the Interpreter config web page of Zeppelin. This can be > observed only after the recent switch from 0.11.0 to 0.11.1. I do see > Interpreters angukar, flink, flink-cmd, influxdb, jupyter, md, python, > r, spark, spark-submit, but no sh unfortunately. Can you reproduce? Is > that expected behavior? > > Thanks for comments or clearification. > > [1] > > https://github.com/apache/zeppelin/commit/194577f49aa77e9ff4117affdba65ed02a8bcc7c > > -- Best regards, Jongyoul Lee
Re: [ANNOUCE] Zeppelin 0.11.0 is released
Hello Pavan, Thank you for reporting it. I'll try to fix it by this weekend. Best regards, Jongyoul Lee 2024년 3월 8일 (금) 오전 11:01, Pavan Kotikalapudi 님이 작성: > Thanks for all the hard work to release the new version of Zeppelin. > > There is a small problem in docker image distributed though > https://hub.docker.com/r/apache/zeppelin/tags, > > Though it says the latest release is 0.11.0, Inside you can see that it is > built with version 0.10.0. > https://hub.docker.com/layers/apache/zeppelin/0.11.0/images/sha256-073da47d69effc8ad281ea5a4c802372e33b45e813440649966a2e4445136fc9?context=explore > . > > Can you validate if that is true? How can we fix this? > > Thank you, > > Pavan > > On 2024/02/11 17:24:29 Jongyoul Lee wrote: > > Hello, > > > > The Apache Zeppelin community is pleased to announce the availability > > of the 0.11.0 release. > > > > Zeppelin is a collaborative data analytics and visualization tool for > > distributed, general-purpose data processing system such as Apache > > Spark, Apache Flink, Apache Cassandra and etc. > > > > It is the first release in two years. The community improved to Java > > 11, Spark 3.5, Flink 1.17, and Python 3.9. > > > > We encourage you to download the latest release > > fromhttp://zeppelin.apache.org/download.html > > > > The easiest way to try Zeppelin is to run the shipped tutorial notes > > via docker image > > > > > https://zeppelin.apache.org/download.html#using-the-official-docker-image > > > > We welcome your help and feedback. For more information on the project > > and how to get involved, visit our website at > > http://zeppelin.apache.org/ > > > > Thank you all users and contributors who have helped to improve Apache > > Zeppelin. Welcome to join our community to discuss with others > > https://zeppelin.apache.org/community.html#mailing-list--slack-channel > > > > > > Best regards, > > > > Jongyoul Lee > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Binary package download
Hello, I fixed it. It was a problem with the legacy code. Best regards, Jongyoul Lee 2024년 3월 4일 (월) 오후 10:33, Jongyoul Lee 님이 작성: > Hello, > > I also checked it. Let me fix it asap. > > Best regards, > Jongyoul Lee > > 2024년 3월 4일 (월) 오후 5:32, Siegfried Lösch 님이 > 작성: > >> Dear all, >> >> I wanted to test apache zeppelin and started to try your latest binary >> packages at: >> >> https://zeppelin.apache.org/download.html >> >> There are two download links: "zeppelin-0.11.0-bin-all.tgz" and >> "zeppelin-0.11.0-bin-netinst.tgz". Both links are dead (tested using >> google-chrome as well as firefox at debian bookworm). For the moment I >> worked around to self compile zeppelin but noticed that my version is >> not fully functional via doing ipython tutorials. >> >> Therefore I appreciate any help! >> >> Thank you >> >> Siegfried Lösch >> >> > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Binary package download
Hello, I also checked it. Let me fix it asap. Best regards, Jongyoul Lee 2024년 3월 4일 (월) 오후 5:32, Siegfried Lösch 님이 작성: > Dear all, > > I wanted to test apache zeppelin and started to try your latest binary > packages at: > > https://zeppelin.apache.org/download.html > > There are two download links: "zeppelin-0.11.0-bin-all.tgz" and > "zeppelin-0.11.0-bin-netinst.tgz". Both links are dead (tested using > google-chrome as well as firefox at debian bookworm). For the moment I > worked around to self compile zeppelin but noticed that my version is > not fully functional via doing ipython tutorials. > > Therefore I appreciate any help! > > Thank you > > Siegfried Lösch > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[ANNOUCE] Zeppelin 0.11.0 is released
Hello, The Apache Zeppelin community is pleased to announce the availability of the 0.11.0 release. Zeppelin is a collaborative data analytics and visualization tool for distributed, general-purpose data processing system such as Apache Spark, Apache Flink, Apache Cassandra and etc. It is the first release in two years. The community improved to Java 11, Spark 3.5, Flink 1.17, and Python 3.9. We encourage you to download the latest release fromhttp://zeppelin.apache.org/download.html The easiest way to try Zeppelin is to run the shipped tutorial notes via docker image https://zeppelin.apache.org/download.html#using-the-official-docker-image We welcome your help and feedback. For more information on the project and how to get involved, visit our website at http://zeppelin.apache.org/ Thank you all users and contributors who have helped to improve Apache Zeppelin. Welcome to join our community to discuss with others https://zeppelin.apache.org/community.html#mailing-list--slack-channel Best regards, Jongyoul Lee
[RESULT] [VOTE] Release Apache Zeppelin 0.11.0 (RC1)
Hello, The vote passes with 4 binding +1 votes, 1 non-binding +1 votes, and not +0 or -1 votes. Thanks to everyone who verified rc and voted. +1: Michael Matsko Jeff Zhang* Philipp Dallig* Guanhua Li* Jongyoul Lee* +0: -1: *binding Vote thread: https://lists.apache.org/thread/dg4nn1dgoh52ss8owvm61jbdt3bo16zx Best regards, Jongyoul Lee
Re: [VOTE] Release Apache Zeppelin 0.11.0 RC1
Hello, Thank you for your approval. Let me release the new version. Best regards, Jongyoul Lee 2024년 2월 7일 (수) 오전 11:05, Guanhua Li 님이 작성: > +1 Thanks Jongyoul > > ---Original--- > *From:* "Jongyoul Lee" > *Date:* Mon, Feb 5, 2024 19:02 PM > *To:* "dev";"users"; > *Subject:* [VOTE] Release Apache Zeppelin 0.11.0 RC1 > > Hi folks, > > I propose the following RC to be released for the Apache Zeppelin 0.11.0 > release. > > The commit id is 523075a498ea6b72109fd817bab95f5d7fcb2829: > > https://github.com/apache/zeppelin/commit/523075a498ea6b72109fd817bab95f5d7fcb2829 > > This corresponds to the tag: v0.11.0-rc1: > https://github.com/apache/zeppelin/tree/v0.11.0-rc1 > > The release archives, signature, and checksums are here: > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.11.0-rc1/ > > The release candidate consists of the following source distributions > in zeppelin-0.11.0.tgz > > In addition, the following supplementary binary distributions are provided > for user convenience at the same location as zeppelin-0.11.0-bin-all-tgz > > You can find the KEYS file here: > https://dist.apache.org/repos/dist/release/zeppelin/KEYS > > Release notes are available at > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316221=12350539 > > Shortly, this release resolves 257 issues including fixing security issues > from the last release. > > Vote will be open for next 72 hours. (close at 4AM 8th Feb PDT) > > [] +1 approve > [] 0 no opinion > [] -1 disapprove (and reason why) > > Best regards, > Jongyoul Lee > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[VOTE] Release Apache Zeppelin 0.11.0 RC1
Hi folks, I propose the following RC to be released for the Apache Zeppelin 0.11.0 release. The commit id is 523075a498ea6b72109fd817bab95f5d7fcb2829: https://github.com/apache/zeppelin/commit/523075a498ea6b72109fd817bab95f5d7fcb2829 This corresponds to the tag: v0.11.0-rc1: https://github.com/apache/zeppelin/tree/v0.11.0-rc1 The release archives, signature, and checksums are here: https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.11.0-rc1/ The release candidate consists of the following source distributions in zeppelin-0.11.0.tgz In addition, the following supplementary binary distributions are provided for user convenience at the same location as zeppelin-0.11.0-bin-all-tgz You can find the KEYS file here: https://dist.apache.org/repos/dist/release/zeppelin/KEYS Release notes are available at https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316221=12350539 Shortly, this release resolves 257 issues including fixing security issues from the last release. Vote will be open for next 72 hours. (close at 4AM 8th Feb PDT) [] +1 approve [] 0 no opinion [] -1 disapprove (and reason why) Best regards, Jongyoul Lee
Re: invitation to slack channel
Hello Günter, I tried to add you to the channel but you are already a space member. Could you please check if you can access the channel named "#zeppelin" by yourself? Bets regards, Jongyoul 2023년 12월 16일 (토) 오후 11:13, guenterh.lists 님이 작성: > Hello, I would like to be invited to the zeppelin community slack channel. > > guenterh.li...@bluewin.ch > > Thanks > Günter > > -- > Günter Hipler > https://openbiblio.social/@vog61 > https://twitter.com/vog61 > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Official slack channel
Hello, I requested to invite you in the channel. Best regards, Jongyoul 2023년 11월 21일 (화) 오전 1:28, Carlos Brito Freitez 님이 작성: > Hi Jongyoul Lee, > > Could you please invite me to the slack channel as well. > > Thank you, > > Best, > > Eduardo Brito > Sent from my iPhone > > On Nov 19, 2023, at 21:03, Jongyoul Lee wrote: > > > Hello, > > I invited you to the channel. You will get some information soon. > > Best regards, > Jongyoul Lee > > 2023년 11월 20일 (월) 오후 1:56, Aescher Ebenalp <9960...@gmail.com>님이 작성: > >> HI, >> >> I want to be invited to the zeppelin community slack channel. >> Can you invite me? >> >> Thanks, >> Asher >> > > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Official slack channel
Hello, I invited you to the channel. You will get some information soon. Best regards, Jongyoul Lee 2023년 11월 20일 (월) 오후 1:56, Aescher Ebenalp <9960...@gmail.com>님이 작성: > HI, > > I want to be invited to the zeppelin community slack channel. > Can you invite me? > > Thanks, > Asher > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[Release] 0.11 preperation
Hello community, I hope this email finds you well. After we release 0.10.x, it took almost one and a half year. There have been a lot of contributions including improvement and changes, and I feel like it's time for the next release. I made the release branch of https://github.com/apache/zeppelin/tree/branch-0.11 I hope the release process could start within ten days. During this period, I'll review as many as I can and willing to accept proper contributions as well. If you can access the wiki of https://cwiki.apache.org/confluence/display/ZEPPELIN/Release+Roadmap, please feel free to check it and leave some comments about the next release. Best regards, Jongyoul Lee
Re: Helium Registry Gone?
Hello, You can install helium.json in your local. I have the latest one and will attach it soon. Then, for the future release, I’ll change the location to github repo or similar location. Best regards, Jongyoul 2023년 9월 22일 (금) 오후 7:13, Vladimir Prus 님이 작성: > > Hi, > > Zeppelin used to fetch Helium visualization plugins from the following URL > > https://s3.amazonaws.com/helium-package/helium.json > > As of now, that URL returns "All access to this object has been disabled" > error, and I don't see > any alternative URLs in the current code. > > We do enjoy some of the Helium visualizations, such as heatmap. Any > suggestions how to get them back in the most future-proof way? > > > -- > Vladimir Prus > http://vladimirprus.com >
Re: A question about pull requests for updates to the web site
Hello, I'm currently moving the source code for Zeppelin's web to https://github.com/apache/zeppelin-site. I want to make it easier for users to review the current code and contribute more easily. Best regards, Jongyoul 2023년 9월 9일 (토) 오전 12:33, Bill Brooks 님이 작성: > Hi Jongyoul, > > I am wrestling with getting Jekyll to generate the documentation website on > Mac OS right now, but I will come back to my first contribution to zeppelin > very soon. > > Thanks for the pointer! > > Bill > > > On Wed, Sep 6, 2023 at 9:08 AM Jongyoul Lee wrote: > > > Hello Bill, > > > > Sorry for the late reply. > > > > In general, Zeppelin's documentation has two different places. > > > > - https://github.com/apache/zeppelin/tree/master/docs > > - https://github.com/apache/zeppelin/tree/gh-pages > > > > I hope this helps you and I appreciate your interest and welcome your > > contribution. > > > > Best regards, > > Jongyoul > > > > > > > > 2023년 8월 31일 (목) 오전 7:30, Bill Brooks 님이 작성: > > > > > Hi All. > > > > > > I am preparing my first PR, for some updates to the website, and had a > > > question: > > > > > > Is the current version of the website, such as the contributors page > > here: > > > https://zeppelin.apache.org/contribution/contributions.html > > > > > > ...actually located in the GitHub repo? My upcoming PR is about the > > > now-outdated reference to travis-ci and I searched for that string, > using > > > the built-in GitHub search, against the GitHub repo here: > > > https://github.com/apache/zeppelin > > > > > > ..and the only result is a single mention in a shell/PTY4J.md > > > < > > > > > > https://github.com/apache/zeppelin/blob/ed958b538d67539cc8bc232668cce1163f51c468/shell/PTY4J.md?plain=1#L22 > > > > > > > file. The same page says that the official site for Zeppelin is > > > http://git.apache.org/zeppelin.git but trying to retrieve that URL > > results > > > in a 404 error. > > > > > > My upcoming PR is a change for the contributors page and I don't see > the > > > source for the website, but of course I could be overlooking it. > Running: > > > grep -c -i "travis-ci" *.md > > > > > > ...in the subdirectory 'docs/development/contribution' returns no > > matches. > > > > > > Thanks in advance for any insight you can provide. > > > > > > Bill > > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Passing a variable from an Angular paragraph to a Pyspark paragraph
Hello,
It's been a long time but do you still have the same issue? I'm not sure
about the feature but someone is using the similar features in my
understanding.
Do you have any update?
Regards,
Jongyoul
2023년 6월 16일 (금) 오전 12:36, alexander.2.kerr.bt.com via users <
users@zeppelin.apache.org>님이 작성:
> Hi,
>
>
>
> I’m trying to do a very simple thing – passing a variable (ideally an
> array, but string will do) from an Angular paragraph to a pyspark
> paragraph. Unfortunately I just get the result “None” when printing the
> value in pyspark.
>
>
>
> Setting the variable in my Angular paragraph (I’m using Javascript to do
> this for reasons related to the bigger project I’m working on and this
> works fine for passing to another %angular or %spark.spark paragraph):
>
>
>
> %angular
>
>
>
>
>
>
>
> $( document ).ready(function() {
>
> angFormScope =
> angular.element(document.getElementById('main_form')).scope(); // Set
> Angular scope variable now document is fully loaded and ready
>
> angFormScope.testvar = "hello"
>
> });
>
>
>
>
>
> Trying to read the variable in my pyspark paragraph:
>
>
>
> %spark.pyspark
>
> testvar = z.get("testvar");
>
> print(testvar);
>
>
>
> Output: None
>
>
>
> However, getting the variable in another angular paragraph works fine.
>
>
>
> I also seem to have access to the Zeppelin context in %pyspark and %python
> paragraphs, e.g. these work:
>
>
>
> %spark.pyspark
>
> print(z.z);
>
> Output: org.apache.zeppelin.spark.SparkZeppelinContext@2cd4a775
>
>
>
> %python
>
> # Gets the user's ID from the system, and puts it in the user_ID bound
> variable
>
> z.z.angularBind("user_ID",
> z.getInterpreterContext().getAuthenticationInfo().getUser())
>
> Output: gives the user id correctly
>
>
>
> %spark.pyspark
>
> z.z.getInterpreterContext().getNoteId()
>
> Output: Gives the note ID correctly.
>
>
>
> I notice from the python Zeppelin context source that get() should be
> available but the function call should be “def get(self, key)” – am I
> supposed to specify self somehow?
> https://github.com/apache/zeppelin/blob/master/python/src/main/resources/python/zeppelin_context.py#L58
>
>
>
> Think I just need the right syntax here unless there’s a bug in Zeppelin
> blocking this from working (unfortunately it’s running in a corporate
> environment with Zeppelin 0.8.2).
>
>
>
> Thanks for any info.
>
>
>
> Alex Kerr
>
>
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Error on approving request about account creation.
Hello, I'm getting error while I'm approving requests. I will handle this issue with INFRA so please wait for a moment. Regards, Jongyoul -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: How to reduce zeppelin docker image size ( from custom build)
Hello Rajat, Thank you for the detailed information. I recommend changing Dockerfile using "AS Builder". Zeppelin's docker images install all related dependencies even if you don't use those ones. Basically, It's fine to include your distribution, Java, and Flink if you want. Can you try to do it? JL 2023년 8월 8일 (화) 오후 5:19, Rajat Ahuja 님이 작성: > Hi, > > I am building my own image. so when i run the image it has following > content which is around *1.2 GB* > > > du -sh * > > > > > > > > > > > > > > > > > *80K bin180K conf758M interpreter24K k8s79M lib20K LICENSE76K licenses8.0K > local-repo166M logs14M notebook4.0K NOTICE104M plugins4.0K README.md4.0K > run400K scripts8.0K webapps28M zeppelin-web-0.11.0-SNAPSHOT.war* > > But docker size is > > > > *ahujarajat261/library_26_again > latest e26aed940485 3 days ago > 9.34GBartifactory.us.bank-dns.com:5000/flink/zeppelin-flink-us-bank-image > <http://artifactory.us.bank-dns.com:5000/flink/zeppelin-flink-us-bank-image> > latest e26aed940485 3 days ago 9.34GB* > > > *Here's the dockerfile > (https://github.com/apache/zeppelin/blob/master/scripts/docker/zeppelin/bin/Dockerfile > <https://github.com/apache/zeppelin/blob/master/scripts/docker/zeppelin/bin/Dockerfile>) > I am using after some modifications to build the image. Any suggestions as > to how I can reduce the size ?* > > https://github.com/apache/zeppelin/blob/master/scripts/docker/zeppelin/bin/Dockerfile > Thanks > > > On Mon, Aug 7, 2023 at 6:55 AM Vishal Rao wrote: > >> Yes I meant for Rajat (email starter) see >> https://zeppelin.apache.org/download.html there is the "all" package >> and the smaller "netinst" version too. >> >> On Mon, 7 Aug 2023 at 19:13, Jongyoul Lee wrote: >> > >> > Thank you. >> > >> > I'll make it as well and get back to you. We have some redundant >> libraries so it can be removed as well. >> > >> > Best regards, >> > Jongyoul >> > >> > 2023년 8월 7일 (월) 오후 10:38, Vishal Rao 님이 작성: >> >> >> >> Yep, instead of starting with the full package and removing stuff, try >> >> the "netinst" (tiny) package and add only what you need? >> >> >> >> On Mon, 7 Aug 2023 at 18:49, Jongyoul Lee wrote: >> >> > >> >> > Hello, >> >> > >> >> > I'm not sure but did you check your tgz? What's inside it? >> >> > >> >> > 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성: >> >> >> >> >> >> Hi team, >> >> >> >> >> >> I am trying to build a zeppelin from source code to run Flink. Here >> are the things i have done to build the image for only Flink interpreter >> only but seems like size is 9gb something of docker. How can i reduce the >> size of the docker image >> >> >> >> >> >> >> >> >> 1) I removed all possible interpreter modules ( like hbase, >> cassandra, mark down) from pom.xml. >> >> >> 2) mvn package clean -DskipTests >> >> >> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to >> zeppelin/scripts/docker/bin >> >> >> 4) change file names and docker file to build the image >> >> >> and image size is 9gb. Any idea how can i reduce this image size ? >> >> >> >> >> >> Thanks >> >> > >> >> > >> >> > >> >> > -- >> >> > 이종열, Jongyoul Lee, 李宗烈 >> >> > http://madeng.net >> >> >> >> >> >> >> >> -- >> >> "The World is a book, and those who do not travel read only a page." - >> >> St. Augustine. >> > >> > >> > >> > -- >> > 이종열, Jongyoul Lee, 李宗烈 >> > http://madeng.net >> >> >> >> -- >> "The World is a book, and those who do not travel read only a page." - >> St. Augustine. >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: helium repository down ?
Hello, Yes, it's down. We also lost some files being served in the same s3. Do you have any backup for the file? If so, I'll upload it to another place. Regards, Jongyoul Lee 2023년 8월 7일 (월) 오후 10:50, Sylvain Gibier 님이 작성: > Hi, > > we noticed that we are no longer able to download/ access to the helium > registry ? It seems https://s3.amazonaws.com/helium-package/helium.json is > no longer available. > > Known issue ? or where can I find the original helium registry to run it > on our own infrastructure? > > Cheers, > > Sylvain > > > *Sylvain Gibie*r > > cell: +49 172 666 2435 > > > > ape factory GmbH > Rumfordstr. 21a, 80469 München > https://apefactory.com' > > > Geschäftsführer: Sylvain Gibier > Sitz: München; Registergericht: München, HRB 221698 > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: How to reduce zeppelin docker image size ( from custom build)
Thank you. I'll make it as well and get back to you. We have some redundant libraries so it can be removed as well. Best regards, Jongyoul 2023년 8월 7일 (월) 오후 10:38, Vishal Rao 님이 작성: > Yep, instead of starting with the full package and removing stuff, try > the "netinst" (tiny) package and add only what you need? > > On Mon, 7 Aug 2023 at 18:49, Jongyoul Lee wrote: > > > > Hello, > > > > I'm not sure but did you check your tgz? What's inside it? > > > > 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성: > >> > >> Hi team, > >> > >> I am trying to build a zeppelin from source code to run Flink. Here are > the things i have done to build the image for only Flink interpreter only > but seems like size is 9gb something of docker. How can i reduce the size > of the docker image > >> > >> > >> 1) I removed all possible interpreter modules ( like hbase, cassandra, > mark down) from pom.xml. > >> 2) mvn package clean -DskipTests > >> 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to > zeppelin/scripts/docker/bin > >> 4) change file names and docker file to build the image > >> and image size is 9gb. Any idea how can i reduce this image size ? > >> > >> Thanks > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > > > > -- > "The World is a book, and those who do not travel read only a page." - > St. Augustine. > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: How to reduce zeppelin docker image size ( from custom build)
Hello, I'm not sure but did you check your tgz? What's inside it? 2023년 8월 6일 (일) 오전 6:10, Rajat Ahuja 님이 작성: > Hi team, > > I am trying to build a zeppelin from source code to run Flink. Here are > the things i have done to build the image for only Flink interpreter only > but seems like size is 9gb something of docker. How can i reduce the size > of the docker image > > > 1) I removed all possible interpreter modules ( like hbase, cassandra, > mark down) from pom.xml. > 2) mvn package clean -DskipTests > 3) Move zeppelin/distribution/zeppelin-0.11.0-snapshot.tgz to > zeppelin/scripts/docker/bin > 4) change file names and docker file to build the image > and image size is 9gb. Any idea how can i reduce this image size ? > > Thanks > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [!!Массовая рассылка][DISCUSSION] 0.11.0 Release
Hello Роза, Thank you for the feedback. My company also uses Zeppelin as an internal tool but I think the reason why the community looks inactive is Zeppelin is quite stable for some use cases. It's good for some aspects but it might be a sign that Zeppelin has no newer features any longer. Anyway, I hope we could talk about this issue further :-) For the release, I'll make a release branch within the next week and will do a release within the next month. For docker support, I totally agree with you that we should improve both the way and the result. I, however, don't know who can help to contribute to it so I hope we can help to improve it. I have permission to submit docker images to docker hub so don't worry about that. Best regards, Jongyoul 2023년 6월 29일 (목) 오후 6:53, Айсина Роза Мунеровна 님이 작성: > Hi Jongyoul Lee! > > > It is cool to see any activity in the community! I was under impression > that both the community and the project itself are dead. > > > Me and my team are currently integrating Zeppelin in production so we are > looking for new release with Spark 3.4.0 very much! > > > > Two questions: > > > - when new version will be released? > > > - "Fluent docker support" - right now on master base Docker image is very > huge (3.44 GB) and there is only one image with all kernels. Right now we > fork Zeppelin repo to our GitLab and build base image manually. It will be > cool that official repo can build several images with different kernel > groups. > > I mean official image on DHub: > https://hub.docker.com/r/apache/zeppelin/tags > > > Thanks in advance! > -- > *От:* Jongyoul Lee > *Отправлено:* 29 июня 2023 г. 12:39:37 > *Кому:* dev; users > *Тема:* [!!Массовая рассылка][DISCUSSION] 0.11.0 Release > > Внимание: Внешний отправитель! > Если вы не знаете отправителя - не открывайте вложения, не переходите по > ссылкам, не пересылайте письмо! > > Hello community, > > I made a wiki page for the next release. > > - https://cwiki.apache.org/confluence/display/ZEPPELIN/Release+Roadmap > > Could you please check the wiki and give some feedback? > > Here is the content of wiki for those who cannot access the page. > >- Junit5 >- Spark 3.4.0 >- Minimize default interpreters > - Remove Shell interpreter to minimize security issues > - Download interpreters easily > - Could use Github release for individual interpreters > - Fluent docker support > > > > Best regards, > Jongyoul Lee > > > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net > > *--* > > *Роза Айсина* > > Старший разработчик ПО > > *СберМаркет* | Доставка из любимых магазинов > > > > Email: roza.ays...@sbermarket.ru > > Mob: +7 (915) 360-62-87 > > Web: sbermarket.ru > > App: iOS > <https://apps.apple.com/ru/app/%D1%81%D0%B1%D0%B5%D1%80%D0%BC%D0%B0%D1%80%D0%BA%D0%B5%D1%82-%D0%B4%D0%BE%D1%81%D1%82%D0%B0%D0%B2%D0%BA%D0%B0-%D0%BF%D1%80%D0%BE%D0%B4%D1%83%D0%BA%D1%82%D0%BE%D0%B2/id1166642457> > и Android > <https://play.google.com/store/apps/details?id=ru.instamart=en=ru> > > > > *УВЕДОМЛЕНИЕ О КОНФИДЕНЦИАЛЬНОСТИ:* это электронное сообщение и любые > документы, приложенные к нему, содержат конфиденциальную информацию. > Настоящим уведомляем Вас о том, что, если это сообщение не предназначено > Вам, использование, копирование, распространение информации, содержащейся в > настоящем сообщении, а также осуществление любых действий на основе этой > информации, строго запрещено. Если Вы получили это сообщение по ошибке, > пожалуйста, сообщите об этом отправителю по электронной почте и удалите это > сообщение. > *CONFIDENTIALITY NOTICE:* This email and any files attached to it are > confidential. If you are not the intended recipient you are notified that > using, copying, distributing or taking any action in reliance on the > contents of this information is strictly prohibited. If you have received > this email in error please notify the sender and delete this email. > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[DISCUSSION] 0.11.0 Release
Hello community, I made a wiki page for the next release. - https://cwiki.apache.org/confluence/display/ZEPPELIN/Release+Roadmap Could you please check the wiki and give some feedback? Here is the content of wiki for those who cannot access the page. - Junit5 - Spark 3.4.0 - Minimize default interpreters - Remove Shell interpreter to minimize security issues - Download interpreters easily - Could use Github release for individual interpreters - Fluent docker support Best regards, Jongyoul Lee -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Invite to Slack? shiro realm not accepting jwt token
Hello Rutland,
I invited you to slack.
BTW, I'm not using Know now so I couldn't help you but how about
mankind jira issue to report it if it's a bug?
Regards,
Jongyoul
2023년 6월 1일 (목) 오전 1:17, Rutland, Nathan (CTR) <
nathan.rutl...@mail.associates.cisa.dhs.gov>님이 작성:
> I'd like to get an invite to the Slack channel for apache zeppelin. I'm
> working a tough shiro config to get PIV / SAML login working into Zeppelin
> 0.10 on AWS EMR. Everything looks okay with the saml request and response
> through Apache Knox. But when the jwt gets back to zeppelin (after the
> redirect to knox and idp), it's saying the realm can't accept the jwt.
>
> From zeppelin:
> DEBUG [2023-05-31 00:12:18,700] ({qtp1521083627-118}
> AuthenticatingRealm.java[getAuthenticationInfo]:583) - No
> AuthenticationInfo found for submitted AuthenticationToken
> [org.apache.zeppelin.realm.jwt.JWTAuthenticationToken@3924c982].
> Returning null.
> org.apache.shiro.authc.UnknownAccountException: Realm
> [org.apache.zeppelin.realm.jwt.KnoxJwtRealm@31a2833d] was unable to find
> account data for the submitted AuthenticationToken
> [org.apache.zeppelin.realm.jwt.JWTAuthenticationToken@3924c982].
>
> The shiro.ini:
>
> [main]
> ### A sample for configuring Knox JWT Realm
> knoxJwtRealm = org.apache.zeppelin.realm.jwt.KnoxJwtRealm
> ## Domain of Knox SSO
> knoxJwtRealm.providerUrl = https://knoxsso.domain:8443/
> knoxJwtRealm.login = gateway/knoxsso/api/v1/websso
> knoxJwtRealm.logout = gateway/knoxssout/api/v1/webssout
> knoxJwtRealm.logoutAPI = true
> knoxJwtRealm.redirectParam = originalUrl
> knoxJwtRealm.cookieName = hadoop-jwt
> knoxJwtRealm.publicKeyPath = /etc/pki/tls/certs/ae-emr-sso.crt
> # This is required if KNOX SSO is enabled, to check if
> "knoxJwtRealm.cookieName" cookie was expired/deleted.
> authc = org.apache.zeppelin.realm.jwt.KnoxAuthenticationFilter
> anyofrolesuser =
> org.apache.zeppelin.utils.AnyOfRolesUserAuthorizationFilter
>
> sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
>
> cookie = org.apache.shiro.web.servlet.SimpleCookie
> cookie.name = JSESSIONID
> cookie.secure = false
> cookie.httpOnly = false
> sessionManager.sessionIdCookie = $cookie
>
> securityManager.sessionManager = $sessionManager
> # 86,400,000 milliseconds = 24 hour
> securityManager.sessionManager.globalSessionTimeout = 8640
> shiro.loginUrl = /api/login
>
>
> [urls]
> /api/version = anon
> /api/interpreter/setting/restart/** = authc
> /api/interpreter/** = authc, anyofrolesuser[admin, nrutland]
> /api/configurations/** = anyofrolesuser[admin, nrutland]
> /api/credential/** = anyofrolesuser[admin, nrutland]
> /** = authc
>
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: Tutorial bank.csv returns access denied
Hello Zoltan, Done. 2023년 6월 29일 (목) 오후 4:57, Zoltan Holoda 님이 작성: > Hello Jongyoul! > > Thank you, I appreciate your help. > I filed a jira ticket about this issue, please resolve it. > https://issues.apache.org/jira/browse/ZEPPELIN-5929 > > Best regards, > Zoltan > > On Thu, Jun 29, 2023 at 9:37 AM Jongyoul Lee wrote: > > > > Hello Zoltan, > > > > Thank you for reporting it. > > > > We changed the link to > https://raw.githubusercontent.com/apache/zeppelin/master/testing/resources/bank.csv > > > > The change has already been merged so you can see the repository as well. > > > > Best regards, > > Jongyoul > > > > 2023년 6월 6일 (화) 오후 7:21, Zoltan Holoda 님이 작성: > >> > >> Hello Team! > >> > >> Links were missing, sorry. > >> - bank.csv s3 link: > >> https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv > >> - bank.csv asf link: > >> > https://github.com/apache/zeppelin/blob/master/livy/src/test/resources/livy_tutorial_1.scala#L11 > >> > >> Regards, > >> Zoltan > >> > >> On 2023/06/06 10:07:39 Zoltan Holoda wrote: > >> > Hello Team! > >> > > >> > Recently Zeppelin Bank Tutorial bank.csv hosted by S3 returns HTTP 403 > >> > (Access Denied). src > >> > Can someone tell me if this change is intentional or some minor ACL > issue? > >> > > >> > Regards, > >> > Zoltan > >> > > > > > > > > > -- > > 이종열, Jongyoul Lee, 李宗烈 > > http://madeng.net > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Custom Docker images: need help!
Hello Айсина Роза Мунеровна, I think you can modify the Dockerfile of https://github.com/apache/zeppelin/blob/master/Dockerfile Can you try this one? Regards, Jongyoul 2023년 4월 29일 (토) 오전 2:04, Айсина Роза Мунеровна 님이 작성: > Hola! > > I am new user to Apache Zeppelin. > My goal is to dynamically lunch Zeppelin setup with Kubernetes: > > - Zeppelin Server > - Spark Kernel == Spark Driver with custom Python environment (run by > poetry package manager) > - Spark executors with the same Python env. > > My problem is that current Zeppelin image has old Python and there is no > way to parametrize Python version > (especially that miniconda is old and does not support 3.11). > Also I need to make sure that Python in driver (== Zeppelin kernel) and > Python on Spark executors is the same. > > What I want to build is Python environment with Spark cluster run by > Zeppelin in k8s. > > Is it possible? > I see some Dockerfiles in Zeppeline pack - is it valid to use them? > There is no word about this Dockerfiles in docs :( > > Thanks in advance! > -- > > Роза Айсина > > Старший разработчик ПО > > СберМаркет | Доставка из любимых магазинов > > > > Email: roza.ays...@sbermarket.ru<mailto:roza.ays...@sbermarket.ru> > > Mob: > > Web: sbermarket.ru<https://sbermarket.ru/> > > App: iOS< > https://apps.apple.com/ru/app/%D1%81%D0%B1%D0%B5%D1%80%D0%BC%D0%B0%D1%80%D0%BA%D0%B5%D1%82-%D0%B4%D0%BE%D1%81%D1%82%D0%B0%D0%B2%D0%BA%D0%B0-%D0%BF%D1%80%D0%BE%D0%B4%D1%83%D0%BA%D1%82%D0%BE%D0%B2/id1166642457> > и Android< > https://play.google.com/store/apps/details?id=ru.instamart=en=ru> > > > УВЕДОМЛЕНИЕ О КОНФИДЕНЦИАЛЬНОСТИ: это электронное сообщение и любые > документы, приложенные к нему, содержат конфиденциальную информацию. > Настоящим уведомляем Вас о том, что, если это сообщение не предназначено > Вам, использование, копирование, распространение информации, содержащейся в > настоящем сообщении, а также осуществление любых действий на основе этой > информации, строго запрещено. Если Вы получили это сообщение по ошибке, > пожалуйста, сообщите об этом отправителю по электронной почте и удалите это > сообщение. > CONFIDENTIALITY NOTICE: This email and any files attached to it are > confidential. If you are not the intended recipient you are notified that > using, copying, distributing or taking any action in reliance on the > contents of this information is strictly prohibited. If you have received > this email in error please notify the sender and delete this email. > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Tutorial bank.csv returns access denied
Hello Zoltan, Thank you for reporting it. We changed the link to https://raw.githubusercontent.com/apache/zeppelin/master/testing/resources/bank.csv The change has already been merged so you can see the repository as well. Best regards, Jongyoul 2023년 6월 6일 (화) 오후 7:21, Zoltan Holoda 님이 작성: > Hello Team! > > Links were missing, sorry. > - bank.csv s3 link: > https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv > - bank.csv asf link: > > https://github.com/apache/zeppelin/blob/master/livy/src/test/resources/livy_tutorial_1.scala#L11 > > Regards, > Zoltan > > On 2023/06/06 10:07:39 Zoltan Holoda wrote: > > Hello Team! > > > > Recently Zeppelin Bank Tutorial bank.csv hosted by S3 returns HTTP 403 > > (Access Denied). src > > Can someone tell me if this change is intentional or some minor ACL > issue? > > > > Regards, > > Zoltan > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Cannot build zeppelin-web due to fedops-grafana-api@^2.0.0
Hello, I'm not a frontend developer but did you check your node version? JL 2022년 11월 10일 (목) 오후 3:40, 이상노 님이 작성: > Hello > > I cannot build zeppelin-web with below error logs : > > npm ERR! npm WARN using --force Recommended protections disabled. > npm ERR! npm ERR! code ETARGET > npm ERR! npm ERR! notarget No matching version found for > fedops-grafana-api@^2.0.0. > npm ERR! npm ERR! notarget In most cases you or one of your dependencies > are requesting > npm ERR! npm ERR! notarget a package version that doesn't exist. > > So I found fedops-grafana-api on > https://www.npmjs.com/package/fedops-grafana-api > but I cannot install that package. > > And I find same issue mailing list > https://www.mail-archive.com/users@zeppelin.apache.org/msg05530.html > > But I cannot find any solution. > > Please give me any feedback > > Thanks! > Sincerely > Sangrho Lee > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Road Map
Hello, Actually, we are discussing the next release in dev@ but it's not decided anything yet. I, however, also think it's been a very long time since we release the latest version. > Is it planned to set Angular application as the default UI? AFAIK, we, strongly, change the default UI for angular UI but haven't tested it well yet. I appreciate it if you test it. > Is it planned to migrate Zeppelin from Java 8 to 11? Some PRs are trying to upgrade JDK to 11. - https://lists.apache.org/thread/bkwx768lo570os1p222z405nllmbthrl - https://github.com/apache/zeppelin/pull/4497 Hope this helps, JL 2023년 2월 6일 (월) 오전 5:34, Baptiste Meynier 님이 작성: > Hi, > Thx a lot for your work, Zeppelin is realy a great application! I would > like to know where I could find the road map of Zeppelin? Inside the > project I find this url > <https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap> > that does not seems to be updated. Then I looking for information inside > the Zeppelin issues tracker for the release 0.11.0. > I see that there are 4 opened jiras for version 0.11.0. It will be the > last Jira for version 0.11.0 or some Jira issues could be added ? > > I have some questions for the next feature of Zeppelin: > Is it planned to set Angular application as the default UI? > Is it planned to migrate Zeppelin from Java 8 to 11? > > Thx a lot ! > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[DISCUSS] Change mailing list scheme
Hello, I suggest some changes to our mailing list. Currently, our dev@ mailing list has a lot of messages including Github logs, Jira comments, and dev issues. I prepared a proposal: https://cwiki.apache.org/confluence/display/ZEPPELIN/(WIP)+Mailing+Scheme Here is the link for dev@: https://lists.apache.org/list.html?d...@zeppelin.apache.org I will appreciate it if you check the link and give some feedback. Best regards, Jongyoul Lee -- 이종열, Jongyoul Lee, 李宗烈
Fwd: TAC supporting Berlin Buzzwords
Hello, Here is useful information for attendees. Please check the email below. Hope this helps, Jongyoul -- Forwarded message - 보낸사람: Gavin McDonald Date: 2023년 3월 24일 (금) 오후 6:57 Subject: TAC supporting Berlin Buzzwords To: PMCs, Please forward to your dev and user lists. Hi All, The ASF Travel Assistance Committee is supporting taking up to six (6) people to attend Berlin Buzzwords In June this year. This includes Conference passes, and travel & accommodation as needed. Please see our website at https://tac.apache.org for more information and how to apply. Applications close on 15th April. Good luck to those that apply. Gavin McDonald (VP TAC) -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Interpreter maintenance
Hello Andrea, I don't have any concrete plan for Neo4j yet but some contributors already submitted the PR to remove ksql interpreter. If there's no plan to maintain ksql interpreter by some contributors, I guess it would be better to remove it and revert if it's necessary. By the way, thank you for the update about the Neo4j interpreter. We will remain it as it is. Best regards, Jongyoul 2022년 7월 31일 (일) 오후 8:51, Andrea Santurbano 님이 작성: > Ho Jongyoul > I created the ksql and Neo4j interpreters. In particular the last one is > widely used by Neo users. Do you plan to cut it out? > > Il giorno dom 31 lug 2022 alle 12:34 Jongyoul Lee ha > scritto: > >> Hello, >> >> I'm currently working with several contributors about the issue of what >> interpreters we will focus on. >> >> My plan is to minimize the number of interpreters and focus on managing >> it very actively so the first step is to remove outdated and less active >> interpreters. Please check the status and let us know if you really want to >> keep some interpreters. We will keep them updated as much as possible so >> please update the version and the usage. Please take a look into it. >> >> Here is the reference: >> >> https://cwiki.apache.org/confluence/display/ZEPPELIN/Interpreter+Maintenance >> >> Regards, >> Jongyoul Lee >> >> -- >> 이종열, Jongyoul Lee, 李宗烈 >> http://madeng.net >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Interpreter maintenance
Hello, I'm currently working with several contributors about the issue of what interpreters we will focus on. My plan is to minimize the number of interpreters and focus on managing it very actively so the first step is to remove outdated and less active interpreters. Please check the status and let us know if you really want to keep some interpreters. We will keep them updated as much as possible so please update the version and the usage. Please take a look into it. Here is the reference: https://cwiki.apache.org/confluence/display/ZEPPELIN/Interpreter+Maintenance Regards, Jongyoul Lee -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: can't build zeppelin
Hello, Zeppelin uses v12.3.1 6.9.0 So I think we'd better use the same version to compile the frontend directly. Regards, Jongyoul 2022년 3월 29일 (화) 오후 1:08, LuNing Wang 님이 작성: > Hi all, > > I try to build zeppelin in 0.10-branch or master. When I run this command > `npm install --no-lockfile` in zeppelin-web dir, throw the following error. > and I run `mvn clean install -Dskiptests` in project dir throw the > following error too. > > ``` > npm ERR! code 1 > npm ERR! git dep preparation failed > npm ERR! command /Users/ada/.nvm/versions/node/v16.14.0/bin/node > /Users/ada/.nvm/versions/node/v16.14.0/lib/node_modules/npm/bin/npm-cli.js > install --force --cache=/Users/ada/.npm --prefer-offline=false > --prefer-online=false --offline=false --no-progress --no-save --no-audit > --include=dev --include=peer --include=optional --no-package-lock-only > --no-dry-run > npm ERR! npm WARN using --force Recommended protections disabled. > npm ERR! npm ERR! code ETARGET > npm ERR! npm ERR! notarget No matching version found for > fedops-grafana-api@^2.0.0. > npm ERR! npm ERR! notarget In most cases you or one of your dependencies > are requesting > npm ERR! npm ERR! notarget a package version that doesn't exist. > npm ERR! > npm ERR! npm ERR! A complete log of this run can be found in: > npm ERR! npm ERR! > /Users/ada/.npm/_logs/2022-03-29T03_49_17_980Z-debug-0.log > > npm ERR! A complete log of this run can be found in: > npm ERR! /Users/ada/.npm/_logs/2022-03-29T03_41_06_493Z-debug-0.log > ``` > > I can not find ` fedops-grafana-api@^2.0.0`, what is that? > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Interpreter maintenance
Hello, I made a proposal for maintaining interpreters for the future. - https://cwiki.apache.org/confluence/display/ZEPPELIN/Interpreter+Maintenance Please check it and leave your comments freely on the thread of this email, comments on the wiki, and the threads of the slack channel. References - https://the-asf.slack.com/archives/C4RDKR51R/p1632495099017000 - https://lists.apache.org/thread/cxzsortyypgh4mnydzxfnljfnwnv2t5c Regards, Jongyoul Lee -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Slack Channel Invitation
I asked for it 2021년 12월 21일 (화) 오후 4:38, zeeshan ali 님이 작성: > Hello Guys > I hope everybody is doing Ok, just wanted an invitation to the Slack > channel. Is this possible? > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [VOTE] Release Apache Zeppelin 0.10.0 RC2
+1 I've checked the basic example about this RC 2021년 8월 18일 (수) 오후 3:28, 님이 작성: > +1 > > > > *From:* Philipp Dallig > *Sent:* Wednesday, August 18, 2021 3:24 PM > *To:* users@zeppelin.apache.org > *Subject:* Re: [VOTE] Release Apache Zeppelin 0.10.0 RC2 > > > > +1 > > Am 18.08.21 um 08:01 schrieb Francois Reynald: > > +1 > > > > On 18 Aug 2021, at 05:52, Jeff Zhang wrote: > > > > > Hi folks, > > > > I propose the following RC to be released for the Apache Zeppelin 0.10.0 > release. > > > > > > The commit id is 8f85476c40df3536718c5aa2c07b519b6bb6607e : > > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=8f85476c40df3536718c5aa2c07b519b6bb6607e > > > > This corresponds to the tag: v0.10.0-rc2 : > > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.10.0-rc2 > > The release archives (tgz), signature, and checksums are here > > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.10.0-rc2/ > > > > The release candidate consists of the following source distribution archive > > zeppelin-0.10.0.tgz > > > > In addition, the following supplementary binary distributions are provided > > for user convenience at the same location > > zeppelin-0.10.0-bin-all.tgz > > > > > > The maven artifacts are here > > https://repository.apache.org/content/repositories/orgapachezeppelin-1302/org/apache/zeppelin/ > > > > You can find the KEYS file here: > > https://dist.apache.org/repos/dist/release/zeppelin/KEYS > > > > Release notes available at > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349536==12316221 > > > > Vote will be open for next 72 hours (close at 10PM 20/Aug PDT). > > > > [ ] +1 approve > > [ ] 0 no opinion > > [ ] -1 disapprove (and reason why) > > > > > > -- > > Best Regards > > Jeff Zhang > > > > *François Reynald | Educational Services* > t. +33 (0) 624 200 125 > cloudera.com <http://www.cloudera.com> > > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [DISCUSS] Zeppelin community sync up meeting
+1 2021년 1월 28일 (목) 오전 5:00, Felix Cheung 님이 작성: > +1 > > -- > *From:* Eric Pugh > *Sent:* Tuesday, January 26, 2021 5:28:59 AM > *To:* users > *Subject:* Re: [DISCUSS] Zeppelin community sync up meeting > > I’d advocate for a regular quarterly run well versus a monthly that ends > up sparsely attended and a scramble to setup. The general idea is > fantastic. > > On Jan 25, 2021, at 10:37 PM, moon soo Lee wrote: > > +1 > > Looking forward to it! > > On Mon, Jan 25, 2021 at 6:20 PM Jeff Zhang wrote: > > Hi Bill, > > That's right, it has been more than one year we haven't do community sync > up. I'd like to continue that to bring the community together for more wide > discussion around zeppelin. > > Bill Brooks 于2021年1月26日周二 上午10:11写道： > > This is a great idea, but I thought that Mei Long was already putting > these events together and prior to the pandemic, they were happening. > > Bill > > On Mon, Jan 25, 2021 at 3:09 PM Jeff Zhang wrote: > > > > Hi folks (users & developers), > > > > I'd like to raise this discussion about regular community sync up > meeting to discuss various things of Zeppelin. You may know me that I have > been active in the Zeppelin community for the last 4 years. I believe > Zeppelin is a very promising and important project in the data science and > big data area. But it needs more community involvement to improve it ,no > matter whether you are zeppelin users or zeppelin developers. I hope this > kind of sync up meeting can gather people who are interested in zeppelin > together, and let's discuss what we can do to make zeppelin standout in the > data science and big data area. > > > > I hope it will be a regular sync up meeting, maybe one time per month. > Let us know whether you are interested in such kind of regular sync up > meeting and what kind of things do you like to discuss. > > > > -- > > Best Regards > > > > Jeff Zhang > > > > -- > Best Regards > > Jeff Zhang > > > ___ > *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 > | http://www.opensourceconnections.com | My Free/Busy > <http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed > <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless > of whether attachments are marked as such. > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [Proposal] Zeppelin Client API (Zeppelin SDK)
Sounds good to me as well.
BTW, don't we have to change some web socket code as well? <- It's just an
idea.
I would like to see PoC for it and I'm willing to help to improve it
because my company needs it too.
2020년 8월 1일 (토) 오전 12:05, Jeff Zhang 님이 작성:
> Good idea, Moon, I think cli would be a good start for users. This kind of
> cli is a super combination of spark-shell, hive beeline, flink scala shell,
> python shell.
>
> e.g.
> >>> %spark sc.version
> >>> %hive show tables
> >>> %python print('hello world')
>
>
>
> moon soo Lee 于2020年7月31日周五 下午10:55写道：
>
> > Looks really good!
> > Do you think to have a CLI on top of high-level API be also useful?
> >
> >
> > On Fri, Jul 31, 2020 at 7:13 AM Jeff Zhang wrote:
> >
> > > Thanks for your feedback, I have created ticket
> > > https://issues.apache.org/jira/browse/ZEPPELIN-4981
> > >
> > > Just like @Alex Ott mentioned, this api may need
> > > several iterations, so I plan to make it as an experimental feature
> > > first, and then refine the api based on user feedback.
> > > .
> > >
> > > Alex Ott 于2020年7月31日周五 下午7:48写道：
> > >
> > >> Idea is very good - I think that we'll need to get several iterations
> of
> > >> API refinement, but current approach looks promising.
> > >>
> > >> On Fri, Jul 24, 2020 at 5:54 PM Jeff Zhang wrote:
> > >>
> > >> > Hi Folks,
> > >> >
> > >> > I'd like to discuss this proposal with you about the zeppelin client
> > api
> > >> > (zeppelin sdk).
> > >> > The background is that now Zeppelin’s main usage scenario is
> > interactive
> > >> > data analysis. Although it provides rest api, it is not easy for an
> > >> > external system (e.g. scheduler system) to integrate Zeppelin for
> the
> > >> > scenario where zeppelin is used as a backend job service. So I
> propose
> > >> to
> > >> > introduce a new module: Zeppelin client api (Zeppelin SDK), whose
> > >> purpose
> > >> > is to provide easy api for external systems to integrate zeppelin.
> > >> >
> > >> > I have created a google doc for the details, welcome any comments
> and
> > >> > feedback.
> > >> >
> > >> >
> > >> >
> > >>
> >
> https://docs.google.com/document/d/1bLLKKxleZlZpP9EFJlLLkJKwDBps-RNvzNwh3LFZWZ4/edit?usp=sharing
> > >> >
> > >> >
> > >> > --
> > >> > Best Regards
> > >> >
> > >> > Jeff Zhang
> > >> >
> > >>
> > >>
> > >> --
> > >> With best wishes,Alex Ott
> > >> http://alexott.net/
> > >> Twitter: alexott_en (English), alexott (Russian)
> > >>
> > >
> > >
> > > --
> > > Best Regards
> > >
> > > Jeff Zhang
> > >
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: [VOTE] Release Apache Zeppelin 0.9.0-preview2 (RC1)
+1 2020년 7월 19일 (일) 08:51, Surjan S Rawat 님이 작성: > +1 > > On Fri, Jul 17, 2020 at 9:05 AM Eric Pugh > wrote: > >> Very excited to see preview2, 0.9.0 has been much more stable then >> previous versions, and the ipyn export makes it easier to use with more >> clients in my work. >> >> +1 (non binding) >> >> On Jul 17, 2020, at 11:27 AM, Yadong Xie wrote: >> >> Thanks, Jeff >> +1 from my side >> >> Jeff Zhang 于2020年7月17日周五 下午11:25写道： >> >>> Hi folks, >>> >>> I propose the following RC to be released for the Apache Zeppelin >>> 0.9.0-preview2 release. >>> >>> >>> The commit id is a74365c0813b451db1bc78def7d1ad1279429224 : >>> >>> https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=commit;h=dd2058395ad4cf08fb6bdc901ec0c426c5095a94 >>> >>> This corresponds to the tag: v0.9.0-preview1-rc2 : >>> >>> https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.9.0-preview2-rc1 >>> >>> The release archives (tgz), signature, and checksums are here >>> >>> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.9.0-preview2-rc1/ >>> >>> The release candidate consists of the following source distribution >>> archive zeppelin-v0.9.0-preview2.tgz >>> >>> In addition, the following supplementary binary distributions are >>> provided >>> for user convenience at the same location >>> zeppelin-0.9.0-preview2-bin-all.tgz >>> >>> >>> The maven artifacts are here >>> >>> https://repository.apache.org/content/repositories/orgapachezeppelin-1279/org/apache/zeppelin/ >>> >>> You can find the KEYS file here: >>> https://dist.apache.org/repos/dist/release/zeppelin/KEYS >>> >>> Release notes available at >>> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342692==12316221 >>> >>> Vote will be open for next 72 hours (close at 8am 20/July PDT). >>> >>> [ ] +1 approve >>> [ ] 0 no opinion >>> [ ] -1 disapprove (and reason why) >>> >>> >>> -- >>> Best Regards >>> >>> Jeff Zhang >>> >> >> ___ >> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 >> | http://www.opensourceconnections.com | My Free/Busy >> <http://tinyurl.com/eric-cal> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >> This e-mail and all contents, including attachments, is considered to be >> Company Confidential unless explicitly stated otherwise, regardless >> of whether attachments are marked as such. >> >> -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [VOTE] Release Apache Zeppelin 0.9.0-preview1 (RC1)
+1 2020년 3월 24일 (화) 오전 12:15, Jeff Zhang 님이 작성: > Hi folks, > > I propose the following RC to be released for the Apache Zeppelin > 0.9.0-preview1 release. > > > The commit id is a74365c0813b451db1bc78def7d1ad1279429224 : > > https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=commit;h=a74365c0813b451db1bc78def7d1ad1279429224 > > This corresponds to the tag: v0.9.0-preview1-rc1 : > > https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.9.0-preview1-rc1 > > The release archives (tgz), signature, and checksums are here > > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.9.0-preview1-rc1/ > > The release candidate consists of the following source distribution archive > zeppelin-v0.9.0-preview1.tgz > > In addition, the following supplementary binary distributions are provided > for user convenience at the same location > zeppelin-0.9.0-preview1-bin-all.tgz > > > The maven artifacts are here > > https://repository.apache.org/content/repositories/orgapachezeppelin-1276/org/apache/zeppelin/ > > You can find the KEYS file here: > https://dist.apache.org/repos/dist/release/zeppelin/KEYS > > Release notes available at > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342692==12316221 > > Vote will be open for next 72 hours (close at 7am 26/March PDT). > > [ ] +1 approve > [ ] 0 no opinion > [ ] -1 disapprove (and reason why) > > > -- > Best Regards > > Jeff Zhang > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [DISCUSS] Release for 0.9.0-preview-1
+1 2020년 3월 17일 (화) 오후 11:29, Sam Nicholson 님이 작성: > +1 > > On Mon, Mar 16, 2020, 10:45 Jeff Zhang wrote: > >> Hi Folks, >> >> I'd like to start the discussion of releasing of 0.9.0-preview-1. As we >> discussed before releasing a preview version first is suitable for a major >> release based on our previous experience[1]. The last major release 0.8.0 >> is in 2018, I do believe we should release 0.9 to allow more people try >> that and gather feedback. What do you think ? >> >> >> [1] >> https://lists.apache.org/thread.html/3812fd7dab53e1fed4140c826fc69fa5b718636371b6c7c677b4ae88%40%3Cdev.zeppelin.apache.org%3E >> >> -- >> Best Regards >> >> Jeff Zhang >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [ANNOUNCE] Welcome Alex Ott as New Committer
That's really great!! I appreciate his activities against Apache Zeppelin and I'm really happy as I have already known how difficult to contribute an opensource continuously. Please keep helping make Apache Zeppelin healthy. Regards, JL 2020년 1월 7일 (화) 오후 6:48, Jeff Zhang 님이 작성: > The Project Management Committee (PMC) for Apache Zeppelin > has invited Alex Ott to become a committer and we are very pleased > to announce that he has accepted. > > We greatly appreciate all of Alex's hard work and generous contributions to > the project. We look forward to continued involvement in the project. > > Congratulations & Welcome aboard Alex ! > > > -- > Best Regards > > Jeff Zhang > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Zeppelin Studio Proposal
Sounds great!! On Tue, Aug 27, 2019 at 1:31 AM ieglonewolf ieglonewolf < ieglonew...@gmail.com> wrote: > I would like to add > > Our key motive for us to start ZEPPELIN-4138 > <https://issues.apache.org/jira/browse/ZEPPELIN-4138> here is to build a > UI > system on top of zeppelin service which is intuitive and very much > functional. > Moreover doing it the right way is the key to scalability. I would like to > request each and every member of this community to help us develop this. > > Please provide your valuable feedback > > Thanks folks! > > On Mon, Aug 26, 2019 at 9:24 PM Xun Liu wrote: > > > Zeppelin is very much in need of a front end developed using VUE.js > > front-end technology. > > Thank you for your contribution. :-) > > > > Xun Liu > > Best Regards > > > > On Aug 26, 2019, at 9:21 PM, Jeff Zhang wrote: > > > > + user mail list > > > > > > Thanks Malay for the proposal. The zeppelin frontend do needs some > rework. > > > > Overall, your proposal make sense to me. User experience and performance > > are the 2 key things we need to improve in the frontend. > > I left some comments in the design doc. > > > > > > Malay Majithia 于2019年8月23日周五 下午6:10写道： > > > >> Hey Folks, > >> > >> Regarding ZEPPELIN-4138 > >> <https://issues.apache.org/jira/browse/ZEPPELIN-4138>, we have come up > >> with the design document(draft) and the task list for the same: > >> > >> > >> Zeppelin Studio - Design Document > >> < > https://docs.google.com/document/d/1FCAVgODwYtpXUxtTdB3JVGrVNEqEzpUA6Dcyi6xvpB4/edit?usp=drive_web > > > >> > >> Zeppelin Studio - Task list > >> < > https://docs.google.com/spreadsheets/d/1cPI4t0Wqm0JVxjkLPNpah1I9vW-XmEcd9eDYISTV0iw/edit?usp=drive_web > > > >> > >> POC code is available on GitHub > >> <https://github.com/malayhm/zeppelin-studio>. > >> > >> Sneak peek of the proposed interface: > >> > >> > >> > >> Please review it and provide your valuable feedback. > >> > >> Best Regards > >> Malay Majithia > >> > >> > > > > -- > > Best Regards > > > > Jeff Zhang > > > > > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: install helium packages
Here is configurations related to Helium:
ZEPPELIN_HELIUM_REGISTRY("zeppelin.helium.registry", "helium," +
HELIUM_PACKAGE_DEFAULT_URL),
ZEPPELIN_HELIUM_NODE_INSTALLER_URL("zeppelin.helium.node.installer.url",
"https://nodejs.org/dist/;),
ZEPPELIN_HELIUM_NPM_INSTALLER_URL("zeppelin.helium.npm.installer.url",
"http://registry.npmjs.org/;),
ZEPPELIN_HELIUM_YARNPKG_INSTALLER_URL("zeppelin.helium.yarnpkg.installer.url",
"https://github.com/yarnpkg/yarn/releases/download/;),
HELIUM_PACKAGE_DEFAULT_URL is not configuratable and the default value is "
https://s3.amazonaws.com/helium-package/helium.json;
You can change ZEPPELIN_HELIUM_REGISTRY to "helium," +
${YOUR_SERVER_HAVING_HELIUM_JSON} And rest of them you can use proxy
setting for your server.
Hope this helps,
JL
On Fri, Mar 22, 2019 at 2:38 PM Lian Jiang wrote:
> Thanks. I saw https://issues.apache.org/jira/browse/ZEPPELIN-3614 is
> still open which make me think zeppelin does not support offline helium.
> However, below posts all indicate otherwise:
>
>
> https://zeppelin.apache.org/docs/0.8.0/development/helium/writing_visualization_basic.html
>
> https://community.hortonworks.com/questions/223589/installing-helium-modules-in-zeppelin.html
>
> Can anybody confirm if zeppelin support using helium offline? Thanks.
>
>
>
> On Thu, Mar 21, 2019 at 2:48 PM Jhon Anderson Cardenas Diaz <
> jhonderson2...@gmail.com> wrote:
>
>> Is there something helium-related in the zeppelin logs once you start it
>> ? If there is some problem it should say.
>>
>> Btw, once you open a notebook, if it has associated helium plugins,
>> zeppelin tries to download and install node and yarn on runtime, that makes
>> me wonder if it will works without internet connection.
>>
>> El jue., 21 mar. 2019 a las 16:24, Lian Jiang ()
>> escribió:
>>
>>> Any clue is highly appreciated!
>>>
>>> On Wed, Mar 20, 2019 at 9:27 PM Lian Jiang
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am using Horton work HDP3.0 which has zeppelin 0.8.0. I followed
>>>> https://zeppelin.apache.org/docs/0.8.0/development/helium/writing_visualization_basic.html
>>>> to install helium viz packages. Since my environment does not have internet
>>>> access, I have to install packages into local registry. Here is what I did:
>>>>
>>>> step 1. added zeppelin.helium.localregistry.default in custom
>>>> zeppelin-site settings, pointing to a local helium folder on the zeppelin
>>>> master host. Restart zeppelin in ambari.
>>>> step 2. download https://s3.amazonaws.com/helium-package/helium.json
>>>> to the helium folder.
>>>> step 3. create a npm package tarball, upload the zeppelin master host,
>>>> unzip into the helium folder. I also updated the artifact to point to local
>>>> npm folder (e.g. from sogou-map-vis@1.0.0 to /u01/helium/sogou-map-vis)
>>>>
>>>> However, zeppelin's helium page does not list any viz packages. I
>>>> searched zeppelin.helium.localregistry.default in zeppelin source code
>>>> and found it only exist in the document writing_visualization_basic.md.
>>>>
>>>> What am I missing? Thanks a lot.
>>>>
>>>>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: zeppelin url append '/#/'
It means Front End. On Fri, Mar 22, 2019 at 3:42 PM Weilan Wu wrote: > Hi Jongyoul, > Could you explain what “FE” means? It’s an important question for us to > solve when running Zeppelin it in Kubernetes... > > Thanks, > Weilan > > Sent from my iPhone 6s > > On Mar 21, 2019, at 7:18 PM, Jongyoul Lee wrote: > > It's not 100% sure but mainly because of the behaviours of FE. > > On Fri, Mar 22, 2019 at 7:25 AM Weilan Wu wrote: > >> Hi, I am observing that the zeppelin web server always append ‘/#/‘ to >> end of the url, what’s the purpose of this? (and which code/module is >> responsible for it) >> >> e.g. if i type in browser: http://host.myzeppelin.com/ the browser ends >> up showing (in address bar): http://host.myzeppelin.com/#/ >> >> Thanks in advance, >> >> Weilan >> >> > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: zeppelin url append '/#/'
It's not 100% sure but mainly because of the behaviours of FE. On Fri, Mar 22, 2019 at 7:25 AM Weilan Wu wrote: > Hi, I am observing that the zeppelin web server always append ‘/#/‘ to end > of the url, what’s the purpose of this? (and which code/module is > responsible for it) > > e.g. if i type in browser: http://host.myzeppelin.com/ the browser ends > up showing (in address bar): http://host.myzeppelin.com/#/ > > Thanks in advance, > > Weilan > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Windows Supports
@Felix, I'd love that someone helps to set Appveyer. :-) On Fri, Mar 1, 2019 at 8:33 AM Felix Cheung wrote: > Ok but was the point about appveyer as CI. It’s not hard to setup. > > > -- > *From:* Jongyoul Lee > *Sent:* Tuesday, February 26, 2019 11:12 PM > *To:* users > *Subject:* Re: Windows Supports > > @Felix > What I meant was the case that running Zeppelin in the Windows environment > natively without a docker and a virtual Linux environment. People could run > Zeppelin through these kinds of ways but in the case where they run > Zeppelin natively, we didn't test that case and couldn't know the potential > problems as well. We could guide to use a docker container or virtual Linux > environment by default for Windows users instead of using native scripts. > > @Jeff, > I also thought of running interpreters in Windows. I don't think it's easy > to set up CI for windows. > > Basically, I agree with that the best way is to support Windows well. But > the more important thing is to keep Window users' UX. WDYT? > > On Wed, Feb 27, 2019 at 12:53 AM Jeff Zhang wrote: > >> I think the issue is about running spark interpreter in windows. This is >> due to some script changes in interpreter launch script interpreter.sh, but >> it is not applied in interpreter.cmd. We could still support windows by >> fixing this issue, but I don't have time on this right now. I would be very >> appreciated if someone else can help on this, and also set up CI in >> appveyer >> >> Thomas Bernhardt 于2019年2月26日周二 下午8:12写道： >> >>> We had no trouble running 0.8.0 on Windows 10 professional. We even set >>> up authentication. Maybe our case is special however since we don't use any >>> of the provided interpreters and only have an own interpreter. >>> -Tom >>> >>> On Monday, February 25, 2019, 9:29:14 PM EST, Jongyoul Lee < >>> jongy...@gmail.com> wrote: >>> >>> >>> Hi Dev and Users, >>> >>> Recently, personally, I've got reports that Z couldn't run under >>> Windows' environments. >>> >>> I think we need to discuss how to handle issues supporting windows. >>> >>> AFAIK, there are not many resources to test Z under Windows by >>> committers or contributors. If we couldn't support Windows well, how about >>> removing bin/*.cmd and focusing on alternatives like dockers. >>> >>> WDYT? >>> >>> JL >>> >>> -- >>> 이종열, Jongyoul Lee, 李宗烈 >>> http://madeng.net >>> >> >> >> -- >> Best Regards >> >> Jeff Zhang >> > > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Windows Supports
@Felix What I meant was the case that running Zeppelin in the Windows environment natively without a docker and a virtual Linux environment. People could run Zeppelin through these kinds of ways but in the case where they run Zeppelin natively, we didn't test that case and couldn't know the potential problems as well. We could guide to use a docker container or virtual Linux environment by default for Windows users instead of using native scripts. @Jeff, I also thought of running interpreters in Windows. I don't think it's easy to set up CI for windows. Basically, I agree with that the best way is to support Windows well. But the more important thing is to keep Window users' UX. WDYT? On Wed, Feb 27, 2019 at 12:53 AM Jeff Zhang wrote: > I think the issue is about running spark interpreter in windows. This is > due to some script changes in interpreter launch script interpreter.sh, but > it is not applied in interpreter.cmd. We could still support windows by > fixing this issue, but I don't have time on this right now. I would be very > appreciated if someone else can help on this, and also set up CI in > appveyer > > Thomas Bernhardt 于2019年2月26日周二 下午8:12写道： > >> We had no trouble running 0.8.0 on Windows 10 professional. We even set >> up authentication. Maybe our case is special however since we don't use any >> of the provided interpreters and only have an own interpreter. >> -Tom >> >> On Monday, February 25, 2019, 9:29:14 PM EST, Jongyoul Lee < >> jongy...@gmail.com> wrote: >> >> >> Hi Dev and Users, >> >> Recently, personally, I've got reports that Z couldn't run under Windows' >> environments. >> >> I think we need to discuss how to handle issues supporting windows. >> >> AFAIK, there are not many resources to test Z under Windows by committers >> or contributors. If we couldn't support Windows well, how about removing >> bin/*.cmd and focusing on alternatives like dockers. >> >> WDYT? >> >> JL >> >> -- >> 이종열, Jongyoul Lee, 李宗烈 >> http://madeng.net >> > > > -- > Best Regards > > Jeff Zhang > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Windows Supports
Hi Dev and Users, Recently, personally, I've got reports that Z couldn't run under Windows' environments. I think we need to discuss how to handle issues supporting windows. AFAIK, there are not many resources to test Z under Windows by committers or contributors. If we couldn't support Windows well, how about removing bin/*.cmd and focusing on alternatives like dockers. WDYT? JL -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [DISCUSS] Deprecate support for Spark 2.2.x and earlier version
+1 On Sun, Feb 17, 2019 at 10:45 AM moon soo Lee wrote: > +1 > > On Sat, Feb 16, 2019 at 06:52 Xun Liu wrote: > >> +1 >> >> >> 在 2019年2月16日，下午2:44，Felix Cheung 写道： >> >> +1 >> >> >> -- >> *From:* Jeff Zhang >> *Sent:* Thursday, February 14, 2019 10:28 PM >> *To:* users >> *Subject:* [DISCUSS] Deprecate support for Spark 2.2.x and earlier >> version >> >> Hi Folks, >> >> Spark 2.2.x will be EOL[1] from January of 2019. So I am considering to >> deprecate support for spark 2.2.x and earlier version in Zeppelin 0.9.0. >> Deprecation means that from Zeppelin 0.9 user is still able to run spark >> 2.2.x and earlier version, but will see a warning message in frontend about >> this deprecation. And in the next major version(maybe 0.10, or 1.0), we >> would totally remove support for spark 2.2.x and earlier version. The >> impact for users is the deprecation message in frontend. It may cause >> issues for users if they use rest api of zeppelin to run paragraph, then >> fetch and parse the result. >> >> Let me know your concern about this. Thanks >> >> [1] https://spark.apache.org/versioning-policy.html >> >> >> -- >> Best Regards >> >> Jeff Zhang >> >> >> -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Override Spark configs in notebook
AFAIK, `%conf` might work for it. On Thu, Jan 3, 2019 at 6:02 AM Kashif Khan wrote: > Hi, > > I am using Zeppelin 0.7.3 and want to know if there is a way to set number > of executors and executor memory in notebook. Per my understanding, these > properties are set up at interpreter level and will be same for all users > and notebooks. > > Would appreciate your help here. > > -- > Thanks, > Kashif > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Per user authorization with the Cassandra Interpreter
AFAIK, It's only implemented on JDBC Interpreter. On Mon, Sep 10, 2018 at 8:16 PM, Rexwell Minnis wrote: > Hi, > > Is it possible to use the Cassandra interpreter similar to the JDBC > interpreter for authorization, where the backend database system handles > the authorization? > > Example: > http://zeppelin.apache.org/docs/0.9.0-SNAPSHOT/setup/security/datasource_ > authorization.html > > Thank you for the feedback. > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: install Helium Packages offline
Hi, I'm not sure but you can download from internet at first, and could copy it inside your zeppelin directory. On Wed, Sep 5, 2018 at 2:05 PM, Lian Jiang wrote: > Hi, > > Our HDP3.0 hadoop is in an environment that has not internet access. How > can I install Helium packages offline? Thanks for any hint. > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln instead of [NOTEID]/note.json
Hi,
I have a bit different thoughts about the conflicts of the name of a new
note created. In a multiuser environment, AFAIK, most teams and companies,
generally, use a prefix for the group policy internally. In my case,
user/{user_id}/{notebook_name_they_want}.zpln. In this case, naming
conflicts rarely happen. And it will be stored under a specific folder. If
someone needed two different same named notes in the same directory, I
might not be appropriate. WDYT?
JL
On Fri, Aug 31, 2018 at 4:44 AM, andreas.we...@gmail.com <
andreas.we...@gmail.com> wrote:
> another reason for keeping noteId is uniqueness in case of multi-user
> environments. In that case users have separate zeppelin workspaces, which
> is something we are using in production: see ZEPPELIN_NOTEBOOK_PUBLIC=false
> in the doc [1]. In that case users might be very confused when they can not
> create notebooks with a name that already exists, but they most likely
> don't see (yet).
>
> So I like the proposal {note_name}_{note_id}.zpln. where note_name could
> contains folders, e.g. folder_1/mynote_abcd.zpln. Even though I like
> {note_name}.{note_id}.zpln (dot in between note_name and note_id) even
> better :-)
>
> Regards
> Andreas
>
>
> [1] http://zeppelin.apache.org/docs/0.8.0/setup/security/
> notebook_authorization.html#separate-notebook-workspaces-public-vs-private
>
> On 2018/08/18 08:42:44, Jeff Zhang wrote:
> > BTW, I also prefer to use note name as identify of note if the issue I
> > mentioned before is acceptable for most of users.
> >
> >
> >
> > Jeff Zhang 于2018年8月18日周六 下午4:40写道：
> >
> > >
> > > I am afraid we can not remove noteId, as noteId is the unique
> identifier
> > > of note and is immutable which is used in a lot places, such as
> paragraph
> > > share and rest api.
> > > If we use note name as note id then it may break user's app if note
> name
> > > is changed
> > >
> > >
> > > Jongyoul Lee 于2018年8月18日周六 下午2:33写道：
> > >
> > >> Hi, thanks for this kind of discussion.
> > >>
> > >> About noteId, How about changing note id to note name? AFAIK, Note id
> is
> > >> just an identifier and we can set any value to it.
> > >>
> > >> There’re two potential problems. We should be more careful to handle
> note
> > >> id as it could have very various type of characters. And Second, in
> case
> > >> where someone changes a note name, those who are seeing and updating
> the
> > >> same note wouldn’t access that note. We could handle it by using
> websockets.
> > >>
> > >> WDYT?
> > >>
> > >> On Tue, 14 Aug 2018 at 6:14 PM Jeff Zhang wrote:
> > >>
> > >>> >>> But I’m still not comfortable with note ids in the name of the
> > >>> notebook itself. Those names would look ugly if you shared your
> notebooks
> > >>> on github for example. You don’t see Jupyter notebooks with names
> like
> > >>> that. If you have to keep the note ids with the notebooks could you
> not
> > >>> simply put the note id at the top of the notebook as Ruslan
> suggested? Then
> > >>> you’d only have to read the first line of each notebook.
> > >>>
> > >>> I know putting note_id in the note file name is not so elegant, but
> this
> > >>> is what we have to compromise to keep compatibility as we use noteId
> to
> > >>> uniquely identify note right now. And I don't think putting noteId
> in the
> > >>> top first line of note would help much. We still have to read note
> files
> > >>> which take much more time than just read the file names via file
> system.
> > >>>
> > >>> Regarding the readability of note file name, I think it won't affect
> > >>> much. E.g. This is the note book file name like: *My Project/My
> Spark
> > >>> Tutorial Note_2A94M5J1Z.zpln*
> > >>> What user see in notebook menu is still *My Project/My Spark
> Tutorial* *Note
> > >>> *which is no difference from what we see now.
> > >>>
> > >>> And thanks again for the feedback and comments, I am so glad to see
> so
> > >>> many discussion in community.
> > >>>
> > >>>
> > >>>
> > >>> Partridge, Lucas (GE Aviation) 于2018年8月14日周二
> > >>> 下午4:29写道：
> > >>>
> > >>>> I agree you’re inviting consistency issues if you maintained a
> separate
>
Re: Env var suport in interpreter-setting.json
You could edit it like:
"zeppelin.spark.uiWebUrl": {
"envName": "SPARK_UI_URL",
"propertyName": "zeppelin.spark.uiWebUrl",
"defaultValue": "",
"description": "Override Spark UI default URL",
"type": "string"
},
Hope this help.
On Wed, Aug 29, 2018 at 10:03 PM, Ricardo Martinelli de Oliveira <
rmart...@redhat.com> wrote:
> Hi,
>
> Does anyone know if there is env var support in interpreter-setting.json?
> I'd like to configure zeppelin.spark.uiWebUrl parameter default value with
> the value of my SPARK_UI_URL env var.
>
> --
> Ricardo Martinelli de Oliveira
> Senior Software Engineer
> T: +55 11 3524-6125 | M: +55 11 9 7069-6531
> Av. Brigadeiro Faria Lima 3900, 8° Andar. São Paulo, Brasil
> <https://maps.google.com/?q=Av.+Brigadeiro+Faria+Lima+3900,+8%C2%B0+Andar.+S%C3%A3o+Paulo,+Brasil=gmail=g>
> .
> <https://red.ht/sig>
> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>
> Red Hat é reconhecida entre as melhores empresas para trabalhar no Brasil
> pelo *Great Place to Work*.
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: "Read replica" for Zeppelin
I like this idea, totally. I have an exactly same issue and thought to provide read-only mode. BTW, currently, Zeppelin server doesn't look proper for now. I, personally, have an idea to make a simple webserver only as a viewer. JL On Wed, Aug 29, 2018 at 9:55 PM, peay wrote: > Hello, > > I have a Zeppelin server with a S3 repo backend for notebooks. > > I would like to have a second Zeppelin instance to access the notebooks in > read only mode. Mostly for security reasons, but also in general some weak > form of HA. > > Any suggestion on how to achieve this? I've set it up naively, but the > missing piece is getting the read-only Zeppelin to reload notebooks from S3 > periodically (or when getting web server requests, for instance). Any help > would be appreciated. > > Thanks! > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: "GC overhead limit exceeded" while running multiple jobs
Yes, that might solve most of the issues in general. On Sat, Aug 18, 2018 at 5:43 PM, Jeff Zhang wrote: > > you can increase driver memory by setting spark.driver.memory > > > Jongyoul Lee 于2018年8月18日周六 下午2:35写道： > >> Hi, >> >> 1. AKAIF, it’s a problem of spark-shell. Z’s spark interpreter uses >> spark-shell internally. Thus it cannot be solved easily. >> >> 2. You could try ‘per user’ setting in your interpreter. >> >> 3. Currently, there’s no way to figure it out. >> >> On Tue, 14 Aug 2018 at 5:59 PM Chintan Patel >> wrote: >> >>> Hello, >>> >>> I'm running Zeppelin in yarn-client mode. I'm using SQL interpreter and >>> pyspark interpreter to run some query and python jobs in shared mode per >>> note. Sometimes when I run multiple jobs at same time. It's using lot's of >>> CPU. I try to check the problem and I found that It's because of It creates >>> spark driver for each notebook. >>> >>> >>> My Question are >>> 1. How I can tune Zeppelin to Handle large amount of concurrent jobs to >>> fix "GC overhead limit exceeded" ? >>> 2. How can I scale the zeppelin with number of users ? >>> 3. If memory or CPU is not available, Is there any way to backlog the >>> jobs ? >>> >>> Thanks & Regards >>> Chintan >>> >> -- >> 이종열, Jongyoul Lee, 李宗烈 >> http://madeng.net >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: "GC overhead limit exceeded" while running multiple jobs
Hi, 1. AKAIF, it’s a problem of spark-shell. Z’s spark interpreter uses spark-shell internally. Thus it cannot be solved easily. 2. You could try ‘per user’ setting in your interpreter. 3. Currently, there’s no way to figure it out. On Tue, 14 Aug 2018 at 5:59 PM Chintan Patel wrote: > Hello, > > I'm running Zeppelin in yarn-client mode. I'm using SQL interpreter and > pyspark interpreter to run some query and python jobs in shared mode per > note. Sometimes when I run multiple jobs at same time. It's using lot's of > CPU. I try to check the problem and I found that It's because of It creates > spark driver for each notebook. > > > My Question are > 1. How I can tune Zeppelin to Handle large amount of concurrent jobs to > fix "GC overhead limit exceeded" ? > 2. How can I scale the zeppelin with number of users ? > 3. If memory or CPU is not available, Is there any way to backlog the jobs > ? > > Thanks & Regards > Chintan > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Zeppelin distributed architecture design
Thank you. I fully agree with you that we need a framework to support distributed version. IMHO, we cannot afford to develop our own. I'll dig into atomix as well. On Tue, Jul 24, 2018 at 1:57 PM, liuxun wrote: > @Jongyoul Lee： > Thank you for your attention. > > Indeed, as you said, the `Copycat` project has been closed and has been > migrated to `https://github.com/atomix/atomix` > <https://github.com/atomix/atomix>. > > I also considered this issue during development. > The main reason was that it was enough to realize Raft using `Copycat` at > the time, and it was not considered too long. > > Today, I took a look at the documentation of atomix, > https://atomix.io/docs/latest/user-manual/ , > which has a lot of features, such as broadcasting messages in the cluster, > detecting cluster events... , > From the perspective of zeppelin's long-term development, it is better to > use atomix. > So, I will switch the Raft protocol algorithm library to atomix, which is > not difficult to modify. > > Struggle for zeppelin!!! :-) > > > 在 2018年7月24日，上午9:35，Jongyoul Lee 写道： > > First of all, thank you for your effort and contribution. > > I read it carefully today, and personally, it's a very nice feature and > idea. > > Let's discuss it and improve more concretely. I also left comments on the > doc. > > And I have a simple question. > > `Copycat`, which you used to implement it, is deprecated by owner[1] and > moved under https://github.com/atomix/atomix/. I'm afraid of it. Do you > have any reason to use this library? It's even SNAPSHOT version. > > Regards, > JL > > [1]: https://github.com/atomix/copycat > > On Sat, Jul 21, 2018 at 2:07 AM, liuxun wrote: > > HI： > > In order to more intuitively express the actual use of distributed > zeppelin clusters. > I updated this design document, starting with the 16th page of the > document, adding 2 GIF animations showing the operation record screen of > the zeppelin cluster we are using now. > https://docs.google.com/document/d/1a8QLSyR3M5AhlG1GIYuDTj6bwazeu > VDKCRRBm-Qa3Bw/edit# <https://docs.google.com/document/d/ > 1a8QLSyR3M5AhlG1GIYuDTj6bwazeuVDKCRRBm-Qa3Bw/edit#> > > Distributed clustered zeppelin is already in use at our company, and the > recorded screens are all real. > The first recorded screens GIF shows the following > Create a cluster of three zeppelin servers > Add 234, 235, 236 to the zeppelin.cluster.addr attribute in > zeppelin-site.xml to create a cluster > Start these 3 servers at the same time > Open the web pages of these 3 servers and prepare for the notebook > operation. > > > The second recorded screens GIF shows the following > Create an interpreter process in the cluster > Create a notebook on host234 and execute it, This action will create an > interpreter process in the server with free resources in the cluster. > You can then continue editing this notebook on host235 and execute it, You > can return results immediately without waiting for the time to create an > interpreter process. > Again, you can continue to edit this notebook on host236. And execute it, > you can return results immediately without waiting for the time to create > the interpreter process > The same notebook will reuse the first created interpreter process, so you > can get the execution result immediately on any server. > By looking at the background server process, you will find that host234, > host235, and host235 use the same interpreter process for the same > notebook. > > Originally, I wanted to record the interpreter process exception. The > cluster re-created the screenshot of the interpreter process in the idle > server, but I am too tired now. > There is time to record later. > > > 在 2018年7月19日，上午7:36，Ruslan Dautkhanov 写道： > > Thank you luxun, > > I left a couple of comments in that google document. > > -- > Ruslan Dautkhanov > > > On Tue, Jul 17, 2018 at 11:30 PM liuxun > neliu...@163.com>> wrote: > > hi，Ruslan Dautkhanov > > Thank you very much for your question. according to your advice, I added > > 3 schematics to illustrate. > > 1. Distributed Zeppelin Deployment architecture diagram. > 2. Distributed zeppelin Server fault tolerance diagram. > 3. Distributed zeppelin Server & intp process fault tolerance diagram. > > > The email attachment exceeded the size limit, so I reorganized the > > document and updated it with Google Docs. > > https://docs.google.com/document/d/1a8QLSyR3M5AhlG1GIYuDTj6bwazeu > > VDKCRRBm-Qa3Bw/edit?usp=sharing <https://docs.google.com/document/d/ > 1a8QLSyR3M5AhlG1GIYuDTj6bwazeuVDKCRRBm-Qa3Bw/edit?usp=sharing> > > > > 在 201
Re: Zeppelin distributed architecture design
or unhealthy, is nginx > >>> supposed to detect (if so, how?) that and reroute users to a survived > >>> instance? > >>> > >>> Thanks, > >>> Ruslan Dautkhanov > >>> > >>> > >>> On Tue, Jul 17, 2018 at 2:46 AM liuxun neliu...@163.com>> wrote: > >>> > >>>> hi: > >>>> > >>>> Our company installed and deployed a lot of zeppelin for data > analysis. > >>>> The single server version of zeppelin could not meet our application > >>>> scenarios, so we transformed zeppelin into a clustered service that > >>>> supports distributed deployment, Have a unified entrance, high > >>>> availability, and High server resource usage. the email attachment > is the > >>>> entire design document, I am very happy to feedback our modified code > back > >>>> to the community. > >>>> > >>>> > >>>> this is the JIRA I submitted in the community, > >>>> > >>>> https://issues.apache.org/jira/browse/ZEPPELIN-3471 < > https://issues.apache.org/jira/browse/ZEPPELIN-3471> > >>>> > >>>> > >>>> Since the design document size exceeds the mail attachment size > limit, the > >>>> document link address has to be sent. > >>>> > >>>> https://issues.apache.org/jira/secure/attachment/12931896/Zeppelin% > 20distributed%20architecture%20design.pdf <https://issues.apache.org/ > jira/secure/attachment/12931896/Zeppelin%20distributed%20architecture% > 20design.pdf> > >>>> > >>>> https://issues.apache.org/jira/secure/attachment/ > 12931895/zepplin%20Cluster%20Sequence%20Diagram.png < > https://issues.apache.org/jira/secure/attachment/ > 12931895/zepplin%20Cluster%20Sequence%20Diagram.png> > >>>> > >>>> > >>>> liuxun > >>>> > >> > > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: ZeppelinContext Not Found in yarn-cluster Mode
I have the same issue. We might need to investigate it deeply. Could you
please file it up?
Regards,
JL
On Tue, Jul 17, 2018 at 7:27 PM, Chris Penny wrote:
> Hi all,
>
> Thanks for the 0.8.0 release!
>
> We’re keen to take advantage of the yarn-cluster support to take the
> pressure off our Zeppelin host. However, I am having some trouble with it.
> The first problem was in following the documentation here:
> https://zeppelin.apache.org/docs/0.8.0/interpreter/spark.html
>
> This suggests that we need to modify the master configuration from
> “yarn-client” to “yarn-cluster”. However, doing so results in the following
> error:
>
> Warning: Master yarn-cluster is deprecated since 2.0. Please use master
> “yarn” with specified deploy mode instead.
> Error: Client deploy mode is not compatible with master “yarn-cluster”
> Run with --help for usage help or --verbose for debug output
>
>
> I got past this error with the following settings:
> master = yarn
> spark.submit.deployMode = cluster
>
> I’m somewhat unclear if I’m straying from the correct (documented)
> configuration or if the documentation needs an update. Anyway;
>
> These settings appear to work for everything except the ZeppelinContext,
> which is missing.
> Code:
> %spark
> z
>
> Output:
> :24: error: not found: value z
>
> Using yarn-client mode I can identify that z is meant to be an instance of
> org.apache.zeppelin.spark.SparkZeppelinContext
> Code:
> %spark
> z
>
> Output:
> res4: org.apache.zeppelin.spark.SparkZeppelinContext =
> org.apache.zeppelin.spark.SparkZeppelinContext@5b9282e1
>
> However, this class is absent in cluster-mode:
> Code:
> %spark
> org.apache.zeppelin.spark.SparkZeppelinContext
>
> Output:
> :24: error: object zeppelin is not a member of package org.apache
>org.apache.zeppelin.spark.SparkZeppelinContext
> ^
>
> Snooping around the Zeppelin installation I was able to locate this class
> in ${ZEPPELIN_INSTALL_DIR}/interpreter/spark/spark-interpreter-0.8.0.jar.
> I then uploaded this jar to HDFS and added it to spark.jars &
> spark.driver.extraClassPath. Relevant entries in driver log:
>
> …
> Added JAR hdfs:/spark-interpreter-0.8.0.jar at
> hdfs:/tmp/zeppelin/spark-interpreter-0.8.0.jar with timestamp
> 1531732774379
> …
> CLASSPATH -> …:hdfs:/tmp/zeppelin/spark-interpreter-0.8.0.jar …
> …
> command:
> …
> file:$PWD/spark-interpreter-0.8.0.jar \
> etc.
>
> However, I still can’t use the ZeppelinContext or
> org.apache.zeppelin.spark.SparkZeppelinContext class. At this point I’ve
> run out of ideas and am willing to ask for help.
>
> Does anyone have thoughts on how I could use the ZeppelinContext in yarn
> cluster mode?
>
> Regards, Chris.
>
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: [DISCUSS] Disable supports on Spark 1.6.3
I think the current release is good enough to use Spark 1.6.x. For the future release, it would be better to focus on 2.x only. And older versions than 1.6, fully agreed. we should do it, personally. On Tue, Jul 17, 2018 at 10:16 AM, Jeff Zhang wrote: > > This might be a little risky. But it depends on how many people still use > spark 1.6 > But at least I would suggest to disable support any spark before 1.6. > There's many legacy code in zeppelin to support very old versions of spark > (e.g. 1.5, 1.4) > Actually we don't have travis job for any spark before 1.6, so we don't > know whether these legacy code still works. Maintain these legacy code is a > extra effort for community. so I would suggest to disable support on spark > before 1.6 at least. > > > Jongyoul Lee 于2018年7月17日周二 上午9:10写道： > >> Hi, >> >> Today, I found that Apache Spark 1.6.3 distribution was removed from >> Apache CDN. We can get the link for 1.6.3 from Apache Spark, but it's only >> available to be download from Apache archive. I'm not sure how many ppl >> still use Spark 1.6.3 in Apache Zeppelin, but in my opinion, it means Spark >> 1.6.3 is not active anymore. >> >> From now, AFAIK, about supporting versions of Apache Spark, we have >> followed for Spark's policy. >> >> I suggest that we also remove Spark 1.6.3 from the officially supported >> version for the next Apache Zeppelin major release - 0.9.0 or 1.0.0. If we >> could focus on support on Spark 2.x only, we could make SparkInterpreter >> more concretely. >> >> WDYT? >> >> Best regards, >> JL >> >> -- >> 이종열, Jongyoul Lee, 李宗烈 >> http://madeng.net >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
[DISCUSS] Disable supports on Spark 1.6.3
Hi, Today, I found that Apache Spark 1.6.3 distribution was removed from Apache CDN. We can get the link for 1.6.3 from Apache Spark, but it's only available to be download from Apache archive. I'm not sure how many ppl still use Spark 1.6.3 in Apache Zeppelin, but in my opinion, it means Spark 1.6.3 is not active anymore. >From now, AFAIK, about supporting versions of Apache Spark, we have followed for Spark's policy. I suggest that we also remove Spark 1.6.3 from the officially supported version for the next Apache Zeppelin major release - 0.9.0 or 1.0.0. If we could focus on support on Spark 2.x only, we could make SparkInterpreter more concretely. WDYT? Best regards, JL -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [DISCUSS] Share Data in Zeppelin
BTW, we need to consider the case where the result is large in a design
time. In my experience, If we implement this feature, users could use it
with large data.
On Fri, Jul 13, 2018 at 11:51 AM, Sanjay Dasgupta wrote:
> I prefer 2.b also. Could we use (save*Result*AsTable=people) instead?
>
> There are a few typos in the example note shared:
>
> 1) The line val peopleDF = spark.read.format("zeppelin").load() should
> mention the table name (possibly as argument to load?)
> 2) The python line val peopleDF = z.getTable("people").toPandas() should
> not have the val
>
>
> The z.getTable() method could be a very good tool to judge
> which use-cases are important in the community. It is easy to implement for
> the in-memory data case, and could be very useful for many situations where
> a small amount of data is being transferred across interpreters (like the
> jdbc -> matplotlib case mentioned).
>
> Thanks,
> Sanjay
>
> On Fri, Jul 13, 2018 at 8:07 AM, Jongyoul Lee wrote:
>
>> Yes, it's similar to 2.b.
>>
>> Basically, my concern is to handle all kinds of data. But in your case,
>> it looks like focusing on table data. It's also useful but it would be
>> better to handle all of the data including table or plain text as well.
>> WDYT?
>>
>> About storage, we could discuss it later.
>>
>> On Fri, Jul 13, 2018 at 11:25 AM, Jeff Zhang wrote:
>>
>>>
>>> I think your use case is the same of 2.b. Personally I don't recommend
>>> to use z.get(noteId, paragraphId) to get the shared data for 2 reasons
>>> 1. noteId, paragraphId is meaningless, which is not readable
>>> 2. The note will break if we clone it as the noteId is changed.
>>> That's why I suggest to use paragraph property to save paragraph's result
>>>
>>> Regarding the intermediate storage, I also though about it and agree
>>> that in the long term we should provide such layer to support large data,
>>> currently we put the shared data in memory which is not a scalable
>>> solution. One candidate in my mind is alluxio [1], and regarding the data
>>> format I think apache arrow [2] is another good option for zeppelin to
>>> share table data across interpreter processes and different languages. But
>>> these are all implementation details, I think we can talk about them in
>>> another thread. In this thread, I think we should focus on the user facing
>>> api.
>>>
>>>
>>> [1] http://www.alluxio.org/
>>> [2] https://arrow.apache.org/
>>>
>>>
>>>
>>> Jongyoul Lee 于2018年7月13日周五 上午10:11写道：
>>>
>>>> I have a bit different idea to share data.
>>>>
>>>> In my case,
>>>>
>>>> It would be very useful to get a paragraph's result as an input of
>>>> other paragraphs.
>>>>
>>>> e.g.
>>>>
>>>> -- Paragrph 1
>>>> %jdbc
>>>> select * from some_table;
>>>>
>>>> -- Paragraph 2
>>>> %spark
>>>> val rdd = z.get("noteId", "paragraphId").parse.makeRddByMyself
>>>> spark.read(table).select
>>>>
>>>> If paragraph 1's result is too big to show on FE, it would be saved in
>>>> Zeppelin Server with proper way and pass to SparkInterpreter when Paragraph
>>>> 2 is executed.
>>>>
>>>> Basically, I think we need to intermediate storage to store paragraph's
>>>> results to share them. We can introduce another layer or extend
>>>> NotebootRepo. In some cases, we might change notebook repos as well.
>>>>
>>>> JL
>>>>
>>>>
>>>>
>>>> On Fri, Jul 13, 2018 at 10:39 AM, Jeff Zhang wrote:
>>>>
>>>>> Hi Folks,
>>>>>
>>>>> Recently, there's several tickets [1][2][3] about sharing data in
>>>>> zeppelin.
>>>>> Zeppelin's goal is to be an unified data analyst platform which could
>>>>> integrate most of the big data tools and help user to switch between
>>>>> tools
>>>>> and share data between tools easily. So sharing data is a very
>>>>> critical and
>>>>> killer feature of Zeppelin IMHO.
>>>>>
>>>>> I raise this ticket to discuss about the scenario of sharing data and
>>>>> how
>>>>> to do that. Although zeppelin already provides tools and api to share
>>>>> data,
>>>>>
Re: [DISCUSS] Share Data in Zeppelin
That would be great.
BTW, does ZEPL's example work for now?
On Fri, Jul 13, 2018 at 11:43 AM, Jeff Zhang wrote:
>
> Sure, we can support plain text as well.
>
> Jongyoul Lee 于2018年7月13日周五 上午10:37写道：
>
>> Yes, it's similar to 2.b.
>>
>> Basically, my concern is to handle all kinds of data. But in your case,
>> it looks like focusing on table data. It's also useful but it would be
>> better to handle all of the data including table or plain text as well.
>> WDYT?
>>
>> About storage, we could discuss it later.
>>
>> On Fri, Jul 13, 2018 at 11:25 AM, Jeff Zhang wrote:
>>
>>>
>>> I think your use case is the same of 2.b. Personally I don't recommend
>>> to use z.get(noteId, paragraphId) to get the shared data for 2 reasons
>>> 1. noteId, paragraphId is meaningless, which is not readable
>>> 2. The note will break if we clone it as the noteId is changed.
>>> That's why I suggest to use paragraph property to save paragraph's result
>>>
>>> Regarding the intermediate storage, I also though about it and agree
>>> that in the long term we should provide such layer to support large data,
>>> currently we put the shared data in memory which is not a scalable
>>> solution. One candidate in my mind is alluxio [1], and regarding the data
>>> format I think apache arrow [2] is another good option for zeppelin to
>>> share table data across interpreter processes and different languages. But
>>> these are all implementation details, I think we can talk about them in
>>> another thread. In this thread, I think we should focus on the user facing
>>> api.
>>>
>>>
>>> [1] http://www.alluxio.org/
>>> [2] https://arrow.apache.org/
>>>
>>>
>>>
>>> Jongyoul Lee 于2018年7月13日周五 上午10:11写道：
>>>
>>>> I have a bit different idea to share data.
>>>>
>>>> In my case,
>>>>
>>>> It would be very useful to get a paragraph's result as an input of
>>>> other paragraphs.
>>>>
>>>> e.g.
>>>>
>>>> -- Paragrph 1
>>>> %jdbc
>>>> select * from some_table;
>>>>
>>>> -- Paragraph 2
>>>> %spark
>>>> val rdd = z.get("noteId", "paragraphId").parse.makeRddByMyself
>>>> spark.read(table).select
>>>>
>>>> If paragraph 1's result is too big to show on FE, it would be saved in
>>>> Zeppelin Server with proper way and pass to SparkInterpreter when Paragraph
>>>> 2 is executed.
>>>>
>>>> Basically, I think we need to intermediate storage to store paragraph's
>>>> results to share them. We can introduce another layer or extend
>>>> NotebootRepo. In some cases, we might change notebook repos as well.
>>>>
>>>> JL
>>>>
>>>>
>>>>
>>>> On Fri, Jul 13, 2018 at 10:39 AM, Jeff Zhang wrote:
>>>>
>>>>> Hi Folks,
>>>>>
>>>>> Recently, there's several tickets [1][2][3] about sharing data in
>>>>> zeppelin.
>>>>> Zeppelin's goal is to be an unified data analyst platform which could
>>>>> integrate most of the big data tools and help user to switch between
>>>>> tools
>>>>> and share data between tools easily. So sharing data is a very
>>>>> critical and
>>>>> killer feature of Zeppelin IMHO.
>>>>>
>>>>> I raise this ticket to discuss about the scenario of sharing data and
>>>>> how
>>>>> to do that. Although zeppelin already provides tools and api to share
>>>>> data,
>>>>> I don't think it is mature and stable enough. After seeing these
>>>>> tickets, I
>>>>> think it might be a good time to talk about it in community and gather
>>>>> more
>>>>> feedback, so that we could provide a more stable and mature approach
>>>>> for
>>>>> it.
>>>>>
>>>>> Currently, there're 3 approaches to share data between interpreters and
>>>>> interpreter processes.
>>>>> 1. Sharing data across interpreter in the same interpreter process.
>>>>> Like
>>>>> sharing data via the same SparkContext in %spark, %spark.pyspark and
>>>>> %spark.r.
>>>>> 2. Sharing data between frontend and backend via angularObject
>>>>> 3. Sharing data
Re: [DISCUSS] Share Data in Zeppelin
Yes, it's similar to 2.b.
Basically, my concern is to handle all kinds of data. But in your case, it
looks like focusing on table data. It's also useful but it would be better
to handle all of the data including table or plain text as well. WDYT?
About storage, we could discuss it later.
On Fri, Jul 13, 2018 at 11:25 AM, Jeff Zhang wrote:
>
> I think your use case is the same of 2.b. Personally I don't recommend to
> use z.get(noteId, paragraphId) to get the shared data for 2 reasons
> 1. noteId, paragraphId is meaningless, which is not readable
> 2. The note will break if we clone it as the noteId is changed.
> That's why I suggest to use paragraph property to save paragraph's result
>
> Regarding the intermediate storage, I also though about it and agree that
> in the long term we should provide such layer to support large data,
> currently we put the shared data in memory which is not a scalable
> solution. One candidate in my mind is alluxio [1], and regarding the data
> format I think apache arrow [2] is another good option for zeppelin to
> share table data across interpreter processes and different languages. But
> these are all implementation details, I think we can talk about them in
> another thread. In this thread, I think we should focus on the user facing
> api.
>
>
> [1] http://www.alluxio.org/
> [2] https://arrow.apache.org/
>
>
>
> Jongyoul Lee 于2018年7月13日周五 上午10:11写道：
>
>> I have a bit different idea to share data.
>>
>> In my case,
>>
>> It would be very useful to get a paragraph's result as an input of other
>> paragraphs.
>>
>> e.g.
>>
>> -- Paragrph 1
>> %jdbc
>> select * from some_table;
>>
>> -- Paragraph 2
>> %spark
>> val rdd = z.get("noteId", "paragraphId").parse.makeRddByMyself
>> spark.read(table).select
>>
>> If paragraph 1's result is too big to show on FE, it would be saved in
>> Zeppelin Server with proper way and pass to SparkInterpreter when Paragraph
>> 2 is executed.
>>
>> Basically, I think we need to intermediate storage to store paragraph's
>> results to share them. We can introduce another layer or extend
>> NotebootRepo. In some cases, we might change notebook repos as well.
>>
>> JL
>>
>>
>>
>> On Fri, Jul 13, 2018 at 10:39 AM, Jeff Zhang wrote:
>>
>>> Hi Folks,
>>>
>>> Recently, there's several tickets [1][2][3] about sharing data in
>>> zeppelin.
>>> Zeppelin's goal is to be an unified data analyst platform which could
>>> integrate most of the big data tools and help user to switch between
>>> tools
>>> and share data between tools easily. So sharing data is a very critical
>>> and
>>> killer feature of Zeppelin IMHO.
>>>
>>> I raise this ticket to discuss about the scenario of sharing data and how
>>> to do that. Although zeppelin already provides tools and api to share
>>> data,
>>> I don't think it is mature and stable enough. After seeing these
>>> tickets, I
>>> think it might be a good time to talk about it in community and gather
>>> more
>>> feedback, so that we could provide a more stable and mature approach for
>>> it.
>>>
>>> Currently, there're 3 approaches to share data between interpreters and
>>> interpreter processes.
>>> 1. Sharing data across interpreter in the same interpreter process. Like
>>> sharing data via the same SparkContext in %spark, %spark.pyspark and
>>> %spark.r.
>>> 2. Sharing data between frontend and backend via angularObject
>>> 3. Sharing data across interpreter processes via Zeppelin's ResourcePool
>>>
>>> For this thread, I would like to talk about the approach 3 (Sharing data
>>> via Zeppelin's ResourcePool)
>>>
>>> Here's my current thinking of sharing data.
>>> 1. What kind of data would be shared ?
>>>IMHO, users would share 2 kinds of data: primitive data (string,
>>> number)
>>> and table data.
>>>
>>> 2. How to write shared data ?
>>> User may want to share data via 2 approches
>>> a. Use ZeppelinContext (e.g. z.put).
>>> b. Share the paragraph result via paragraph properties. e.g. user may
>>> want to read data from oracle database via jdbc interpreter and then do
>>> plotting in python interpreter. In such scenario. he can save the jdbc
>>> result in ResourcePool via paragraph property and then read it it via
>>> z.get. Here's one simple example (Not implemented yet)
>>>
>
Re: [DISCUSS] Share Data in Zeppelin
I have a bit different idea to share data.
In my case,
It would be very useful to get a paragraph's result as an input of other
paragraphs.
e.g.
-- Paragrph 1
%jdbc
select * from some_table;
-- Paragraph 2
%spark
val rdd = z.get("noteId", "paragraphId").parse.makeRddByMyself
spark.read(table).select
If paragraph 1's result is too big to show on FE, it would be saved in
Zeppelin Server with proper way and pass to SparkInterpreter when Paragraph
2 is executed.
Basically, I think we need to intermediate storage to store paragraph's
results to share them. We can introduce another layer or extend
NotebootRepo. In some cases, we might change notebook repos as well.
JL
On Fri, Jul 13, 2018 at 10:39 AM, Jeff Zhang wrote:
> Hi Folks,
>
> Recently, there's several tickets [1][2][3] about sharing data in zeppelin.
> Zeppelin's goal is to be an unified data analyst platform which could
> integrate most of the big data tools and help user to switch between tools
> and share data between tools easily. So sharing data is a very critical and
> killer feature of Zeppelin IMHO.
>
> I raise this ticket to discuss about the scenario of sharing data and how
> to do that. Although zeppelin already provides tools and api to share data,
> I don't think it is mature and stable enough. After seeing these tickets, I
> think it might be a good time to talk about it in community and gather more
> feedback, so that we could provide a more stable and mature approach for
> it.
>
> Currently, there're 3 approaches to share data between interpreters and
> interpreter processes.
> 1. Sharing data across interpreter in the same interpreter process. Like
> sharing data via the same SparkContext in %spark, %spark.pyspark and
> %spark.r.
> 2. Sharing data between frontend and backend via angularObject
> 3. Sharing data across interpreter processes via Zeppelin's ResourcePool
>
> For this thread, I would like to talk about the approach 3 (Sharing data
> via Zeppelin's ResourcePool)
>
> Here's my current thinking of sharing data.
> 1. What kind of data would be shared ?
>IMHO, users would share 2 kinds of data: primitive data (string, number)
> and table data.
>
> 2. How to write shared data ?
> User may want to share data via 2 approches
> a. Use ZeppelinContext (e.g. z.put).
> b. Share the paragraph result via paragraph properties. e.g. user may
> want to read data from oracle database via jdbc interpreter and then do
> plotting in python interpreter. In such scenario. he can save the jdbc
> result in ResourcePool via paragraph property and then read it it via
> z.get. Here's one simple example (Not implemented yet)
>
> %jdbc(saveAsTable=people)
> select * from oracle_table
>
> %python
> z.getTable("people).toPandas()
>
> 3. How to read shared data ?
> User can also have 2 approaches to read the shared data.
> a. Via ZeppelinContext. (e.g. z.get, z.getTable)
> b. Via variable substitution [1]
>
> Here's one sample note which illustrate the scenario of sharing data.
> https://www.zepl.com/viewer/notebooks/bm90ZTovL3pqZmZkdS8zMzkxZjg3Ym
> FhMjg0MDY3OGM1ZmYzODAwODAxMGJhNy9ub3RlLmpzb24
>
> This is just my current thinking of sharing data in zeppelin, it definitely
> doesn't cover all the scenarios, so I raise this thread to discuss about in
> community, welcome any feedback and comments.
>
>
> [1]. https://issues.apache.org/jira/browse/ZEPPELIN-3377
> [2]. https://issues.apache.org/jira/browse/ZEPPELIN-3596
> [3]. https://issues.apache.org/jira/browse/ZEPPELIN-3617
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
Re: Paragraphs collapsing
19cqdYpTZcNwm3sE4B-NIAyP0cMfmH-QTH8iUtZ6579WTCl8VNVYIqXw_I7Z-FzYfB4rfduIxMnHcTCXR5Fn6RPuIGUXHuVrSTYMk2DuI04TB0HDj6qiLiC_wVyJOZoNbJjsyyHipvZng0xFnYNWuH4GnpcDuUu2y4fOkg==>[image: >> PlaceIQ:CES 2018] >> <https://share.polymail.io/v1/z/b/NWI0M2I5NWU4NmJm/1rVo9iKinRysHler5gdcH0DC_77dWkc-NKChI464_6MwZ_CPflgB3zN7x8Tb19cqdYpTZcNwm3sE4B-NIAyP0cMfmH-QTH8iUtZ6579WTCl8VNVYIqXw_I7Z-FzYfB4rfduIxMnHcTCXR5Fn6RPuIGUXHuVrSTYMk2DuI04TdjShirfaeAuXw0ucJJEOE7D8zC_0prFhkAx-5gN30UdVqF0GH2_GohYJv_55PzhYSuLAiNPSmQUuB5WzyHRfC2YmvRmj> >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [DISCUSS] Is interpreter binding necessary ?
z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXF_0aLHKo78IZnVEhMSU95ZRlzGOU6E3h9YHPMTBJ1C0W3OugeM-PlV9y9UPbVWH-48AFuBh2Kl-rPeSWwK3V5Z2WDTsXOFNdsm1FHYgSBohH7ZvJLDfNGNaulXP4qbfXG2HpYq-QfysDvXs> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXF_0aLHKo78IZnVEhMSU95ZRlzGOU6E3h9YHPMTBJ1C0W3OugeM-PlV9y9UPbVWH-48AFuBh2Kl-rPeSWwK3V5Z2WDTsXOFNdsm1FHYgSBohH7ZvJLDfNGNaulXP4qbfXG2HpYq-QfysDvXs> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXFLybqeNov0Kan0Cx9rGrtRO73HcAfouhNlJHczEaxeaUX6o3vY1IlVExsgMZ1eNposWTcF8265hqviRWAOtU44gTDTodfcPbdq_XCoADEx_TrZye7icLHVSlxcGKgTDv-xtsFxHK1yYgLIAwmXEcCJuX7Se1nCJqqjL7Ce1UybLz0Pobvr1D9lcl4D1sPQ=> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXF_0aLHKo78IZnVEhMSU95ZRmzGPU6E3h9YHPMTBJ1i0Xm6pyeUldhR8zcAMa0iC-4cKFOB_wKlqoPeWE0apUoNrXy_3R6kXcdy0FHoyFl90XaFme7mXMG9MoR0LOh6Z0T_oMx9aVrq3yuLjLMxg7A==> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXF_0aLHKo78IZnVEhMSU95ZQkzGNVqEzg9JJNMzbY1S8H3K7gfZ7Nxdwz9URbUnBso8QAah6wqR4tu2HD0a0V456SjTqROFP2XDF2vaZPDduEFYBPNSULA==> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXF_0aLHKo78IZnVEhMSU95ZQkTGPV6E3h9YHPMTBJ1O1Rm-yyOI1Pgs-wsAWZkqNpIVJAah03q94q_2XBA64VYtxXm3qTOIFasGyXjonDUplAqZiZbTMPHNM5RsALQjFofxcshBaLx_AnLY4yWHadG52TbvP_meT8rbF7D-WahZRyGUwIkax-u0Gez-1LHbWmD8spk4MVqFW7B7xBIFsnzIO30oi1yzBh2RdqBjN_an6Zv5x5MhmA-p7DzyK43s-dvD54LtD1c-UaG8=> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXF_0aLHKo78IZnVEhMSU95ZQkTGPV6E3h9YHPMTBJ1O1Rm-yyOI1Pgs-wsAWZkqNpIVJAah03q94q_2XBA64VYtxXm3qTOIFasGyXjonDUplAqZiZbTMPHNM5RsALQjFofxcshBaLx_AnLY4yWHadG52TbvP_meT8rbF7D-WahZRyGUwIkax-u0Gez-1LHbWmD8spk4MVqFW7B7xBIFsnzIO30oi1yzBh2RdqBjN_an6Zv5x5MhmA-p7DzyK43s-dvD54LtD1c-UaG8=> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stTsANWA0TifPrTvvAIbHEamJfKrJZTky-IT75yxMcIOM7VY0uoH3G8wvM7Ogp4g9EXdUKepcMID-Zy2Kliq7SXFxi8Ws92VzPsTewUNcGyV3glBF94QKwqb7LIdGBRuxMBIgibtaJcqxBGJgyMh_IyxGDAYW5zUL-O-naOtruH5DeWYBFaiywwOEOx8e0GM3mCt5DNBtBcRh1QFP1e1WV9> >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXFLybqeNov0Kan0Cx9rGrtRO4FvsUr520-gpPMjEY1S8YHis2fIlL1Z72swUdUPRbyySupDdl6O4R2KObQ==>[image: >>>>>> PlaceIQ:CES 2018] >>>>>> <https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kVB6OMitKz7Pqm5NuaQG4awyekI99c0gkhUdEDv2kriJlBnjhW1b_KV6uy_oCGrO25Ri9NpPRXrqb1oYLH5U3-CVKF4hGqzYaRjNEfcRIrXjFevdugvD0d2pmfVXx7uzGRDDgZjI9stT-RZWXFLybqeNov0Kan0Cx9rGrtROkS6OV6MOhcMBPt_Rfl-_H1C83vwzLxF9yYwvak6Ys54FEOBh84xdmt2aAQW1UYN7FyjxR-i4tcfIzqeCHMRwEvO3G63G> >>>>>> >>>>>> On Fri, Jul 06, 2018 at 6:02 AM Jongyoul Lee >>>>> > wrote: >>>>>> >>>>>>> It looks like a good idea to show the default interpreter group - >>>>>>> and it should be also changeable - and store it to `note.json`. It could >>>>>>> remove interpreter-binding information, keep default interpreter when >>>>>>> extracting note. Moreover it also helps to reduce our server code as >>>>>>> well. >>>>>>> >>>>>>> What do you guys think of? >>>>>>> >>>>>>> On Fri, Jul 6, 2018 at 6:16 PM, Sanjay Dasgupta < >>>>>>> sanjay.dasgu...@gmail.com> wrote: >>>>>>> >>>>>>>> If there is no per-interpreter overhead of binding all the >>>>>>>> interpreters from the beginning, we sh
Re: [DISCUSS] Is interpreter binding necessary ?
It looks like a good idea to show the default interpreter group - and it should be also changeable - and store it to `note.json`. It could remove interpreter-binding information, keep default interpreter when extracting note. Moreover it also helps to reduce our server code as well. What do you guys think of? On Fri, Jul 6, 2018 at 6:16 PM, Sanjay Dasgupta wrote: > If there is no per-interpreter overhead of binding all the interpreters > from the beginning, we should definitely do it. This will simplify the GUI > somewhat. > > Regards, > - Sanjay > > > On Fri, Jul 6, 2018 at 1:49 PM, Partridge, Lucas (GE Aviation) < > lucas.partri...@ge.com> wrote: > >> “So usually we would recommend user to specify the full qualified >> interpreter name.” >> >> - I usually recommend the exact opposite to our users. We frequently >> change interpreter groups to allow for different Spark cluster settings >> (number of executors, memory, etc). Users with more demanding requirements >> are asked to use custom interpreter groups with more allocated resources. >> If users included the interpreter group name at the start of every >> paragraph they would then have to manually edit the start of every >> paragraph before they could run their note using a different interpreter >> group. Very tedious! >> >> >> >> But I agree the short names without the interpreter group are often >> ambiguous and can cause confusion. Maybe somewhere in the execution output >> of each paragraph there should be some discrete text giving the fully >> qualified name of the interpreter that was actually used to produce that >> output. Or a clearly defined ‘default interpreter group’ text in the >> toolbar at the top of each notebook. Make it a dropdown so it would be easy >> to change the default. >> >> >> >> *From:* Jeff Zhang >> *Sent:* 06 July 2018 08:53 >> *To:* users@zeppelin.apache.org >> *Cc:* dev >> *Subject:* EXT: Re: [DISCUSS] Is interpreter binding necessary ? >> >> >> >> >> >> We already allow setting default interpreter when creating note. Another >> way to set default interpreter is to reorder the interpreter setting >> binding in note page. >> >> >> >> But personally I don't recommend user to use short interpreter name >> because of default interpreter. 2 Reaons: >> >> 1. It introduce in-accurate info. e.g. In our product, we have 2 spark >> interpreters (`spark`: for spark 1.x & `spark2` for spark 2.x). Then user >> often specify `%spark` for spark interpreter. But it could mean both >> `%spark.spark` and `%spark2.spark`, So usually it is very hard to tell >> what's wrong when user expect to work spark2 but actually he still use >> spark 1.x. So usually we would recommend user to specify the full qualified >> interpreter name. Just type several more characters which just cost 2 >> seconds but make it more clear and readable. >> >> 2. Another issue is that interpreter binding is stored in >> interpreter.json, that means if they export this note to another zeppelin >> instance, the default interpreter won't work. >> >> >> >> So I don't think setting default interpreter via interpreter binding is >> valuable for users. If user really want to do that, I would suggest to >> store it in note.json instead of interpreter.json >> >> >> >> >> >> Jongyoul Lee 于2018年7月6日周五 下午3:36写道： >> >> There are two purposes of interpreter binding. One is what you mentioned >> and another one is to manage a default interpreter. If we provide a new way >> to set default interpreter, I think we can remove them :-) We could set >> permissions in other ways. >> >> >> >> Overall, +1 >> >> >> >> On Fri, Jul 6, 2018 at 4:24 PM, Jeff Zhang wrote: >> >> Hi Folks, >> >> >> >> I raise this thread to discuss whether we need the interpreter binding. >> Currently when user create notes, they have to bind interpreters to their >> notes in note page. Otherwise they will hit interpreter not found issue. >> Besides that in zeppelin server side, we maintain the interpreter binding >> info in memory as well as in interpreter.json. >> >> >> >> IMHO, it is not necessary to do interpreter binding. Because it just add >> extra burden to maintain the interpreter binding info in zeppelin server >> side, and doesn't introduce any benefits. The only benefit is that we will >> check whether user have permission to use this interpreter, but actually >> zeppelin will check the permission when running paragraph, so I don't think >> we need to introduce interpreter binding just for this kind of permission >> check that we will do later. >> >> >> >> So overall, I would suggest to remove interpreter binding feature. What >> do you think ? >> >> >> >> >> >> -- >> >> 이종열, Jongyoul Lee, 李宗烈 >> >> http://madeng.net >> >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [DISCUSS] Is interpreter binding necessary ?
There are two purposes of interpreter binding. One is what you mentioned and another one is to manage a default interpreter. If we provide a new way to set default interpreter, I think we can remove them :-) We could set permissions in other ways. Overall, +1 On Fri, Jul 6, 2018 at 4:24 PM, Jeff Zhang wrote: > Hi Folks, > > I raise this thread to discuss whether we need the interpreter binding. > Currently when user create notes, they have to bind interpreters to their > notes in note page. Otherwise they will hit interpreter not found issue. > Besides that in zeppelin server side, we maintain the interpreter binding > info in memory as well as in interpreter.json. > > IMHO, it is not necessary to do interpreter binding. Because it just add > extra burden to maintain the interpreter binding info in zeppelin server > side, and doesn't introduce any benefits. The only benefit is that we will > check whether user have permission to use this interpreter, but actually > zeppelin will check the permission when running paragraph, so I don't think > we need to introduce interpreter binding just for this kind of permission > check that we will do later. > > So overall, I would suggest to remove interpreter binding feature. What > do you think ? > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Clean up PRs
I'll remove three more PRs below: Closes #2713 How to create an universal filter for a dashboard using date filter? By Manituti Closes #2742 [ZEPPELIN-3168] Interpreter Settings Authorization By necosta Closes #3040 Branch 0.7 By niuguidong On Wed, Jul 4, 2018 at 4:51 PM, Jeff Zhang wrote: > Thanks Jongyoul > > > > Jongyoul Lee 于2018年7月4日周三 下午2:41写道： > > > Hi, > > > > Keeping clean on PR page, I will remove outdated PR by the end of 2017 > > soon. Herer is the list of PRs: > > > > Closes #1896 implementation test case for connect to exists interpreter > > process By cloverhearts > > Closes #1928 ZEPPELIN-598 ] Dynamic loading for Interpreter and API By > > cloverhearts > > Closes #1930 [ZEPPELIN-1886] implementation z.getZeppelinJobStatus By > > cloverhearts > > Closes #2271 [ZEPPELIN-2414] Memory leak under scoped mode of > > SparkInterpreter caused by inapproprately setting > Thread.contextClassLoader > > By hammertank > > Closes #2473 [fix] Check runtimeInfos is not null By ebuildy > > Closes #2399 [ZEPPELIN-2620] fa-area chart cannot be resized smaller By > > 0q > > Closes #2041 ZEPPELIN-2137 Changed "show line chart with focus" to "zoom" > > so that … By samthebest > > Closes #2282 [ZEPPELIN-2447] Fix python interpreter as to use max result > > setting By paularmand > > Closes #2859 [ZEPPELIN-3320] upgrade the flink interpreter from 1.1.3 to > > 1.4.0 By jianran > > Closes #1873 [ZEPPELIN-1922] Exclude jackson-core and jackson-databind > > from aws dep By epahomov > > Closes #1893 [ZEPPELIN-451] Save codes and messages as multi-line By > sixmen > > Closes #1897 Can't remove interpreter repository. By astroshim > > Closes #1907 [MINOR] Add enough padding at the bottom of notebook By > > cuspymd > > Closes #2019 [ZEPPELIN-2117] jdbc autocomplete for exasol database By > > sotnich > > Closes #2151 [ZEPPELIN-465] Be able to run paragraph and the following > > ones By Remilito > > Closes #2195 [ZEPPELIN-2319] new methods for ZeppelinContext By meniluca > > Closes #2249 ZEPPELIN-2356. Improvement for z.angularBind and > > z.angularWatch By zjffdu > > Closes #2250 [ZEPPELIN-2085] Interpret scala code in paste mode By DrIgor > > Closes #2227 [ZEPPELIN-2359] Support Spell as Display By echarles > > Closes #2321 [ZEPPELIN-2498] add more info and config for Ldap > > authentication By khalidhuseynov > > Closes #2286 Jenkins CI test By Leemoonsoo > > Closes #2294 [ZEPPELIN-2438]: Add local winutils.exe in build step and > use > > if required … By cfries > > Closes #2301 [ZEPPELIN-1625] Override Interpreter settings at User level > > By benoyantony > > Closes #2345 [MINOR] Hide pagination button if there is only 1 page in > > Helium menu By AhyoungRyu > > Closes #2372 [ZEPPELIN-2581] Shell ansi codes By DrIgor > > Closes #2374 [ZEPPELIN-2593] Add storage settings to persist on run and > > commit By khalidhuseynov > > Closes #2396 [ZEPPELIN-2451]: Add JDBC config option for calling > > connection.commit after paragraph execution By randerzander > > Closes #2402 [ZEPPELIN-2636] User role lookup via interfaces By volumeint > > Closes #2427 Rinterpreter By test2016new > > Closes #2389 [ZEPPELIN-2612] Remove duplicates from ZeppelinConfiguration > > By DrIgor > > Closes #2434 ZEPPELIN-2687: Show code in description section of Spark UI > > By karuppayya > > Closes #2439 [ZEPPELIN-2680] allow opening notebook as a reader By herval > > Closes #2440 [ZEPPELIN-2587] allow logging in if you're anonymous By > herval > > Closes #2448 [ZEPPELIN-2702] save notes in reader-friendly format with > > zpln extension for VFS and Git repo By khalidhuseynov > > Closes #2479 [ZEPPELIN-2762] Use regex to retrieve interpreter/repl name, > > and to retrieve property key; add and fix tests By dabaitu > > Closes #2508 [ZEPPELIN-2817] Support default interpreter setting in > create > > note re… By reminia > > Closes #2484 [ZEPPELIN-2711] basic metrics for paragraphs & notebook > > view/create/run By herval > > Closes #2489 [Refactoring] some opportunities to use diamond operator By > > desmorto > > Closes #2503 [ZEPPELIN-2808] remember me support By herval > > Closes #2523 [ZEPPELIN-2764] update packages and documentation for > > ZeppelinHubRepo storage By khalidhuseynov > > Closes #2528 ZEPPELIN-2834 Show Interpreter list as expand collapse > blocks > > By malayhm > > Closes #2548 fix tiny typos By kepricon > > Closes #2558 sc.setLocalProperty(...) should be more deterministic By > > ruseel > > Closes
Clean up PRs
e header By sjoerdmulder Closes #2676 [ZEPPELIN-3048] Add the option which prevents paragraphs from being edited while their notebook is cron scheduled By kjmrknsn Closes #2695 [ZEPPELIN-3089] Create user folders under Trash folder By yu74n Closes #2704 ZEPPELIN-3061: Updated the SecurityUtils to add Shiro's custom realm functionality By jithinchandranj Closes #2714 [ZEPPELIN-3121] Fix to create log per each interpreter By woowahan-jaehoon Regards, JL -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released
Great work!! On Fri, Jun 29, 2018 at 9:49 AM, Jeff Zhang wrote: > Thanks Patrick, I have fixed the broken link. > > > > Patrick Maroney 于2018年6月29日周五 上午7:13写道： > > > Install guides: > > > > http://zeppelin.apache.org/docs/0.8.0/install/install.html > > Not Found > > > > The requested URL /docs/0.8.0/install/install.html was not found on this > > server. > > > > http://zeppelin.apache.org/docs/0.8.0/manual/ > interpreterinstallation.html > > > > Not Found > > > > The requested URL /docs/0.8.0/manual/interpreterinstallation.html was > not > > found on this server. > > > > > > Patrick Maroney > > Principal Engineer - Data Science & Analytics > > Wapack Labs > > > > > > On Jun 28, 2018, at 6:59 PM, Jianfeng (Jeff) Zhang < > jzh...@hortonworks.com> > > wrote: > > > > Hi Patrick, > > > > Which link is broken ? I can access all the links. > > > > Best Regard, > > Jeff Zhang > > > > > > From: Patrick Maroney > > Reply-To: > > Date: Friday, June 29, 2018 at 4:59 AM > > To: > > Cc: dev > > Subject: Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released > > > > Great work Team/Community! > > > > Links on the main download page are broken: > > > > http://zeppelin.apache.org/download.html > > > > ...at least the ones I need ;-) > > > > *Patrick Maroney* > > Principal Engineer - Data Science & Analytics > > Wapack Labs LLC > > > > > > Public Key: http://pgp.mit.edu/pks/lookup?op=get= > 0x7C810C9769BD29AF > > > > On Jun 27, 2018, at 11:21 PM, Prabhjyot Singh > > wrote: > > > > Awesome! congratulations team. > > > > > > > > On Thu 28 Jun, 2018, 8:39 AM Taejun Kim, wrote: > > > >> Awesome! Thanks for your great work :) > >> > >> 2018년 6월 28일 (목) 오후 12:07, Jeff Zhang 님이 작성: > >> > >>> The Apache Zeppelin community is pleased to announce the availability > of > >>> the 0.8.0 release. > >>> > >>> Zeppelin is a collaborative data analytics and visualization tool for > >>> distributed, general-purpose data processing system such as Apache > Spark, > >>> Apache Flink, etc. > >>> > >>> This is another major release after the last minor release 0.7.3. > >>> The community put significant effort into improving Apache Zeppelin > since > >>> the last release. 122 contributors fixed totally 602 issues. Lots of > >>> new features are introduced, such as inline configuration, ipython > >>> interpreter, yarn-cluster mode support , interpreter lifecycle manager > >>> and etc. > >>> > >>> We encourage you to download the latest release > >>> fromhttp://zeppelin.apache.org/download.html > >>> > >>> Release note is available > >>> athttp://zeppelin.apache.org/releases/zeppelin-release-0.8.0.html > >>> > >>> We welcome your help and feedback. For more information on the project > >>> and > >>> how to get involved, visit our website at http://zeppelin.apache.org/ > >>> > >>> Thank you all users and contributors who have helped to improve Apache > >>> Zeppelin. > >>> > >>> Regards, > >>> The Apache Zeppelin community > >>> > >> -- > >> Taejun Kim > >> > >> Data Mining Lab. > >> School of Electrical and Computer Engineering > >> University of Seoul > >> > > > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Credentials for JDBC
We have a trick to get credential information from a credential page. I'll take into it. On Thu, Jun 7, 2018 at 7:53 AM, Benjamin Kim wrote: > I created a JDBC interpreter for AWS Athena, and it passes the access key > as UID and secret key as PWD in the URL connection string. Does anyone know > if I can setup each user to pass their own credentials in a, sort of, > credentials file or config? > > Thanks, > Ben -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: WARNING: A provider org.apache.zeppelin.rest.LoginRestApi registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime
I also thought it's weird. On Sat, May 26, 2018 at 4:20 AM, Ruslan Dautkhanov <dautkha...@gmail.com> wrote: > Filed https://issues.apache.org/jira/browse/ZEPPELIN-3504 > > Thanks. > > > > -- > Ruslan Dautkhanov > > On Tue, May 22, 2018 at 11:26 AM, Ruslan Dautkhanov <dautkha...@gmail.com> > wrote: > >> Testing Zeppelin from master snapshot.. getting new below errors/warnings >> [1]. >> Never seen these. >> Is this something we should be concerned about? >> If not, how to disable them? >> They show up at stdout when launching Zeppelin and not to a log file. >> >> >> [1] >> >> >>> WARNING: A provider org.apache.zeppelin.rest.SecurityRestApi registered >>> in SERVER runtime does not implement any provider interfaces applicable in >>> the SERVER runtime. Due to constraint configuration problems the provider >>> org.apache.zeppelin.rest.SecurityRestApi will be ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.InterpreterRestApi >>> registered in SERVER runtime does not implement any provider interfaces >>> applicable in the SERVER runtime. Due to constraint configuration problems >>> the provider org.apache.zeppelin.rest.InterpreterRestApi will be >>> ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.LoginRestApi registered in >>> SERVER runtime does not implement any provider interfaces applicable in the >>> SERVER runtime. Due to constraint configuration problems the provider >>> org.apache.zeppelin.rest.LoginRestApi will be ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.NotebookRepoRestApi >>> registered in SERVER runtime does not implement any provider interfaces >>> applicable in the SERVER runtime. Due to constraint configuration problems >>> the provider org.apache.zeppelin.rest.NotebookRepoRestApi will be >>> ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.HeliumRestApi registered >>> in SERVER runtime does not implement any provider interfaces applicable in >>> the SERVER runtime. Due to constraint configuration problems the provider >>> org.apache.zeppelin.rest.HeliumRestApi will be ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.NotebookRestApi registered >>> in SERVER runtime does not implement any provider interfaces applicable in >>> the SERVER runtime. Due to constraint configuration problems the provider >>> org.apache.zeppelin.rest.NotebookRestApi will be ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.ConfigurationsRestApi >>> registered in SERVER runtime does not implement any provider interfaces >>> applicable in the SERVER runtime. Due to constraint configuration problems >>> the provider org.apache.zeppelin.rest.ConfigurationsRestApi will be >>> ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.CredentialRestApi >>> registered in SERVER runtime does not implement any provider interfaces >>> applicable in the SERVER runtime. Due to constraint configuration problems >>> the provider org.apache.zeppelin.rest.CredentialRestApi will be ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.inject.Providers >>> checkProviderRuntime >>> WARNING: A provider org.apache.zeppelin.rest.ZeppelinRestApi registered >>> in SERVER runtime does not implement any provider interfaces applicable in >>> the SERVER runtime. Due to constraint configuration problems the provider >>> org.apache.zeppelin.rest.ZeppelinRestApi will be ignored. >>> >> >> >>> May 22, 2018 11:21:57 AM org.glassfish.jersey.internal.Errors logErrors >>> WARNING: The following warnings have been detected: WARNING: A HTTP GET >>> method, public javax.ws.rs.core.Response org.apache.zeppelin.rest.Inter >>> preterRestApi.listInterpreter(java.lang.String), should not consume any >>> entity. >>> >> >> >>> WARNING: A HTTP GET method, public javax.ws.rs.core.Response >>> org.apache.zeppelin.rest.CredentialRestApi.getCredentials(java.lang.String) >>> throws java.io.IOException,java.lang.IllegalArgumentException, should >>> not consume any entity. >> >> >> >> -- >> Ruslan Dautkhanov >> >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
