Usually 2-3 interpreter running and use, where multiple users might be
using the same interpreter in Per Note, Scoped setting.
Though it might also happen with just 1-2 interpreter running and only
single user on the UI.
Belousov Maksim Eduardovich <m.belou...@tinkoff.ru> schrieb am Fr., 13.
Okt. 2017, 10:44:
> Paul, Ben, Fabian,
>
> please share your workload at time when notes are not loading.
>
>
>
> How much interpreters were started at that moment?
>
>
>
> You can find all started interpreters in linux command line with:
>
> ps -ef | grep ZeppelinServer | grep -v grep | awk "NR==1" | awk -F' '
> '{print $2}' | xargs ps -f --ppid | wc -l
>
>
>
> And spark started interpreters:
>
> ps -ef | grep ZeppelinServer | grep -v grep | awk "NR==1" | awk -F' '
> '{print $2}' | xargs ps -f --ppid | grep spark | wc -l
>
>
>
>
>
>
>
> *Максим Белоусов *Архитектор
>
> Отдел отчетности и витрин данных
>
> Управление хранилищ данных и отчетности
> Тел.: +7 495 648-10-00, доб. 2271
>
>
>
> *From:* Paul Brenner [mailto:pbren...@placeiq.com]
> *Sent:* Thursday, October 12, 2017 7:45 PM
> *To:* Geoffrey Cheng <geoffrey.ch...@gmail.com>; users@zeppelin.apache.org
> *Subject:* Re: Zeppelin Stops Loading Notes
>
>
>
> Does this issue need a Jira ticket? The problem is that I have no idea how
> to reproduce and I’m not sure if there is anything in the logs that is
> relevant.
>
>
>
> Any ideas how we can produce an actionable Jira ticket out of this?
>
> * <http://www.placeiq.com/>*
>
> *Paul Brenner*
>
> * <https://twitter.com/placeiq>*
>
> * <https://www.facebook.com/PlaceIQ>*
>
> * <https://www.linkedin.com/company/placeiq>*
>
> *DATA SCIENTIST*
>
> *(217) 390-3033 *
>
>
> [image: PlaceIQ:Landmark by PlaceIQ]
> <https://www.placeiq.com/2017/05/placeiqs-landmark-powers-location-based-insight-innovation-for-ansible-gstv-havas-media-the-media-kitchen-and-more/>
>
> On Thu, Oct 12, 2017 at 8:31 AM Geoffrey Cheng <Geoffrey Cheng
> <geoffrey%20cheng%20%3cgeoffrey.ch...@gmail.com%3e>> wrote:
>
> we have the same issue. usually when multiple ppl using it, only header
> loads.
>
>
>
> we tried couldn't find solution, so we restart every single time. in
> fact , we have to restart daily at least.
>
>
>
> On Oct 12, 2017 2:40 AM, "Fabian Böhnlein" <fabian.boehnl...@gmail.com>
> wrote:
>
> Hi Paul, Ben,
>
>
>
> we also see this happen regularly. It's more likely to happen when a
> handful of people are using it.
>
>
>
> We mostly run one spark interpreter per person. We also don't observe
> anything in the logs. The 'header' that you mentioned is actually still in
> the cache.
>
>
>
> Sometimes it's specific notes that don't load.
>
> Sometimes there's a hanging Spark interpreter, once it's killed notes load
> again.
>
>
>
> We're pretty clueless about it.
>
>
>
> Any front-end related logs we could enable to find out more?
>
>
>
> On Sat, 19 Aug 2017 at 20:19 Ben Vogan <b...@shopkick.com> wrote:
>
> I have seen Zeppelin get into this state once. I restarted it without
> investigating the logs however so I don't have anything useful to go on as
> to why.
>
>
>
> --Ben
>
>
>
> On Sat, Aug 19, 2017 at 8:17 AM, Paul Brenner <pbren...@placeiq.com>
> wrote:
>
> You were correct. We had "export ZEPPELIN_SSL_PORT=false” in our
> zeppelin-env.sh. I’m going to comment that out. I suspect it is actually
> unrelated to the behavior we are seeing where pages stop loading though.
> Anyone else see this happen?
>
>
>
> I’ll report back if that happens again after the fix.
>
> * <http://www.placeiq.com/>*
>
> *Paul Brenner <http://www.placeiq.com/>*
>
> * <http://www.placeiq.com/>*
>
> * <http://www.placeiq.com/>*
>
> * <http://www.placeiq.com/>*
>
> *DATA SCIENTIST <http://www.placeiq.com/>*
>
> *(217) 390-3033 <http://www.placeiq.com/>*
>
>
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> On Fri, Aug 18, 2017 at 6:37 PM moon soo Lee <*moon soo Lee *> wrote:
> <http://www.placeiq.com/>
>
> Hi, <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> One of configuration value in your conf/zeppelin-env.sh or
> conf/zeppelin-site.xml seems "false" which expected to be to a number.
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> Do you have any environment variable or property set to "false" for the
> configurations below? <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> ZEPPELIN_PORT, zeppelin.server.port <http://www.placeiq.com/>
>
> ZEPPELIN_SSL_PORT, zeppelin.server.ssl.port <http://www.placeiq.com/>
>
> ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT, zeppelin.interpreter.connect.timeout
> <http://www.placeiq.com/>
>
> ZEPPELIN_INTERPRETER_MAX_POOL_SIZE, zeppelin.interpreter.max.poolsize
> <http://www.placeiq.com/>
>
> ZEPPELIN_INTERPRETER_OUTPUT_LIMIT, zeppelin.interpreter.output.limit
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> Thanks,
> moon <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> On Fri, Aug 18, 2017 at 2:30 PM Paul Brenner <*pbren...@placeiq.com*>
> wrote: <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> We have a team of 5 users who all use the same zeppelin server. Lately a
> few times we have run into a case where zeppelin notes stop responding and
> then when we try refreshing the webpage for the note all that loads is the
> zeppelin header with no note. When I look at the logs I see:
> <http://www.placeiq.com/>
>
> INFO [2017-08-18 21:23:06,569] ({qtp1286783232-14114}
> NotebookServer.java[sendNote]:705) - New operation from 10.201.12.26 :
> 55178 : nshah : GET_NOTE : 2CR2ANDEX <http://www.placeiq.com/>
>
> INFO [2017-08-18 21:24:05,740] ({qtp1286783232-14115}
> NotebookServer.java[onClose]:363) - Closed connection to 10.201.12.22 :
> 57366. (1001) Idle Timeout <http://www.placeiq.com/>
>
> INFO [2017-08-18 21:24:08,084] ({qtp1286783232-14121}
> NotebookServer.java[onClose]:363) - Closed connection to 10.201.12.22 :
> 57461. (1001) Idle Timeout <http://www.placeiq.com/>
>
> INFO [2017-08-18 21:25:10,133] ({qtp1286783232-14122}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained. <http://www.placeiq.com/>
>
> INFO [2017-08-18 21:25:10,157] ({qtp1286783232-14122}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained. <http://www.placeiq.com/>
>
> INFO [2017-08-18 21:25:10,172] ({qtp1286783232-14122}
> AuthorizingRealm.java[getAuthorizationCacheLazy]:248) - No cache or
> cacheManager properties have been set. Authorization cache cannot be
> obtained. <http://www.placeiq.com/>
>
> WARN [2017-08-18 21:25:10,192] ({qtp1286783232-14122}
> SecurityRestApi.java[ticket]:87) -
> {"status":"OK","message":"","body":{"principal":"pbrenner","ticket":"5f717a1a-46df-4178-bc9a-690f66064d0e","roles":"[]"}}
> <http://www.placeiq.com/>
>
> INFO [2017-08-18 21:25:10,252] ({qtp1286783232-14123}
> NotebookServer.java[onOpen]:156) - New connection from 10.201.12.48 : 62413
> <http://www.placeiq.com/>
>
> ERROR [2017-08-18 21:25:10,353] ({qtp1286783232-14122}
> NotebookServer.java[onMessage]:357) - Can't handle message
> <http://www.placeiq.com/>
>
> java.lang.NumberFormatException: For input string: "false"
> <http://www.placeiq.com/>
>
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> <http://www.placeiq.com/>
>
> at java.lang.Integer.parseInt(Integer.java:580) <http://www.placeiq.com/>
>
> at java.lang.Integer.parseInt(Integer.java:615) <http://www.placeiq.com/>
>
> at
> org.apache.zeppelin.conf.ZeppelinConfiguration.getInt(ZeppelinConfiguration.java:213)
> <http://www.placeiq.com/>
>
> at
> org.apache.zeppelin.conf.ZeppelinConfiguration.getInt(ZeppelinConfiguration.java:208)
> <http://www.placeiq.com/>
>
> at
> org.apache.zeppelin.conf.ZeppelinConfiguration.dumpConfigurations(ZeppelinConfiguration.java:496)
> <http://www.placeiq.com/>
>
> at
> org.apache.zeppelin.socket.NotebookServer.sendAllConfigurations(NotebookServer.java:1680)
> <http://www.placeiq.com/>
>
> at
> org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:318)
> <http://www.placeiq.com/>
>
> at
> org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:59)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:122)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:161)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:309)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:214)
> <http://www.placeiq.com/>
>
> at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220)
> <http://www.placeiq.com/>
>
> at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:258)
> <http://www.placeiq.com/>
>
> at org.eclipse.jetty.websocket.*common.io*
> .AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632)
> <http://www.placeiq.com/>
>
> at org.eclipse.jetty.websocket.*common.io*
> .AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480)
> <http://www.placeiq.com/>
>
> at *org.eclipse.jetty.io*
> .AbstractConnection$2.run(AbstractConnection.java:544)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> <http://www.placeiq.com/>
>
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> <http://www.placeiq.com/>
>
> at java.lang.Thread.run(Thread.java:745) <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> Anyone have any idea what is going on or how we could trouble shoot?
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> * <http://www.placeiq.com/>*
>
> *Paul Brenner <http://www.placeiq.com/>*
>
> * <http://www.placeiq.com/>*
>
> * <http://www.placeiq.com/>*
>
> * <http://www.placeiq.com/>*
>
> *DATA SCIENTIST <http://www.placeiq.com/>*
>
> *(217) 390-3033 <http://www.placeiq.com/>*
>
>
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
>
>
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> -- <http://www.placeiq.com/>
>
> *BENJAMIN VOGAN | Data Platform Team Lead <http://www.placeiq.com/>*
>
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
> <http://www.placeiq.com/>
>
>
