Zeppelin failing?
Hi, I’m running 7.2 compiled against MapR 5.2 I have nginx in front of Zeppelin so I run zeppelin on the local host port and use nginx as a proxy w ssh. After running zeppelin for approximately 24 hours, if I login and try to run a paragraph in any notebook, I get a java error, connection refused. Even trying to restart the interpreter (any interpreter results in a failure.) I’m going to assume its human error on setting up the configuration. Any suggestions? Thx -Mike
Re: Zeppelin failing?
The error: ERROR [2017-10-18 00:00:00,003] ({pool-2-thread-20} Job.java[run]:188) - Job failed org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:354) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:406) at org.apache.zeppelin.scheduler.Job.run(Job.java:175) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) And this in the zeppelin spark interpreter log: ERROR [2017-10-18 10:39:29,582] ({BoneCP-pool-watch-thread} PoolWatchThread.java[fillConnections]:118) - Error in trying to obtain a connection. Retrying in 7000ms java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source) at com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324) at com.jolbox.bonecp.ConnectionHandle.(ConnectionHandle.java:262) at com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115) at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) > On Oct 18, 2017, at 10:36 AM, Michael Segel wrote: > > Hi, > > I’m running 7.2 compiled against MapR 5.2 > > I have nginx in front of Zeppelin so I run zeppelin on the local host port > and use nginx as a proxy w ssh. > > After running zeppelin for approximately 24 hours, if I login and try to run > a paragraph in any notebook, I get a java error, connection refused. > > Even trying to restart the interpreter (any interpreter results in a > failure.) > > I’m going to assume its human error on setting up the configuration. > > Any suggestions? > > Thx > > -Mike >
Re: Zeppelin failing?
I went back and rebuilt 0.7.2 and it seems that the issue has gone away. Not sure what changed or which setting could have been causing the issue with socket connections. > On Oct 18, 2017, at 10:40 AM, Michael Segel wrote: > > The error: > ERROR [2017-10-18 00:00:00,003] ({pool-2-thread-20} Job.java[run]:188) - Job > failed > org.apache.zeppelin.interpreter.InterpreterException: > org.apache.zeppelin.interpreter.InterpreterException: > org.apache.thrift.transport.TTransportException: java.net.ConnectException: > Connection refused (Connection refused) >at > org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:354) >at > org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97) >at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:406) >at org.apache.zeppelin.scheduler.Job.run(Job.java:175) >at > org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329) >at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >at java.util.concurrent.FutureTask.run(FutureTask.java:266) >at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) >at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >at java.lang.Thread.run(Thread.java:748) > > And this in the zeppelin spark interpreter log: > ERROR [2017-10-18 10:39:29,582] ({BoneCP-pool-watch-thread} > PoolWatchThread.java[fillConnections]:118) - Error in trying to obtain a > connection. Retrying in 7000ms > java.sql.SQLException: A read-only user or a user in a read-only database is > not permitted to disable read-only mode on a connection. >at > org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown > Source) >at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown > Source) >at > org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown > Source) >at > org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown > Source) >at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown > Source) >at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown > Source) >at > com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324) >at com.jolbox.bonecp.ConnectionHandle.(ConnectionHandle.java:262) >at > com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115) >at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82) >at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >at java.lang.Thread.run(Thread.java:748) > Caused by: java.sql.SQLException: A read-only user or a user in a read-only > database is not permitted to disable read-only mode on a connection. >at > org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) > > >> On Oct 18, 2017, at 10:36 AM, Michael Segel >> wrote: >> >> Hi, >> >> I’m running 7.2 compiled against MapR 5.2 >> >> I have nginx in front of Zeppelin so I run zeppelin on the local host port >> and use nginx as a proxy w ssh. >> >> After running zeppelin for approximately 24 hours, if I login and try to run >> a paragraph in any notebook, I get a java error, connection refused. >> >> Even trying to restart the interpreter (any interpreter results in a >> failure.) >> >> I’m going to assume its human error on setting up the configuration. >> >> Any suggestions? >> >> Thx >> >> -Mike >> >