Re: NPE when using REST APIs that take a body (POSTs and PUTs)

[Jonathan Kelly]

I should have mentioned the JIRA issue ID so that this issue might be more
searchable in the mailing list archives. Here it is:
https://issues.apache.org/jira/browse/ZEPPELIN-837

On Mon, Jul 18, 2016 at 10:29 PM Jonathan Kelly 
wrote:

> Through a handy dandy `git bisect` command, I found that the commit that
> caused this behavior is
> https://github.com/apache/zeppelin/commit/57555f5ba16dd8109942513181d1330e9958744e.
> Is this something that we might be able to get fixed before 0.6.1? Or am I
> simply doing something wrong now that Shiro security has been added?
>
> Thanks,
> Jonathan
>
> On Mon, Jul 18, 2016 at 2:34 PM Jonathan Kelly 
> wrote:
>
>> I'm running into an issue with the REST API in Zeppelin 0.6.0 that I did
>> not experience using the same exact workflow on Zeppelin 0.5.6.
>>
>> When I call a method that takes a message body (i.e., POSTs and PUTs), I
>> get an NPE in the Zeppelin server. Somehow the message String being
>> passed to the API methods is always an empty string, causing GSON to
>> deserialize it into a null reference, ultimately resulting in an NPE soon
>> afterward. I have not yet figured out how and why the message is ending up
>> as an empty string but am looking into that now.
>>
>> I have reproduced this with both InterpreterRestApi.updateSetting()
>> (which uses a PUT) and NotebookRestApi.createNote() (which uses a POST).
>>
>> I am calling the API by writing a JSON object to a file, then passing it
>> to curl with -XPOST (or -XPUT) --data @file.json. e.g.,
>>
>> % cat deps/settings.json
>> {
>>   "dependencies": [
>> {
>>   "groupArtifactVersion": "/tmp/zeppelin/deps/commons-csv-1.1.jar",
>>   "local": true
>> },
>> {
>>   "groupArtifactVersion":
>> "/tmp/zeppelin/deps/spark-csv_2.10-1.3.0.jar",
>>   "local": true
>> }
>>   ]
>> }
>>
>> % curl -s -XPUT --data @/tmp/zeppelin/deps/settings.json
>> http://localhost:8080/interpreter/setting/2BS95QDGK
>>
>> The resulting NPE (which I find in the Zeppelin server logs) looks like
>> the following:
>>
>> javax.servlet.ServletException: java.lang.RuntimeException:
>> org.apache.cxf.interceptor.Fault
>> at
>> org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196)
>> at
>> org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148)
>> at
>> org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
>> at
>> org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
>> at
>> org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
>> at
>> org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
>> at
>> org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
>> at
>> org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
>> at
>> org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
>> at
>> org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
>> at
>> org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>> at org.apache.zeppelin.server.CorsFilter.doFilter(CorsFilter.java:72)
>> at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>> at
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>> at
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>> at
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>> at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>> at
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>> at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>> at org.eclipse.jetty.server.Server.handle(Server.java:499)
>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
>> at
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>> at
>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>> at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>> at 

Re: NPE when using REST APIs that take a body (POSTs and PUTs)

[Jonathan Kelly]

Through a handy dandy `git bisect` command, I found that the commit that
caused this behavior is
https://github.com/apache/zeppelin/commit/57555f5ba16dd8109942513181d1330e9958744e.
Is this something that we might be able to get fixed before 0.6.1? Or am I
simply doing something wrong now that Shiro security has been added?

Thanks,
Jonathan

On Mon, Jul 18, 2016 at 2:34 PM Jonathan Kelly 
wrote:

> I'm running into an issue with the REST API in Zeppelin 0.6.0 that I did
> not experience using the same exact workflow on Zeppelin 0.5.6.
>
> When I call a method that takes a message body (i.e., POSTs and PUTs), I
> get an NPE in the Zeppelin server. Somehow the message String being
> passed to the API methods is always an empty string, causing GSON to
> deserialize it into a null reference, ultimately resulting in an NPE soon
> afterward. I have not yet figured out how and why the message is ending up
> as an empty string but am looking into that now.
>
> I have reproduced this with both InterpreterRestApi.updateSetting() (which
> uses a PUT) and NotebookRestApi.createNote() (which uses a POST).
>
> I am calling the API by writing a JSON object to a file, then passing it
> to curl with -XPOST (or -XPUT) --data @file.json. e.g.,
>
> % cat deps/settings.json
> {
>   "dependencies": [
> {
>   "groupArtifactVersion": "/tmp/zeppelin/deps/commons-csv-1.1.jar",
>   "local": true
> },
> {
>   "groupArtifactVersion":
> "/tmp/zeppelin/deps/spark-csv_2.10-1.3.0.jar",
>   "local": true
> }
>   ]
> }
>
> % curl -s -XPUT --data @/tmp/zeppelin/deps/settings.json
> http://localhost:8080/interpreter/setting/2BS95QDGK
>
> The resulting NPE (which I find in the Zeppelin server logs) looks like
> the following:
>
> javax.servlet.ServletException: java.lang.RuntimeException:
> org.apache.cxf.interceptor.Fault
> at org.apache.shiro.web.servlet.AdviceFilter.cleanup(AdviceFilter.java:196)
> at
> org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:148)
> at
> org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
> at
> org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
> at
> org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
> at
> org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
> at
> org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
> at
> org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
> at
> org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
> at
> org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
> at
> org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at org.apache.zeppelin.server.CorsFilter.doFilter(CorsFilter.java:72)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: org.apache.cxf.interceptor.Fault
> at
> org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:116)
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:333)
> at
> 

Re: Change default interpreter to pyspark

[Ahyoung Ryu]

Hi Jayant,

I tested as you said, and can't change the default interpreter as well. So
I created issue for this in ZEPPELIN-1209
.
Thanks for reporting the issue.

Best regards,
Ahyoung

2016년 7월 19일 (화) 오전 6:54, Jayant Raj 님이 작성:

> Zeppelin has the Scala interpreter assigned as the default for Spark
> notebooks. This default setting creates an additional step if you are to
> write code using PySpark. You will need to insert a %pyspark at the
> beginning of each row of the notebook, for Zeppelin to understand that this
> is PySpark code.
>
> I modified the 'zeppelin.interpreters' property in zeppelin-site.xml and
> restarted the Zeppelin process. However, I do not see the default
> interpreter change. Am I missing something?
>
> Thanks for any help​
>
> .
>

Change default interpreter to pyspark

[Jayant Raj]

Zeppelin has the Scala interpreter assigned as the default for Spark
notebooks. This default setting creates an additional step if you are to
write code using PySpark. You will need to insert a %pyspark at the
beginning of each row of the notebook, for Zeppelin to understand that this
is PySpark code.

I modified the 'zeppelin.interpreters' property in zeppelin-site.xml and
restarted the Zeppelin process. However, I do not see the default
interpreter change. Am I missing something?

Thanks for any help​

.

Re: Forbid creating new notes

[Krzysztof Zarzycki]

Hi Vinay. Thank you for your advice.
But the thing I'm looking is not how to limit a particular note. I would
like to disallow *creation* of new notes to some users. I don't know how to
do it currently.  For now I can either disallow complete access to
Zeppelin, or give access together with creating new nodes.
Do you have an idea how to achieve it?

Thanks,
Krzysztof





pt., 15.07.2016 o 17:20 użytkownik Vinay Shukla 
napisał:

> Krzysztof,
>
> Today access control is configured at the Note level & not done a-priori
> in the shiro.ini.
>
> You configure authentication in shiro.ini & the notebook creator can set
> access control by clicking the lock icon on the notebook.
>
> Thanks,
> Vinay
>
>
>
> On Friday, July 15, 2016, Krzysztof Zarzycki  wrote:
>
>> Hi,
>> I'm configuring security for Zeppelin and I would like to be able to
>> restrict access the following way:
>> 1. only admins are able to create new notebooks
>> 2. everyone logged in can read notebooks. (Actually, finally this should
>> be also limited to the group).
>> 3. anonymous access is forbidden.
>>
>> I thought that I can do it with urls and roles, by specifying which urls
>> a particular role can access, but I'm unable to make it work.
>>
>>  My current configuration of shiro.ini looks like this:
>> [main]
>> adRealm = org.apache.shiro.realm.activedirectory.ActiveDirectoryRealm
>> adRealm.url = ldap://ad.server:389
>> adRealm.groupRolesMap = "CN=Admins,DC=example,DC=com":"admin"
>> adRealm.searchBase = dc=example,dc=com
>> adRealm.systemUsername= systemUser
>> adRealm.systemPassword= systemPassword
>> adRealm.principalSuffix= @example.com
>> adRealm.authorizationCachingEnabled = true
>> sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
>> securityManager.sessionManager = $sessionManager
>> securityManager.sessionManager.globalSessionTimeout = 8640
>> cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
>> securityManager.cacheManager = $cacheManager
>> securityManager.realms = $adRealm
>> shiro.loginUrl = /api/login
>>
>> [roles]
>>
>> admin = *
>>
>> [urls]
>> /api/version = anon
>> # I've tried multiple url filters:
>> /api/interpreter/** = authcBasic, roles[admin]
>> /api/notebook** = authcBasic, roles[admin]
>> /api/notebook/** = authcBasic, roles[admin]
>> /"#"/notebook/** = authcBasic, roles[admin]
>> /** = authcBasic
>> # I tried also above lines with authc, no success.
>>
>> # Only this option successfully limits access to admins, but of whole
>> Zeppelin, not only notebook creation.
>> /** = authcBasic, roles[admin]
>>
>> With this configuration, I log in as non-admin user and I'm still able to
>> create new notebooks...
>> Could you please help me configure Zeppelin to achieve my goal?
>>
>> Thanks a lot,
>> Krzysztof
>>
>