Great, you got it work, touching the topology is a bit that can be so easy to miss. I don't think we have working configs for Jupyter notebook, we would love to have one though. It would be really awesome if you can contribute it to the community when you get it to work !
Best, Sandeep On Wed, Sep 12, 2018 at 6:41 PM Lian Jiang <[email protected]> wrote: > Thanks Sandeep. > > I got clue from > http://mail-archives.apache.org/mod_mbox/knox-user/201805.mbox/%3ccacrbfygjshmcc+qm0wnwzhcjcb9ge2galrtngory-zklgag...@mail.gmail.com%3E > . > > Restarting knox is not enough and I must touch the topology file to reload > it every time even if the topology is not changed. Now I see that from the > auditing log that knox_load_balancer.com is replaced with yahoo.com. > > My goal is to expose Jupyter via KNOX. It is hard to compose rewrite.xml > and service.xml from the scratch for proxying Jupyter. I tried to refer > other UIs (e.g. ambari, ranger and zeppelin)'s routing files but it is > still hard. I need to inspect the response, header in details which is time > consuming and unreliable. Do you know where I can find working > service.xml/rewrite.xml for jupyter? I believe this should be a common > requirement to expose Jupyter via Knox. Thanks a lot for any hint! > > > On Wed, Sep 12, 2018 at 11:31 AM Sandeep Moré <[email protected]> > wrote: > >> No idea why Knox is redirecting back to itself, perhaps turning on Debug >> logging will help understand what is happening. I remember I got the >> weather service working at some point, let me know if you would like the >> service and XML files for it. >> >> On Wed, Sep 12, 2018 at 11:52 AM Lian Jiang <[email protected]> >> wrote: >> >>> This is from gateway.log. It is expected that knox cannot connect to >>> knox_load_balancer.com because rewriting knox_load_balancer.com to >>> yahoo.com did not happen. Thanks for any hint. >>> >>> 2018-09-12 15:48:38,273 WARN knox.gateway >>> (DefaultDispatch.java:executeOutboundRequest(147)) - Connection exception >>> dispatching request: http:/// >>> knox_load_balancer.com:80/gateway/ui/weather >>> org.apache.http.conn.ConnectTimeoutException: Connect to / >>> knox_load_balancer.com:80 [/knox_load_balancer.com/130.35.0.245] >>> failed: connect timed out >>> org.apache.http.conn.ConnectTimeoutException: Connect to / >>> knox_load_balancer.com:80 [/knox_load_balancer.com/130.35.0.245] >>> failed: connect timed out >>> at >>> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) >>> at >>> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373) >>> at >>> org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) >>> at >>> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) >>> at >>> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) >>> at >>> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) >>> at >>> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) >>> at >>> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) >>> at >>> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) >>> at >>> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) >>> at >>> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) >>> at >>> org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:130) >>> at >>> org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:116) >>> at >>> org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:278) >>> at >>> org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:170) >>> at >>> org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:122) >>> at >>> org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) >>> at >>> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >>> at >>> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >>> at >>> org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:166) >>> at >>> org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:110) >>> at >>> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >>> at >>> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >>> at >>> org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) >>> at >>> org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) >>> at >>> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >>> at >>> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >>> at >>> org.apache.knox.gateway.filter.AnonymousAuthFilter$1.run(AnonymousAuthFilter.java:76) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:422) >>> at >>> org.apache.knox.gateway.filter.AnonymousAuthFilter.continueWithEstablishedSecurityContext(AnonymousAuthFilter.java:71) >>> at >>> org.apache.knox.gateway.filter.AnonymousAuthFilter.doFilter(AnonymousAuthFilter.java:66) >>> at >>> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >>> at >>> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >>> at >>> org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30) >>> at >>> org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) >>> at >>> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >>> at >>> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >>> at >>> org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:171) >>> at >>> org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:94) >>> at >>> org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:141) >>> at >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) >>> at >>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) >>> at >>> org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:201) >>> at >>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) >>> at >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) >>> at >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) >>> at >>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) >>> at >>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) >>> at >>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) >>> at >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) >>> at >>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) >>> at >>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) >>> at >>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) >>> at >>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) >>> at >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >>> at >>> org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51) >>> at >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >>> at >>> org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41) >>> at >>> org.eclipse.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479) >>> at >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >>> at >>> org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:152) >>> at >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >>> at >>> org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:112) >>> at >>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) >>> at >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >>> at org.eclipse.jetty.server.Server.handle(Server.java:499) >>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) >>> at >>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) >>> at >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) >>> at >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) >>> at java.lang.Thread.run(Thread.java:745) >>> Caused by: java.net.SocketTimeoutException: connect timed out >>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>> at >>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >>> at >>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >>> at >>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>> at java.net.Socket.connect(Socket.java:589) >>> at >>> org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75) >>> at >>> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) >>> ... 72 more >>> >>> >>> On Wed, Sep 12, 2018 at 6:19 AM Sandeep Moré <[email protected]> >>> wrote: >>> >>>> Hello Lian, >>>> >>>> What do you see in gateway.log ? >>>> Knox replacing knox_load_balancer.com:80 >>>> <http://knox_load_balancer.com/> with http://yahoo.com/ might be the >>>> log output of rewritten URL which is expected given >>>> >>>> <service> >>>> <role>WEATHER</role> >>>> <url>http://yahoo.com/</url> >>>> </service> >>>> >>>> >>>> >>>> >>>> On Wed, Sep 12, 2018 at 2:22 AM Lian Jiang <[email protected]> >>>> wrote: >>>> >>>>> I am following >>>>> http://kminder.github.io/knox/2015/11/16/adding-a-service-to-knox.html >>>>> to add a weather service to knox. >>>>> >>>>> data/services/weather/0.0.1/rewrite.xml: >>>>> <rules> >>>>> <rule dir="IN" name="WEATHER/weather/inbound" >>>>> pattern="*://*:*/**/weather/{path=**}?{**}"> >>>>> <rewrite template="{$serviceUrl[WEATHER]}/{path=**}?{**}"/> >>>>> </rule> >>>>> </rules> >>>>> >>>>> data/services/weather/0.0.1/service.xml: >>>>> <service role="WEATHER" name="weather" version="0.0.1"> >>>>> <policies> >>>>> <policy role="webappsec"/> >>>>> <policy role="authentication" name="Anonymous"/> >>>>> <policy role="rewrite"/> >>>>> <policy role="authorization"/> >>>>> </policies> >>>>> <routes> >>>>> <route path="/weather/**"> >>>>> </route> >>>>> </routes> >>>>> </service> >>>>> >>>>> in topology ui.xml: >>>>> <service> >>>>> <role>WEATHER</role> >>>>> <url>http://yahoo.com/</url> >>>>> </service> >>>>> >>>>> Accessing https://*knox_load_balancer.com/gateway/ui/weather >>>>> <http://knox_load_balancer.com/gateway/ui/weather>* got below output >>>>> in gateway-audit.log, I see: >>>>> >>>>> 18/09/12 06:15:21 >>>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER||||access|uri|/gateway/ui/weather|unavailable|Request >>>>> method: GET >>>>> 18/09/12 06:15:21 >>>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||authentication|uri|/gateway/ui/weather|success| >>>>> 18/09/12 06:15:21 >>>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||dispatch|uri|http://*knox_load_balancer.com:80 >>>>> <http://knox_load_balancer.com:80>*/gateway/ui/weather|unavailable|Request >>>>> method: GET >>>>> 18/09/12 06:15:41 >>>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||dispatch|uri|http://*knox_load_balancer.com:80 >>>>> <http://knox_load_balancer.com:80>*/gateway/ui/weather|failure| >>>>> 18/09/12 06:15:41 >>>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||access|uri|/gateway/ui/weather|failure| >>>>> >>>>> Looks like url rewrite (replace knox_load_balancer.com:80 with >>>>> http://yahoo.com/) does not work. Any idea? Appreciate any clue. >>>>> >>>>>
