Thanks Sandeep. I got clue from http://mail-archives.apache.org/mod_mbox/knox-user/201805.mbox/%3ccacrbfygjshmcc+qm0wnwzhcjcb9ge2galrtngory-zklgag...@mail.gmail.com%3E .
Restarting knox is not enough and I must touch the topology file to reload it every time even if the topology is not changed. Now I see that from the auditing log that knox_load_balancer.com is replaced with yahoo.com. My goal is to expose Jupyter via KNOX. It is hard to compose rewrite.xml and service.xml from the scratch for proxying Jupyter. I tried to refer other UIs (e.g. ambari, ranger and zeppelin)'s routing files but it is still hard. I need to inspect the response, header in details which is time consuming and unreliable. Do you know where I can find working service.xml/rewrite.xml for jupyter? I believe this should be a common requirement to expose Jupyter via Knox. Thanks a lot for any hint! On Wed, Sep 12, 2018 at 11:31 AM Sandeep Moré <[email protected]> wrote: > No idea why Knox is redirecting back to itself, perhaps turning on Debug > logging will help understand what is happening. I remember I got the > weather service working at some point, let me know if you would like the > service and XML files for it. > > On Wed, Sep 12, 2018 at 11:52 AM Lian Jiang <[email protected]> wrote: > >> This is from gateway.log. It is expected that knox cannot connect to >> knox_load_balancer.com because rewriting knox_load_balancer.com to >> yahoo.com did not happen. Thanks for any hint. >> >> 2018-09-12 15:48:38,273 WARN knox.gateway >> (DefaultDispatch.java:executeOutboundRequest(147)) - Connection exception >> dispatching request: http:///knox_load_balancer.com:80/gateway/ui/weather >> org.apache.http.conn.ConnectTimeoutException: Connect to / >> knox_load_balancer.com:80 [/knox_load_balancer.com/130.35.0.245] failed: >> connect timed out >> org.apache.http.conn.ConnectTimeoutException: Connect to / >> knox_load_balancer.com:80 [/knox_load_balancer.com/130.35.0.245] failed: >> connect timed out >> at >> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) >> at >> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373) >> at >> org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) >> at >> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) >> at >> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) >> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) >> at >> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) >> at >> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) >> at >> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) >> at >> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) >> at >> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) >> at >> org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:130) >> at >> org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:116) >> at >> org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:278) >> at >> org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:170) >> at >> org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:122) >> at >> org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) >> at >> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >> at >> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >> at >> org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:166) >> at >> org.apache.ranger.authorization.knox.RangerPDPKnoxFilter.doFilter(RangerPDPKnoxFilter.java:110) >> at >> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >> at >> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >> at >> org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:60) >> at >> org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) >> at >> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >> at >> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >> at >> org.apache.knox.gateway.filter.AnonymousAuthFilter$1.run(AnonymousAuthFilter.java:76) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:422) >> at >> org.apache.knox.gateway.filter.AnonymousAuthFilter.continueWithEstablishedSecurityContext(AnonymousAuthFilter.java:71) >> at >> org.apache.knox.gateway.filter.AnonymousAuthFilter.doFilter(AnonymousAuthFilter.java:66) >> at >> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >> at >> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >> at >> org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30) >> at >> org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61) >> at >> org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:372) >> at >> org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:272) >> at >> org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:171) >> at >> org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:94) >> at >> org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:141) >> at >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) >> at >> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) >> at >> org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:201) >> at >> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) >> at >> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> at >> org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> at >> org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41) >> at >> org.eclipse.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> at >> org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:152) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> at >> org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:112) >> at >> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> at org.eclipse.jetty.server.Server.handle(Server.java:499) >> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) >> at >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:258) >> at >> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.net.SocketTimeoutException: connect timed out >> at java.net.PlainSocketImpl.socketConnect(Native Method) >> at >> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >> at >> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >> at >> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >> at java.net.Socket.connect(Socket.java:589) >> at >> org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75) >> at >> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) >> ... 72 more >> >> >> On Wed, Sep 12, 2018 at 6:19 AM Sandeep Moré <[email protected]> >> wrote: >> >>> Hello Lian, >>> >>> What do you see in gateway.log ? >>> Knox replacing knox_load_balancer.com:80 >>> <http://knox_load_balancer.com/> with http://yahoo.com/ might be the >>> log output of rewritten URL which is expected given >>> >>> <service> >>> <role>WEATHER</role> >>> <url>http://yahoo.com/</url> >>> </service> >>> >>> >>> >>> >>> On Wed, Sep 12, 2018 at 2:22 AM Lian Jiang <[email protected]> >>> wrote: >>> >>>> I am following >>>> http://kminder.github.io/knox/2015/11/16/adding-a-service-to-knox.html >>>> to add a weather service to knox. >>>> >>>> data/services/weather/0.0.1/rewrite.xml: >>>> <rules> >>>> <rule dir="IN" name="WEATHER/weather/inbound" >>>> pattern="*://*:*/**/weather/{path=**}?{**}"> >>>> <rewrite template="{$serviceUrl[WEATHER]}/{path=**}?{**}"/> >>>> </rule> >>>> </rules> >>>> >>>> data/services/weather/0.0.1/service.xml: >>>> <service role="WEATHER" name="weather" version="0.0.1"> >>>> <policies> >>>> <policy role="webappsec"/> >>>> <policy role="authentication" name="Anonymous"/> >>>> <policy role="rewrite"/> >>>> <policy role="authorization"/> >>>> </policies> >>>> <routes> >>>> <route path="/weather/**"> >>>> </route> >>>> </routes> >>>> </service> >>>> >>>> in topology ui.xml: >>>> <service> >>>> <role>WEATHER</role> >>>> <url>http://yahoo.com/</url> >>>> </service> >>>> >>>> Accessing https://*knox_load_balancer.com/gateway/ui/weather >>>> <http://knox_load_balancer.com/gateway/ui/weather>* got below output >>>> in gateway-audit.log, I see: >>>> >>>> 18/09/12 06:15:21 >>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER||||access|uri|/gateway/ui/weather|unavailable|Request >>>> method: GET >>>> 18/09/12 06:15:21 >>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||authentication|uri|/gateway/ui/weather|success| >>>> 18/09/12 06:15:21 >>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||dispatch|uri|http://*knox_load_balancer.com:80 >>>> <http://knox_load_balancer.com:80>*/gateway/ui/weather|unavailable|Request >>>> method: GET >>>> 18/09/12 06:15:41 >>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||dispatch|uri|http://*knox_load_balancer.com:80 >>>> <http://knox_load_balancer.com:80>*/gateway/ui/weather|failure| >>>> 18/09/12 06:15:41 >>>> ||f7b1ea3a-73ec-464a-87f4-dbb22e22867c|audit|160.34.88.239|WEATHER|anonymous|||access|uri|/gateway/ui/weather|failure| >>>> >>>> Looks like url rewrite (replace knox_load_balancer.com:80 with >>>> http://yahoo.com/) does not work. Any idea? Appreciate any clue. >>>> >>>>
