Great suggestions! Thanks Larry
I will work on getting the web.xml and the servlet integrated
Completely agreed on the vulnerability side. We may expose this in a DEBUG
version and not the release or provide a config value...
________________________________
From: larry mccay <[email protected]>
Sent: Friday, September 6, 2019 7:25 PM
To: [email protected] <[email protected]>
Subject: Re: Adding a web.xml to gateway.jar
Hi Jeff -
This is an interesting idea and we should consider discussing this as a feature
of Knox rather than just something that you are trying to hack into an existing
release/deployment.
In order to get this to work, I would first change the web.xml in the
deployments directory for a given topology and add the servlet to the in a jar
within {GATEWAY_HOME}/ext directory.
Stop and start the server and it should hopefully pickup the changed web.xml
file.
In order to cause a 500, I think just dispatching to an invalid URL would
result in a 500 with a connection exception.
See if that web.xml will work and we can take it from there.
It should be noted that surfacing the details of a webappexception may expose
sensitive information about the server and you may not want to always have this
enabled.
HTH.
--larry
On Fri, Sep 6, 2019 at 9:37 PM jeff saremi
<[email protected]<mailto:[email protected]>> wrote:
Ultimately I am trying to make sure when an HTTP 500 error happens the
exception message and stacktrace are returned in the response, in the gateway
So I decided to add a web.xml and overwrite parts of error handling there to
the gateway project. (added to
gateway-server-launcher/src/main/resources/META-INF/web.xml)
root@gateway-0:/opt/knox/bin# cat META-INF/web.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to ...
-->
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xmlns="http://java.sun.com/xml/ns/javaee";
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"; version="3.0">
<error-page>
<error-code>400</exception-type>
<location>/ExceptionHandler</location>
</error-page>
<error-page>
<error-code>401</exception-type>
<location>/ExceptionHandler</location>
</error-page>
<error-page>
<error-code>404</exception-type>
<location>/ExceptionHandler</location>
</error-page>
<error-page>
<exception-type>java.lang.Throwable</exception-type>
<location>/ExceptionHandler</location>
</error-page>
<error-page>
<exception-type>javax.servlet.ServletException</exception-type>
<location>/ExceptionHandler</location>
</error-page>
</web-app>
And then I added the following Servlet (added it to
gateway-util-common/src/main/java/org/apache/knox/gateway/servlet/ExceptionHandlerServlet.java)
@WebServlet("/ExceptionHandler")
public class ExceptionHandlerServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
protected void service(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
// Analyze the servlet exception
Throwable throwable = (Throwable)
request.getAttribute("javax.servlet.error.exception");
Integer statusCode = (Integer)
request.getAttribute("javax.servlet.error.status_code");
String servletName = (String)
request.getAttribute("javax.servlet.error.servlet_name");
if (servletName == null) {
servletName = "Unknown";
}
String requestUri = (String)
request.getAttribute("javax.servlet.error.request_uri");
if (requestUri == null) {
requestUri = "Unknown";
}
// Set response content type
response.setContentType("text/html");
PrintWriter out = response.getWriter();
out.write("<html><head><title>Exception/Error
Details</title></head><body>");
if(statusCode != 500){
out.write("<h3>Error Details</h3>");
out.write("<strong>Status Code</strong>:"+statusCode+"<br>");
out.write("<strong>Requested URI</strong>:"+requestUri);
} else {
out.write("<h3>Exception Details</h3>");
out.write("<ul><li>Servlet Name:"+servletName+"</li>");
out.write("<li>Exception Name:"+throwable.getClass().getName()+"</li>");
out.write("<li>Requested URI:"+requestUri+"</li>");
out.write("<li>Exception Message:"+throwable.getMessage()+"</li>");
out.write("</ul>");
}
out.write("<br><br>");
out.write("</body></html>");
}
}
I see that the application is launched using gateway.jar. And i also see my
web.xml inside that jar. However I'm not able to get anything returned from
this servlet!
I honestly don't know how to repro a 500. But I could do a 400, 401, and 404.
Neither of them got intercepted by the Exception servlet i wrote.
Here are some examples I ran. Note in the first one, a 400 is returned along
with some exception message. that's what i want to do for 500 or verify that
it's being done. However I haven't been able to (using text search) find out
where in the code this response is formed like this
root@clustertest:/tests/knox# curl -iku root:goodpassword -X GET
https://gateway-svc:8443/gateway/default/webhdfs/v1/?op=STATUS
HTTP/1.1 400 Bad Request
Date: Fri, 06 Sep 2019 23:42:42 GMT
Set-Cookie:
KNOXSESSIONID=node01q5krk3jp1c9dzv3fc3t5tkgh4.node0;Path=/gateway/default;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Thu,
05-Sep-2019 23:42:43 GMT
Date: Fri, 06 Sep 2019 23:42:43 GMT
Cache-Control: no-cache
Expires: Fri, 06 Sep 2019 23:42:45 GMT
Date: Fri, 06 Sep 2019 23:42:45 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Content-Type: application/json;charset=utf-8
Transfer-Encoding: chunked
Server: Jetty(9.4.12.v20180830)
{"RemoteException":{"exception":"IllegalArgumentException","javaClassName":"java.lang.IllegalArgumentException","message":"Invalid
value for webhdfs parameter \"op\": STATUS is not a valid GET operation."}}
root@clustertest:/tests/knox# curl -iku root:goodpassword -X GET
https://gateway-svc:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS
HTTP/1.1 200 OK
Date: Fri, 06 Sep 2019 23:43:29 GMT
Set-Cookie:
KNOXSESSIONID=node0iz11bxvbn318h7zow5z977pc5.node0;Path=/gateway/default;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Thu,
05-Sep-2019 23:43:30 GMT
Date: Fri, 06 Sep 2019 23:43:30 GMT
Cache-Control: no-cache
Expires: Fri, 06 Sep 2019 23:43:30 GMT
Date: Fri, 06 Sep 2019 23:43:30 GMT
Pragma: no-cache
X-FRAME-OPTIONS: SAMEORIGIN
Content-Type: application/json;charset=utf-8
Transfer-Encoding: chunked
Server: Jetty(9.4.12.v20180830)
{"FileStatuses":{"FileStatus":[{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":16411,"group":"supergroup","length":0,"modificationTime":1567812978306,"owner":"root","pathSuffix":"jar","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":6,"fileId":16389,"group":"supergroup","length":0,"modificationTime":1567812975255,"owner":"root","pathSuffix":"livy","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16386,"group":"supergroup","length":0,"modificationTime":1567812943856,"owner":"root","pathSuffix":"spark","permission":"775","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16387,"group":"supergroup","length":0,"modificationTime":1567813293988,"owner":"root","pathSuffix":"spark-events","permission":"733","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":2,"fileId":16395,"group":"supergroup","length":0,"modificationTime":1567813273907,"owner":"root","pathSuffix":"tmp","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"},{"accessTime":0,"blockSize":0,"childrenNum":1,"fileId":16412,"group":"supergroup","length":0,"modificationTime":1567813267540,"owner":"root","pathSuffix":"user","permission":"777","replication":0,"storagePolicy":0,"type":"DIRECTORY"}]}}
root@clustertest:/tests/knox# curl -iku root:goodpassword -X GET
https://gateway-svc:8443/gateway/default/webhdfs/v2/?op=LISTSTATUS
HTTP/1.1 404 Not Found
Date: Fri, 06 Sep 2019 23:43:53 GMT
Content-Length: 0
Server: Jetty(9.4.12.v20180830)
root@clustertest:/tests/knox# curl -iku root:badpassword -X GET
https://gateway-svc:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS
HTTP/1.1 401 Unauthorized
Date: Fri, 06 Sep 2019 23:44:17 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Thu,
05-Sep-2019 23:44:17 GMT
WWW-Authenticate: BASIC realm="application"
Content-Length: 0
Server: Jetty(9.4.12.v20180830)
