Re: Error Regarding RFC 7230 and RC 3986
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 10/18/18 09:09, Mark H. Wood wrote: > On Thu, Oct 18, 2018 at 11:55:24AM +0100, M. Manna wrote: >> Thanks a bunch Mark. >> >> "The correct fix is to ensure that the user agents are sending >> specification compliant requests." - Do you mean at browser level >> ? If so, is there any specific browser/update we can use? We've >> checked a few browsers so far (Firefox, Edge, Chrome) and none of >> them seem to have this option (or we might've missed it). > > [snip] > The URI we have for this problem has the following param (did work with 8.5.28) defaultMessageType=true&locale=en_US&action=[key:label.edit] > > The browser did not actually *compose* that parameter, did it? > > If I had this problem, given only what I know from this thread, I > would suppose that the page which contained an href having such a > parameter is the source of the problem. Some link is improperly > encoded. Nope. Because if the application %-encodes those bracket characters, the browser will double-%-encode them. There is literally no way to get a browser to send the proper RFC-compliant character string, which is why these hacks are necessary. > I would say it is debatable whether browsers should be > "correcting" hrefs which are handed to them by some site. They are required to do so by the HTML specification. Well... the HTML/HTTP spec also require them to do other things which they aren't doing, so I suppose it's all the wild west out there. Did I mention that ignoring standards hurts everybody? IMO, HTTP/2 should have done-away with all of this and 100% required clients and servers to implement the spec TO THE LETTER. Basically a lot of "servers RFC-MUST cause the client to catch fire if the encoding is incorrect" or something similar. But we all missed an opportunity, there. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvIqsYACgkQHPApP6U8 pFi8zA/+KtWKJrOPn85xsW59lGBvZrxRP49DoLT7Smjkolpni99zCnHbR/QYfEqP 4iAo/GS1v11OtgJythTc26Ywy70Tm4KRWyrLwb+b4NfiChBaqzcg2k1mPL3N+spp XDC7lfjzEHRtECuO9C/P1BJZ1+yQPaqBtZlczwanCLps7rveOc2z2Tysfo8AaFAy 2aURDpn/BsUrcS08FJMHaqBBsrBbQnvQ2fOjgV1v7umE21C/oZSEZ2/DMKfw+Tx4 92TwszW+cJloWRpSYVI0GA7ydSpw8GAfBbhci9VTc4PqHTxgUVkTQcem+8D90RtR sa8aHy1CC6VILdtunApjcJCztnKn+6+XKqNEb5gHnI7uaEcU/I/hiGxqvfGv3fDh o+gXD3YDwyJiG3v5ldJtP5GyIlmST0ihhDS8B1PyTHaYhEabU9flZ5Ul+pibU35X aTPfBuN3+zHqAuNCZDSekHWTOHhwCHxeMhEBSYKe/8BxSXkfDpf+n1xGQWPtI/u2 gRQeI1BsqN+PAZDPXbeSMc2ArZoqHVxQ5WCUAQfdD37PC2L4s/t72jonRXNplRLt X7EmwuaowURf7tJMetsWLmkWxbxaHQG0Cb4BwEOl3pST2vbNg9eZ4L6yW8+VCojx WsvwSBrbZl6k55A19ZnRnoEdCTyWZh+8cObB8HZPrCGHgTsws4E= =2/qX -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Error Regarding RFC 7230 and RC 3986
On Thu, 18 Oct 2018 at 13:38, Mark Thomas wrote: > On 18/10/18 12:17, Johan Compagner wrote: > > how is the browser to blame for " > > defaultMessageType=true&locale=en_US&action=[key:label.edit]" > > > > that url is not generated by a browser but by some software that uses a > > browser... > > Browsers these days try to be helpful and show the user the un-encoded > URI in the address bar but send the encoded version to the server. > > If this is really the case that you have a webpage where we have a href with the right url (so the above url is encoded) and when you click on it, the browser loads it (and i guess that should be fine then) But then in the browser you see the un-encoded version (as a bove) and when you do then a refresh it goes wrong? That would be bad... That something the browser really needs to fix... So yes if the browser really sets the un-encoded version in the url bar that would be quite annoying (bookmarking it and so on)
Re: Error Regarding RFC 7230 and RC 3986
We already know that the parameter is the issue. Having to change all the parameter i.e. refactoring code is always the answer. The question was more about the recommended way of handling this issue without exposing application to any specific vulnerability. I believe Mark T has answered this already. Thanks, On Thu, 18 Oct 2018 at 14:09, Mark H. Wood wrote: > On Thu, Oct 18, 2018 at 11:55:24AM +0100, M. Manna wrote: > > Thanks a bunch Mark. > > > > "The correct fix is to ensure that the user agents are sending > > specification compliant requests." - Do you mean at browser level ? If > so, > > is there any specific browser/update we can use? We've checked a few > > browsers so far (Firefox, Edge, Chrome) and none of them seem to have > this > > option (or we might've missed it). > > [snip] > > > > > The URI we have for this problem has the following param (did work > with > > > > 8.5.28) > > > > > > > > defaultMessageType=true&locale=en_US&action=[key:label.edit] > > The browser did not actually *compose* that parameter, did it? > > If I had this problem, given only what I know from this thread, I > would suppose that the page which contained an href having such a > parameter is the source of the problem. Some link is improperly > encoded. > > I would say it is debatable whether browsers should be "correcting" > hrefs which are handed to them by some site. > > -- > Mark H. Wood > Lead Technology Analyst > > University Library > Indiana University - Purdue University Indianapolis > 755 W. Michigan Street > Indianapolis, IN 46202 > 317-274-0749 > www.ulib.iupui.edu >
Re: Error Regarding RFC 7230 and RC 3986
On Thu, Oct 18, 2018 at 11:55:24AM +0100, M. Manna wrote: > Thanks a bunch Mark. > > "The correct fix is to ensure that the user agents are sending > specification compliant requests." - Do you mean at browser level ? If so, > is there any specific browser/update we can use? We've checked a few > browsers so far (Firefox, Edge, Chrome) and none of them seem to have this > option (or we might've missed it). [snip] > > > The URI we have for this problem has the following param (did work with > > > 8.5.28) > > > > > > defaultMessageType=true&locale=en_US&action=[key:label.edit] The browser did not actually *compose* that parameter, did it? If I had this problem, given only what I know from this thread, I would suppose that the page which contained an href having such a parameter is the source of the problem. Some link is improperly encoded. I would say it is debatable whether browsers should be "correcting" hrefs which are handed to them by some site. -- Mark H. Wood Lead Technology Analyst University Library Indiana University - Purdue University Indianapolis 755 W. Michigan Street Indianapolis, IN 46202 317-274-0749 www.ulib.iupui.edu signature.asc Description: PGP signature
Re: Error Regarding RFC 7230 and RC 3986
On 18/10/18 12:17, Johan Compagner wrote: > how is the browser to blame for " > defaultMessageType=true&locale=en_US&action=[key:label.edit]" > > that url is not generated by a browser but by some software that uses a > browser... Browsers these days try to be helpful and show the user the un-encoded URI in the address bar but send the encoded version to the server. Personally, that sort of behaviour bugs me as I like to know exactly what is going on - not have the technology perform 'magic' for me under the covers to get stuff to work. But I suspect that is just me. And that is why I do a lot of my testing in this area with telnet. In this instance, the browser should encode the '[' and the ']' but it does not. See this link for how browsers should (as per the RFC) encode characters in the path and query string compared to how they actually behave. https://cwiki.apache.org/confluence/display/TOMCAT/Encoding+and+URIs Mark > > > On Thu, 18 Oct 2018 at 12:55, M. Manna wrote: > >> Thanks a bunch Mark. >> >> "The correct fix is to ensure that the user agents are sending >> specification compliant requests." - Do you mean at browser level ? If so, >> is there any specific browser/update we can use? We've checked a few >> browsers so far (Firefox, Edge, Chrome) and none of them seem to have this >> option (or we might've missed it). >> >> We are using relaxedQueryChars for now - but would like to understand the >> fix you've proposed above. >> >> On Thu, 18 Oct 2018 at 10:39, Mark Thomas wrote: >> >>> On 18/10/18 09:52, M. Manna wrote: Hello, We received in error in our application after we have upgraded to >> 8.5.34 INFO: Error parsing HTTP request header Note: further occurrences of HTTP header parsing errors will be logged >> at DEBUG level. java.lang.IllegalArgumentException: Invalid character found in the >>> request target. The valid characters are defined in RFC 7230 and RFC 3986 The URI we have for this problem has the following param (did work with 8.5.28) defaultMessageType=true&locale=en_US&action=[key:label.edit] The issue is the action parameter value. Could someone help me >> understand the following? 1) Since the issue didn't happen for 8.5.28 - this means some CVE has triggered this change to be in place. I am just trying to confirm if >> this is CVE-2016-681 ? If not, could you please let me know which one that >> is? >>> >>> The change in request parsing was prompted by CVE-2016-6816. There >>> wasn't a specific attack that prompted this particular change. >>> CVE-2016-6816 was caused by not parsing the request line as per the >>> spec. Therefore, to reduce the risk of future vulnerabilities, we have >>> been tightening up the parsing of the request line. >>> 2) Apart from refactoring code, is there any recommended corrective >>> action? >>> >>> The correct fix is to ensure that the user agents are sending >>> specification compliant requests. >>> >>> The work-around is to use relaxedPathChars and/or relaxedQueryChars on >>> the Connector. >>> >>> Mark >>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >>> >> > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Error Regarding RFC 7230 and RC 3986
I understand. We will use the connector patch for now. But thanks again for sharing your thoughts. And the link to apache Confluence is really helpful! Thanks, On Thu, 18 Oct 2018 at 12:12, Mark Thomas wrote: > On 18/10/18 11:55, M. Manna wrote: > > Thanks a bunch Mark. > > > > "The correct fix is to ensure that the user agents are sending > > specification compliant requests." - Do you mean at browser level ? If > so, > > is there any specific browser/update we can use? We've checked a few > > browsers so far (Firefox, Edge, Chrome) and none of them seem to have > this > > option (or we might've missed it). > > The browsers should fix it but I doubt they will. I've seen at least one > browser vendor refuse to accept the current behaviour is a bug and claim > that the browsers are working to their own, different spec. > > Investigations show that each browser has slightly different behaviour. > > https://cwiki.apache.org/confluence/display/TOMCAT/Encoding+and+URIs > > So even if the browsers are following a different spec, they aren't > implementing that one correctly either. > > Sigh. > > Mark > > > > > We are using relaxedQueryChars for now - but would like to understand the > > fix you've proposed above. > > > > On Thu, 18 Oct 2018 at 10:39, Mark Thomas wrote: > > > >> On 18/10/18 09:52, M. Manna wrote: > >>> Hello, > >>> > >>> We received in error in our application after we have upgraded to > 8.5.34 > >>> > >>> INFO: Error parsing HTTP request header > >>> Note: further occurrences of HTTP header parsing errors will be logged > at > >>> DEBUG level. > >>> java.lang.IllegalArgumentException: Invalid character found in the > >> request > >>> target. The valid characters are defined in RFC 7230 and RFC 3986 > >>> > >>> > >>> The URI we have for this problem has the following param (did work with > >>> 8.5.28) > >>> > >>> defaultMessageType=true&locale=en_US&action=[key:label.edit] > >>> > >>> The issue is the action parameter value. Could someone help me > understand > >>> the following? > >>> > >>> 1) Since the issue didn't happen for 8.5.28 - this means some CVE has > >>> triggered this change to be in place. I am just trying to confirm if > this > >>> is CVE-2016-681 ? If not, could you please let me know which one that > is? > >> > >> The change in request parsing was prompted by CVE-2016-6816. There > >> wasn't a specific attack that prompted this particular change. > >> CVE-2016-6816 was caused by not parsing the request line as per the > >> spec. Therefore, to reduce the risk of future vulnerabilities, we have > >> been tightening up the parsing of the request line. > >> > >>> 2) Apart from refactoring code, is there any recommended corrective > >> action? > >> > >> The correct fix is to ensure that the user agents are sending > >> specification compliant requests. > >> > >> The work-around is to use relaxedPathChars and/or relaxedQueryChars on > >> the Connector. > >> > >> Mark > >> > >> - > >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > >> > >> > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Error Regarding RFC 7230 and RC 3986
how is the browser to blame for " defaultMessageType=true&locale=en_US&action=[key:label.edit]" that url is not generated by a browser but by some software that uses a browser... On Thu, 18 Oct 2018 at 12:55, M. Manna wrote: > Thanks a bunch Mark. > > "The correct fix is to ensure that the user agents are sending > specification compliant requests." - Do you mean at browser level ? If so, > is there any specific browser/update we can use? We've checked a few > browsers so far (Firefox, Edge, Chrome) and none of them seem to have this > option (or we might've missed it). > > We are using relaxedQueryChars for now - but would like to understand the > fix you've proposed above. > > On Thu, 18 Oct 2018 at 10:39, Mark Thomas wrote: > > > On 18/10/18 09:52, M. Manna wrote: > > > Hello, > > > > > > We received in error in our application after we have upgraded to > 8.5.34 > > > > > > INFO: Error parsing HTTP request header > > > Note: further occurrences of HTTP header parsing errors will be logged > at > > > DEBUG level. > > > java.lang.IllegalArgumentException: Invalid character found in the > > request > > > target. The valid characters are defined in RFC 7230 and RFC 3986 > > > > > > > > > The URI we have for this problem has the following param (did work with > > > 8.5.28) > > > > > > defaultMessageType=true&locale=en_US&action=[key:label.edit] > > > > > > The issue is the action parameter value. Could someone help me > understand > > > the following? > > > > > > 1) Since the issue didn't happen for 8.5.28 - this means some CVE has > > > triggered this change to be in place. I am just trying to confirm if > this > > > is CVE-2016-681 ? If not, could you please let me know which one that > is? > > > > The change in request parsing was prompted by CVE-2016-6816. There > > wasn't a specific attack that prompted this particular change. > > CVE-2016-6816 was caused by not parsing the request line as per the > > spec. Therefore, to reduce the risk of future vulnerabilities, we have > > been tightening up the parsing of the request line. > > > > > 2) Apart from refactoring code, is there any recommended corrective > > action? > > > > The correct fix is to ensure that the user agents are sending > > specification compliant requests. > > > > The work-around is to use relaxedPathChars and/or relaxedQueryChars on > > the Connector. > > > > Mark > > > > - > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > > -- Johan Compagner Servoy
Re: Error Regarding RFC 7230 and RC 3986
On 18/10/18 11:55, M. Manna wrote: > Thanks a bunch Mark. > > "The correct fix is to ensure that the user agents are sending > specification compliant requests." - Do you mean at browser level ? If so, > is there any specific browser/update we can use? We've checked a few > browsers so far (Firefox, Edge, Chrome) and none of them seem to have this > option (or we might've missed it). The browsers should fix it but I doubt they will. I've seen at least one browser vendor refuse to accept the current behaviour is a bug and claim that the browsers are working to their own, different spec. Investigations show that each browser has slightly different behaviour. https://cwiki.apache.org/confluence/display/TOMCAT/Encoding+and+URIs So even if the browsers are following a different spec, they aren't implementing that one correctly either. Sigh. Mark > > We are using relaxedQueryChars for now - but would like to understand the > fix you've proposed above. > > On Thu, 18 Oct 2018 at 10:39, Mark Thomas wrote: > >> On 18/10/18 09:52, M. Manna wrote: >>> Hello, >>> >>> We received in error in our application after we have upgraded to 8.5.34 >>> >>> INFO: Error parsing HTTP request header >>> Note: further occurrences of HTTP header parsing errors will be logged at >>> DEBUG level. >>> java.lang.IllegalArgumentException: Invalid character found in the >> request >>> target. The valid characters are defined in RFC 7230 and RFC 3986 >>> >>> >>> The URI we have for this problem has the following param (did work with >>> 8.5.28) >>> >>> defaultMessageType=true&locale=en_US&action=[key:label.edit] >>> >>> The issue is the action parameter value. Could someone help me understand >>> the following? >>> >>> 1) Since the issue didn't happen for 8.5.28 - this means some CVE has >>> triggered this change to be in place. I am just trying to confirm if this >>> is CVE-2016-681 ? If not, could you please let me know which one that is? >> >> The change in request parsing was prompted by CVE-2016-6816. There >> wasn't a specific attack that prompted this particular change. >> CVE-2016-6816 was caused by not parsing the request line as per the >> spec. Therefore, to reduce the risk of future vulnerabilities, we have >> been tightening up the parsing of the request line. >> >>> 2) Apart from refactoring code, is there any recommended corrective >> action? >> >> The correct fix is to ensure that the user agents are sending >> specification compliant requests. >> >> The work-around is to use relaxedPathChars and/or relaxedQueryChars on >> the Connector. >> >> Mark >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Error Regarding RFC 7230 and RC 3986
Thanks a bunch Mark. "The correct fix is to ensure that the user agents are sending specification compliant requests." - Do you mean at browser level ? If so, is there any specific browser/update we can use? We've checked a few browsers so far (Firefox, Edge, Chrome) and none of them seem to have this option (or we might've missed it). We are using relaxedQueryChars for now - but would like to understand the fix you've proposed above. On Thu, 18 Oct 2018 at 10:39, Mark Thomas wrote: > On 18/10/18 09:52, M. Manna wrote: > > Hello, > > > > We received in error in our application after we have upgraded to 8.5.34 > > > > INFO: Error parsing HTTP request header > > Note: further occurrences of HTTP header parsing errors will be logged at > > DEBUG level. > > java.lang.IllegalArgumentException: Invalid character found in the > request > > target. The valid characters are defined in RFC 7230 and RFC 3986 > > > > > > The URI we have for this problem has the following param (did work with > > 8.5.28) > > > > defaultMessageType=true&locale=en_US&action=[key:label.edit] > > > > The issue is the action parameter value. Could someone help me understand > > the following? > > > > 1) Since the issue didn't happen for 8.5.28 - this means some CVE has > > triggered this change to be in place. I am just trying to confirm if this > > is CVE-2016-681 ? If not, could you please let me know which one that is? > > The change in request parsing was prompted by CVE-2016-6816. There > wasn't a specific attack that prompted this particular change. > CVE-2016-6816 was caused by not parsing the request line as per the > spec. Therefore, to reduce the risk of future vulnerabilities, we have > been tightening up the parsing of the request line. > > > 2) Apart from refactoring code, is there any recommended corrective > action? > > The correct fix is to ensure that the user agents are sending > specification compliant requests. > > The work-around is to use relaxedPathChars and/or relaxedQueryChars on > the Connector. > > Mark > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Error Regarding RFC 7230 and RC 3986
On 18/10/18 09:52, M. Manna wrote: > Hello, > > We received in error in our application after we have upgraded to 8.5.34 > > INFO: Error parsing HTTP request header > Note: further occurrences of HTTP header parsing errors will be logged at > DEBUG level. > java.lang.IllegalArgumentException: Invalid character found in the request > target. The valid characters are defined in RFC 7230 and RFC 3986 > > > The URI we have for this problem has the following param (did work with > 8.5.28) > > defaultMessageType=true&locale=en_US&action=[key:label.edit] > > The issue is the action parameter value. Could someone help me understand > the following? > > 1) Since the issue didn't happen for 8.5.28 - this means some CVE has > triggered this change to be in place. I am just trying to confirm if this > is CVE-2016-681 ? If not, could you please let me know which one that is? The change in request parsing was prompted by CVE-2016-6816. There wasn't a specific attack that prompted this particular change. CVE-2016-6816 was caused by not parsing the request line as per the spec. Therefore, to reduce the risk of future vulnerabilities, we have been tightening up the parsing of the request line. > 2) Apart from refactoring code, is there any recommended corrective action? The correct fix is to ensure that the user agents are sending specification compliant requests. The work-around is to use relaxedPathChars and/or relaxedQueryChars on the Connector. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Error Regarding RFC 7230 and RC 3986
Hello, We received in error in our application after we have upgraded to 8.5.34 INFO: Error parsing HTTP request header Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level. java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986 The URI we have for this problem has the following param (did work with 8.5.28) defaultMessageType=true&locale=en_US&action=[key:label.edit] The issue is the action parameter value. Could someone help me understand the following? 1) Since the issue didn't happen for 8.5.28 - this means some CVE has triggered this change to be in place. I am just trying to confirm if this is CVE-2016-681 ? If not, could you please let me know which one that is? 2) Apart from refactoring code, is there any recommended corrective action? Thanks,
