Mark,
On 11/17/2016 1:06 PM, Mark Thomas wrote:
> On 17/11/2016 21:29, Mark Eggers wrote:
>> Mark,
>>
>>
>> On 11/17/2016 2:00 AM, Mark Thomas wrote:
>>> On 16/11/2016 20:05, Mark Eggers wrote:
>>>> Mark,
>>>>
>>>> On 11/16/2016 12:23 AM, Mark Thomas wrote:
>>>>> On 15/11/2016 22:36, Zdeněk Henek wrote:
>>>>>> Hi,
>>>>>>
>>>>>> we are using tomcat 8.0.30 without problems.
>>>>>>
>>>>>> I have tested upgrade to 8.0.38 today and I got this error
>>>>>> More env. details JDK 8, tested on both Linux and Windows using different
>>>>>> JDK 8 updates (71, 111).
>>>>>>
>>>>>> 15-Nov-2016 17:14:51.189 INFO [http-nio-8080-exec-2]
>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process Error parsing
>>>>>> HTTP
>>>>>> request header
>>>>>> Note: further occurrences of HTTP header parsing errors will be logged
>>>>>> at
>>>>>> DEBUG level.
>>>>>> java.lang.IllegalArgumentException: Invalid character found in the
>>>>>> request
>>>>>> target. The valid characters are defined in RFC 7230 and RFC 3986
>>>>>
>>>>> <snip/>
>>>>>
>>>>>> The parameter in the request is this
>>>>>>
>>>>>> /list?criteria={%22$type%22:%22Equal%22,%22attr%22:%22id%22,%22value%22:101}
>>>>>
>>>>> Neither '{' nor '}' are permitted characters in a URI and that includes
>>>>> the query string.
>>>>>
>>>>>> Looks like this commit caused the exception
>>>>>> https://github.com/apache/tomcat80/commit/779d5d34e68e50d2f721897050b147106992f566
>>>>>>
>>>>>> The commit message says:
>>>>>> Add additional checks for valid characters to the HTTP request line
>>>>>> parsing so invalid request lines are rejected sooner.
>>>>>>
>>>>>> We don't get any error in 8.0.30 using same request.
>>>>>>
>>>>>> The state in 8.0.30 was bug or 8.0.38 should process parameter
>>>>>>
>>>>>> criteria={%22$type%22:%22Equal%22,%22attr%22:%22id%22,%22value%22:101}
>>>>>>
>>>>>> ?
>>>>>
>>>>> Technically, 8.0.30 should have rejected the request but didn't.
>>>>>
>>>>> As per the commit message, Tomcat has tightened up validation of
>>>>> incoming HTTP requests to reject any that are not specification compliant.
>>>>>
>>>>> For the query string, the relevant extracts from RFC 3986 are:
>>>>>
>>>>> query = *( pchar / "/" / "?" )
>>>>>
>>>>> pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
>>>>>
>>>>> unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
>>>>>
>>>>> sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
>>>>> / "*" / "+" / "," / ";" / "="
>>>>>
>>>>>
>>>>> Hence, '{' and '}' are rejected.
>>>>>
>>>>> Mark
>>>>
>>>> Based on your explanation above, shouldn't the following query parameter
>>>> be rejected?
>>>>
>>>> http://somehost/someurl?plist=tagA=valueA|tagB=valueB|tagC=valueC
>>>>
>>>> where tagA, tagB, tagC, valueA, valueB, valueC are all ALPHA or DIGIT.
>>>>
>>>> I didn't see "|" listed as acceptable anywhere in RFC 3986.
>>>
>>> I agree, such a request should be rejected.
>>>
>>>> However, above URL works in Tomcat 8.0.39.
>>>
>>> I've just tested 9.0.x and 8.0.x and both rejected it. I don't think
>>> there have been any changes since those releases. Are you sure that:
>>> a) you are using 8.0.39
>>> b) the client isn't encoding the '|' before it is sent to Tomcat
>>>
>>>> I ask this because a developer has used the pipe symbol to separate
>>>> components. It plays havoc with mod_security rules, among other things.
>>>>
>>>> . . . a bit puzzled
>>>
>>> Me too. Any light you can shed would be helpful.
>>
>> I did a Wireshark capture. The client is not encoding '|' before
>> sending. The '=' is not being encoded either.
>>
>> I figured it out. I have Apache 2.2 (on Linux) or Apache 2.4 (on
>> Windows) in front of Tomcat.
>>
>> I connect the two using mod_jk. When going through the following:
>>
>> browser --> apache httpd (2.2, 2.4) -->(AJP) Tomcat (8.0.39, 8.5.8)
>>
>> the request works ('|', '=', and other hideousness).
>>
>> When going through the following:
>>
>> browser --> Tomcat (8.0.39, 8.5.8)
>>
>> the request fails with the error message as posted by the original author.
>>
>> I'll go through the Apache HTTPD and mod_jk configurations carefully to
>> see what's going on.
>>
>> However, both are pretty stock configurations.
>
> The AJP checks are much less rigorous since it is assumed that the
> front-end server will validate the data before forwarding. It looks like
> httpd isn't as strict as Tomcat in this case.
>
> MarkAlso, the default for mod_jk JkOptions is: JkOptions +ForwardURIProxy which according to the documentation does a partial encoding before sending the request off to Tomcat. So in summary: 1. Apache HTTPD 2.2 and Apache HTTPD 2.4 are lenient when parsing URIs 2. Default JkOptions partially encode the request before sending 3. The resulting encoded URI is happily parsed by Tomcat Removing Apache HTTPD 2.2 / Apache HTTPD 2.4 with the default mod_jk configuration (JkOptions) and the URI will no longer work with Tomcat 8.x. Time to get the developers to fix their code. . . . just my two cents /mde/
signature.asc
Description: OpenPGP digital signature
