Content-Length header for HTTP 204 and 1xx status codes

[Luca Toscano]

Hi everybody,

while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350 a
user asked why httpd send the "Content-Length: 0" header for HTTP 204
responses given the following statement in the RFC:

https://tools.ietf.org/html/rfc7230#page-30
"A server MUST NOT send a Content-Length header field in any response with
a status code of 1xx (Informational) or 204 (No Content)."

I tried with a simple PHP script returning an HTTP 204 header (via
mod_proxy_fcgi) and indeed I can see the Content-Length: 0. After a bit of
digging it seems that ap_content_length_filter in protocol.c adds the
header when it evaluates:

if (!(r->header_only
  && !r->bytes_sent
  && (r->sent_bodyct
  || conf->http_cl_head_zero != AP_HTTP_CL_HEAD_ZERO_ENABLE
  || apr_table_get(r->headers_out, "Content-Length" {
ap_set_content_length(r, r->bytes_sent);
}

An idea to fix the issue after a bit of chat on IRC could be the following
snippet, even if it might no be the right move. Since I am not expert
enough to attempt any code change to http_filters.c or protocol.c, I'll
wait for some feedback about how to proceed (that might also be "don't do
anything, it is fine in this way" :)

Thanks!

Luca

Index: modules/http/http_filters.c
===
--- modules/http/http_filters.c (revision 1772052)
+++ modules/http/http_filters.c (working copy)
@@ -1296,6 +1296,10 @@
 apr_table_unset(r->headers_out, "Content-Length");
 }

+if ((r->status == HTTP_NO_CONTENT || ap_is_HTTP_INFO(r->status)) &&
!r->bytes_sent) {
+apr_table_unset(r->headers_out, "Content-Length");
+}
+
 ctype = ap_make_content_type(r, r->content_type);
 if (ctype) {
 apr_table_setn(r->headers_out, "Content-Type", ctype);

Re: Content-Length header for HTTP 204 and 1xx status codes

[Luca Toscano]

2016-11-30 18:46 GMT+01:00 Luca Toscano :

> Hi everybody,
>
> while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350 a
> user asked why httpd send the "Content-Length: 0" header for HTTP 204
> responses given the following statement in the RFC:
>
> https://tools.ietf.org/html/rfc7230#page-30
> "A server MUST NOT send a Content-Length header field in any response with
> a status code of 1xx (Informational) or 204 (No Content)."
>
> I tried with a simple PHP script returning an HTTP 204 header (via
> mod_proxy_fcgi) and indeed I can see the Content-Length: 0. After a bit of
> digging it seems that ap_content_length_filter in protocol.c adds the
> header when it evaluates:
>
> if (!(r->header_only
>   && !r->bytes_sent
>   && (r->sent_bodyct
>   || conf->http_cl_head_zero != AP_HTTP_CL_HEAD_ZERO_ENABLE
>   || apr_table_get(r->headers_out, "Content-Length" {
> ap_set_content_length(r, r->bytes_sent);
> }
>
> An idea to fix the issue after a bit of chat on IRC could be the following
> snippet, even if it might no be the right move. Since I am not expert
> enough to attempt any code change to http_filters.c or protocol.c, I'll
> wait for some feedback about how to proceed (that might also be "don't do
> anything, it is fine in this way" :)
>
> Thanks!
>
> Luca
>
> Index: modules/http/http_filters.c
> ===
> --- modules/http/http_filters.c (revision 1772052)
> +++ modules/http/http_filters.c (working copy)
> @@ -1296,6 +1296,10 @@
>  apr_table_unset(r->headers_out, "Content-Length");
>  }
>
> +if ((r->status == HTTP_NO_CONTENT || ap_is_HTTP_INFO(r->status)) &&
> !r->bytes_sent) {
> +apr_table_unset(r->headers_out, "Content-Length");
> +}
> +
>  ctype = ap_make_content_type(r, r->content_type);
>  if (ctype) {
>  apr_table_setn(r->headers_out, "Content-Type", ctype);
>
>
Any feedback?

Thanks!

Luca

Re: Content-Length header for HTTP 204 and 1xx status codes

[Yann Ylavic]

Hi Luca,

sorry for the delay (overwhelmed these times)...

On Mon, Dec 5, 2016 at 1:21 PM, Luca Toscano  wrote:
>
>
> 2016-11-30 18:46 GMT+01:00 Luca Toscano :
>>
>> Hi everybody,
>>
>> while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350 a
>> user asked why httpd send the "Content-Length: 0" header for HTTP 204
>> responses given the following statement in the RFC:
>>
>> https://tools.ietf.org/html/rfc7230#page-30
>> "A server MUST NOT send a Content-Length header field in any response with
>> a status code of 1xx (Informational) or 204 (No Content)."
>>
>> I tried with a simple PHP script returning an HTTP 204 header (via
>> mod_proxy_fcgi) and indeed I can see the Content-Length: 0. After a bit of
>> digging it seems that ap_content_length_filter in protocol.c adds the header
>> when it evaluates:
>>
>> if (!(r->header_only
>>   && !r->bytes_sent
>>   && (r->sent_bodyct
>>   || conf->http_cl_head_zero !=
>> AP_HTTP_CL_HEAD_ZERO_ENABLE
>>   || apr_table_get(r->headers_out, "Content-Length" {
>> ap_set_content_length(r, r->bytes_sent);
>> }

How about adding (yet) another condition to the above:

Index: server/protocol.c
===
--- server/protocol.c(revision 1772657)
+++ server/protocol.c(working copy)
@@ -1766,7 +1766,9 @@ AP_CORE_DECLARE_NONSTD(apr_status_t) ap_content_le
  * such filters update or remove the C-L header, and just use it
  * if present.
  */
-if (!(r->header_only
+if (!((r->header_only
+   || r->status == HTTP_NO_CONTENT
+   || r->status == HTTP_NOT_MODIFIED)
   && !r->bytes_sent
   && (r->sent_bodyct
   || conf->http_cl_head_zero != AP_HTTP_CL_HEAD_ZERO_ENABLE
?

Regards,
Yann.

Re: Content-Length header for HTTP 204 and 1xx status codes

[Luca Toscano]

Hi Yann,

2016-12-05 13:54 GMT+01:00 Yann Ylavic :

> Hi Luca,
>
> sorry for the delay (overwhelmed these times)...
>

thanks a lot for the help!


>
> On Mon, Dec 5, 2016 at 1:21 PM, Luca Toscano 
> wrote:
> >
> >
> > 2016-11-30 18:46 GMT+01:00 Luca Toscano :
> >>
> >> Hi everybody,
> >>
> >> while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350 a
> >> user asked why httpd send the "Content-Length: 0" header for HTTP 204
> >> responses given the following statement in the RFC:
> >>
> >> https://tools.ietf.org/html/rfc7230#page-30
> >> "A server MUST NOT send a Content-Length header field in any response
> with
> >> a status code of 1xx (Informational) or 204 (No Content)."
> >>
> >> I tried with a simple PHP script returning an HTTP 204 header (via
> >> mod_proxy_fcgi) and indeed I can see the Content-Length: 0. After a bit
> of
> >> digging it seems that ap_content_length_filter in protocol.c adds the
> header
> >> when it evaluates:
> >>
> >> if (!(r->header_only
> >>   && !r->bytes_sent
> >>   && (r->sent_bodyct
> >>   || conf->http_cl_head_zero !=
> >> AP_HTTP_CL_HEAD_ZERO_ENABLE
> >>   || apr_table_get(r->headers_out, "Content-Length"
> {
> >> ap_set_content_length(r, r->bytes_sent);
> >> }
>
> How about adding (yet) another condition to the above:
>
> Index: server/protocol.c
> ===
> --- server/protocol.c(revision 1772657)
> +++ server/protocol.c(working copy)
> @@ -1766,7 +1766,9 @@ AP_CORE_DECLARE_NONSTD(apr_status_t) ap_content_le
>   * such filters update or remove the C-L header, and just use it
>   * if present.
>   */
> -if (!(r->header_only
> +if (!((r->header_only
> +   || r->status == HTTP_NO_CONTENT
> +   || r->status == HTTP_NOT_MODIFIED)
>&& !r->bytes_sent
>&& (r->sent_bodyct
>|| conf->http_cl_head_zero !=
> AP_HTTP_CL_HEAD_ZERO_ENABLE
> ?


This was the other solution that I had in mind (fixing the issue at its
origin rather than patching it afterwards) but I wasn't confident to make
changes to the already crowded if (that seems to be related to a specific
use case).

One caveat that I realized only now: if the backend sets a C-L header (for
a 204 response) it will not be handled by the above if, so if we want to
patch this use case too we'd probably want to add a stricter condition
in ap_http_header_filter.

Follow up: I got tricked by the "r->header_only" condition in the
beginning, I thought that it would have been applied to all the responses
requiring headers and no body, but it applies only to HEAD requests. I
didn't find any trace in the code about how to prevent a HTTP 204 response
body to be sent, except for mod_proxy_http that explicitly handle this
case. I tested the presence of a body in a simple 204 response from a
Perl/PHP cgi/fcgi script with telnet and I confirmed my suspicion.

So this might help:

Index: modules/http/http_filters.c
===
--- modules/http/http_filters.c (revision 1772510)
+++ modules/http/http_filters.c (working copy)
@@ -1297,6 +1297,10 @@
 apr_table_unset(r->headers_out, "Content-Length");
 }

+if (r->status == HTTP_NO_CONTENT || ap_is_HTTP_INFO(r->status)){
+apr_table_unset(r->headers_out, "Content-Length");
+}
+
 ctype = ap_make_content_type(r, r->content_type);
 if (ctype) {
 apr_table_setn(r->headers_out, "Content-Type", ctype);
@@ -1368,7 +1372,7 @@

 ap_pass_brigade(f->next, b2);

-if (r->header_only) {
+if (r->header_only || r->status == HTTP_NO_CONTENT) {
 apr_brigade_cleanup(b);
 ctx->headers_sent = 1;
 return OK;


Does it make any sense?

Thanks,

Luca

Re: Content-Length header for HTTP 204 and 1xx status codes

[Luca Toscano]

2016-12-06 9:45 GMT+01:00 Luca Toscano :

> Hi Yann,
>
> 2016-12-05 13:54 GMT+01:00 Yann Ylavic :
>
>> Hi Luca,
>>
>> sorry for the delay (overwhelmed these times)...
>>
>
> thanks a lot for the help!
>
>
>>
>> On Mon, Dec 5, 2016 at 1:21 PM, Luca Toscano 
>> wrote:
>> >
>> >
>> > 2016-11-30 18:46 GMT+01:00 Luca Toscano :
>> >>
>> >> Hi everybody,
>> >>
>> >> while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350
>> a
>> >> user asked why httpd send the "Content-Length: 0" header for HTTP 204
>> >> responses given the following statement in the RFC:
>> >>
>> >> https://tools.ietf.org/html/rfc7230#page-30
>> >> "A server MUST NOT send a Content-Length header field in any response
>> with
>> >> a status code of 1xx (Informational) or 204 (No Content)."
>> >>
>> >> I tried with a simple PHP script returning an HTTP 204 header (via
>> >> mod_proxy_fcgi) and indeed I can see the Content-Length: 0. After a
>> bit of
>> >> digging it seems that ap_content_length_filter in protocol.c adds the
>> header
>> >> when it evaluates:
>> >>
>> >> if (!(r->header_only
>> >>   && !r->bytes_sent
>> >>   && (r->sent_bodyct
>> >>   || conf->http_cl_head_zero !=
>> >> AP_HTTP_CL_HEAD_ZERO_ENABLE
>> >>   || apr_table_get(r->headers_out,
>> "Content-Length" {
>> >> ap_set_content_length(r, r->bytes_sent);
>> >> }
>>
>> How about adding (yet) another condition to the above:
>>
>> Index: server/protocol.c
>> ===
>> --- server/protocol.c(revision 1772657)
>> +++ server/protocol.c(working copy)
>> @@ -1766,7 +1766,9 @@ AP_CORE_DECLARE_NONSTD(apr_status_t) ap_content_le
>>   * such filters update or remove the C-L header, and just use it
>>   * if present.
>>   */
>> -if (!(r->header_only
>> +if (!((r->header_only
>> +   || r->status == HTTP_NO_CONTENT
>> +   || r->status == HTTP_NOT_MODIFIED)
>>&& !r->bytes_sent
>>&& (r->sent_bodyct
>>|| conf->http_cl_head_zero !=
>> AP_HTTP_CL_HEAD_ZERO_ENABLE
>> ?
>
>
> This was the other solution that I had in mind (fixing the issue at its
> origin rather than patching it afterwards) but I wasn't confident to make
> changes to the already crowded if (that seems to be related to a specific
> use case).
>
> One caveat that I realized only now: if the backend sets a C-L header (for
> a 204 response) it will not be handled by the above if, so if we want to
> patch this use case too we'd probably want to add a stricter condition
> in ap_http_header_filter.
>
> Follow up: I got tricked by the "r->header_only" condition in the
> beginning, I thought that it would have been applied to all the responses
> requiring headers and no body, but it applies only to HEAD requests. I
> didn't find any trace in the code about how to prevent a HTTP 204 response
> body to be sent, except for mod_proxy_http that explicitly handle this
> case. I tested the presence of a body in a simple 204 response from a
> Perl/PHP cgi/fcgi script with telnet and I confirmed my suspicion.
>
> So this might help:
>
> Index: modules/http/http_filters.c
> ===
> --- modules/http/http_filters.c (revision 1772510)
> +++ modules/http/http_filters.c (working copy)
> @@ -1297,6 +1297,10 @@
>  apr_table_unset(r->headers_out, "Content-Length");
>  }
>
> +if (r->status == HTTP_NO_CONTENT || ap_is_HTTP_INFO(r->status)){
> +apr_table_unset(r->headers_out, "Content-Length");
> +}
> +
>  ctype = ap_make_content_type(r, r->content_type);
>  if (ctype) {
>  apr_table_setn(r->headers_out, "Content-Type", ctype);
> @@ -1368,7 +1372,7 @@
>
>  ap_pass_brigade(f->next, b2);
>
> -if (r->header_only) {
> +if (r->header_only || r->status == HTTP_NO_CONTENT) {
>  apr_brigade_cleanup(b);
>  ctx->headers_sent = 1;
>  return OK;
>
>
> Does it make any sense?
>

 Updated version:
http://home.apache.org/~elukey/httpd-trunk-core-http_204_1xx_nocontent.patch

Luca

Re: Content-Length header for HTTP 204 and 1xx status codes

[William A Rowe Jr]

On Nov 30, 2016 11:46 AM, "Luca Toscano"  wrote:

Hi everybody,

while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350 a
user asked why httpd send the "Content-Length: 0" header for HTTP 204
responses given the following statement in the RFC:

https://tools.ietf.org/html/rfc7230#page-30
"A server MUST NOT send a Content-Length header field in any response with
a status code of 1xx (Informational) or 204 (No Content)."


I was looking at the spec for 101 and 100 responses and think we are going
way overboard on replying with a 100 response. Looking at the 101 example,
we should send a reply of 0 or a few very explicit header fields and save
the balance of output headers for the final response code. Otherwise these
all seem to be wasted network bytes.

WDYT?

Re: Content-Length header for HTTP 204 and 1xx status codes

[William A Rowe Jr]

On Wed, Dec 7, 2016 at 9:05 AM, William A Rowe Jr 
wrote:

> On Nov 30, 2016 11:46 AM, "Luca Toscano"  wrote:
>
> Hi everybody,
>
> while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350 a
> user asked why httpd send the "Content-Length: 0" header for HTTP 204
> responses given the following statement in the RFC:
>
> https://tools.ietf.org/html/rfc7230#page-30
> "A server MUST NOT send a Content-Length header field in any response with
> a status code of 1xx (Informational) or 204 (No Content)."
>
>
> I was looking at the spec for 101 and 100 responses and think we are going
> way overboard on replying with a 100 response. Looking at the 101 example,
> we should send a reply of 0 or a few very explicit header fields and save
> the balance of output headers for the final response code. Otherwise these
> all seem to be wasted network bytes.
>


The 101 Upgrading response has a very short list of necessary
headers, only Connection: and Upgrade: fields are informative;
https://tools.ietf.org/html/rfc7230#section-6.7

I did not find anything in these sections on useful 100 Continue
response headers, and believe there are none;
https://tools.ietf.org/html/rfc7231#section-6.2.1
https://tools.ietf.org/html/rfc7231#section-5.1.1

Does anyone have pointers to legitimizing any 100 response
headers?

Re: Content-Length header for HTTP 204 and 1xx status codes

[Jacob Champion]

On 12/07/2016 08:31 AM, William A Rowe Jr wrote:

Does anyone have pointers to legitimizing any 100 response
headers?


Date is called out explicitly (RFC 7231, sec. 7.1.1.2):

   [...] An origin server MAY
   send a Date header field if the response is in the 1xx
   (Informational) or 5xx (Server Error) class of status codes.

It was even more explicit in 2616 (sec. 14.18):

1. If the response status code is 100 (Continue) or 101 (Switching
   Protocols), the response MAY include a Date header field, at
   the server's option.

What's your end goal? I don't think we can *prohibit* modules from 
sending extra headers in 100 responses, but it does make sense to limit 
what we send by default, especially if we're currently sending 
Content-Length (which IIUC is meaningless in that context).


*Are* we currently sending Content-Length in 100 responses? Luca's bug 
mentions 204 only.


--Jacob

Re: Content-Length header for HTTP 204 and 1xx status codes

[Stefan Eissing]

From a pure protocol point of view, all responses can have headers, I think. 
But there might be several implementations that do not cope well with them. But 
if we get some from an upstream server, I think we should forward them. 

> Am 07.12.2016 um 17:31 schrieb William A Rowe Jr :
> 
> On Wed, Dec 7, 2016 at 9:05 AM, William A Rowe Jr  wrote:
> On Nov 30, 2016 11:46 AM, "Luca Toscano"  wrote:
> Hi everybody,
> 
> while working on https://bz.apache.org/bugzilla/show_bug.cgi?id=51350 a user 
> asked why httpd send the "Content-Length: 0" header for HTTP 204 responses 
> given the following statement in the RFC:
> 
> https://tools.ietf.org/html/rfc7230#page-30
> "A server MUST NOT send a Content-Length header field in any response with a 
> status code of 1xx (Informational) or 204 (No Content)."
> 
> I was looking at the spec for 101 and 100 responses and think we are going 
> way overboard on replying with a 100 response. Looking at the 101 example, we 
> should send a reply of 0 or a few very explicit header fields and save the 
> balance of output headers for the final response code. Otherwise these all 
> seem to be wasted network bytes.
> 
> 
> The 101 Upgrading response has a very short list of necessary
> headers, only Connection: and Upgrade: fields are informative; 
> https://tools.ietf.org/html/rfc7230#section-6.7
> 
> I did not find anything in these sections on useful 100 Continue
> response headers, and believe there are none;
> https://tools.ietf.org/html/rfc7231#section-6.2.1
> https://tools.ietf.org/html/rfc7231#section-5.1.1
> 
> Does anyone have pointers to legitimizing any 100 response
> headers?
> 

Re: Content-Length header for HTTP 204 and 1xx status codes

[Luca Toscano]

Hi everybody,

thanks a lot for the useful feedback. Quoting Jacob from another thread:

2016-12-08 0:04 GMT+01:00 Jacob Champion :
>
>
> I thought the original bug report was related to 204 processing only, and
> then Luca asked if his patch should also include informational-status
> checks as well.


My understanding of the 1xx status handling was not correct, I created a
new patch only for the 204 bug:

http://home.apache.org/~elukey/httpd-trunk-core-http_204_nocontent.patch

This should avoid returning a C-L header (set by ap_content_length_filter
or a CGI backend) and any message-body for HTTP 204 responses.

If nobody disagrees, I'd like to commit it and create a specific test in
the httpd test suite.

Thanks!

Luca