Re: http/2 PUT's without content-length fail to http 1.1 backend

[Willy Tarreau]

Hi Robert,

On Sat, Mar 10, 2018 at 06:16:21PM +, Robert Samuel Newson wrote:
> Hi,
> 
> Whenever you look into this, I noticed that a streamed upload also fails, but 
> differently;
> 
> cat  | acurl https://rnewson.cloudant.com/db1/doc1/att1 -XPUT -T-
> 
> curl chooses http/2 and sends the Transfer-Encoding: chunked header, and then 
> streams the file.
> 
> The result is that the body is not received, the resulting upload is zero 
> bytes.
> 
> I mention this for completeness but it would seem curl is at fault here, it
> should not send the Transfer-Encoding header, but I'm not sure haproxy is
> entirely innocent either?

The header should indeed not be sent, but regardless, haproxy is culprit here
for not being able to process DATA frames without a content-length. I have
already started to look into this and am trying to figure how to best address
the various issues currently pending in the same area without having to
reinvent everything each time ;-)

Thanks for the heads up, it will definitely help me.

Willy

Re: http/2 PUT's without content-length fail to http 1.1 backend

[Robert Samuel Newson]

Hi,

Whenever you look into this, I noticed that a streamed upload also fails, but 
differently;

cat  | acurl https://rnewson.cloudant.com/db1/doc1/att1 -XPUT -T-

curl chooses http/2 and sends the Transfer-Encoding: chunked header, and then 
streams the file.

The result is that the body is not received, the resulting upload is zero bytes.

I mention this for completeness but it would seem curl is at fault here, it 
should not send the Transfer-Encoding header, but I'm not sure haproxy is 
entirely innocent either?

B.

* acurl is simple a curl alias that adds my user:pass credentials.

> On 1 Mar 2018, at 11:06, Willy Tarreau  wrote:
> 
> On Thu, Mar 01, 2018 at 09:38:10AM +, Robert Samuel Newson wrote:
>> Yup, agreed, the frame-only transfers are only really implied in the spec
>> (8.1.2.6's, "A request or response that includes a payload body _can_ include
>> a content-length header field", my emphasis). The http 2 spec does
>> specifically prohibit the transfer-encoding: chunked header, again implying
>> that the framing mechanism subsumes it.
> 
> In fact it's even worse for me : I identified I had to do it, as indicated
> by the comments on top of function h2_frt_transfer_data() which claim that
> when no C-L nor tunnel is found, chunks are emitted. But apparently relying
> too much on the checks I had on the paper-written RFC allowed me to completely
> forget about this part! I've checked if I had a temporary implementation in
> one of my previous dev branches but no, so I think I never wrote that code.
> The good point is that the code was written with this in mind, so I hope I
> won't face too big a surprise (and none of the usual "ah now I remember why").
> 
>> There are a few libraries (like https://github.com/go-kivik/couchdb) which
>> implement PUT without sending Content-Length, but other tools, like good old
>> curl, which do send it.
> 
> Good to know, thanks for the links. Indeed, curl always used C-L.
> 
>> The basic plan to transform to a chunked transfer encoding sounds right to
>> me, and I'll keep an eye out for commits, I'm happy to test it out.
> 
> Thanks!
> Willy

Re: http/2 PUT's without content-length fail to http 1.1 backend

[Willy Tarreau]

On Thu, Mar 01, 2018 at 09:38:10AM +, Robert Samuel Newson wrote:
> Yup, agreed, the frame-only transfers are only really implied in the spec
> (8.1.2.6's, "A request or response that includes a payload body _can_ include
> a content-length header field", my emphasis). The http 2 spec does
> specifically prohibit the transfer-encoding: chunked header, again implying
> that the framing mechanism subsumes it.

In fact it's even worse for me : I identified I had to do it, as indicated
by the comments on top of function h2_frt_transfer_data() which claim that
when no C-L nor tunnel is found, chunks are emitted. But apparently relying
too much on the checks I had on the paper-written RFC allowed me to completely
forget about this part! I've checked if I had a temporary implementation in
one of my previous dev branches but no, so I think I never wrote that code.
The good point is that the code was written with this in mind, so I hope I
won't face too big a surprise (and none of the usual "ah now I remember why").

> There are a few libraries (like https://github.com/go-kivik/couchdb) which
> implement PUT without sending Content-Length, but other tools, like good old
> curl, which do send it.

Good to know, thanks for the links. Indeed, curl always used C-L.

> The basic plan to transform to a chunked transfer encoding sounds right to
> me, and I'll keep an eye out for commits, I'm happy to test it out.

Thanks!
Willy

Re: http/2 PUT's without content-length fail to http 1.1 backend

[Robert Samuel Newson]

Hi,

Yup, agreed, the frame-only transfers are only really implied in the spec 
(8.1.2.6's, "A request or response that includes a payload body _can_ include a 
content-length header field", my emphasis). The http 2 spec does specifically 
prohibit the transfer-encoding: chunked header, again implying that the framing 
mechanism subsumes it.
 
There are a few libraries (like https://github.com/go-kivik/couchdb) which 
implement PUT without sending Content-Length, but other tools, like good old 
curl, which do send it.

The basic plan to transform to a chunked transfer encoding sounds right to me, 
and I'll keep an eye out for commits, I'm happy to test it out.

Thanks!

B.

> On 1 Mar 2018, at 06:16, Willy Tarreau  wrote:
> 
> Hi Robert,
> 
> On Tue, Feb 27, 2018 at 08:26:01PM +, Robert Samuel Newson wrote:
>> Hi,
>> 
>> I use haproxy (1.8.4) with http/2 support in front of a server that speaks
>> http 1.1. This is working great with one exception. Several http/2 client
>> libraries are sending PUT requests without sending the Content-Length header
>> (as it' not strictly needed due to the framing).
> 
> At first I thought this was not valid but in fact it obviously is since
> it is the equivalent of chunked encoding for HTTP/1. Interestingly, just
> like chunked uploads are not much documented in the HTTP/1 specs, this
> frame-only transfer is never mentionned in the HTTP/2 spec and I totally
> overlooked it.
> 
>> The http 1.1 request issued
>> to the backend is therefore malformed as the header is required there.
>> 
>> I think the right thing here is for haproxy to convert the PUT to a chunked
>> transfer request to the backend.
> 
> Yes I think so as well. It will be a bit tricky though. From what I'm
> imagining, we'll have to add "Transfer-encoding: chunked" if the HEADERS
> frame doesn't carry the END_STREAM flag and there is no Content-length
> header, then all DATA frames will have to be prefixed with a chunk size
> and the frame carrying the ES flag will have to cause the final empty
> chunk to be sent. This one will be a bit delicate I think, as we don't
> want to lose it if the output buffer is full and we want not to miss the
> event.
> 
> Also we must not do this for the CONNECT method!
> 
> I'm adding this to the todo list of fixes for H2. I don't promise to
> have it by 1.8.5 though, but I'll try.
> 
> Thanks!
> Willy

http/2 PUT's without content-length fail to http 1.1 backend

[Robert Samuel Newson]

Hi,

I use haproxy (1.8.4) with http/2 support in front of a server that speaks http 
1.1. This is working great with one exception. Several http/2 client libraries 
are sending PUT requests without sending the Content-Length header (as it' not 
strictly needed due to the framing). The http 1.1 request issued to the backend 
is therefore malformed as the header is required there.

I think the right thing here is for haproxy to convert the PUT to a chunked 
transfer request to the backend.

Thoughts?

B.