Re: Associating a server push with the original client request
On Sun, 15 Jul 2018, at 9:58 PM, Oleg Kalnichevski wrote: > On Sun, 2018-07-15 at 21:41 +1000, Alex Osborne wrote: > > Here's a simplified example of what I want to do. I want to make two > > requests each having its own directory and save all the corresponding > > responses (both the main response and any push responses) to the > > appropriate directory. > > > > client.register("*", () -> new > > FileSavingDataConsumer("/tmp/request??/"))); > > client.execute(request1, new > > FileSavingDataConsumer("/tmp/request1/"), null); > > client.execute(request2, new > > FileSavingDataConsumer("/tmp/request2/"), null); > > > Given the example you can clearly use the original request request URI > as a correlation id. Sorry if I'm missing something obvious. I still don't see how to do it. :-( Say request1 is for GET /page1.html and for that page the server pushes GET /cat.jpg. Say request2 is for GET /page2.html and for that page the server pushes GET /dog.jpg. I'm sending the requests to an arbitrary server I have no control over and I don't know in advance which resources its going push so I can't client.register("/cat.jpg") specifcially. The Supplier register receives no arguments when it's called so it doesn't know the appropriate original request URI. The AsyncPushConsumer it returns receives: HttpRequest[GET /cat.jpg], HttpResponse[200 image/jpeg] HttpRequest[GET /dog.jpg], HttpResponse[200 image/jpeg] So there's still no association with the original request URIs /page1.html and /page2.html. > I am sure it is possible. Please do refer me to a section of the > specification that states _how_ _exactly_ those messages are associated > other than through the promised request that server pushes to the > client. Expacing the quote from section 8.2.1: Pushed responses are always associated with an explicit request from the client. The PUSH_PROMISE frames sent by the server are sent on that explicit request's stream. The PUSH_PROMISE frame also includes a promised stream identifier, chosen from the stream identifiers available to the server The PUSH_PROMISE frame is what associates them. It's sent on the stream the client sent the original request on and includes the id of a new stream the pushed response will be sent on. [stream 1] Client: GET /page1.html, END_STREAM (half-closes) [stream 3] Client: GET /page2.html, END_STREAM (half-closes) [stream 1] Server: PUSH_PROMISE /cat.jpg promising stream 2 [stream 3] Server: PUSH_PROMISE /dog.jpg promising stream 4 Then interleaved: [stream 1] Server: the response HEADERS, DATA for page1.html [stream 2] Server: the response HEADERS, DATA for cat.jpg [stream 3] Server: the response HEADERS, DATA for page2.html [stream 4] Server: the response HEADERS, DATA for dog.jpg > We happily take patches. Will be more than happy to commit a change-set > that makes correlation of push response messages easier. Thanks. I'll attempt to do so and send a patch if I get it working. :-) Alex - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Associating a server push with the original client request
On Sun, 2018-07-15 at 21:41 +1000, Alex Osborne wrote: > On Sun, 15 Jul 2018, at 8:53 PM, Oleg Kalnichevski wrote: > > On Sun, 2018-07-15 at 16:36 +1000, Alex Osborne wrote: > > > If I receive a server push is it possible to determine which > > > client > > > request it was sent in response to? > > > > Not unless the server is willing to help with that. > > > > Why do not you add a correlation id of a sort to the promised > > request > > from AsyncPushProducer? > > Here's a simplified example of what I want to do. I want to make two > requests each having its own directory and save all the corresponding > responses (both the main response and any push responses) to the > appropriate directory. > > client.register("*", () -> new > FileSavingDataConsumer("/tmp/request??/"))); > client.execute(request1, new > FileSavingDataConsumer("/tmp/request1/"), null); > client.execute(request2, new > FileSavingDataConsumer("/tmp/request2/"), null); > > While I could certainly put a counter in the AsyncPushProducer and > mint ids I have no guarantee it'd be called in any particular order > so I still wouldn't know which was which. As request1 and request2 > are executing concurrently I believe they may produce pushes in any > order, possibly even interleaved. > Given the example you can clearly use the original request request URI as a correlation id. > I'm pretty sure this is possible in the protocol as RFC 7540 says: > > Pushed responses are always associated with an explicit request > from > the client. The PUSH_PROMISE frames sent by the server are sent > on > that explicit request's stream. > I am sure it is possible. Please do refer me to a section of the specification that states _how_ _exactly_ those messages are associated other than through the promised request that server pushes to the client. > I don't think it's possible in the current implementation though as > AbstractHttp2StreamMultiplexer doesn't seem to do anything with the > explicit request's stream other than checking it hasn't been closed. > > So I guess I need to figure out how to modify > AbstractHttp2StreamMultiplexer to plumb through the information. > We happily take patches. Will be more than happy to commit a change-set that makes correlation of push response messages easier. Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Associating a server push with the original client request
On Sun, 15 Jul 2018, at 8:53 PM, Oleg Kalnichevski wrote: > On Sun, 2018-07-15 at 16:36 +1000, Alex Osborne wrote: > > If I receive a server push is it possible to determine which client > > request it was sent in response to? > > Not unless the server is willing to help with that. > > Why do not you add a correlation id of a sort to the promised request > from AsyncPushProducer? Here's a simplified example of what I want to do. I want to make two requests each having its own directory and save all the corresponding responses (both the main response and any push responses) to the appropriate directory. client.register("*", () -> new FileSavingDataConsumer("/tmp/request??/"))); client.execute(request1, new FileSavingDataConsumer("/tmp/request1/"), null); client.execute(request2, new FileSavingDataConsumer("/tmp/request2/"), null); While I could certainly put a counter in the AsyncPushProducer and mint ids I have no guarantee it'd be called in any particular order so I still wouldn't know which was which. As request1 and request2 are executing concurrently I believe they may produce pushes in any order, possibly even interleaved. I'm pretty sure this is possible in the protocol as RFC 7540 says: Pushed responses are always associated with an explicit request from the client. The PUSH_PROMISE frames sent by the server are sent on that explicit request's stream. I don't think it's possible in the current implementation though as AbstractHttp2StreamMultiplexer doesn't seem to do anything with the explicit request's stream other than checking it hasn't been closed. So I guess I need to figure out how to modify AbstractHttp2StreamMultiplexer to plumb through the information. Cheers, Alex - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Associating a server push with the original client request
On Sun, 2018-07-15 at 16:36 +1000, Alex Osborne wrote: > If I receive a server push is it possible to determine which client > request it was sent in response to? Hi Alex Not unless the server is willing to help with that. Why do not you add a correlation id of a sort to the promised request from AsyncPushProducer? Oleg > I'm not actually after the data of the request just some way to map > it back. Some opaque id, object, callback or whatever I can supply > with the request and later get back when a push comes in would do > just fine. > > I figured there might be a reference to original context in the push > request's context, but unfortunately AsyncPushConsumer doesn't > receive the HttpContext and even if I get at it by backtracking up > the stack in a debugger it seems to be a fresh context with no > reference to the original one. > > Does that mean the only way to associate a server push with the > original client request is to create a new HttpClient for each > request? (As then there'd be only be one request for each client.) > That wouldn't be so bad if they could share resources. I see a > promising looking setConnectionManagerShared() but I can't see a way > to get two HttpClients to use the same IOReactor and so would end up > with a lot more threads than I really want. > > Thanks, > > Alex > - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Associating a server push with the original client request
If I receive a server push is it possible to determine which client request it was sent in response to? I'm not actually after the data of the request just some way to map it back. Some opaque id, object, callback or whatever I can supply with the request and later get back when a push comes in would do just fine. I figured there might be a reference to original context in the push request's context, but unfortunately AsyncPushConsumer doesn't receive the HttpContext and even if I get at it by backtracking up the stack in a debugger it seems to be a fresh context with no reference to the original one. Does that mean the only way to associate a server push with the original client request is to create a new HttpClient for each request? (As then there'd be only be one request for each client.) That wouldn't be so bad if they could share resources. I see a promising looking setConnectionManagerShared() but I can't see a way to get two HttpClients to use the same IOReactor and so would end up with a lot more threads than I really want. Thanks, Alex - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org