Sangjin Lee wrote:
> What I've seen with AHC is that the configuration is often the most
> challenging aspect. One metaphor I used is that HttpClient is more like a
> browser. Things like keep-alive, user-agent, accept-encoding, etc. normally
> belong to the browser and not at the individual request level. I'm sure
> there are many delicate decisions to make that don't get solved easily by
> this metaphor, but I think it's certainly useful.
This is the metaphor I would like to follow is well which is why in my
API proposal I didn't provide a mechanism for sending a raw HttpRequest
object through the HttpClient.
After reviewing all the feedback and thinking about the problem more,
I'm thinking that if the user submits a MutableHttpRequest, then the
HttpClient will modify that request as appropriate. If the HttpRequest
does not implement MutableHttpRequest, then the request will be sent
unmodified. I think this should solve the problem adequately. WDT?
> I've mentioned this before, but one thing I like with AHC is handling
> multiple requests with a completion queue (not unlike CompletionService).
> This addresses a use case which is a variation of one of Mike's use cases.
>
> Suppose you want to send requests to N URLs concurrently. You want to limit
> the overall duration to a certain time, and you want to place a standard
> error result for the URL for which the response did not get in time. This
> is a pretty typical situation with a "scatter-and-gather" scenario (e.g.
> portal with multiple server-side content aggregation, etc.).
>
> With a completion queue, one can do things like the following:
>
> CompletionQueue q;
> client.send(request1, q);
> client.send(request2, q);
> ...
> client.send(requestN, q);
>
> for (int i = 0; i < N; i++) {
> Future f = q.take();
> Response r = f.get();
> }
>
> This can be done by the user in terms of a listener/callback, but it would
> certainly be nice to provide support for this... My 2 cents.
I too really like the idea of using a Queue for handling futures. It
opens up a lot of interesting possibilities.
The problem I see is for each method in HttpClient, do we provide an
overloaded version that accepts a Queue? This would make the API very
cluttered IMO.
We use an HttpListener to add the completed future to the queue. See
the following example.
BlockingQueue q;
queueListener = new QueueListener(q);
client.send(request1).addListener(queueListener);
client.send(request2).addListener(queueListener);
client.send(request3).addListener(queueListener);
client.send(request4).addListener(queueListener);
for (int i = 0; i < N; i++) {
Future f = q.take();
HttpResponse r = f.get();
}
The problem with this approach is that with the AsyncWeb client API, as
I've proposed it, there would be no way to know which request the future
object represents because we don't know the order in which each
HttpListener will be invoked.
I will try to refine this in the API that I'm proposing.
-Mike
> On Feb 12, 2008 10:53 AM, Mike Heath <[EMAIL PROTECTED]> wrote:
>
>> Rick McGuire wrote:
>>> Mike Heath wrote:
>>>> Rick McGuire wrote:
>>>>
>>>>> The one feature I like about the AHC client that appears to be missing
>>>>> here is the higher-level abstraction of an HTTP request. The one
>>>>> drawback of doing everything with URLs is the requirement that the
>> user
>>>>> of the client needs to be responsible for encoding all of the
>> parameter
>>>>> information in the URL. In the AHC approach, a request is configured
>> as
>>>>> an operation to a particular address and additional specifics of the
>>>>> operation are attached to the request (parameters, credentials needed
>>>>> for authentication, proxy configuration etc.). The AHC client then
>> uses
>>>>> that information to handle the URL encoding, authentication
>> challenges,
>>>>> proxy connection, etc. There are times where a straighforward "fetch
>> me
>>>>> this URL mode" is sufficient. There are other situations where that
>>>>> becomes awkward to use.
>>>>>
>>>> Thanks for the feedback Rick. That's one of the issues I've been going
>>>> back and forth on. In the HttpConnection interface, you have no choice
>>>> but to build and send an HttpRequest manually. So you have some
>>>> flexibility there. I've been debating whether or not to put support
>> for
>>>> sending a raw HttpRequest in the 'simplified' HttpClient. I've been
>>>> leaning toward add that functionality to the HttpClient your reply is
>>>> making me lean even further in that direction.
>>>>
>>>> -Mike
>>>>
>>> Yes, it is a bit of a balancing act. After a certain point, it becomes
>>> difficult to continue adding all of the possibilities to the method
>>> signatures. I think a good balance point is to support having a request
>>> object that's got all of the bells and whistles and a few methods that
>>> handle the most typical use scenarios (but in the end, are just creating
>>> the same type of request object on the caller's behalf).
>>> Rick
>> Yes I totally agree. A lot of the methods I put in HttpClient were
>> brainstorming ideas. I was trying to support the most common cases as
>> well as some of the more complex cases (like doing a PUT with a file,
>> for example.)
>>
>> We need to find a balance of simplicity and function and I'm not sure
>> where that point is at. Some of the issues that need to be addressed
>> with supporting sending raw HttpRequest objects using the HttpClient
>> (the way I've modeled it, that is) are:
>> - If the HttpClient has connection pooling enabled but a request is
>> sent without KeepAlive and Connection headers, do we automatically
>> change the request to turn keepAlive on and use connection pooling or do
>> we create a new connection?
>> - If the HttpClient has a cookie manager, do we automatically add
>> cookies to a manually created HttpRequest?
>> - Are there other headers that we would want to automatically populate
>> if their absent (User-Agent, Accept*, etc.) ?
>> - Do we even want to modify a manually created HttpRequest? Should
>> this be configurable? If so how?
>>
>> I don't have answers to these questions which is why I decided to just
>> put support for sending raw HttpRequest messages in the HttpConnection
>> interface because that's the interface I intended to make available for
>> complex use cases (COMET, dealing with large data streams, etc.)
>> However, I think we need to find some middle ground. I'm just not sure
>> where that middle ground is.
>>
>> Thanks again for your feedback, Rick.
>>
>> -Mike
>>
>>>
>>>>
>>>>
>>>>> Rick
>>>>>
>>>>> Mike Heath wrote:
>>>>>
>>>>>> I posted some use cases here:
>>>>>> http://cwiki.apache.org/confluence/display/AWEB/ClientUseCases They
>>>>>> still need some refinement to properly convey what I want but they're
>> a
>>>>>> decent start.
>>>>>>
>>>>>> I've also posted a hypothetical AsyncWeb Client API at
>>>>>> http://swamp.homelinux.net/mina/asyncweb/client/api/ with the intent
>> to
>>>>>> further promote discussion and foster more innovative ideas. I would
>>>>>> love to here some feedback on this API. What do you like, dislike,
>> not
>>>>>> understand? Where do you see room for improvement? The API is
>> really
>>>>>> rough in places but for the most part it conveys the ideas I've had
>>>>>> over
>>>>>> the past week or so. Any suggestions for name changes to classes
>>>>>> and/or
>>>>>> methods are welcome.
>>>>>>
>>>>>> -Mike
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>
>
