[WWWOFFLE-Users] wget requesting with HTTP Authorization

[Paul Slootman]

I've discovered that when using wget to download an object that needs
authorization, wget first tries without authorization, and when a 401 is
returned, it retries with the authorization.

>From tcpdump:

request:
    GET /whatever/some/bla.jpg HTTP/1.1
    User-Agent: Wget/1.9.1
    Accept: */*
    Host: members.easynews.com
    Connection: close
    TE: chunked

answer:

    HTTP/1.1 401 Authorization Required
    Date: Mon, 05 Apr 2004 16:40:13 GMT
    Server: Microsoft-IIS/5.0
    WWW-Authenticate: Basic realm="EasyNews"
    Connection: close
    Transfer-Encoding: chunked
    Content-Type: text/html

    1a7
    <font face="arial" size=+1>
    <p align=center>
    Your username and/or password could not be verified.<br>
    Please try to <a href="http://www.easynews.com/login/";>re-login</a> for
    more information.
    </p>
    <p align=center>Login <a href="http://www.easynews.com/login/";>HERE</a>
    to get your account status.</p>
    <p align=center><a href="http://www.easynews.com/getpass.phtml";>Forgot your 
password?</a></p>
    </font>
    <!-- nicemaster -->

Wwwoffle logs this:

    wwwoffles[30790]: Server has used 'Transfer-Encoding: chunked'.
    wwwoffles[30790]: Cache Access Status='New Page'.

retry request:

    GET /whatever/some/bla.jpg HTTP/1.1
    User-Agent: Wget/1.9.1
    Accept: */*
    Host: members.easynews.com
    Connection: close
    Authorization: Basic xxxxxxxxxxxxxxxxxxx=
    TE: chunked

answer:

    HTTP/1.1 200 OK
    Date: Mon, 05 Apr 2004 16:40:13 GMT
    Server: Microsoft-IIS/5.0
    Last-Modified: Thu, 25 Mar 2004 16:02:02 GMT
    ETag: "1720af1-4501e-4063327a"
    Accept-Ranges: bytes
    Content-Length: 283422
    Connection: close
    Content-Type: image/jpeg

    [EMAIL PROTECTED]

At this point, wwwoffle says:

    wwwoffles[30790]: Server has used 'Transfer-Encoding: (null)'.
    wwwoffles[30790]: Cache Access Status='New Page'.
    wwwoffles[30790]: Error reading reply body from remote host [IO(chunked): not 
chunked encoding; not hex digit].

It looks like it's expecting the chunked TE from the first transfer to
also apply to the second, even though it isn't used then.
This means the transfer fails. Retrying works, because now the first
request to the server already contains the Authorization info.

This happens from within mozilla as well (with wget it was easier to
replay). The effect is that the first access fails, and reloading
succeeds.


Paul Slootman