bug in LWP::Simple::head (may return less that 5 values)

[David Dyck]

LWP::Simple::head is documented as returning 5 values:

       head($url)
          Get document headers. Returns the following 5 values if
          successful: ($content_type, $document_length, $modi<AD<AD<AD<AD>
          fied_time, $expires, $server)

          Returns an empty list if it fails.  In scalar context
          returns TRUE if successful.

On an internal url it seemed to only return 4 values:
Here's what I get back if I request the header via telnet
HTTP/1.0 200 OK
Server: Netscape-Communications/1.1
Date: Saturday, 11-Mar-00 22:12:02 GMT
Last-modified: Saturday, 11-Mar-00 01:08:44 GMT
Content-length: 4175
Content-type: text/html


For the sake of discussion, here is the code for LWP::Simple::head 

     1  sub head ($)
     2  {
     3      my($url) = @_;
     4      _init_ua() unless $ua;
     5  
     6      my $request = HTTP::Request->new(HEAD => $url);
     7      my $response = $ua->request($request);
     8  
     9      if ($response->is_success) {
    10          return $response unless wantarray;
    11          return (scalar $response->header('Content-Type'),
    12                  scalar $response->header('Content-Length'),
    13                  HTTP::Date::str2time($response->header('Last-Modified')),
    14                  HTTP::Date::str2time($response->header('Expires')),
    15                  scalar $response->header('Server'),
    16                 );
    17      }
    18      return;
    19  }

When run under the debugger I looked at $response at line 11

  DB<2> x $response
0  HTTP::Response=HASH(0x85e7230)
   '_content' => ''
   '_headers' => HTTP::Headers=HASH(0x85e7200)
      'client-date' => 'Sat, 11 Mar 2000 21:01:00 GMT'
      'client-peer' => '129.196.128.19:80'
      'content-length' => 4175
      'content-type' => 'text/html'
      'date' => 'Saturday, 11-Mar-00 21:00:45 GMT'
      'last-modified' => 'Saturday, 11-Mar-00 01:08:44 GMT'
      'server' => 'Netscape-Communications/1.1'
   '_msg' => 'OK'
   '_protocol' => 'HTTP/1.0'
   '_rc' => 200
   '_request' => HTTP::Request=HASH(0x842f664)
      '_content' => ''
      '_headers' => HTTP::Headers=HASH(0x815edb0)
         'user-agent' => 'LWP::Simple/5.47'
      '_method' => 'HEAD'
      '_uri' => URI::URL=ARRAY(0x84279d4)
         0  URI::http=SCALAR(0x838bfa8)
            -> 'http://fww/'
         1  undef

Notice no Expires header in %_headers, so on line 14
the void return from HTTP::Date::str2time
yields only 4 entries in the list.


Here's a one liner that shows the void v() sub not filling the
slot in the list.
perl -lwe 'sub v() { } sub l() { return ( "a", v, "c" ); } print join ":",l'
a:c

if v() returned undef then it would print a warning and you would detect
the empty entry in the list
a::c


Changing line 14 to read
  HTTP::Date::str2time($response->header('Expires')) || undef,
or
  scalar( HTTP::Date::str2time($response->header('Expires')) ),

reserves the slot, so that the server header in returned
in the 5'th position.

Which HTTP headers are optional, and may also need the treatment
to preserve their slot in the returned list?

David Dyck