I have no smoking gun to indicate this is a mechanize problem. just
instinct. I use mechanize daily as part of my job and its an old dear
friend =)
current problem: for a particular site, for a particular request, the
response i get with mechanize is extremely truncated in the body of the
response. the headers seems to come back fine, but instead of getting 20k
of response i get 2700 bytes or so. comparing it to a view source from a
browser, they are line for line identical - until the mechanize object
simply ends as if it gets an EOF too early, or some control character or
something.
does anyone have advice on debugging this? I am using a modified version
of Mechanize thats a bit old, so if there have been any known problems in
the past with truncation, well im probably not using their fixes.
any advice on how to debug this? heres a sample of what I mean. We start
to get some correct html, but then it just stops about 10 percent in. you
will see it get a few lines of javascript then suddenly end the response.
from putting the same url (non edited) into a web browser, the page is ten
times larger. the line right after the </script> tag (the last we get) is
a blank line.
# COMMENT: From [some site] via submit()
$response = bless( {
'_protocol' => 'HTTP/1.1',
'_content' => '<!-- Generated by WebSpeed - http://www.webspeed.com/ -->
<html>
<HTML>
<SCRIPT>
var mykey;
</SCRIPT>
<SCRIPT LANGUAGE="JavaScript">
function updateVendor(newVend)
{
var i;
for(i = 0; i <
parent.frames[0].document.user.selectedvendor.options.length; i++)
{
if (parent.frames[0].document.user.selectedvendor.options[i].value ==
newVend)
{
parent.frames[0].document.user.selectedvendor.options.selectedIndex =
i;
parent.frames[0].document.user.submit();
break;
}
}
}
</SCRIPT>
<script language="JavaScript" type="text/JavaScript">
<!--
var monthsF = new Array (31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31);
function doLeapYearF (inY) {
if ((inY % 400 == 0) || (inY % 4 == 0) && (!(inY % 100 == 0))) {
monthsF[1] = 29;
} else {
monthsF[1] = 28;
}
}
function setFDays (ipt, mn, inY) {
var sl = 0;
if (ipt.selectedIndex) {
sl = ipt.selectedIndex;
}
doLeapYearF (inY);
ipt.length = 0;
for (var i = 0; i < monthsF[mn-1]; i++) {
ipt[ipt.length] = new Option (i+1, i+1);
}
if (sl > (ipt.length-1)) {
sl = (ipt.length-1);
}
ipt.options[sl].selected = true;
}
function modFDays () {
with (document.details) {
setFDays (slFromDay, slFromMonth[slFromMonth.selectedIndex].value,
slFromYear[slFromYear.selectedIndex].valu
e);
setFDays (slToDay, slToMonth[slToMonth.selectedIndex].value,
slToYear[slToYear.selectedIndex].value);
}
}
//-->
</script>
',
'_rc' => '200',
'_headers' => bless( {
'connection' => 'close',
'client-response-num' => 1,
'date' => 'Mon, 22 Dec 2003 15:52:41 GMT',
'vary' => 'Accept-Encoding',
'client-peer' => '62.254.203.12:80',
'client-date' => 'Mon, 22 Dec 2003 15:51:52 GMT',
'content-type' => 'text/html',
'server' => 'Apache/1.3.27 (Unix) (Red-Hat/Linux) mod_ssl/2.8.12
OpenSSL/0.9.6b mod_perl/1.26',
'client-transfer-encoding' => 'chunked'
}, 'HTTP::Headers' ),
'_msg' => 'OK',
'_request' => bless( {
'_content' => 'area=7',
'_uri' => bless( do{\(my $o =
'http://www.somesite.com/cgi-bin/liveweb.sh/MainFrame.w?UckUcZllaaIpkkac=&POlfi
gXkjkjkWbdq=&SblzdbFStskKccak=&RkkFlPbjklpsIaUb=FRA&aff=&ctryref=FRA%E2%8C%A9=FR&currcode=&ClerkID=')},
'URI::http'
),
'_headers' => bless( {
'user-agent' => 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
5.0)',
'content-type' => 'application/x-www-form-urlencoded',
'content-length' => 6,
'referer' =>
'http://www.somesite.com/cgi-bin/liveweb.sh/MainFrame.w?ctryref=FRA&lang=FR'
}, 'HTTP::Headers' ),
'_method' => 'POST'
}, 'HTTP::Request' )
}, 'HTTP::Response' );
--
gedanken
