Liam Quinn <[EMAIL PROTECTED]> writes:
> WWW::RobotRules attempts to trim the robot's User-Agent before comparing
> it with the User-agent field of a robots.txt file:
>
> # Strip it so that it's just the short name.
> # I.e., "FooBot" => "FooBot"
> # "FooBot/1.2" => "FooBot"
> # "FooBot/1.2 [http://foobot.int; [EMAIL PROTECTED]" => "FooBot"
>
> delete $self->{'loc'}; # all old info is now stale
> $name = $1 if $name =~ m/(\S+)/; # get first word
> $name =~ s!/?\s*\d+.\d+\s*$!!; # loose version
>
> My robot's name is "WDG_SiteValidator/1.5.6". The above code changes the
> name to "WDG_SiteValidator/1.", which causes it not to match a robots.txt
> User-agent field of "WDG_SiteValidator".
>
> I've attached a patch against libwww-perl 5.76 (WWW::RobotRules 1.26) that
> replaces the last line above with
>
> $name =~ s!/.*!!; # lose version
>
> which seems to cover the various cases correctly.
Agree. Patch applied. Thanks!
Regards,
Gisle
> --- WWW/RobotRules.pm.orig 2003-10-23 15:11:33.000000000 -0400
> +++ WWW/RobotRules.pm 2004-04-03 18:06:01.000000000 -0500
> @@ -187,7 +187,7 @@
>
> delete $self->{'loc'}; # all old info is now stale
> $name = $1 if $name =~ m/(\S+)/; # get first word
> - $name =~ s!/?\s*\d+.\d+\s*$!!; # loose version
> + $name =~ s!/.*!!; # lose version
> $self->{'ua'}=$name;
> }
> $old;