At 11:26 AM 03/03/02 +0800, Stas Bekman wrote:
>> # optionally validate external links
>> validate_links => $ENV{VALIDATE_LINKS} || 0,
>>
>> So if you set that env variable the spider will try and check all links,
>> even external links. It does a HEAD request so it won't catch all, but it
>> might help a little.
>
>yup, I saw this. Though if I remember correctly it didn't work. At least
>it wasn't reporting anything, while I know there were many broken
>external links.
It takes a while (on my machine instead of indexing the site in less then a
minute it takes about 18 minutes to check all the links.
Some don't make much sense, check out the www.modperl.com error?? -- I'll look
into it next week.
Again, it's just doing simple HEAD requests with LWP. Clearly a lot of those
work in browsers, but
they are returning errors here.
./swish-e -S prog -c swish.conf
Indexing Data Source: "External-Program"
Indexing "./spider.pl"
./spider.pl: Reading parameters from 'SwishSpiderConfig.pl'
Summary for: http://mardy:40994/dst_html/index.html
Duplicates: 4,977 (5.0/sec)
MD5 Duplicates: 1 (0.0/sec)
Off-site links: 1,833 (1.8/sec)
Skipped: 347 (0.3/sec)
Total Bytes: 3,884,170 (3864.8/sec)
Total Docs: 1,279 (1.3/sec)
Unique URLs: 252 (0.3/sec)
Bad Links:
On page: http://mardy:40994/dst_html/about/about.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/about/help.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://validator.w3.org/check/referer 301 Moved Permanently
http://validator.w3.org/
http://jigsaw.w3.org/css-validator/ 300 Multiple Choices
On page: http://mardy:40994/dst_html/contribute/docs.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/contribute/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/contribute/maillist.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/contribute/share_code.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/api/Apache.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.apache.org/docs/. 301 Moved Permanently
http://httpd.apache.org/docs/
http://www.apache.org/docs/ 301 Moved Permanently
http://httpd.apache.org/docs/
On page: http://mardy:40994/dst_html/docs/1.0/api/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.apache.org/docs/. 301 Moved Permanently
http://httpd.apache.org/docs/
On page: http://mardy:40994/dst_html/docs/1.0/faqs/cgi_to_mod_perl.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/faqs/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl_api.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl_cgi.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl_cvs.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://dev.apache.org/anoncvs.txt 302 Found
http://httpd.apache.org/dev/
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl_faq.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.perl.com/CPAN/modules/by-module/Apache/ 302 Found
http://www.perl.com/CPAN/modules/by-module/Apache/
http://www.perl.com/CPAN/src/latest.tar.gz 302 Found
http://www.perl.com/CPAN/src/latest.tar.gz
http://www.example.com/perl-status 404 Not Found
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl_method_handlers.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.hip.dk/date_bar 404 Not Found
http://www.perl.com/CPAN/doc/FMTEYEWTK/perltoot.html 302 Found
http://www.perl.com/CPAN/doc/FMTEYEWTK/perltoot.html
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl_traps.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/faqs/mod_perl_tuning.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.example.com/programs 404 Not Found
http://www.example.com/programs/printenv 404 Not Found
http://www.example.com/programs/printenv 404 Not Found
http://localhost:8042/programs/printenv 500 Can't connect to localhost:8042
(Timeout)
http://squid.nlanr.net/Squid/ 302 Found
http://www.squid-cache.org/Squid/
On page: http://mardy:40994/dst_html/docs/1.0/faqs/perl_myth.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://valueclick.com/ 302 Found http://www.valueclick.com/
http://www.cnation.com/ 501 Method Not Implemented
On page: http://mardy:40994/dst_html/docs/1.0/guide/advocacy.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/browserbugs.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://example.com/foo.pl?foo=bar%AE=foobar 403 Forbidden
http://example.com/foo.pl?foo=bar 403 Forbidden
On page: http://mardy:40994/dst_html/docs/1.0/guide/config.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.example.com/foo/test.pl 404 Not Found
http://www.example.com/perl/test.pl 404 Not Found
http://www.example.com/cgi-bin/test.pl 404 Not Found
http://www.example.com/cgi-perl/test.pl 404 Not Found
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/control.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.apache.org/docs/stopping.html 301 Moved Permanently
http://httpd.apache.org/docs/stopping.html
http://www.example.com/perl/test.pl 404 Not Found
http://www.example.com:8000/perl/test.pl 500 timed out
http://www.example.com:8001/perl/test.pl 500 timed out
ftp://koobera.math.uic.edu/www/daemontools.html 500 timed out
On page: http://mardy:40994/dst_html/docs/1.0/guide/correct_headers.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/databases.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/dbm.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/debug.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.myserver.com:81/perl-status 500 Can't connect to
www.myserver.com:81 (Bad hostname 'www.myserver.com')
http://www.perl.com/CPAN/authors/id/A/AE/AEPAGE/ 302 Found
http://www.perl.com/CPAN/authors/id/A/AE/AEPAGE/
On page: http://mardy:40994/dst_html/docs/1.0/guide/download.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.perl.com/ 302 Found http://www.perl.com/
http://www.perl.com/pace/pub/perldocs/latest.html 302 Found
http://www.perl.com/pace/pub/perldocs/latest.html
http://perl.apache.org/snapshots/modperl/ 404 Not Found
http://perl.apache.org/snapshots/. 404 Not Found
http://squid.nlanr.net/ 302 Found http://www.squid-cache.org/
http://modules.apache.org/search?id=124 500 EOF instead of reponse status
line
http://www.ask-the-guru.com/ 302 Found
http://www.httpservice.de/ask-the-guru.html
http://www.henge.com/~alanr/ha/ 500 timed out
http://www.csn.tu-chemnitz.de/~mha/linux-ip-nat/diplom/node4.html 301 Moved
Permanently https://www.csn.tu-chemnitz.de/~mha/linux-ip-nat/diplom/node4.html
http://coombs.anu.edu.au/~avalon/ipf3.4beta3.tgz 404 Not Found
http://home-of-linux.org/gnome/libgtop/ 404 Not Found
On page: http://mardy:40994/dst_html/docs/1.0/guide/frequent.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/hardware.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://deja.com/ 302 Found http://groups.google.com/
http://egroups.com/ 302 Found http://groups.yahoo.com/
On page: http://mardy:40994/dst_html/docs/1.0/guide/help.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://perl.apache.org/guide 301 Moved Permanently
http://perl.apache.org/guide/
http://www.davin.ottawa.on.ca/archive/modperl/ 301 Moved Permanently
http://archive.davin.ottawa.on.ca/modperl/
http://www.egroups.com/group/modperl/ 302 Found
http://groups.yahoo.com/group/modperl/
http://www.perl.com/CPAN/doc/FAQs/FAQ/PerlFAQ.html 302 Found
http://www.perl.com/CPAN/doc/FAQs/FAQ/PerlFAQ.html
http://www.perl.com/ 302 Found http://www.perl.com/
http://www.perl.com/CPAN/doc/FAQs/cgi/perl-cgi-faq.html 302 Found
http://www.perl.com/CPAN/doc/FAQs/cgi/perl-cgi-faq.html
http://www.perl.com/CPAN/doc/FAQs/cgi/idiots-guide.html 302 Found
http://www.perl.com/CPAN/doc/FAQs/cgi/idiots-guide.html
http://www.apache.org/docs/misc/FAQ.html 301 Moved Permanently
http://httpd.apache.org/docs/misc/FAQ.html
http://www.apache.org/docs/ 301 Moved Permanently
http://httpd.apache.org/docs/
http://www.apache.org/docs/handler.html 301 Moved Permanently
http://httpd.apache.org/docs/handler.html
http://www.securityfocus.com/infocus/1356 301 Moved Permanently
http://online.securityfocus.com/infocus/1356
http://www.symbolstone.org/technology/perl/DBI/ 302 Found
http://dbi.symbolstone.org/
http://www.fugue.com/dbi/ 302 Found
http://www.isc.org/services/public/lists/dbi-lists.html
http://squid.nlanr.net/ 302 Found http://www.squid-cache.org/
http://squid.nlanr.net/Squid/FAQ/FAQ.html 302 Found
http://www.squid-cache.org/Squid/FAQ/FAQ.html
http://squid.nlanr.net/Squid/Users-Guide/ 302 Found
http://www.squid-cache.org/Squid/Users-Guide/
http://squid.nlanr.net/Squid/mailing-lists.html 302 Found
http://www.squid-cache.org/Squid/mailing-lists.html
http://www.sourcegear.com/CVS/Docs/online 301 Moved Permanently
http://www.cvshome.org/Docs/online
http://www.sourcegear.com/CVS/Docs/ref 301 Moved Permanently
http://www.cvshome.org/Docs/ref
http://www.sourcegear.com/CVS/Docs/books 301 Moved Permanently
http://www.cvshome.org/Docs/books
http://www.sourcegear.com/CVS/Docs/docfaq 301 Moved Permanently
http://www.cvshome.org/Docs/docfaq
http://www.sourcegear.com/CVS/Docs/blandy 301 Moved Permanently
http://www.cvshome.org/Docs/blandy
On page: http://mardy:40994/dst_html/docs/1.0/guide/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/install.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://perl.apache.org/dist/mod_perl-x.xx.tar.gz 404 Not Found
http://perl.apache.org/dist 301 Moved Permanently
http://perl.apache.org/dist/
http://perl.apache.org/download/distributions.html 404 Not Found
http://www.davideous.com/modperlrpm/distrib/libapreq-0.31_include.patch 302
Found http://perl.apache.org/rpm/distrib/libapreq-0.31_include.patch
http://hex.tamu.edu/build-apache 404 Not Found
http://hex.tamu.edu/generic.conf 404 Not Found
http://www.example.com/perl-status 404 Not Found
http://www.example.com:8080/perl/test.pl 500 timed out
http://www.example.com:8080/ 500 timed out
On page: http://mardy:40994/dst_html/docs/1.0/guide/intro.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://valueclick.com/ 302 Found http://www.valueclick.com/
http://www.deja.com/ 302 Found http://groups.google.com/
http://www.flash.net/ 302 Found
http://home.flash.net/index.html
http://www.mediametrix.com/ 302 Found
http://www.mediametrix.com/landing.html
http://www.commissioner.com/ 302 Found
http://members2.fantasy.sportsline.com/
http://www.slashdot.org/ 301 Moved Permanently
http://slashdot.org/
http://members.hotbot.com/ 302 Found
http://members.hotbot.lycos.com/
On page: http://mardy:40994/dst_html/docs/1.0/guide/modules.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.home-of-linux.org/gnome/libgtop/ 404 Not Found
On page: http://mardy:40994/dst_html/docs/1.0/guide/multiuser.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.s390.ibm.com/linux/vif/ 301 Moved Permanently
http://www.ibm.com/link/redirect.www.ibm.com/www.s390.ibm.com/linux/vif/
On page: http://mardy:40994/dst_html/docs/1.0/guide/performance.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.realtime.net/~parkerm/perl/conf98/sld006.htm 404 File Not Found
http://www.home-of-linux.org/gnome/libgtop/ 404 Not Found
http://www.example.com/perl/access/access.cgi 404 Not Found
http://www.example.com/perl/access/access.cgi?do_sub=query_form 404 Not Found
http://www.example.com/perl/access/access.cgi 404 Not Found
http://www.modperl.com/chapters/ch6.html 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/perl.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.plover.com/~mjd/perl/FAQs/Namespaces.html 301 Moved Permanently
http://perl.plover.com/FAQs/Namespaces.html
http://www.avrasoft.com/perl/rfc/try-1136.zip 404 Not Found
On page: http://mardy:40994/dst_html/docs/1.0/guide/porting.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.example.com/perl-status?inc 404 Not Found
On page: http://mardy:40994/dst_html/docs/1.0/guide/scenario.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.example.com/modperl/foo.pl 404 Not Found
http://backend.example.com/modperl/foo.pl 500 Can't connect to
backend.example.com:80 (Bad hostname 'backend.example.com')
http://backend.example.com/ 500 Can't connect to
backend.example.com:80 (Bad hostname 'backend.example.com')
http://www.example.com/modperl/foo.pl 404 Not Found
http://backend.example.com/modperl/foo.pl 500 Can't connect to
backend.example.com:80 (Bad hostname 'backend.example.com')
http://www.example.com/modperl/foo.pl 404 Not Found
On page: http://mardy:40994/dst_html/docs/1.0/guide/security.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/snippets.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www2.example.com/news/ 500 Can't connect to
www2.example.com:80 (Bad hostname 'www2.example.com')
On page: http://mardy:40994/dst_html/docs/1.0/guide/start.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/guide/strategy.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.example.com:8080/perl/index.pl 500 timed out
http://www.example.com/icons/arrow.gif 404 Not Found
http://www.fenrus.demon.nl/. 302 Found
http://www.fenrus.demon.nl/./
http://www.example.com/images/test.gif 404 Not Found
http://www.example.com:8080/perl/test.pl 500 timed out
http://example.com/modperl/ 404 Not Found
http://localhost:81/modperl/ 500 Can't connect to localhost:81
(Timeout)
http://www.iagora.com/about/software/lingerd/ 500 timed out
On page: http://mardy:40994/dst_html/docs/1.0/guide/troubleshooting.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/win32/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/win32/win32_binaries.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/win32/win32_compile.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/1.0/win32/win32_multithread.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/api/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/api/mod_perl-2.0/APR/PerlIO.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/api/mod_perl-2.0/APR/Table.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page:
http://mardy:40994/dst_html/docs/2.0/api/mod_perl-2.0/Apache/RequestRec.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page:
http://mardy:40994/dst_html/docs/2.0/api/mod_perl-2.0/Apache/ServerUtil.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page:
http://mardy:40994/dst_html/docs/2.0/api/mod_perl-2.0/Apache/compat.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page:
http://mardy:40994/dst_html/docs/2.0/devel/core_explained/core_explained.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/devel/debug_c/debug_c.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/devel/debug_perl/debug_perl.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/devel/help/help.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://sunsite.ualberta.ca/Documentation/Misc/perl-5.6.1/Porting/patching.html.
404 Not Found
news://news.perl.com/perl.porters-gw/ 500 Can't connect to nntp server
On page: http://mardy:40994/dst_html/docs/2.0/devel/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page:
http://mardy:40994/dst_html/docs/2.0/devel/modperl_style/modperl_style.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://dev.apache.org/styleguide.html 302 Found
http://httpd.apache.org/dev/
On page: http://mardy:40994/dst_html/docs/2.0/devel/perf_sizeof/perf_sizeof.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page:
http://mardy:40994/dst_html/docs/2.0/devel/porting_from_1.x/porting_from_1.x.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/devel/testing/testing.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/coding/coding.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/compat/compat.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/config/config.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/design/design.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/help/help.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/install/install.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/intro/start_fast.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/user/overview/overview.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/world/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/2.0/world/templates/choosing.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/docs/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/download/binaries.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://perl.apache.org/dist 301 Moved Permanently
http://perl.apache.org/dist/
http://theoryx5.uwinnipeg.ca/cgi-bin/ppmserver?urn:/PPMServer 405 Method Not
Allowed
http://www.davideous.com/modperlrpm/distrib/ 302 Found
http://perl.apache.org/rpm/distrib/
On page: http://mardy:40994/dst_html/download/bundles.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.perl.com/CPAN/modules/by-module/CPAN/ 302 Found
http://www.perl.com/CPAN/modules/by-module/CPAN/
On page: http://mardy:40994/dst_html/download/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/download/stable.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.cpan.org/modules/by-module/Apache 301 Moved Permanently
http://cpan.valueclick.com/modules/by-module/Apache/
http://mardy:40994/dst_html/download/dist/ 404 Not Found
On page: http://mardy:40994/dst_html/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.slashdot.org/ 301 Moved Permanently
http://slashdot.org/
On page: http://mardy:40994/dst_html/maillist/contact.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/email-etiquette.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://stason.org/guide-snapshots 301 Moved Permanently
http://www.stason.org/guide-snapshots/
On page: http://mardy:40994/dst_html/maillist/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-advocacy.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-announce.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-asp.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-cvs.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-dev.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-docs-cvs.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-docs-dev.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-embperl.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://perl.apache.org/mail/embperl 301 Moved Permanently
http://perl.apache.org/mail/embperl/
On page: http://mardy:40994/dst_html/maillist/list-test-cvs.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-test-dev.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/maillist/list-users.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://groups.yahoo.com/group/modperl/ 302 Found
http://groups.yahoo.com/group/modperl/
http://www.davin.ottawa.on.ca/archive/modperl/ 301 Moved Permanently
http://archive.davin.ottawa.on.ca/modperl/
http://www.egroups.com/group/modperl/ 302 Found
http://groups.yahoo.com/group/modperl/
On page: http://mardy:40994/dst_html/products/apache-modlist.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.perl.com/CPAN/modules/00modlist.long.html 302 Found
http://www.perl.com/CPAN/modules/00modlist.long.html
http://www.perl.com/CPAN/modules/00modlist.long.html 302 Found
http://www.perl.com/CPAN/modules/00modlist.long.html
http://www.perl.com/CPAN/modules/by-module/Apache/ 302 Found
http://www.perl.com/CPAN/modules/by-module/Apache/
http://www.perl.com/CPAN/modules/by-module/Apache/ 302 Found
http://www.perl.com/CPAN/modules/by-module/Apache/
On page: http://mardy:40994/dst_html/products/app-server.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/products/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/products/products.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://interact.canoe.ca/~bsugars/nsapi_perl.html 500 timed out
http://www.activestate.com/software/perlEx/default.htm 403 Access Forbidden
http://www.olab.com/doc/books/perl/perltoc.htm 500 timed out
http://www.binevolve.com/bine/vep/ 500 timed out
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
http://www.perl.org/CPAN/modules/by-module/Apache/ 302 Found
http://www.cpan.org/modules/by-module/Apache/
https://www.sgi.com/toolbox/public/apache/ 501 Protocol scheme 'https' is not
supported
http://modules.apache.org/ 500 EOF instead of reponse status
line
http://www.c2.net/ 302 Found
http://www.redhat.com/software/apache/index.html
On page: http://mardy:40994/dst_html/search/swish.cgi
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://mardy:40994/dst_html/search/searchresults.html.gz 403 Forbidden
On page: http://mardy:40994/dst_html/stats/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stats/netcraft.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.netcraft.co.uk/up/graph/ 302 Found http://uptime.netcraft.com/
On page: http://mardy:40994/dst_html/stats/securityspace.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.securityspace.com/ 302 Found
http://www.securityspace.com/sspace/index.html
On page: http://mardy:40994/dst_html/stories/adultad.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/allakhazam.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/bsat.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/calmaeth.maths.uwa.edu.au.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/colbychem.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/iagore.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/idl-net.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/imdb.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/openscape.org.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/presto.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/rent.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/seds.org.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/singlesheaven.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/sites.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.perl.com/CPAN/modules/04pause.html 302 Found
http://www.perl.com/CPAN/modules/04pause.html
http://www.kulturbox.de/perl/berlininfo 404 Object Not Found
http://www.perl.com/CPAN/modules/by-module/GD/ 302 Found
http://www.perl.com/CPAN/modules/by-module/GD/
http://www.webpersonals.com/ 302 Moved Temporarily
http://www.lavalife.com/
http://www.etoys.com/ 302 Moved Temporarily
http://www.etoys.com/index.html
http://www.enews.com/ 302 Moved Temporarily
http://www.enews.com/personalization/process/?returnUri=/
http://www.sol.no/ 500 timed out
http://kvasir.sol.no/ 500 timed out
http://www.assoziations-blaster.de/statistik 301 Moved Permanently
http://www.assoziations-blaster.de/statistik/
http://perlmonth.com/ 500 timed out
http://singlesheaven.com/ 500 Can't connect to
singlesheaven.com:80 (Bad hostname 'singlesheaven.com')
http://www.filepile.com/ 500 Can't connect to
www.filepile.com:80 (Bad hostname 'www.filepile.com')
http://www.warlords3.com/guild/maps 301 Moved Permanently
http://www.warlords3.com/guild/maps/
http://www.gobig12.com/ 302 Found
http://www.austin360.com/sports/college/hookem/
http://theory.uwinnipeg.ca/search/cpan-search.html 302 Found
http://theoryx5.uwinnipeg.ca/CPAN/cpan-search.html
http://online-books.oreilly.com/books/ 404 Not Found
http://home.wired.com/ 302 Found
http://hotwired.lycos.com/home/
http://members.hotbot.com/ 302 Found
http://members.hotbot.lycos.com/
http://www.hotwired.com/ 302 Found http://hotwired.lycos.com/
http://www.webmonkey.com/ 301 Moved Permanently
http://hotwired.lycos.com/webmonkey/
http://www.medimatch.com/ 302 Object moved http://suzy.3x.ca/
http://www.filemine.com/ 302 Found http://itprodownloads.com/
http://mardy:40994/dst_html/stories/netcraft/ 404 Not Found
On page: http://mardy:40994/dst_html/stories/tamu.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/tgix.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/winamillion.msn.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/wmboerse.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/www.afp-direct.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/www.bivio.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://www.bivio.com/ 302 Found
http://www.bivio.com/index.html
On page: http://mardy:40994/dst_html/stories/www.lind-waldock.com.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/stories/www.mobile.de.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/support/commercial.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/support/index.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://mardy:40994/dst_html/conferences/index.html 404 Not Found
On page: http://mardy:40994/dst_html/support/isps.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://hosting.devcon.net/ 500 Can't connect to
hosting.devcon.net:80 (Timeout)
http://www.bsb-software.com/ 302 Found
http://www.prodyna.de/unternehmen/news/namensaenderung/
http://www.oneway.net/ 500 Can't connect to
www.oneway.net:80 (Bad hostname 'www.oneway.net')
http://www.dwc.ch/ 500 Can't connect to www.dwc.ch:80
(Bad hostname 'www.dwc.ch')
http://www.highpoint.co.uk/hosting.html 500 timed out
http://www.speakeasy.org/ 302 Found http://www.speakeasy.net/
http://www.digiforest.com/ 500 timed out
http://www.onehost.net/ 500 timed out
http://www.alaska.net/ 500 timed out
http://www.greatland.net/ 404 Object Not Found
http://www.adgrafix.com/ 302 Found
http://www.adgrafix.com/index3.html
http://www.iconnet.net/ 500 timed out
http://www.psn.net/ 500 timed out
http://www.visionhost.net/ 500 Can't connect to
www.visionhost.net:80 (Bad hostname 'www.visionhost.net')
On page: http://mardy:40994/dst_html/support/jobs.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
On page: http://mardy:40994/dst_html/support/training.html
http://www.modperl.com/ 302 Found
http://www.no-ip.com/hopto-404.php
http://training.gbdirect.co.uk/courses/linux/customized_and_bespoke.html 404
Not Found
Removing very common words...
no words removed.
Writing main index...
Sorting words ...
Sorting 10651 words alphabetically
Writing header ...
Writing index entries ...
Writing word text: Complete
Writing word hash: Complete
Writing word data: Complete
10651 unique words indexed.
5 properties sorted.
1279 files indexed. 3884170 total bytes. 318386 total words.
Elapsed time: 00:16:47 CPU time: 00:00:08
Indexing done!
--
Bill Moseley
mailto:[EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
