I am trying to exclude some Search spiders.  Some are working but others are not.

Here are my excludes:
BROWEXCLUDE EMonitor*
BROWEXCLUDE GomezAgent*
BROWEXCLUDE "Ask Jeeves/Teoma*"
BROWEXCLUDE psbot*
BROWEXCLUDE "-DIE-KRAEHE*"
Log entry for the last one:
217.160.94.14 - - [01/Feb/2004:06:00:39 -0600] "GET / HTTP/1.0" 200 58386 "-" "-DIE-KRAEHE- META-SEARCH-ENGINE/1.0 http://www.die-kraehe.de" GET / - "HTTP/1.0" 200 58386 - - 113 133 230 133 1 DIRECT FIN FIN DO-NOT-CACHE -

These two are still getting through:

   5: Mozilla/2.0 (compatible; Ask Jeeves/Teoma)
   1: Mozilla/4.0 (compatible; EMonitor 6.1 Windows NT)

Here are the log file entries:
65.214.36.53 - - [01/Feb/2004:06:00:33 -0600] "GET /sensors/library/CAD/Limit/802tw18.dwg HTTP/1.0" 200 20684 "-" "Mozilla/2.0 (compatible; Ask Jeeves/Teoma)" GET /sensors/library/CAD/Limit/802tw18.dwg - "HTTP/1.0" 200 20684 - - 192 224 312 224 1 DIRECT FIN FIN DO-NOT-CACHE -


10.90.4.15 - - [01/Feb/2004:06:00:40 -0600] "GET /products/information-pdf/large-inventory/ HTTP/1.0" 200 58386 "-" "Mozilla/4.0 (compatible; EMonitor 6.1 Windows NT)" GET / - "HTTP/1.0" 200 58386 - - 208 133 309 133 1 DIRECT FIN FIN DO-NOT-CACHE -

Reply via email to