On Wed, 10 May 2000, William Martin wrote:
BROWEXCLUDE *spider*,*spyder*,*crawler* etc.
BROWEXCLUDE *Spider*,*Spyder*
this gives much better results, but it's cumbersome. is there a way to make
only the browser field case insensitive (say a BROWCASE command, similar to
USERCASE)? I
On Thu, 11 May 2000, you wrote:
On Wed, 10 May 2000, William Martin wrote:
BROWEXCLUDE *spider*,*spyder*,*crawler* etc.
BROWEXCLUDE *Spider*,*Spyder*
this gives much better results, but it's cumbersome. is there a way to make
only the browser field case insensitive (say a BROWCASE
On Thu, 11 May 2000, Alejandro Fernandez wrote:
That makes me think:
can I go:
USEREXCLUDE REGEXPI:Tom.*Jones
or is it
USEREXCLUDE REGEXPI:Tom\ Jones
?
Either. They mean different things. The first matches (case insensitively)
Tom followed by any characters followed by Jones.
hi there,
I've been struggling to reconcile traffic reports from analog and webtrends
for a while now. I strongly favor analog, but I've found one item where
webtrends seems to have the upper hand: screening spiders. I've found some
references on your FAQ but they don't quite answer my question.
William Martin wrote:
BROWEXCLUDE *spider*,*spyder*,*crawler* etc.
BROWEXCLUDE *Spider*,*Spyder*
There has been threads on the list at varying times about a collaborative list
of browsers and spiders. I believe Marco Bernardini is hosting those at the
moment. Check the Analog Helper
Alle 13.26 Wednesday 10/05/2000 -0700, Jeremy Wadsack ha mandato a Marco
questo messaggio:
William Martin wrote:
BROWEXCLUDE *spider*,*spyder*,*crawler* etc.
BROWEXCLUDE *Spider*,*Spyder*
There has been threads on the list at varying times about a collaborative list
of browsers and
On Sun, 6 Feb 2000, Marco Bernardini wrote:
Supposing I find every robot (doing a grep for robots.txt, looking manually
for heavy movement or annoying engines webmasters) how can I group them
separately into the BROWSER report?
I try things like
BROWALIAS Scooter* "Robot Scooter$1"
At 12.32 15/12/99 -0500, Boris Goldowsky wrote:
I've been asked to report page-view statistics for our web site
which eliminate page views from search-engine spiders and other robots.
I've tried to do some of this by coming up with a list of User-Agent
strings that look like spiders, but it seems
Stephen Turner wrote:
On Thu, 16 Dec 1999, ikong wrote:
Sorry, I meant to say "10 requests to the same page in 10 minutes from
the same IP".
10 requests to the same page wouldn't identify spiders because they only
visit each page once.
Oops, this algorithm seems to be for banner
Aengus Lawlor wrote:
ikong fu wrote:
I have heard of an algorithm of 10 requests within 10 minutes from the
same IP being used to flag potential spiders. However, that still
leaves potential proxy server requests.
IE allows you to capture a page or sequence of pages for off-line use
I have heard of an algorithm of 10 requests within 10 minutes from the
same IP being used to flag potential spiders. However, that still
leaves potential proxy server requests.
Considering that spiders are always changing and mutating, I think the best way
to keep track of them is to join
I've been asked to report page-view statistics for our web site
which eliminate page views from search-engine spiders and other robots.
I've tried to do some of this by coming up with a list of User-Agent
strings that look like spiders, but it seems like a hit-or-miss sort
of approach.
Does
Boris Goldowsky wrote:
I've been asked to report page-view statistics for our web site
which eliminate page views from search-engine spiders and other robots.
I've tried to do some of this by coming up with a list of User-Agent
strings that look like spiders, but it seems like a hit-or-miss
On Wed, 15 Dec 1999, Boris Goldowsky wrote:
I've been asked to report page-view statistics for our web site
which eliminate page views from search-engine spiders and other robots.
I've tried to do some of this by coming up with a list of User-Agent
strings that look like spiders, but it
I have heard of an algorithm of 10 requests within 10 minutes from the
same IP being used to flag potential spiders. However, that still leaves
potential proxy server requests.
ikong fu
[EMAIL PROTECTED]
Stephen Turner wrote:
On Wed, 15 Dec 1999, Boris Goldowsky wrote:
I've been asked
ikong fu wrote:
I have heard of an algorithm of 10 requests within 10 minutes from the
same IP being used to flag potential spiders. However, that still
leaves potential proxy server requests.
IE allows you to capture a page or sequence of pages for off-line use
that would easily fall afoul
Jim Foley wrote:
Date sent: Wed, 03 Mar 1999 11:58:36 -0800
To: [EMAIL PROTECTED]
From: "Glen W. Forister" [EMAIL PROTECTED]
Subject: [analog-help] Spiders and Analog
Send reply to: [EMAIL PROTECTED]
On Fri, 5 Mar 1999, Jeremy Wadsack wrote:
A cautionary note here. Some spiders (not the top search engines, I don't
think) mimick a mozilla useragent to ensure they are spidering the content
likely to be presented to a user and not some content you may have chosen to
present only to
Date sent: Wed, 03 Mar 1999 11:58:36 -0800
To: [EMAIL PROTECTED]
From: "Glen W. Forister" [EMAIL PROTECTED]
Subject: [analog-help] Spiders and Analog
Send reply to: [EMAIL PROTECTED]
In looking at the results
19 matches
Mail list logo
