I was looking into the server logs and I discovered a
curious difference between MSN and Google. Over the past
7 days, the MSN bot has visited www.sqlite.org 114550 times.
The Google bot, on the other hand, has only visited 4070
times.
In spite of this, when I type "Embedded SQL database" into
Google, www.sqlite.org is the #1 hit. But www.sqlite.org
appears nowhere on the first 2 pages of results from MSN.
Of the 114550 visits that the MSN bot made to www.sqlite.org,
42320 were to a single URL:
http://www.sqlite.org/cvstrac/captcha
You would think that after a few hundred visits, the bot would
figure out what this page contains and move on....
Another 59115 hits from the MSN bot contain the name "luggle.com"
in the URL. No such page exists on the SQLite website, so I
am puzzled about why MSN finds this such an attractive place
to visit. Does anybody know what "luggle.com" is? Here is
a typical luggle.com URL:
http://www.sqlite.org/cvstrac/wiki/luggle.com/attach_get/241/wiki
In this URL, everything from the "luggle.com" onward is bogus.
So the wiki server ignores it all and serves up the wiki home
page. So MSN has apparently indexed the SQLite wiki homepage
some 59115 times over the past 7 days. Any ideas why? Where
is MSN getting this "luggle.com" stuff?
I consider the behavior of the MSN bot to be abusive. I'm
sorely tempted to ban the MSN bot from the entire sqlite.org
website. If SQLite ceased to be indexed by MSN, would that
seriously inconvenience any users?
--
D. Richard Hipp <[EMAIL PROTECTED]>
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
