Ok I have been getting some really weird hits on my websites and they don't
contain a IP or host in the log.
. - - [02/Jan/2006:01:59:37 -0800] GET /robots.txt HTTP/1.0 200 54 -
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Has anyone else been getting these as well? And does anyone
Scott, what's the mystery, the missing IP or the GET robots.txt?GerryOn 1/2/06, Scott Grayban [EMAIL PROTECTED]
wrote:Ok I have been getting some really weird hits on my websites and they don't
contain a IP or host in the log.. - - [02/Jan/2006:01:59:37 -0800] GET /robots.txt HTTP/1.0 200 54
The missing IP/Host of course. I have never seen this before and I
know it didn't come from anything I do here. This just started to happen.
Scott
On Mon 2 January 2006 12:59, Gerry Danen wrote:
Scott, what's the mystery, the missing IP or the GET robots.txt?
Gerry
On 1/2/06, Scott
Does it happen with real visitors also, or just bots?On 1/2/06, Scott Grayban [EMAIL PROTECTED] wrote:
The missing IP/Host of course. I have never seen this before and Iknow it didn't come from anything I do here. This just started to happen.
ScottOn Mon 2 January 2006 12:59, Gerry Danen wrote:
Looks like only bots. I have a few hits of robots.txt getting hit first then
it crawls the site but thats but I have more hits to the robots.txt without a
crawl.
It almost feels virus like but I have no windows computer here in my network.
Scott
On Mon 2 January 2006 13:28, Gerry Danen wrote:
When it crawls the site, if you get always a hit on a specific page, you could capture more info than you would get in the log. Sometimes the browser field has the name of the bot in it. You want to pursue this off-list?
GerryOn 1/2/06, Scott Grayban [EMAIL PROTECTED] wrote:
Looks like only bots.
Sure. I would like to know who, what, how on this one.
Scott
On Mon 2 January 2006 14:44, Gerry Danen wrote:
When it crawls the site, if you get always a hit on a specific page, you
could capture more info than you would get in the log. Sometimes the
browser field has the name of the bot in
[EMAIL PROTECTED]On 1/2/06, Scott Grayban [EMAIL PROTECTED] wrote:
Sure. I would like to know who, what, how on this one.ScottOn Mon 2 January 2006 14:44, Gerry Danen wrote:
When it crawls the site, if you get always a hit on a specific page, you could capture more info than you would get in the
