Hello Scott,
I have had similar problem. Can you let me know if this is resolved on your
end. Sometimes the email response coming back to me gets buried in another
folder and I never get to see the resolutions.
I can't seem to get search engines to see my site, as well. I do not know
how to resolve this....

Thanks
Ed
[EMAIL PROTECTED]


-----Original Message-----
From: Scott Purcell [mailto:[EMAIL PROTECTED]
Sent: Friday, February 10, 2006 8:40 PM
To: Tomcat Users List
Subject: Access log to see where robots go.


I have had trouble getting search engines to see my site. I built it with
struts, and use some tags from the index.html page to get business logic, to
finally get to my page. The url is http://www.theuniquepear.com

Anyway, upon talking to some co-workers, they suggested I watch my access
log, so I can see what files they are indexing. I thought I had the access
log turned on for the site, and see when someone hits my web site, but as
far as the searchbots go, I only see this in my logs daily.

$ cat  localhost_access_log.2006-02-07.txt | less
67.15.16.30 - - [07/Feb/2006:03:44:55 -0600] "GET /robots.txt HTTP/1.0" 404
985
67.15.16.30 - - [07/Feb/2006:03:46:21 -0600] "GET / HTTP/1.0" 200 844
67.15.16.30 - - [07/Feb/2006:03:51:57 -0600] "GET /robots.txt HTTP/1.0" 404
985
62.114.208.233 - - [07/Feb/2006:03:52:42 -0600] "GET
/unique/welcome.do?OVRAW=home%20decorating%20ideas&OVKEY=home
62.114.208.233 - - [07/Feb/2006:03:52:44 -0600] "GET
/unique/includes/siteWide.css HTTP/1.1" 200 15402
62.114.208.233 - - [07/Feb/2006:03:52:44 -0600] "GET
/unique/images/header_pear.jpg HTTP/1.1" 200 11227


I see the entry for robots.txt, but I have no idea where they are going, or
what they are doing.

I turned on access log like this in the server.xml like so:
        <Valve className="org.apache.catalina.valves.AccessLogValve"
                 directory="logs"  prefix="localhost_access_log."
suffix=".txt"
                 pattern="common" resolveHosts="false"/>

And that is a snippet of the log from above.

Does anyone know how to get more involved text, or can anyone tell me what
the robots.txt above is doing?


Thanks,
Scott


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to