Looks like you are running the default access log settings.
For logging of user-agent and referer (when given) try:
CustomLog logs/access_log combined
http://httpd.apache.org/docs/1.3/logs.html#accesslog
HTH,
Jon
Scott Purcell wrote:
I really would like to find out how my (and if my site) is being
indexed. I am using Tomcat 5.5 and I am running a ecommerce site. I have
had nothing but trouble getting seen in search engines, so I would like
to be able to somehow trace what pages robots are indexing.
I did add a robots.txt and allow my whole domain to be indexed. So that
being said, I added an access log but by default, I cannot really tell
if I have users hitting my site, or when robots are visiting my site.
Below is my access_log from Saturday. What I would like to be able to
do, is distinguish between normal users, and robots. If I can verify
that the robots are indexing my site, I would be happy, and then know I
need to work more meta. But if they are not indexing my site, then I
need to find out why.
Does this make sense? Has anyone been through this before? Thanks ahead
of time,
Regards
Scott
Access log Saturday
152.163.100.73 - - [26/Mar/2006:02:27:13 -0600] "GET
/unique/viewThumbProducts.do?type=category&searchString=Candles%20&%20Ar
oma&categoryID=13&pageNumber=1 HTTP/1.0" 200 17139
152.163.100.72 - - [26/Mar/2006:02:27:14 -0600] "GET
/unique/includes/siteWide.css HTTP/1.0" 200 15402
152.163.100.134 - - [26/Mar/2006:02:27:14 -0600] "GET
/unique/images/head_text.jpg HTTP/1.0" 200 18661
152.163.100.138 - - [26/Mar/2006:02:27:16 -0600] "GET
/unique/images/spacer_transparent.gif HTTP/1.0" 200 49
152.163.100.133 - - [26/Mar/2006:02:27:16 -0600] "GET
/unique/images/browseProducts.gif HTTP/1.0" 200 659
152.163.100.66 - - [26/Mar/2006:02:27:16 -0600] "GET
/unique/images/moregreatfinds.gif HTTP/1.0" 200 564
152.163.100.69 - - [26/Mar/2006:02:27:16 -0600] "GET
/unique/images/searchText.gif HTTP/1.0" 200 624
152.163.100.138 - - [26/Mar/2006:02:27:16 -0600] "GET
/unique/images/go.gif HTTP/1.1" 200 308
152.163.100.204 - - [26/Mar/2006:02:27:16 -0600] "GET
/unique//images/thumb/UP00130.jpg HTTP/1.0" 200 10044
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
