On Friday, May 12, 2006 2:10 PM [EDT],
Moshe <[EMAIL PROTECTED]> wrote:
I need your help to clear an issue for me:
Let's say that google has crawled my website.
The log should have many many entries for that day or week caused by
the Google crawler. I am confused about if and how these entries will
show in the different reports.
General:
Successful requests: Included? Not Included?
A request is a request is a request. If it's a request, it's included in the
Request Reort, unless you explicitly exclude it.
Can I exclude crawler
requests from this item? How?
By HOSTEXCLUDE-ing the IP address of the spider/robot, or BROWEXCLUDE-ing
the User Agent string used by the spider/robot.
Successful requests for pages: and Distinct files requested:
Included? Not Included? Can I exclude crawler requests from this
item? How?
I assume that answers for General will apply to the Daily, monthly
and so on.
Referer Report:
I assume that crawlers entries in the log do not have referes? Right?
That depends on the spider/robot.
Browser summary report:
What command controls this report. I have on my report browsers such
as GoogleBot, msnbot. How can I exclude these from this report and
only
show "real" browsers?
http://analog.cx/docs/include.html#outputexcludes
You can use BROWREPEXCLUDE and BROWSUMEXCLUDE to exclude specific User
Agents from the Browser Reports. These are "Output Excludes", and only
remove the specified lines from those specific reports - the requests would
still be counted in other reports. Of course, if you use the "Input Exclude"
BROWEXCLUDE, the requests with the specified User agent are ignored before
they are processed, so they don't show up in the Browser Reports either.
You can also use ROBOTINCLUDE to tell Analog which User Agents are Robots
for the Operating System Report.
Also, if I excluded crawlers request in the
first place (see the first section of my questions) the browser
report will then not have lines for these crawlers right?
Right.
Request Report:
How can make sure that crawlers entries in the log will not be
counted in this report?
See above.
Aengus
+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------
