Re: [analog-help] Re: Ranking logic; characters; total referrals
Just to be clear, you are talking about adding a column to each of the time reports that counts the number of unique referrers for the specified time period. In order to do this, Analog would need to aggregate the referrers and count the number of lines for each time period that each unique referrer has. It may seem different than the cross-referencing, but it's exactly the same problem as asking to count the number of hosts visiting on each day. Yes, this can be done, but it will require a lot more memory. Both hosts and referrers have *lots* of unique entries in most logs (whereas requests have much fewer), so it would significantly increase the memory overhead of the processing. There are lots of existing arguments in this list archive about when such things could or could not be done and whether the option of enabling or disabling such reports could be determined by condiguration rather than design. Right now, the design spec (as far as I know) is to not include any such reports. -- Jeremy Wadsack Seven Simple Machines Szegő Ákos wrote: Re: Referral Time Reports "Can I find out which files each referrer pointed to? or Can I find out which files each host has read? or Can I find out which hosts have read each file? or Can I find out the number of hosts visiting on each day? or lots of similar questions." [from FAQ] I feel my suggestion is not analogous with these. In such cases analog would first aggregate entries that belong together (e.g. log lines belonging to the same host or same file) then sort them by another parameter (e.g date or file). To create a referral time report, analog does not have to aggregate entries that belong together because each referral is on its own. A referrer field is a straightforward, easily recognizable, one-off information, like a page request. I am not a programmer (not by far), but to me it seems that a time report should not need cross-referencing as long as it counts individual, discrete entries, without any pre-aggregation. Perhaps I am wrong, I only wonder. Thanks for your help. + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
RE: [analog-help] arguments to request
Just to follow up on that response. There are 4 different type of search strings, all of which do show up on the request report. It is just that that request is not sub categorized into showing the individual strings. On tht note I also set REQARGSFLOOR 1R and it does not show the query still. Funny thing is that setting does not show up when I do the "analog -setting" command. Abhi. --- Jeff Ross <[EMAIL PROTECTED]> wrote: > Are you sure the query string is present in your log > files? > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf > Of > [EMAIL PROTECTED] > Sent: Thursday, June 10, 2004 11:04 AM > To: [EMAIL PROTECTED] > Subject: [analog-help] arguments to request > > Hi my server recives requets in the form > > "GET > /vafwebapp/sportsWatch.do?category=sportMain&assID=2&sports=Football > HTTP/1.1" > > > By default, argumens showing in REQUEST REPORT is > supposed to be enabled. But apaprently, after > reading > the entire docs and tyring various things including > ARGSINCLE (after reading the docs) did not show it > in > the report. > > Please help. > > > > > __ > Do you Yahoo!? > Friends. Fun. Try the all-new Yahoo! Messenger. > http://messenger.yahoo.com/ > +--- > - > | TO UNSUBSCRIBE from this list: > | > http://lists.isite.net/listgate/analog-help/unsubscribe.html > | > | Digest version: > http://lists.isite.net/listgate/analog-help-digest/ > | Usenet version: > news://news.gmane.org/gmane.comp.web.analog.general > | List archives: > http://www.analog.cx/docs/mailing.html#listarchives > +--- > - > + > | TO UNSUBSCRIBE from this list: > | > http://lists.isite.net/listgate/analog-help/unsubscribe.html > | > | Digest version: > http://lists.isite.net/listgate/analog-help-digest/ > | Usenet version: > news://news.gmane.org/gmane.comp.web.analog.general > | List archives: > http://www.analog.cx/docs/mailing.html#listarchives > + __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
RE: [analog-help] arguments to request
Yes. My initial email is showing the exact request field copy pasted from the log file. The query string is shown as part of the request field and not seprately in the log file. Please tell me what arguments would you use to display it. In the mean time I also used the "analog -settings" command Found some interesting stuff: Although the doc says, search strings are enabled by default, it is just the opposite. Search strings are disabled by default. So I enabled them using ARGSINCLUDE * and found the folowing confirmation .. .. Including (+) and excluding (-) the following search arguments: All excluded, then + * .. .. I also 1. made the floor for items and sub items as one request 2. I also included the all LINKS for REQUEST REPORTT they are confirmed as follows: Request Report [ON] Columns: #requests %bytes last-time Sorting by number of requests Floor is: 1 request Sorting sub-items by number of requests Floor for sub-items is: 1 request Plotting pie chart by number of requests Including (+) and excluding (-) the following in the report: All included, then - * + *.htm* + /vafwebapp/sportsWatch.do Including (+) and excluding (-) the following links in the report: All excluded, then + * --- Jeff Ross <[EMAIL PROTECTED]> wrote: > Are you sure the query string is present in your log > files? > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf > Of > [EMAIL PROTECTED] > Sent: Thursday, June 10, 2004 11:04 AM > To: [EMAIL PROTECTED] > Subject: [analog-help] arguments to request > > Hi my server recives requets in the form > > "GET > /vafwebapp/sportsWatch.do?category=sportMain&assID=2&sports=Football > HTTP/1.1" > > > By default, argumens showing in REQUEST REPORT is > supposed to be enabled. But apaprently, after > reading > the entire docs and tyring various things including > ARGSINCLE (after reading the docs) did not show it > in > the report. > > Please help. > > > > > __ > Do you Yahoo!? > Friends. Fun. Try the all-new Yahoo! Messenger. > http://messenger.yahoo.com/ > +--- > - > | TO UNSUBSCRIBE from this list: > | > http://lists.isite.net/listgate/analog-help/unsubscribe.html > | > | Digest version: > http://lists.isite.net/listgate/analog-help-digest/ > | Usenet version: > news://news.gmane.org/gmane.comp.web.analog.general > | List archives: > http://www.analog.cx/docs/mailing.html#listarchives > +--- > - > + > | TO UNSUBSCRIBE from this list: > | > http://lists.isite.net/listgate/analog-help/unsubscribe.html > | > | Digest version: > http://lists.isite.net/listgate/analog-help-digest/ > | Usenet version: > news://news.gmane.org/gmane.comp.web.analog.general > | List archives: > http://www.analog.cx/docs/mailing.html#listarchives > + __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
Re: [analog-help] Problem with a query string url
On Thu, 10 Jun 2004, Paul Richards-Training wrote: > Hi... > > Trying to run a report on a url which we cannot seem to isolate, ie. So > we just get an html report on this link alone. > > The url is exactly as follows (it's an intranet url): > http://learn.gateway.bbc.co.uk/searchResults.asp?delivery=ONLINECOURSE > Did you try FILEINCLUDE http://learn.gateway.bbc.co.uk/searchResults.asp?delivery=ONLINECOURSE (all on one line)? -- Stephen Turner, Cambridge, UKhttp://homepage.ntlworld.com/adelie/stephen/ "Low Priced Cambridge Clare College. Big selection at eBay UK!" (Ad after Google search for Clare College Cambridge) + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
RE: [analog-help] arguments to request
Are you sure the query string is present in your log files? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, June 10, 2004 11:04 AM To: [EMAIL PROTECTED] Subject: [analog-help] arguments to request Hi my server recives requets in the form "GET /vafwebapp/sportsWatch.do?category=sportMain&assID=2&sports=Football HTTP/1.1" By default, argumens showing in REQUEST REPORT is supposed to be enabled. But apaprently, after reading the entire docs and tyring various things including ARGSINCLE (after reading the docs) did not show it in the report. Please help. __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ +--- - | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +--- - + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
[analog-help] arguments to request
Hi my server recives requets in the form "GET /vafwebapp/sportsWatch.do?category=sportMain&assID=2&sports=Football HTTP/1.1" By default, argumens showing in REQUEST REPORT is supposed to be enabled. But apaprently, after reading the entire docs and tyring various things including ARGSINCLE (after reading the docs) did not show it in the report. Please help. __ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/ + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
[analog-help] Problem with a query string url
Title: Problem with a query string url Hi… Trying to run a report on a url which we cannot seem to isolate, ie. So we just get an html report on this link alone. The url is exactly as follows (it's an intranet url): http://learn.gateway.bbc.co.uk/searchResults.asp?delivery=ONLINECOURSE We have tried all sorts of variants of fileinclude, reqinclude and arginclude but just cannot extract this url. The best we have so far got is the following set of commands in the config file: FILEINCLUDE /searchresults.asp*delivery=* REQINCLUDE /searchresults.asp?*delivery=* REQINCLUDE /searchresults.asp* This at least gives us all the query which include "delivery=", but when we add the letter "O*" after the "=" we get nowhere again…Ie, a report with null results. We have tried in both 5.32 and 5.91 BETA with identical issues arising. Would welcome any guidance in trying to extract this particular query-string url. Paul Very best wishes, Paul Richards BBC Gateway Team - Statistics and Search Internal ext: 89354 External phone number: 020 7208 9354 Room 301, 35 Marylebone High St, London W1U 4PX http://www.bbc.co.uk/ - World Wide WonderlandThis e-mail (and any attachments) is confidential and may containpersonal views which are not the views of the BBC unless specificallystated.If you have received it in error, please delete it from your system. Do not use, copy or disclose the information in any way nor act inreliance on it and notify the sender immediately. Please note that theBBC monitors e-mails sent or received. Further communication will signify your consent to this.
[analog-help] RE: Browser Report/Summary - None Displayed - Problem Solved
Thanks Aengus! That was the problem. Sincerely, Paul Dowd Special Systems Technician C/W MARS, Inc 1 Sunset Lane Paxton, Massachusetts 01612-1197 Phone: (508) 755-3323 x.28 Fax: (508) 755-3721 Email: [EMAIL PROTECTED] Web: http://www.cwmars.org On 2004-06-10 at 17:35:00 you wrote: >Paul Dowd <[EMAIL PROTECTED]> wrote: > >> When processing my logfile I cannot get analog to display the >> browsers used to access my pages in the report. Instead I see [not >> listed: 4 browsers] in the report and [not listed: 1 browser] in the >> summary. I'm not sure what is wrong. > >The line immediately above the report should show what the "floor" for >that report is, for example: > >Listing browsers with at least 1 request for a page, sorted by the >number of requests for pages. > >You can change the floor by using the BROWREPFLOOR and BROWSUMFLOOR >directives: > >BROWREPFLOOR 1r > >to list all browsers with at least 1 request. (Note that the default >seems to be for browsers with a request for _pages_ - if you are using >IIS, for example, you shoule make sure to use PAGEINCLUDE *.asp, as not >all requests are requests for pages). > >Aengus > >+ >| TO UNSUBSCRIBE from this list: >|http://lists.isite.net/listgate/analog-help/unsubscribe.html >| >| Digest version: http://lists.isite.net/listgate/analog-help-digest/ >| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general >| List archives: http://www.analog.cx/docs/mailing.html#listarchives >+ >. = = = = = = = = = = = = = = = = = = = = = = + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
Re: [analog-help] RE: Browser Report/Summary - None Displayed
Paul Dowd <[EMAIL PROTECTED]> wrote: > When processing my logfile I cannot get analog to display the > browsers used to access my pages in the report. Instead I see [not > listed: 4 browsers] in the report and [not listed: 1 browser] in the > summary. I'm not sure what is wrong. The line immediately above the report should show what the "floor" for that report is, for example: Listing browsers with at least 1 request for a page, sorted by the number of requests for pages. You can change the floor by using the BROWREPFLOOR and BROWSUMFLOOR directives: BROWREPFLOOR 1r to list all browsers with at least 1 request. (Note that the default seems to be for browsers with a request for _pages_ - if you are using IIS, for example, you shoule make sure to use PAGEINCLUDE *.asp, as not all requests are requests for pages). Aengus + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
[analog-help] RE: Browser Report/Summary - None Displayed
Hi Everyone, I'm new to the list and I'm currently engaged in learning Analog. I find the program to be awesome in its ability to process logs and to do so quickly. But, I'm faced with a small problem right now. I've already RTFM, but could not find my answer (possibly I missed it). When processing my logfile I cannot get analog to display the browsers used to access my pages in the report. Instead I see [not listed: 4 browsers] in the report and [not listed: 1 browser] in the summary. I'm not sure what is wrong. My log is in the following format: LOGFORMAT (%s %S %u [%d/%M/%Y:%h:%n:%j] "%j %r %j" %c %b "%B") A couple of examples of the "%B" field from the logfile itself: "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)" "Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; .NET CLR 1.1.4322)" I should note here that the OS report populates nicely showing the various OSes used. Any help would be GREATLY appreciated. Sincerely, Paul Dowd Special Systems Technician C/W MARS, Inc 1 Sunset Lane Paxton, Massachusetts 01612-1197 Phone: (508) 755-3323 x.28 Fax: (508) 755-3721 Email: [EMAIL PROTECTED] Web: http://www.cwmars.org + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
Re: [analog-help] Re: Ranking logic; characters; total referrals
There is no secondary sort criterion, and I don't have any desire to add one. It's much more cumbersome than it's worth. -- Stephen Turner, Cambridge, UKhttp://homepage.ntlworld.com/adelie/stephen/ "Low Priced Cambridge Clare College. Big selection at eBay UK!" (Ad after Google search for Clare College Cambridge) + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
Re: [analog-help] Re: Ranking logic; characters; total referrals
Szegő Ákos <[EMAIL PROTECTED]> wrote: > My question referred to > entries that cannot be sorted by the parameter given there, because > the parameter has the same value for them (i.e. pages each requested > once, referrers each referring once etc.). I cannot see a consistent > logic behind ranking such entries. I have a (very!) vague recollection that this question came up some time back on the list, but I can't remember what the answer was. Without looking into it in any way, I would imagine that the list is created as items are encounterd in the log, and therefore "unsortable" items default to sorting by date. You might want to add the "Date and time of last access" (D) and "Date and time of first access" (E) columns to your request report to see if it at least explains what is happening, even if it doesn't allow you to control it REQCOLS NPpDE http://analog.cx/docs/othreps.html#othCOLS Aengus + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
Re: [analog-help] Can you change the "Listing the top # "
Tom W <[EMAIL PROTECTED]> wrote: > On the report file, underneath some graphs it says (for example), > 'Listing the top 40 browsers'. > > Is it possible to change this number so that it would list the top 60 > browsers, for example? > > (I have been through the documentation and FAQ's etc. but have not > found an answer. You use the FLOOR directives: BROWREPFLOOR -60r will list the top 60 Browsers (by number of requests). BROWREPFLOOR 60r will list all browsers with at least 60 requests. http://analog.cx/docs/othreps.html#FLOOR http://analog.cx/docs/hierreps.html#SUBFLOOR Aengus + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
[analog-help] Re: Ranking logic; characters; total referrals
Dear Mr Wadsack, Thank you for your answers. Re: SORTBY I do know and use the SORTBY command. My question referred to entries that cannot be sorted by the parameter given there, because the parameter has the same value for them (i.e. pages each requested once, referrers each referring once etc.). I cannot see a consistent logic behind ranking such entries. Running analog one day might produce: no.: pages: %pages: site ---: -: --: 85:1: 0.24%: http://images.google.com/ 86:1: 0.24%: http://www.google.com.au/ 87:1: 0.24%: http://www.google.ca/ and the next day (run on fresh log files): 87:1: 0.22%: http://images.google.com/ 88:1: 0.22%: http://www.google.ca/ 89:1: 0.22%: http://www.google.com.au/ Nothing changes in terms of these 3 items, still they are shown in a different order. Why is this happening, and what can be done to make such entries rank according to some rule consistently? Re: Referral Time Reports "Can I find out which files each referrer pointed to? or Can I find out which files each host has read? or Can I find out which hosts have read each file? or Can I find out the number of hosts visiting on each day? or lots of similar questions." [from FAQ] I feel my suggestion is not analogous with these. In such cases analog would first aggregate entries that belong together (e.g. log lines belonging to the same host or same file) then sort them by another parameter (e.g date or file). To create a referral time report, analog does not have to aggregate entries that belong together because each referral is on its own. A referrer field is a straightforward, easily recognizable, one-off information, like a page request. I am not a programmer (not by far), but to me it seems that a time report should not need cross-referencing as long as it counts individual, discrete entries, without any pre-aggregation. Perhaps I am wrong, I only wonder. Thanks for your help. Best regards, Akos + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
[analog-help] Can you change the "Listing the top # "
On the report file, underneath some graphs it says (for example), 'Listing the top 40 browsers'. Is it possible to change this number so that it would list the top 60 browsers, for example? (I have been through the documentation and FAQ's etc. but have not found an answer. Many Thanks, Tom. _ It's fast, it's easy and it's free. Get MSN Messenger today! http://www.msn.co.uk/messenger + | TO UNSUBSCRIBE from this list: |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | Digest version: http://lists.isite.net/listgate/analog-help-digest/ | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general | List archives: http://www.analog.cx/docs/mailing.html#listarchives +
