Re: [analog-help] Is there any chance we can get the analog.cfg and rmagic config files that you use on the analog.cx web site?
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > Also, the mdocumentation refers to "Helper applications." There is a > link which points to the analog web page. From there, the only > helper App I found was RMAGIC - a great program for showing analog > data. Is there another helper app page? Are there any other helper > apps? http://www.analog.cx/helpers/ -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] I would like to run a report that only analyzes data for a range of IP addresses
Gonzalo Briceno ([EMAIL PROTECTED]): > 1. Does anybody know if analog can track "visits". A 24 hour period is a good > enough delimiter. No. See http://www.analog.cx/docs/webworks.html for reasons why. > 2. I cannot remove a certain Ip from the "Referrer Report". I've tried a line > like: > REFREPEXCLUDE149.34.*.* since we don't care about people from > within our organization referring to our site. What gives? REFREPEXCLUDE is used to remove items by referrer from the referrer report. To remove items by IP number use HOSTEXCLUDE. But note that this will remove those requests from ALL your reports. See http://www.analog.cx/docs/include.html for details on the INCLUDE/EXCLUDE commands. You can't remove particular items from a single report by any filter other than the data in the report. Analog doesn't maintain these connections. See http://www.analog.cx/docs/faq.html#faq128 for a discussion (it's the same issue described there). > 3. Lastly, is there any way to decide the order of the actual > reports? Use the REPORTORDER command, http://www.analog.cx/docs/output.html#REPORTORDER. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] I would like to run a report that only analyzes data for a range of IP addresses
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I have reason to believe that a user is logging in to our server as other > users. I know this user's static IP. I would like to generate an Analog > report that only includes a range of addresses such as: > IPINCLUDE 208.1.40.* or IPINCLUDE 208.1.40.94 > I could not find the specific command to use to include a range of IP > addresses. HOSTINCLUDE 208.1.40.* > Likewise, my network monitor probes this server and creates an entry with > each probe. I assume that Analog has a way to allow me to define an address > to exclude? Is it IPEXCLUDE? HOSTEXCLUDE > I could find no such commands in the manual however I am sure that you must > have allowed for this. See http://www.analog.cx/docs/include.html -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] How to change the 'floor' for request report from 20 to 10
Bridget Poulter ([EMAIL PROTECTED]): > Hello, > Now, I know I can change the 'floor' from 20 to 10 for the Request Report, > by doing something with ARGSINCLUDE, REQARGSFLOOR, r (code for request > report) but, I don't know where I should be putting all this as I can't > find '20' anywhere to start off with. > So, one very basis question to start off with - is this command code > contained in 'analog.cfg' or, in 'anlgform.pnl or somewhere else entirely? > (I didn't set any of this up so it's all a bit of a mystery to me). If I > knew the answer to this, i.e. which file holds all the commands, then maybe > I can teach myself a bit more but even this basic point is baffling me. > Thank you towhoever can help me; I'm sure it's a doddle once you know how. REQFLOOR 10r See http://www.analog.cx/docs/othreps.html#FLOOR for details. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Analog upgrade
melinda Yuan ([EMAIL PROTECTED]): > I had analog-5.02 in my RedHat 6.2, because I want > to use the latest analog version, so I installed the analog-5.22 into my > RedHat 6.2 machine. I did > following steps > (1)"make"-compiling, > (2)run analog > (3) edit analog.cfg file by setting > LOGFILE /usr/local/apahce/logs/access.log*, > OUTFILE index.html, This will put the report in the Analog directory. Most likely, you should put it somewhere in your web space. You need to run Analog after you make the changes to update the report. Then you can verify the report -- it should contain the version of Analog at the bottom. > and then I edit the httpd.conf file by setting "Alias /analog/ > /usr/local/analog-5.22/" and restart the Apache Server, but when I > open the web browser, and type http://www.bestvocab.com/analog/, it > came out still with analog-5.02 version. How do I do to remove the > version 5.02 and to make my v5.22 work properly? Does my > configuration process for v5.22 correct? Any suggestion is welcome. This may not be safe. You should not put Analog anywhere in your web server space. Although on Apache / Unix it may not be a problem, it's not recommended. Try running Analog again and see if that fixes it. If not, make sure that the version of analog you have installed and are running is 5.22, not 5.02 -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] My Log Files won't read!
Simon Smyth ([EMAIL PROTECTED]): > I have downloaded, unzipped and run analog 5.22. I have managed to get the > sample log files running, but when I import my own log file, I get a blank > report page. I think this is because the config file does not recognize my > log files. Any ideas? Did you specify your log file in the configuration file? Is it a format Analog can auto-detect? Did Analog produce any warning or error messages that might help diagnose the problem? -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] I am having trouble setting up ANALOG.CFG to read USERNAMES my DOMINO logfiles.
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > Jeremy, thanks for the help. I have most of analog working. > Can you tell me what code I would use to tell LOGFORMAT to ignore the rest > of the line when I don't know how many arguments there will be on the line? > Do I just use a bunch of %j %j %j %j ? As I said before Analog looks for the first match of the next character immediately following the variable. So if the last variable on the line is %j it will ignore everything up until the end of the line. In other words, you only need one %j at the end of a line. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] I am having trouble setting up ANALOG.CFG to read USERNAMES my DOMINO logfiles.
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I have 2 years of .TXT log files created by a Lotus Domino server. I > am trying to create a suitable configuration so that ANALOG can > process these. > The main problem that I am having is with the username. The username, when > present, is displayed in the text file as follows: > CN=Mickey M Mouse/O=Disney > "CN" will always be present at the start of the name. "/O=Disney" will > always be present at the end of the name. > The problem that I am having is the spaces - Aanlog tries to take each word > separated by spaces as a new field. Actually Analog looks for the next character that matches after a variable. If the variable is followed by a space, then it looks for the next space. If followed by a '/' it will look for a '/'. > EXAMPLE: > 4.35.83.178 www.disney.com CN=Mickey M Mouse/O=Disney [08/Mar/2002:00:12:15 > +0800] "GET (REMAINDER OF LINE CLIPPED) > 4.35.83.178 www.disney.com CN=Daffy Duck/O=Disney [08/Mar/2002:00:12:15 > +0800] "GET (REMAINDER OF LINE CLIPPED) > 4.35.83.178 www.disney.com - [08/Mar/2002:00:12:15 +0800] "GET > (REMAINDER OF LINE CLIPPED) > Notice in the examples above that in the first case I have a name with a > middle initial. In the second example, I have a first and last name but no > middle initial. In the third example, I have only a "-" meaning that there > is no username. > CAN ANYONE ASSIST ME IN CREATING A LOGFORMAT LINE THAT WILL WORK WITH ALL > THREE OF THE ABOVE? You need two LOGFORMAT commands; one for the line with '-' and one for the lines with names. Assuming the lines are similar to CLF, using the formats in the docs, I would suggest these to start: LOGFORMAT (%S %j - [%d/%M/%Y:%h:%n:%j] "%j%w%r%wHTTP%j" %c %b) LOGFORMAT (%S %j CN=%u/0=Disney [%d/%M/%Y:%h:%n:%j] "%j%w%r%wHTTP%j" %c %b) This will work as long as no usernames have a '/' in them. > I am not sure if the file attachment will go through but I have > attached a snippet of a log file called SampleDominoLog.TXT It is preferable that you include 3 or 4 COMPLETE lines in the text of your message than send a 28kb text attachment to all the members of the list. The lines above (especially if complete) would have been enough. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Analog help: Search requests displaying URL?
Woody Hanscom ([EMAIL PROTECTED]): > I'm trying to find out if it's possible to configure Analog to include > the referred URL in the Search Query report. > The answer to question 25 seemed to indicate that those answers were > essentially 'no' because people were trying to cross-reference > different parts of the logfile. However, I don't believe this is > the case for what I want to do - to be sure, I just checked my > logfile, and one of the two hits I'm using as an example is > displayed thusly: > tnt1-199-199-239-73.langdon.utma.com - - [11/Apr/2002:19:24:47 -0700] > "GET /archives/000131.shtml HTTP/1.1" 200 9393 > "http://google.yahoo.com/bin/query?p=suicide+how+to+commit+painless&hc=0&hs= > 0" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)" > Looking at that line, it looks like all the ingredients are there for > what I want to do (the search term ["suicide how to commit painless"] > and the resulting referenced URL ["/archives/000131.shtml"]) - is there > some reason I'm not seeing as to why this can't be done? Or, if it can - > could someone point me in the right direction? Getting the referenced URL is the search words report is the saem as question 25. It requires significantly more memory and processing time to cross-reference all the fields that are in the log file so Analog doesn't do that. It would require more memory than the original log files, in general. Getting the referred URL (note the different term) is possible with the ALIASes already suggested. I am guessing that you wanted to do the previous. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] How to clear/erase failed request stats
Randy Reines ([EMAIL PROTECTED]): > I'm sorry if I missed the answer to this in the documentation, I > couldn't find it. My question is, what is the best way to clear the > failed requests report? I would rather not clear everything, just > the failed requests. I am running analog 5.22 and apache on linux > 7.2. I tried clearing the error_log restarting apache, then i > re-ran analog, but the errors were still there. Analog does not read error_log. It gets all the error information from access_log. You could clear the log, but that will clear all reports. Alternately, you can use FAILFLOOR to tell Analog to only report on failures since a particular data. See http://www.analog.cx/docs/othreps.html#FLOOR for details. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Analog 5.1
SpydersWeb Internet Solutions Ltd ([EMAIL PROTECTED]): > I have made a complete hash of installingn and setting up Analog on > my web server. How do I uninstall the package from the web server > and then how can install and set it up to use on different web > roots? If you installed Analog from source, then you just need to remove the directory. If you used a package tool, such as rpm, you need to use the remove or un-install feature of that tool to remove it. Analog is controlled by a configuration file (default is analog.cfg in the same location as the executable). From there you an specify settings for particular reports. You can add additional configuration files with the +G command line option or set configuration settings directly on the command line options. You run Analog once per report. So if you want to create statistics for different web sites you need to run analog with different settings for each one. There are some examples of setting up Analog for virtual hosts in the documentation HOWTO pages. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] unknown domain
SUSANA PUMAR PAVON ([EMAIL PROTECTED]): > When does Analog say [unknown domain] in the Domain Report? It really > resolves the IPs. I have seen it at the dnscache file. It usually means the IP resolved to a host that does not have a domain. Like 'workstation1' or something. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] logrotate and analog
Daniel Bondurant ([EMAIL PROTECTED]): > I'm having a slight problem with using rotated logs in analog. > I am loading in the logfiles with these commands > LOGFILE /var/log/apache/mysite-access.log > LOGFILE /var/log/apache/mysite-access.log.*.gz > only problem is logrotate numbers the logs > mysite-acces.log.0.gz > mysite-acces.log.1.gz > mysite-acces.log.2.gz > ... > mysite-acces.log.11.gz > mysite-acces.log.12.gz > but when they are loaded in, they are in the order > mysite-acces.log.0.gz > mysite-acces.log.1.gz > mysite-acces.log.11.gz > mysite-acces.log.12.gz > mysite-acces.log.2.gz > ... > so analog gets confused, and the last date I get is in 9.gz > is there anyway around this? I'm sure it a pretty common problem. I believe Analog will handle requests out of order. I've never had a problem with this kind of rotation before. You might want to run analog with '-settings' to see what log files it is loading and whether there are any constraints on dates (e.g. FROM or TO commands) in effect. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Filealias
SUSANA PUMAR PAVON ([EMAIL PROTECTED]): > In order to get more understandable names in the Request Report I have > written in the Analog configfile several lines with FILEALIAS. > The Request Report is OK now, but in the File Type Report, Analog uses > these aliases in a wrong way, for instance, it considers any of the .gif > files as a different type with its different alias. Use REQALIAS instead of FILEALIAS. REQALIAS affects only the Request Report. FILEALIAS affects all reports. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Logformat
Max Gut ([EMAIL PROTECTED]):
> am 10.04.2002 16:32 Uhr schrieb Aengus unter [EMAIL PROTECTED]:
>> Max Gut <[EMAIL PROTECTED]> wrote:
>>
>>> analog says to the new logformat of my Provider:
>>>
>>> Warning C: Bad argument in configuration command: ignoring it:
>>> LOGFORMAT (%A %h %l %u %t \"%r\" %S %b %{Host}i \"%{Referer}i\"
>>> \"%{User-agent}i\" \"%{X-Forwarded-For}i\")
>>>
>>> The string was send to me like this:
>>>
>>> "%A %h %l %u %t \"%r\" %s %b %{Host}i \"%{Referer}i\" \"%{User-
>>> agent}i\" \"%{X-Forwarded-For}i\""
>>
>> That's an Apache Logformat string, so use the APACHELOGFORMAT
>> directive, not the LOGFORMAT directive.
>>
>> Aengus
> Thank you, but
> APACHELOGFORMAT (%A %h %l %u %t \"%r\" %s %b %{Host}i \"%{Referer}i\"
> \"%{User-agent}i\" \"%{X-Forwarded-For}i\")
> shows the following result:
> Warning L: Large number of corrupt lines
> try different LOGFORMAT
Uh... so try a different log format? :-) Apparently the
APACHELOGFORMAT doesn't match (most of) the lines in the log file.
Look at the warnings that Analog gives. You can turn on "DEBUG C" to
see which lines are considered corrupt and where Analog is failing to
parse them.
--
Jeremy Wadsack
Wadsack-Allen Digital Group
+
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
|http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
|http://www.mail-archive.com/analog-help@lists.isite.net/
|http://lists.isite.net/listgate/analog-help/archives/
|http://www.tallylist.com/archives/index.cfm/mlist.7
+
Re: [analog-help] analog: Fatal error: language file lang\uk.lng too short: exiting
Maff Rigby ([EMAIL PROTECTED]): > My next problem - I keep getting this error when trying to run analog: > analog: Fatal error: language file lang\uk.lng too short: exiting > I have been running it fine for weeks, and don't know why this is > happening... > Please help? You have language files from a previous version of Analog. Did you upgrade your version and not change the language files? You should re-install the current version, removing all existing language files. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] report pie and list sort
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > En réponse à Stephen Turner <[EMAIL PROTECTED]>: >> No, sorry. The pie chart will only plot items which are included on the >> main listing. This is deliberate! > Is there a way to limit the size of the list that is displayed, without > changing content of the list itself (i.e. the items not displayed in the list > could still be displayed on the chart)? The list and the pie chart are designed to show identical data. > If not, what are other possibility to display a chart for highest traffic > requests and a list for highest number of requests? Run analog twice? I'm not sure what "highest traffic" means -- most bytes? But if you want two different reports, then run analog twice. > Is there a way I can add an "other" request report in the analog > report file? You mean "[not listed: ###]"? It's already there. Use the REQFLOOR command to tell Analog how many items to display in the report. See http://www.analog.cx/docs/othreps.html#FLOOR for details on the command. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Sorting in Request Report not okay (?)
Marcel ([EMAIL PROTECTED]): >> This is exactly what it's supposed to do. Notice that the items with >> spaces before the names are the same page, with different combinations >> of arguments. You have told Analog (REQARGSFLOOR) to include to top 40 >> entries for arguments for each request. If you don't want it to show >> request arguments you can turn them off with ARGSEXCLUDE *. > and how can i configure it so that the arguments don't interest analog? > i mean that analoge handle complete part of url as page, not as page with > arguments. > schould be: > 271: 4,13%: /board/topic.cgi > 112: 2,53%: /board/topic.cgi?forum=5&topic=1 > 58: 0,57%: /board/topic.cgi?forum=5&topic=2 > 20: 0,13%: /board/topic.cgi?forum=2&topic=4 > 12: 0,19%: /board/topic.cgi?forum=7&topic=2 > 12: 0,07%: /board/topic.cgi?forum=7&topic=4 > not: > 271: 4,13%: /board/topic.cgi > 112: 2,53%: /board/topic.cgi?forum=5&topic=1 > 58: 0,57%: /board/topic.cgi?forum=5&topic=2 > 20: 0,13%: /board/topic.cgi?forum=2&topic=4 > 12: 0,19%: /board/topic.cgi?forum=7&topic=2 > 12: 0,07%: /board/topic.cgi?forum=7&topic=4 > is there a way? The only way I can think of to do this is to fool Analog into thinking those are not CGI arguments. You can do that with a pair of aliases. The first converts all file names with '?' to some other character that looks like a file name character: FILEALIAS *?* $1:$2 If any of your requests have ':' in them, then you'll need to find some other character for this. Next, convert them back in the Request Report to the normal names, with this: REQALIAS *:* $1?$2 I haven't tested this, but I believe it will do the trick for you. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] establishing 7-day period
Duke Hillard ([EMAIL PROTECTED]): > In my original message, I mentioned that I had experienced lag times > of an hour or less in the past. So, I was surprised when the lag > time jumped up to 7 hours after upgrading to 5.22. But, I don't > know that the increase is related to analog. I had hoped that using > the ERRFILE command would help me track analog's progress, but the > ERRFILE is overwritten every time I run analog. I didn't see a > command line equivalent of ERRFILE in the manual. A command line > equivalent could help analog users who have a large number of runs > or config files by allowing a command line argument like the > following example in Unix > for cfg in /web/utils/analog/cfg/*; do > /web/utils/analog/analog +g$cfg +ERR/tmp/analogerrors/$cfg > done You can add any Analog command to the command line with +C: /web/utils/analog/analog +g$cfg +C"ERRFILE /tmp/analogerrors/$cfg" You can also use redirection on the command line to accomplish this (which is why there's no command-line argument for it): /web/utils/analog/analog +g$cfg 2> /tmp/analogerrors/$cfg Finally, if you want to trace the processing, use the PROGRESSFREQ command to get messages every so often during the run. See http://www.analog.cx/docs/debug.html#PROGRESSFREQ for details -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Internal Hosts only
Madelon Brennan ([EMAIL PROTECTED]): > What would be the best way to structure the "Host Report" so that I only see > the internal IPs (141.161.*). I basically want to get an idea of how may > accesses are coming from links within our site. HOSTREPINCLUDE 141.161.* Or to change all reports to only show that traffic HOSTINCLUDE 141.161.* See http://www.analog.cx/docs/include.html -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Sorting in Request Report not okay (?)
Marcel ([EMAIL PROTECTED]): > hi, > i've installed and configured analog 5.22 > all is fine - but one thing i don't understand. > the request report is configured as following: > REQCOLS Rb > REQFLOOR -40r > REQARGSFLOOR -40r > REQSORTBY REQUESTS > the request report should be sorted by request, but > it looks like this: > This report lists the files on the site. > Listing the top 40 files by the number of requests, sorted by the > number of requests. > reqs: %bytes: file > -: --: > 1342: 0.43%: /css-js/site.css > 1260: 0.48%: /grafiken/valid-html401.gif > 1252: 0.20%: /grafiken/frames_free.gif > 1252: 0.50%: /grafiken/free_speech.gif > 1246: 0.41%: /grafiken/valid-css.gif > 1244: 0.27%: /grafiken/gzip.gif > 957: 0.39%: /css-js/color/blau/farben.css > 923: 0.16%: /grafiken/color/blau/gesicht.gif > 921: 0.78%: /grafiken/color/blau/logo.gif > 921: 0.05%: /grafiken/color/blau/ecke_lo.gif > 918: 0.05%: /grafiken/color/blau/ecke_ro.gif > 915: 0.05%: /grafiken/color/blau/ecke_ru.gif > 915: 0.05%: /grafiken/color/blau/ecke_lu.gif > 727: 2.32%: /software/ftpupper/screenshot.gif > 683: 3.52%: /software/ftpupper/ > 2: : /software/ftpupper/? > 607: 30.77%: /software/ftpupper/download/ftpupper.zip >11: 0.20%: /software/ftpupper/download/ftpupper.zip?Cache > 385: 0.16%: /css-js/color/orange/farben.css > 347: 0.31%: /grafiken/color/orange/logo.gif > 344: 0.02%: /grafiken/color/orange/ecke_ro.gif > 342: 0.02%: /grafiken/color/orange/ecke_lo.gif > 342: 0.06%: /grafiken/color/orange/gesicht.gif > 339: 0.02%: /grafiken/color/orange/ecke_ru.gif > 337: 0.02%: /grafiken/color/orange/ecke_lu.gif > 320: 1.18%: / > 2: 0.01%: /?page=win > 301: 0.06%: /grafiken/board/images/logo.gif > 288: 12.34%: /software/htmlkette/download/htmlkette.zip > 8: 0.15%: /software/htmlkette/download/htmlkette.zip?Cache > 271: 4.16%: /board/topic.cgi > 112: 2.55%: /board/topic.cgi?forum=5&topic=1 >58: 0.58%: /board/topic.cgi?forum=5&topic=2 >20: 0.14%: /board/topic.cgi?forum=2&topic=4 >12: 0.19%: /board/topic.cgi?forum=7&topic=2 >12: 0.07%: /board/topic.cgi?forum=7&topic=4 > 9: 0.21%: /board/topic.cgi?forum=6&topic=1 > 5: 0.03%: /board/topic.cgi?forum=7&topic=6 > 4: 0.03%: /board/topic.cgi?forum=1&topic=1 > 4: 0.04%: /board/topic.cgi?forum=2&topic=1 > 4: 0.02%: /board/topic.cgi?forum=1&topic=2 > 4: 0.03%: /board/topic.cgi?forum=2&topic=2 > 4: 0.05%: /board/topic.cgi?forum=4&topic=1 > 4: 0.04%: /board/topic.cgi?forum=2&topic=3 > 4: 0.05%: /board/topic.cgi?forum=8&topic=1 > 4: 0.03%: /board/topic.cgi?forum=8&topic=3 > 4: 0.03%: /board/topic.cgi?forum=7&topic=5 > 2: 0.02%: /board/topic.cgi?forum=8&topic=2 > 2: 0.01%: /board/topic.cgi?forum=7&topic=3 > 2: 0.02%: /board/topic.cgi?forum=8&topic=4 > 1: 0.01%: /board/topic.cgi?forum=7&topic=1 > 271: 1.35%: /ebay/luefter.jpg > 261: 0.57%: /software/htmlkette/image1.png > 247: 0.01%: /grafiken/board/images/bar.gif > 242: 0.11%: /grafiken/board/images/openfold.gif > 241: 0.02%: /grafiken/board/images/closedfold.gif > 237: 1.23%: /software/ftpupper > 231: 0.04%: /grafiken/board/images/newthread.gif > 212: 0.03%: /grafiken/board/images/emailtofriend.gif > 211: 0.03%: /grafiken/board/images/printpage.gif > 210: 0.03%: /grafiken/board/images/replytothread.gif > 209: 0.10%: /grafiken/board/images/edit.gif > 208: 0.01%: /grafiken/board/images/part.gif > 207: 1.19%: /software/htmlkette/ > 10212: 36.48%: [not listed: 400 files] > there are some groups of items who doesn't fit there. but why ??? > in the sourcecode of the html file there are also 2 spaces before the url > part. > in the logfile there are no extra spaces before! whats going on here? This is exactly what it's supposed to do. Notice that the items with spaces before the names are the same page, with different combinations of arguments. You have told Analog (REQARGSFLOOR) to include to top 40 entries for arguments for each request. If you don't want it to show request arguments you can turn them off with ARGSEXCLUDE *. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Request floor
Craig Westerman ([EMAIL PROTECTED]): > Sounds like there is no way to do what I want AUTOMATICALLY. Sure, you can automatically run Analog as many times as you want. You can have different configuration files or command-line settings each time. It's just going to create three separate html files. If you want those reports all in one file, you'll need to do some scripting to combine them. It's still possible to do it all automatically, though. > My web host runs analog once every night at mid-night. It is not a > cron job that I can find. If it's a Unix system then it probably is a cron-job. It just may not be one you have access to. > How do I run Analog from command line? analog If it's in your path. /path/to/analog On Unix if it's not in your path. Then add command-line parameters as needed. See the entry for "command-line arguments" in the index for a reference to all the possible arguments. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Analog gives different results
Maff Rigby ([EMAIL PROTECTED]): > Hi there, > I ran analog a couple of weeks ago and got a load of results. I then ran it > again today, changing only the time period settings (to run it over the > exact same period as before) and get different results! The date lines are: > First run > # Do only last 16 weeks - 112 > FROM -00-00-112 > 2nd run > # Do specific time period > FROM 011204 > TO 020326:0404 > Both reports show the same time period: > Analysed requests from Tue-04-Dec-2001 10:12 to Tue-26-Mar-2002 04:04 > (111.74 days). > I don't have a clue what has happened - the log files haven't been changed, > so has anyone seen this before or has any suggestions? What results are different, the General Summary or other pages? Are you certain the logfile doesn't contain new entries with dates in that range (sometimes web servers add log entries with invalid dates)? Have you added any EXCLUDE, INCLUDE or ALIAS commands to the configuration file since the original run? -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Request floor
Craig Westerman ([EMAIL PROTECTED]): >>> a Daily report that shows current month only (ie January, not last 30 >> days) > >> DAILYROWS 31 > >> (approximately) > Won't DAILYROWS 31 display last 31 days? Uh...yeah. Sorry, didn't finish reading that line. In that case you'll need to create another report run with that set of data. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Request floor
Craig Westerman ([EMAIL PROTECTED]): > Will FROM -01-00-00 also affect the Monthy and Daily Report? Yes. > What I'm after is a Monthly Report that shows all months and pages (no req), FILEINCLUDE *.html > a Daily report that shows current month only (ie January, not last 30 days) DAILYROWS 31 (approximately) > Daily to be just date and pages (no req) FILEINCLUDE above already does this. > and Request report that shows just > HTML pages requested in a 24 hour period that were requested more than 20 > times? You'll need to do this as a separate run. That's the only way. You are asking Analog to analyze two different sets of data for the time reports and Request Report, so you'll have to do two runs. You can then combine the information into a single reports with a cut and paste. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Request floor
Craig Westerman ([EMAIL PROTECTED]): > How would I retreive just HTML pages requested in a 24 hour period that were > requested more than 20 times? > I have > FILEEXCLUDE */images/* > FILEEXCLUDE /*.gif > FILEEXCLUDE /*.jpg > FILEEXCLUDE /*.png > FILEEXCLUDE */adimages/* > FILEEXCLUDE /*.js > FILEEXCLUDE /*.class > FILEEXCLUDE /*.pl > REQFLOOR 20p > REQFLOOR -01-00-00e > Is there a easier way? FILEINCLUDE *.html REQFLOOR 20p FROM -01-00-00 Assuming no previous FILEINCLUDE/FILEEXCLUDE. Otherwise start with FILEEXCLUDE * Also, FROM/TO control the time period for all reports, not just the request report. You can't have two FLOOR requirements for a report. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] top 100, can this request be fulfilled by the ROWS option
Gonzalo Briceno ([EMAIL PROTECTED]): > Can a top 100 request be carried out by the ROWS option. Furthermore, I > did not find a FAILUREROWS or REFERRERROWS > option on the site. Do these exist? How else can I chose the top x for > a certain report? *ROWS is for time reports. *FLOOR is for other reports. See http://www.analog.cx/docs/othreps.html#FLOOR for details. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Report results comparison (Analog vs Web Trends)
Berigan, Matthew ([EMAIL PROTECTED]): > Hi! > I'm finally past bad log data and I think that I'm getting inconsistencies > between WebTrends and Analog that can't be explained by INCLUDES or > EXCLUDES. > First, here's the summary results: > ANALOG > Program started at Wed-03-Apr-2002 09:38. > Analysed requests from Sun-24-Mar-2002 00:04 to Sat-30-Mar-2002 23:58 (7.00 > days). > Successful requests: 76,490 <<<<<<<<<>>>>>>>>> > Average successful requests per day: 10,933 <<<<<<<<<>>>>>>>>> > Successful requests for pages: 25,765 <<<<<<<<<>>>>>>>>> > Average successful requests for pages per day: 3,682 <<<<<<<<<>>>>>>>>> > Failed requests: 760 > Redirected requests: 680 > Distinct files requested: 2,047 > Distinct hosts served: 5,009 <<<<<<<<<>>>>>>>>> > Corrupt logfile lines: 4 > Data transferred: 792.723 megabytes > Average data transferred per day: 113.313 megabytes > WEBTRENDS > Date & Time This Report was Generated Wednesday April 03, 2002 - 09:17:45 > Timeframe 03/24/02 00:04:41 - 03/30/02 23:58:54 > Number of Hits for Home Page N/A > Number of Successful Hits for Entire Site 77138 <<<<<<<<<>>>>>>>>> > Number of Page Views (Impressions) 25816 <<<<<<<<<>>>>>>>>> > Number of User Sessions 8488 <<<<<<<<<>>>>>>>>> > User Sessions from United States 0% > International User Sessions 0% > User Sessions of Unknown Origin 100% > Average Number of Hits per Day 11019 <<<<<<<<<>>>>>>>>> > Average Number of Page Views Per Day 3688 <<<<<<<<<>>>>>>>>> > Average Number of User Sessions per Day 1212 > Average User Session Length 00:08:36 > Most everything (A-D)appears to be close enough that I don't even consider > it worthy of question. Well, to answer anyway, WT counts some redirects as "successful requests" that Analog does not. > Item E, however, bugs me. Can anybody guess why > Distinct Hosts [Analog] and User Sessions [WT] should be so different? Because they are different quantities. > I think this is supposed to be the same data and my assumption is > that it is the number of distinct IP addresses listed in the log (I > suppose I could count those (ck!!)). "Sessions" is something that WT estimates (and by most accounts very poorly.) It does not represent the number of distinct IP addresses in the logs. It is meant to represent the number of distinct user sessions. Read http://www.analog.cx/docs/webworks.html as to why that's not easily counted and why Analog does not do it. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Operating System Report
Matthew Mencel ([EMAIL PROTECTED]): > Greetings Everyone... > I'm using Analog to create reports for an Icecast MP3 stream that I > broadcast. For the most part it works great. The only issue I have > is that I have something close to 60% of the Operating Systems > showing up as "OS Unknown". I might think this is normal except for > the fact that Windows and it's sub categories (95/98/xp/etc...) are > just about the only entries. There are a few Mac entries and even > some WebTV entries. There are NO Unix/Linux/BSD entries however. > So I'm wondering if I've got something configured incorrectly and it > can't determine some of the OSes (Unix mainly). > I'm thinking there SHOULD be entries for the Unix variants because > under the Browser Summary I have listeners using "XAudio" and "xmms" > which as far as I know are Unix utilities for listening to MP3 > streams. Analog determines the Operating System from the browser's user agent string. Analog is pre-configured to know what to look for and only recognizes a distinct set of OSes in a few major browsers. Analog is unaware of the XAudio and xmms "browsers" because they are not web browsers. Unfortunately, there's not much you can do about this, except perhaps use an alias to change those browsers into something that Analog will recognize the OS of, which may affect your browser stats. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] FILEINCLUDE/EXCLUDE
Maff Rigby ([EMAIL PROTECTED]): > Hi All, > I am having problems excluding .asp files from my request reports. If I do > "FILEEXCLUDE /common/dir/file.asp" and run analog, then the file is still > in the request report! I can exclude the whole directory using "FILEEXCLUDE > /common/dir/*" but I want to specify the file as there are other files in > this directory I want. > Any suggestions? You must have some other INCLUDE statement that is including it. Run analog -settings to see what include statements Analog is processing. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Request Report
SUSANA PUMAR PAVON ([EMAIL PROTECTED]): > I have got a problem with my Request Report. The only file name that > appears is / and it has the 100% of the bytes. It must be wrong. > What can I do? Do you have an INCLUDE or EXCLUDE statement that is removing everything else? Check 'analog -settings' to see what it says for inclusions. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Not recognizing log file
You need all three to support all possible HTTP protocols. If all your requests are HTTP/1.0 or HTTP/1.1 then you only need the first one. The final %j tells Analog to ignore everything until the new line. -- Jeremy Wadsack Wadsack-Allen Digital Group [EMAIL PROTECTED] ([EMAIL PROTECTED]): > Do I need all three or should I just pick one of them? > Also like you I do not care for anything after the referrer so do I need > "%j" for each of the entry's? > Thanks so much because putting all three in worked I just want some > additional information. > John > "Jeremy Wadsack" <[EMAIL PROTECTED]>@lists.isite.net on 04/01/2002 > 04:51:58 PM > Please respond to [EMAIL PROTECTED] > Sent by: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > cc: > Subject: Re: [analog-help] Not recognizing log file > Your email client mangled some of the lines, it looks like. So I can't > tell what LOGFORMAT you already tried but these should work for you > (I'm choosing to ignore everything after the referrrer): > LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%j%w%r%wHTTP%j" %c %b "%f" "%B" > %j) > LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%j%w%r" %c %b "%f" "%B" %j) > LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%r" %c %b "%f" "%B" %j) > NOTE that each of those should be on one line starting with LOGFORMAT. > -- > Jeremy Wadsack > Wadsack-Allen Digital Group > [EMAIL PROTECTED] ([EMAIL PROTECTED]): >> Here it a sample of my log file. Analog says it cannot recognise the >> format. I need help filling out the LogFile Format command. This is > what >> I have tried and analog says it does not recognize the format >> 203.124.0.247 - - [01/Dec/2001:09:00:02 -0600] "GET > /images/template/sub_abjournal01.gif HTTP/1.1" 200 2565 "http://www.ab.com/ > " "Mozilla/4.0 >> (compatible; MSIE 5.5; Windows 98; MYSINGTEL)" GET > /images/template/sub_abjournal01.gif - "HTTP/1.1" 200 2565 - - 305 223 390 > 223 0 DIRECT FIN FIN >> DO-NOT-CACHE - >> (%s -%j- [%d/%M/%Y:%h:%n:%j %j] "%j %r > %j" %c %b "%f" "%B" >>%j %j > "%j" %j %j -%j- %j %j %j %j %j %j%j %j %j -) >> 203.124.0.247 - - [01/Dec/2001:09:00:02 -0600] "GET > /images/template/vertbar.gif HTTP/1.1" 200 71 "http://www.ab.com/"; > "Mozilla/4.0 (compatible; MSIE >> 5.5; Windows 98; MYSINGTEL)" GET /images/template/vertbar.gif - > "HTTP/1.1" 200 71 - - 297 221 382 221 0 DIRECT FIN FIN DO-NOT-CACHE - >> 213.42.0.243 - - [01/Dec/2001:09:00:02 -0600] "GET > /images/template/ltgray.gif HTTP/1.0" 200 43 "http://www.ab.com/"; > "Mozilla/4.0 (compatible; MSIE >> 5.5; Windows 98; Win 9x 4.90)" GET /images/template/ltgray.gif - > "HTTP/1.0" 200 43 - - 292 221 377 221 0 DIRECT FIN FIN DO-NOT-CACHE - >> 213.42.0.243 - - [01/Dec/2001:09:00:02 -0600] "GET > /images/template/shadowdown.gif HTTP/1.0" 200 79 "http://www.ab.com/"; > "Mozilla/4.0 (compatible; >> MSIE 5.5; Windows 98; Win 9x 4.90)" GET /images/template/shadowdown.gif - > "HTTP/1.0" 200 79 - - 296 221 381 221 0 DIRECT FIN FIN DO-NOT-CACHE - + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Memory control?
D. Alvarado ([EMAIL PROTECTED]): > Our network admin was running an analog report on our Unix server > which had 4GB RAM and he noticed that a particular analog process > took up 3.8 MB RAM. My question is: is there any way to put a cap > on the maximum amount of memory analog utilizes? Perhaps with some Unix tools, but not with Analog. Analog does have memory control configuration commands through: http://www.analog.cx/docs/lowmem.html. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Not recognizing log file
Your email client mangled some of the lines, it looks like. So I can't tell what LOGFORMAT you already tried but these should work for you (I'm choosing to ignore everything after the referrrer): LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%j%w%r%wHTTP%j" %c %b "%f" "%B" %j) LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%j%w%r" %c %b "%f" "%B" %j) LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j] "%r" %c %b "%f" "%B" %j) NOTE that each of those should be on one line starting with LOGFORMAT. -- Jeremy Wadsack Wadsack-Allen Digital Group [EMAIL PROTECTED] ([EMAIL PROTECTED]): > Here it a sample of my log file. Analog says it cannot recognise the > format. I need help filling out the LogFile Format command. This is what > I have tried and analog says it does not recognize the format > 203.124.0.247 - - [01/Dec/2001:09:00:02 -0600] "GET >/images/template/sub_abjournal01.gif HTTP/1.1" 200 2565 "http://www.ab.com/"; >"Mozilla/4.0 > (compatible; MSIE 5.5; Windows 98; MYSINGTEL)" GET >/images/template/sub_abjournal01.gif - "HTTP/1.1" 200 2565 - - 305 223 390 223 0 >DIRECT FIN FIN > DO-NOT-CACHE - > (%s -%j- [%d/%M/%Y:%h:%n:%j %j] "%j %r >%j" %c %b "%f" "%B" >%j %j > "%j" %j %j -%j- %j %j %j %j %j %j%j %j %j -) > 203.124.0.247 - - [01/Dec/2001:09:00:02 -0600] "GET /images/template/vertbar.gif >HTTP/1.1" 200 71 "http://www.ab.com/"; "Mozilla/4.0 (compatible; MSIE > 5.5; Windows 98; MYSINGTEL)" GET /images/template/vertbar.gif - "HTTP/1.1" 200 71 - >- 297 221 382 221 0 DIRECT FIN FIN DO-NOT-CACHE - > 213.42.0.243 - - [01/Dec/2001:09:00:02 -0600] "GET /images/template/ltgray.gif >HTTP/1.0" 200 43 "http://www.ab.com/"; "Mozilla/4.0 (compatible; MSIE > 5.5; Windows 98; Win 9x 4.90)" GET /images/template/ltgray.gif - "HTTP/1.0" 200 43 - >- 292 221 377 221 0 DIRECT FIN FIN DO-NOT-CACHE - > 213.42.0.243 - - [01/Dec/2001:09:00:02 -0600] "GET /images/template/shadowdown.gif >HTTP/1.0" 200 79 "http://www.ab.com/"; "Mozilla/4.0 (compatible; > MSIE 5.5; Windows 98; Win 9x 4.90)" GET /images/template/shadowdown.gif - "HTTP/1.0" >200 79 - - 296 221 381 221 0 DIRECT FIN FIN DO-NOT-CACHE - + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] New to this stuff. Does anyone know the answer to this?
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > Hello, > I am new to the process of analyzing web logs and to Analog 5.22. > Can anyone help me with the following questions? > 1.) Can Analog display the number of successful and unsuccessful page > requests for a specific page per day? > 2.) If so, what is the command. Use FILEINCLUDE to tell Analog to create a report on for a single file. See http://www.analog.cx/docs/include.html for details. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] help
Gianni Accogli ([EMAIL PROTECTED]):
> Hi,
> i use analog 5.21-1 and i have this problem when i start it:
> analog: Warning L: Large number of corrupt lines in logfile
> /home/sites/site77/web/weblogs/access.log: turn debugging on or try different
> LOGFORMAT in my httpd.conf is this:
> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" %{User-Agent}i\" combined
There is a problem with this format. In fact Apache shouldn't even
process this line. There's no trailing quote to match the opening one.
Usually 'combined' format is defined like this:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
If you have actually used that format (and, for some reason, Apache
accepted if) you may have trouble processing your logs, as the User
Agent field will not be quoted. You may be able to match it with an
APACHELOGFORMAT command in Analog or you may want to process those log
files with a tool like Perl or sed to clean them up.
> LogFormat "%h %l %u %t \"%r\" %>s %b" common
> LogFormat "%{Referer}i -> %U" referer
> LogFormat "%{User-agent}i" agent
Which of these is in use? (Look for the CustomLog command.)
> and in my analog.cfg i have this:
> LOGFORMAT COMBINED
> LOGFORMAT COMMON
> LOGFORMAT BROWSER
Analog should auto-detect any of these anyway.
> LOGFORMAT REFERER
REFERRER is spelled with two R's in the middle.
> LOGFORMAT ([%d/%M/%Y:%h:%n:%j] %B)
This only provides browser information and doesn't match any of the
LogFormat lines you show above,
> how can i resolve my problem?
First try it without any *LOGFORMAT commands. Analog will autodetect
common and combined format and many others.
If that doesn't work, use APACHELOGFORMAT command with the format from
above that is in use.
Full details on the *LOGFORMAT commands can be found here:
http://www.analog.cx/docs/logfmt.html.
--
Jeremy Wadsack
Wadsack-Allen Digital Group
+
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
|http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
|http://www.mail-archive.com/analog-help@lists.isite.net/
|http://lists.isite.net/listgate/analog-help/archives/
|http://www.tallylist.com/archives/index.cfm/mlist.7
+
Re: [analog-help] Wild Cards in Log files (linux) --really confused
Stephen Turner ([EMAIL PROTECTED]): > On Wed, 27 Mar 2002, Jeremy Wadsack wrote: >> >> > I re-downloaded the source tar and replaced my anlgform.pl. Again, I'm >> > confused because it works on the linux command line, but not in my web >> > browser. >> >> For security, the web interface does not support wildcards in log file >> specifications. See http://www.analog.cx/docs/form.html#security. >> > No, that's not true. It does accept them from 5.0. Have you tried with just > a single named logfile? Maybe it's a permissions issue -- remember the web > server is probably running as special user www or something. I suppose that's why I found no mention of that limitation in the docs. :-) -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Expanding folders using DIRCHARTEXPAND
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > Hello! > My first post to the list so I hope I'm following any unwritten (and > written) codes of conduct... > Situation: > www.thegrid.org.uk/learning/ has several subfolders... > www.thegrid.org.uk/learning/re/ > www.thegrid.org.uk/learning/ict/ > www.thegrid.org.uk/learning/mfl/ > but I still get a single pie-chart with no sub-categories. The top of the > output Directory Report looks like: > reqs: %bytes: directory > --: --: - > 146386: 100%: /learning/ > 50365: 6.14%: /learning/re/ > 28236: 9.88%: /learning/literacy/ > 13891: 35.06%: /learning/ict/ > I have read the manual as best I can and searched the list archives, but I > can't seem to pinpoint my problem. I get no error messages (other than > 'empty pie chart' and 'one wedge pie chart' messages) when I run analog > from the command line in a command window under Win2000. Use a *CHARTEXPAND command, http://www.analog.cx/docs/hierreps.html#CHARTEXPAND. You will need to be running Analog 5.2 or later. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Wild Cards in Log files (linux) --really confused
Joe Petsche ([EMAIL PROTECTED]): > Alright, I'm really confused now. I've checked my settings with --settings > and everything seems to be good (It's seeing all the logfiles). > I run either /usr/bin/analog or my perl webscript anlgform.pl and I get > output that is consistent with my statistics. > However, as soon as I open my web brower and run the anlgform.pl script I > get back a page that has very little information (Title, Gerneral > Summary[with 0 successful requests] and footer information). > I re-downloaded the source tar and replaced my anlgform.pl. Again, I'm > confused because it works on the linux command line, but not in my web > browser. For security, the web interface does not support wildcards in log file specifications. See http://www.analog.cx/docs/form.html#security. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] sorting special files by size second try.....
Aengus ([EMAIL PROTECTED]): > "Jeremy Wadsack" <[EMAIL PROTECTED]> wrote: >> Aengus ([EMAIL PROTECTED]): >> >> > I think your best bet will be to grep the logs for matching >> > requests. Does anyone know how to create a regular expression >> > to pick out lines with a field that is between 2 numerical values? >> >> You have to know the numbers. To get a field that is between 200 and >> 500 (inclusive) bytes, use something like this: >> >> [2345]00 > That will only find request for 200, 300, 400 or 500 bytes, it won't find > a request for 345 bytes (as far as I know - I still don't do regular > expressions!). Oops! You're right, of course. > Carsten is looking for kbytes, but the principal is the > same. > I suppose [2-5][0-9][0-9][0-9][0-9][0-9] find 20 to 50 > inclusive, but can you tell grep to only look at the nth field in a > file? > Maybe you need perl (or even sed) to do this? Perl or sed would be easier. You can't tell grep to look at the nth field. You might be able to use variables and cut, but that would get complicated. You could create a regular expression for grep if you knew the surrounding data well enough that you could define only one location in the file that the byte count could be found. Or you could define the whole log format with a regular expression as Stephen's anonlog does for some kinds of formats (common, combined and iis.). -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Counting multiple fields
You are getting both:
User | Reqs
--
[not listed: 435,765] | 1,276,501
The number in the 'not listed' tells you how many unique cookies were
given. The number in the Reqs column tells you how many total
(cookied) requests cookied users made.
(5) As I mentioned before, I think you should be able to do that with
a LOGFORMAT command that includes references to both cookies. You
would 'hijack' two variables and associated reports for this. User and
Virtual Host being most likely. You might have to use two command as
the cookies could be in different orders in different requests:
LOGFORMAT ...%jidl_visitor=%u;%jidl_visit=%v;%j...
LOGFORMAT ...%jidl_visit=%v;%jidl_visitor=%u;%j...
(2) Once a user receives a cookie, it should be sent on all requests.
If you track request and response cookie headers in your log format
you can get cookied information for all requests. If cookies are
handled by your server, then they should be sent for graphics requests
too.
--
Jeremy Wadsack
Wadsack-Allen Digital Group
[EMAIL PROTECTED] ([EMAIL PROTECTED]):
> Okay, let me put it this way:
>I'm interested in two cookies, both of which have unique values for each
>visitor
>These cookies will not be present in every line on the logfile (this
>isn't causing problems so far) as they're set only by pages
>I don't want to count the number of times these cookies appear in the
>logfile
>I do want to count the number of unique values for each cookie in the
>logfile
>I would like to identify the two numbers in different reports by running
>Analog once
> What I think I'm getting at the moment is (3): the number of times the text
> string "idl_visit=" or "idl_visitor=" appears in the logfiles. How can I
> get (4)?
> thanks, Adam
> - Forwarded by Adam Browne/Digital/Independent News and Media on
> 26/03/02 16:53 -
> "Jeremy Wadsack"
>
> [EMAIL PROTECTED]
> .com> cc:
>
> Sent by: Subject: Re: [analog-help]
>Counting multiple fields
> owner-analog-help@lists
>
> .isite.net
>
> 25/03/02 15:19
>
> Please respond to
>
> analog-help
>
> I'm not sure if I'm following you, but if you're saying you have lines
> with this cookie and lines that don't have the cookie, then you'll
> need to add an additional LOGFORMAT to read the lines that don't have
> the cookie (otherwise Analog will tell you they are corrupt.)
> I had assumed you had already done this, as you would have needed to
> in your original format anyway.
> --
> Jeremy Wadsack
> Wadsack-Allen Digital Group
> [EMAIL PROTECTED] ([EMAIL PROTECTED]):
>> Sorry, I wasn't clear. It gives the total number of _all_ cookies - what
> I
>> need is the total number of the cookies individually. In other words, the
>> %jidl_visitor=%u;%j string doesn't seem to be making any difference.
>> Adam
+
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
|http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
|http://www.mail-archive.com/analog-help@lists.isite.net/
|http://lists.isite.net/listgate/analog-help/archives/
|http://www.tallylist.com/archives/index.cfm/mlist.7
+
Re: [analog-help] sorting special files by size second try.....
Aengus ([EMAIL PROTECTED]): > I think your best bet will be to grep the logs for matching requests. Does > anyone know how to create a regular expression to pick out lines with a > field that is between 2 numerical values? You have to know the numbers. To get a field that is between 200 and 500 (inclusive) bytes, use something like this: [2345]00 -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Automating analog
I don't have a Solaris box at my disposal, so I can't really help you any further. Maybe someone else with S-2.6 can suggest an alternate approach. The only other suggestions I can think of is to code it by hand: write in the name of each log file and then run the command. Being only 7 files, it's not that much work. -- Jeremy Wadsack Wadsack-Allen Digital Group SUSANA PUMAR PAVON ([EMAIL PROTECTED]): > I'm running Analog on Solaris 2.6. I think that is the problem. I don´t > know how to treat the relative dates. I´ve looked it up with the command > 'man date' but it didn't help me. > Thanks anyway. > -Mensaje original- > De: Jeremy Wadsack [mailto:[EMAIL PROTECTED]] > Enviado el: martes, 26 de marzo de 2002 16:21 > Para: [EMAIL PROTECTED] > Asunto: Re: [analog-help] Automating analog > Well, the shell script was just an off-the-cuff suggestions. But I see > no reason why it wouldn't work. Did you put it all on one line or make > sure the include the '\' to tell your shell that the line continues? > I also don't know what shell you are using, but I tested this on bash. > Finally, it's possible that Solaris's date command doesn't support the > relative dates or formats in the same way. Check 'man date' for > details. + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: AW: [analog-help] sorting special files by size second try.....
Carsten Böttcher ([EMAIL PROTECTED]): > That would be finei don`t need more than just the number of users ore > files in this range or floor... It won't work that way. *FLOOR commands only apply to reports and you can only apply one FLOOR per report. So you can limit the request report to all files > 200kb OR < 500kb. Or you can limit the user report to users who downloaded > 200kb OR < 500kb. But you can't limit the user report to users who downloaded requests > 200kb and you can't set a range (> and <). > have you an example of configuring a "floor" in the config-file ?? http://www.analog.cx/docs/othreps.html#FLOOR -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Automating analog
Well, the shell script was just an off-the-cuff suggestions. But I see no reason why it wouldn't work. Did you put it all on one line or make sure the include the '\' to tell your shell that the line continues? I also don't know what shell you are using, but I tested this on bash. Finally, it's possible that Solaris's date command doesn't support the relative dates or formats in the same way. Check 'man date' for details. -- Jeremy Wadsack Wadsack-Allen Digital Group SUSANA PUMAR PAVON ([EMAIL PROTECTED]): > I tried to run Analog with your shell script but unfortunately it > doesn't work. It only reads the logs from today. Any suggestion? > -Mensaje original- > De: Jeremy Wadsack [mailto:[EMAIL PROTECTED]] > Enviado el: lunes, 25 de marzo de 2002 16:24 > Para: [EMAIL PROTECTED] > Asunto: Re: [analog-help] Automating analog > SUSANA PUMAR PAVON ([EMAIL PROTECTED]): >> What I'd like to do is to get information from the logs of the last >> seven days. The way I try to do that is as follows: >> FROM -00-00-07 >> TO -00-00-01 >> LOGFILE /log/http/iobox.com/access.log-%Y-%M* >> The problem is that Analog reads all the log files of the current > month, >> although it only gets the information from those of the last seven > days, >> so it goes slower. There is another problem at the begining of the >> month, because it won't read the log files of the last seven days, > only >> those that belong to this month. >> If I don't write %M in the LOGFILE sentence it runs too slow. >> Could you please help me? > You will need to write a shell script to feed Analog the last seven > days' log files from the command line. Analog has no ability to figure > this out within its command structure. > Depending on your version of 'date' you may be able to use something > like this: > ./analog \ > /log/http/iobox.com/access.log-`date +%Y-%m-%d -d` \ > /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-1 day'` \ > /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-2 days'` \ > /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-3 days'` \ > /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-4 days'` \ > /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-5 days'` \ > /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-6 days'` + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] BIND Log-File analysis
Phil White ([EMAIL PROTECTED]): > As a very quick question, > before I attempt to sort this out myself, has anyone attempted to analyze a > BIND (v9) log file using analog? > If so, can you let me know how it might be best achieved? > (for reference: Log file output = > Feb 02 22:50:10.892 client 195.25.255.1#53: query: microsoft.com IN SOA > Feb 02 22:50:11.012 client 195.25.255.1#53: query: www.microsoft.com IN A > Current thoughts: > there is no YEAR presentin the current log. Should be able to change this. > There is no status code. I don't think Analog will process a log file without either of these. You may need to pre-process it to add these fields. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Automate Output log format
Susanne Ma ([EMAIL PROTECTED]): > Running IIS5 and trying out Analog to generate logs for users viewing. > How do I designate the CFG file to pull the logfile > EX<2-digit_current_year<2-digit_current_month.log? It doesn't appear > to like ex%y%M.log format. I would like to automate generating a > new report for every month. Analog does not support %y%M in log file statement. You can specify the log file on the command line. You may be able to write a batch file to automate this. Check the list archives for examples. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Help Understanding Bug Found
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I am trying to decode what was meant by this: > It is easy for an attacker to insert arbitrary strings into any web > server logfile. If these strings are then analysed by analog, they can > appear in the report. By this means an attacker can introduce > arbitrary Javascript code, for example, into an analog report produced > by someone else and read by a third person. Analog already attempted > to encode unsafe characters to avoid this type of attack, but the > conversion was incomplete. > Can someone give me an example of how this is or can be a real problem. It is basically an extension if this exploit: http://www.cert.org/advisories/CA-2000-02.html > I am not a JAVA developer but have an extensive C/UNIX background. Java has no bearing on this. JavaScript is a different language entirely and can be embedded in HTML pages to be run on the client. http://developer.netscape.com/js/ -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] File size floors?
Szymon Nitka ([EMAIL PROTECTED]): > I can see in the file size report: > 0: ... >1b- 10b: ... > 11b- 100b: ... > 101b- 1kb: ... > 1kb- 10kb: ... > 10kb-100kb: ... > 100kb- 1Mb: ... > 1Mb- 10Mb: ... > 10Mb-100Mb: ... > Is it possible to change size floors? I can't see > it at http://www.analog.cx/docs/Readme.html... What would such a FLOOR command effect? The histogram shows the distribution of files by size. I'm not sure why you would not want to include all buckets in that report. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] logformat
j.bottoms ([EMAIL PROTECTED]): > I'm still havin trouble understanding logformats so here is a question > ascii27.net myl-c-24-159-181-226.chartertn.net - - [10/Mar/2002:14:52:09 -0500] "-" >408 - 0 - "-" "-" - - > the above is a line from my log file. By looking at it it should break down in this >manner. > (%v %S %j %j [%d/%M/%Y:%h:%n:%j %j] "%r" %c %j %b "%v" "%B) > Is my understanding correct? Tom, You'll need to provide a few more lines of your log file before we can guess what all the '-' fields are. But most apparently, your log format has a " before the %B and no tailing one, while the log file line you show has trailing quote and two more fields. There is also another field between the end of the first quoted on (presumably the request field) and the second quoted one. You have two fields marked as %v. The first looks likely to be correct. The second I would guess should be a referrer (%f.) -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Error in script or dumb newbie?
Chris, This would appear to be an error in your shell script. As you didn't include the script I can't give you any suggestions on it but it appears you have a '[' where bash does not want it. -- Jeremy Wadsack Wadsack-Allen Digital Group Chris McQuade ([EMAIL PROTECTED]): > Hi all. I am quite a newbie with this. I have gotten all setup perfectly > so far but I have one problem. When I go and try to run the script > ./fwanalog it fails with an error "./fwanalog.sh[828]: [: analog:: > unexpected operator/operand". It fails with this 3 times. I checked the > directory and see that the fwanalog.all.log is created from my ipflog in > /var/log/ . What gives? I have even run it as root as well to see if > this would correct the issue. I get the same over and over any help > would be appreciated. Further info below: > ERROR: > bash# ./fwanalog.sh > ./fwanalog.sh[828]: [: analog:: unexpected operator/operand > ./fwanalog.sh[828]: [: analog:: unexpected operator/operand > ./fwanalog.sh[828]: [: analog:: unexpected operator/operand > bash# > ENVIRONMENT: > OpenBSD > Thanks, > Chris + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Automating analog
SUSANA PUMAR PAVON ([EMAIL PROTECTED]): > What I'd like to do is to get information from the logs of the last > seven days. The way I try to do that is as follows: > FROM -00-00-07 > TO -00-00-01 > LOGFILE /log/http/iobox.com/access.log-%Y-%M* > The problem is that Analog reads all the log files of the current month, > although it only gets the information from those of the last seven days, > so it goes slower. There is another problem at the begining of the > month, because it won't read the log files of the last seven days, only > those that belong to this month. > If I don't write %M in the LOGFILE sentence it runs too slow. > Could you please help me? You will need to write a shell script to feed Analog the last seven days' log files from the command line. Analog has no ability to figure this out within its command structure. Depending on your version of 'date' you may be able to use something like this: ./analog \ /log/http/iobox.com/access.log-`date +%Y-%m-%d -d` \ /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-1 day'` \ /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-2 days'` \ /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-3 days'` \ /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-4 days'` \ /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-5 days'` \ /log/http/iobox.com/access.log-`date +%Y-%m-%d -d '-6 days'` -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Counting multiple fields
I'm not sure if I'm following you, but if you're saying you have lines with this cookie and lines that don't have the cookie, then you'll need to add an additional LOGFORMAT to read the lines that don't have the cookie (otherwise Analog will tell you they are corrupt.) I had assumed you had already done this, as you would have needed to in your original format anyway. -- Jeremy Wadsack Wadsack-Allen Digital Group [EMAIL PROTECTED] ([EMAIL PROTECTED]): > Sorry, I wasn't clear. It gives the total number of _all_ cookies - what I > need is the total number of the cookies individually. In other words, the > %jidl_visitor=%u;%j string doesn't seem to be making any difference. > Adam >> Jeremy Wadsack wrote: >Isn't that what you want to do? Count the total number of unique > cookies for the period or log files? >-- >Jeremy Wadsack >Wadsack-Allen Digital Group >[EMAIL PROTECTED] ([EMAIL PROTECTED]): >> I thought something like this would be the solution - but I can't seem >to >> get it to work. I've made a specific logfile (W3C format): >>LOGFORMAT >>%Y-%m-%d\t%h:%n%:%j\t%s\t%j\t%r\t%c\t%b\t%t\t%f\t%B\t%jidl_visitor >=%u;%j >> but it just gives me the total number of cookies during the period. >Any >> more suggestions? >> I haven't attempted hijacking another variable yet, but the FLOOR >setting >> works fine. >> thanks, Adam + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Can Analog handle Redirects as Requests?
Did you read the FAQ item that I linked to? That's the double counting I'm talking about. -- Jeremy Wadsack Wadsack-Allen Digital Group Michael Rabe ([EMAIL PROTECTED]): > Thanks for the feedback - could you elaborate on the "double counting"? > What is that potential? A lot of the redirects I see correspond to POSTS > to a servlet (which appears like a directory in the Url field of the log) > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED]]On Behalf Of Jeremy Wadsack > Sent: Friday, March 22, 2002 2:22 PM > To: [EMAIL PROTECTED] > Subject: Re: [analog-help] Can Analog handle Redirects as Requests? > Michael Rabe ([EMAIL PROTECTED]): >> Is there any way to break down Analog's Redirects report into the Monthly, >> Daily, >> and Hourly summaries as it does for the Requests? >> I would like to filter on log entries using FILEINCLUDE and REFINCLUDE >> config >> entries, but it seems that analog does not include a redirect as a > request. >> Ideally, if every entry was treated as a request, I would be fine. But, > to >> my >> knowledge, Analog is too smart and treats all the entries with HTTP 300 >> status codes as redirects. >> My current workaround is to use "sed" to change all the "300" status codes >> into 200s, then run the appropriate Analog reports. >> Any thoughts on this? > http://www.analog.cx/docs/faq.html#faq181 > If you are ok with the potential double counting on directories, then > your sed approach is probably the best. > -- > Jeremy Wadsack > Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Can Analog handle Redirects as Requests?
Michael Rabe ([EMAIL PROTECTED]): > Is there any way to break down Analog's Redirects report into the Monthly, > Daily, > and Hourly summaries as it does for the Requests? > I would like to filter on log entries using FILEINCLUDE and REFINCLUDE > config > entries, but it seems that analog does not include a redirect as a request. > Ideally, if every entry was treated as a request, I would be fine. But, to > my > knowledge, Analog is too smart and treats all the entries with HTTP 300 > status codes as redirects. > My current workaround is to use "sed" to change all the "300" status codes > into 200s, then run the appropriate Analog reports. > Any thoughts on this? http://www.analog.cx/docs/faq.html#faq181 If you are ok with the potential double counting on directories, then your sed approach is probably the best. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Counting multiple fields
Isn't that what you want to do? Count the total number of unique cookies for the period or log files? -- Jeremy Wadsack Wadsack-Allen Digital Group [EMAIL PROTECTED] ([EMAIL PROTECTED]): > I thought something like this would be the solution - but I can't seem to > get it to work. I've made a specific logfile (W3C format): >LOGFORMAT >%Y-%m-%d\t%h:%n%:%j\t%s\t%j\t%r\t%c\t%b\t%t\t%f\t%B\t%jidl_visitor=%u;%j > but it just gives me the total number of cookies during the period. Any > more suggestions? > I haven't attempted hijacking another variable yet, but the FLOOR setting > works fine. > thanks, Adam > Jeremy Wadsack wrote: >> So, two questions: >> 1. How can I count the number of instances of a cookie? > You'll need a more specific LOGFORMAT, if that's possible. The > LOGFORMAT needs to look for the cookie name that you want to track and > dumb the rest of the cookies. Something like this (within the cookie > field) might do the trick: > %jidl_visitor=%u;%j >> 2. How can I run Analog once and get the same report twice with two >> different *INCLUDE values? > You can't. You can hijack another variable, such as the virtual host > one, if you don't need it, and use the Virtual Host Report to track > the other field. Perhaps something like this in your LOGFORMAT: > %jidl_visitor=%u;%jidl_visit=%v;%j > Also, you can set the FLOOR for the User (and VHost) reports to > something very large to get all the values totaled in the [not listed] > section. In your report above USERFLOOR 2000r would do the trick. + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Counting multiple fields
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > For each page served, our webserver creates two cookies with unique IDs. > One is persistent (to "identify" v***tors), and the other expires 30 > minutes after the end of the last page requested (to "identify" v***ts). I > need to count the number of individual instances for each cookie over a > given period. > By using the USERREPINCLUDE argument, I can get a Users report which tells > me how many pages/requests/bytes have been accessed by one or the other of > these, but that's not what I'm looking for. The only way I can do it at the > moment is to run Analog twice (once for each cookie) and refer to the [not > listed: XXX,XXX users] note at the end of each report. > The output (values abbreviated for clarity) is: >pages: user >-: > 1087: "idl_visitor=xxx; idl_visit=yyy" > 686: "idl_visitor=xxx; idl_visit=yyy" > 679: "idl_visitor=xxx; idl_visit=yyy" > 679: "idl_visitor=xxx; idl_visit=yyy" > 674: "idl_visitor=xxx; idl_visit=yyy" > 660: "idl_visitor=xxx; idl_visit=yyy" > 640: "idl_visitor=xxx; idl_visit=yyy" > 636: "idl_visitor=xxx; idl_visit=yyy" > 607: "idl_visitor=xxx; idl_visit=yyy" > 584: "idl_visitor=xxx; idl_visit=yyy" >69725: [not listed: 110,732 users] > What I'd like is something like: >instances: user >-: > 123087: idl_visitor > 834586: idl_visit > So, two questions: > 1. How can I count the number of instances of a cookie? You'll need a more specific LOGFORMAT, if that's possible. The LOGFORMAT needs to look for the cookie name that you want to track and dumb the rest of the cookies. Something like this (within the cookie field) might do the trick: %jidl_visitor=%u;%j > 2. How can I run Analog once and get the same report twice with two > different *INCLUDE values? You can't. You can hijack another variable, such as the virtual host one, if you don't need it, and use the Virtual Host Report to track the other field. Perhaps something like this in your LOGFORMAT: %jidl_visitor=%u;%jidl_visit=%v;%j Also, you can set the FLOOR for the User (and VHost) reports to something very large to get all the values totaled in the [not listed] section. In your report above USERFLOOR 2000r would do the trick. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] exclusions for successful requests
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I'm using Analog 5.1 on Apache logs, and have just added a list of > Bots to my ROBOTINCLUDE, REQEXCLUDE and BROWSUMEXCLUDE directives. > The number of requests by "robots" in the Operating System report > has jumped, which is good, but nothing has happened to the > "Successful requests" and "Successful requests for pageviews" under > the General Summary. I'm hoping to decrease requests here by a > similar amount . . . > What sort of EXCLUDE can I use to exclude requests from these user > agents in the "Successful requests" summaries? FILEEXCLUDE and BROWEXCLUDE will affect the numbers in the General Summary along with ALL OTHER REPORTS. These will tell Analog to ignore all log file lines that contain the matching criteria. If you only want to remove those hits from the Request Report and Browser Summary, stay with the EXCLUDEs you have and accept the fact that the General Summary (and all other reports) are including that traffic. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] domain not given
SUSANA PUMAR PAVON ([EMAIL PROTECTED]): > Why does this sentence [domain not given] appear when I run Analog in > the ORGANISATION and DOMAIN report? > I don't really understand it well. It means the the IP number of the host did not resolve to any domain name, so it can't be listed as a domain or organization. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Need some help with errors
Kevin Everts ([EMAIL PROTECTED]): > When I run reports, I am getting these errors: > analog.exe: Warning M: Logfile c:\WINNT\System32\logfiles\w3svc3\ex011216.log > contains lines with no bytes: byte counts may be low > analog.exe: Warning M: Logfile c:\WINNT\System32\logfiles\w3svc3\ex011217.log > contains lines with no bytes: byte counts may be low > analog.exe: Warning M: Logfile c:\WINNT\System32\logfiles\w3svc3\ex011218.log > contains lines with no bytes: byte counts may be low These are not errors, they are warnings. > Is this affecting my reports at all? I am using IIS5 and W3C logs. Thanks! It's only affecting reports in that the byte columns, if you have any, may not represent bytes for all requests in the request column. It's a warning to let you know that the values may not be consistent, but it still lists counts all requests. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] ancrawl-1.04
Phil White ([EMAIL PROTECTED]): > Been playing with analog for a little while now, and last night > decided to take a look at the ancrawl spider. > Assuming it was taking a gental perusal of my site, I left it to run > overnight. Ha! This afternoon, as it was STILL running, I killed it, and > decided to actually read the code. By default AnalogCrawler will pause 5 seconds between each request to the server. If you have a 15,000 page site, it could easily take 20 hours to crawl it. > Not surprising it had got lost. There is no support for either the > /robots.txt file, or the robots meta tag. AnalogCrawler does have support for /robots.txt and should behave by the rules of the Standard for Robot Exclusion. Technically, AnalogCrawler implements the LWP::RobotUA Perl module which should handle the processing and implementation of the rules in your /robots.txt file. > It had got lost in an endless loop of dynamically generated pages! This is a more sticky problem. If your site dynamically generates pages that can produce new, unique URLs, then the crawler (or any robot, for that matter) can't really know where to stop. For that matter, it would seem possible that your visitors could hit any of those pages as well and therefore you'd want the title information from AnalogCrawler to correlate with the request information in your log files. The best solution is to rework the dynamic section of your site to generate the same URL for any page that produces the same content. AnalogCrawler will keeps track of the pages it has crawled so it will not get lots in link loops. There must be a finite amount of information that your site can generate and therefore there should be an end to the dynamic pages at some point. > Question: Has anyone addressed this, or am I going to have to start from > scratch? If you are certain that the crawler is not behaving properly, can you contact me off the list with more information about your site, so I can work out any bugs that may be in the software? Thanks, -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Domain Resolution
Mark Challender ([EMAIL PROTECTED]): > I am also using Report Magic but get unresolved names. I can't figure out > how to get these names resolved. This can be done by Analog with 'DNS WRITE' command. See http://www.analog.cx/docs/dns.html.; Or you can use one of the third-party tools on the Helper Applications page, http://www.analog.cx/helpers/.; -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Domain Resolution
Mark Challender ([EMAIL PROTECTED]): > I've looked for a way to show the unique visitors through resolved Domain > names but can't seem to find anything. > Can someone help? Analog shows Unique Hosts in the General Summary. Is this what you're looking for? If you want actual visitors, you can't really get that number. See http://www.analog.cx/docs/webworks.html for reasons why. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Excluding all imnage files
Madelon Brennan ([EMAIL PROTECTED]): > I would like to exclude all image files. I have the following code in > the analog.cfg file: > PAGEEXCLUDE *.gif > PAGEEXCLUDE *.jpg > PAGEEXCLUDE *.css > PAGEEXCLUDE *.js This just makes those not included in the definition of 'pages.' It doesn't change what's listed in the Request Report. > but the "Request Report" is still listing image files. How can I do > this? If you want to remove all traffic generated by images from your reports, then use this: FILEEXCLUDE *.gif,*.jpg,*.css,*.js If you want to just remove those entries from the Request Report, then use this: REQEXCLUDE *.gif,*.jpg,*.css,*.js -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] anlgform.pl IIS5.1 configuration
John Pavel ([EMAIL PROTECTED]): > I am having trouble configuring the CGI interface to analog on IIS5.1 > (XP). > The result that I get is the outputting of a downloadable file > (attached). cgi-bin>>perl anlgform.pl > Content-Type: text/plain > x VE analog 5.21 This is COMPUTER output format. You probably want HTML for your browser to read it. Change your Analog configuration file to OUTPUT HTML (or remove the OUTPUT COMPUTER line) or add that directive to your form. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Organization Report Help
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > would like to get a description (organization name) associated with the ip's in > the organization report. > So it currently shows as this in the dat file: > Z lRr 1 1772715 34.026 155.118 > Z lRr 1 68455013.140 143.21 > I would like to see a text string associated with it so if 155.118 is the >accounting dept and 143.21 is the finance dept, > then can I see in the output as: > Z lRr 1 1772715 34.026 155.118 Accounting Dept > Z lRr 1 68455013.140 143.21Finance Dept > Can analog now link an Organization name to the IP or is there a way > to do this within Analog? I answered this question when you posted it yesterday, Michael: http://www.mail-archive.com/analog-help@lists.isite.net/msg11243.html -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Output Templating
James Linden ([EMAIL PROTECTED]): | >> Does analog have any (hidden) support for true output | >> templating? The header, footer, stylesheet, etc. are great, but | >> are pretty restrictive. I browsed through much of the | >> documentation, and was unable to find anything definitive on the | >> subject. After playing with the computer readable output format, | >> I found that it /could/ be used to export to various templates, | >> but would take some interesting parsing. CSV could be used, but | >> parsing it is much more complex than it needs to be. If | >> suggestions for a solution will be entertained, I am willing to | >> put some time and effort into developing a modular XML based | >> output format that will be more easily machine readable, and | >> thus, simplistic to make scripts to convert to whatever templated | >> output is desired. If this isn't an option, please let me | >> know so I can start writing the parsers for the other formats. | >> :o) | >> > | > | Jeremy's answer is quite correct, but I should add some comments. > | > | Output templates are an idea I've been pondering for some months, but I > | haven't resolved satisfactorily yet. At the moment I have a freeze on more > | output formats, because it's become unscalable to add any more, but there > | are several more I should like to add. The problem is that I can't work out > | how to construct a template which includes even all of the current output > | formats as instances. They all seem to have ugly special cases. > | > | James, if you have any ideas on this, then let's talk further. > Steve, > I am not going to claim to know anything special or be able to provide an > idea that hasn't been previously noted before. I do seem to have a knack for > data conversion of various kinds. I think that is basically what you are > presenting as the issue - right? In my personal experience, I have found that a > straight text or XML format as the core has solved 85% of any templating issues > I have worked with. Admittedly, I haven't worked with the specific needs of log > data, but I have worked with some other "everything is a special case" data. If > you want to give it a shot, I would like to have some discussion about the > templates, and maybe an idea will come out of it. I have a lot to say on this topic: http://www.mail-archive.com/analog-help@lists.isite.net/msg10487.html http://www.mail-archive.com/analog-help@lists.isite.net/msg10869.html http://www.mail-archive.com/analog-help@lists.isite.net/msg10466.html and a vested interest in this discussion: http://www.reportmagic.org/ If you choose to take this discussion off the list, can you please include me in it? Thanks, -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Output Templating
James Linden ([EMAIL PROTECTED]): > Does analog have any (hidden) support for true output templating? The header, >footer, stylesheet, etc. are great, but are pretty > restrictive. I browsed through much of the documentation, and was unable to find >anything definitive on the subject. After playing > with the computer readable output format, I found that it /could/ be used to export >to various templates, but would take some > interesting parsing. CSV could be used, but parsing it is much more complex than it >needs to be. > If suggestions for a solution will be entertained, I am willing to put some time >and effort into developing a modular XML based > output format that will be more easily machine readable, and thus, simplistic to >make scripts to convert to whatever templated > output is desired. > If this isn't an option, please let me know so I can start writing the parsers >for the other formats. :o) Analog does not have any templating built into it, just the predefined output formats. There is a tool on the Helper Applications page (http://www.analog.cx/helpers/) called anaml that converts CRO to XML. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Organization Report Help
Michael, PLEASE do not cross post to lists. If you have a question about Analog, post it to the Analog list. If you have a question about Report Magic, post it to that list. [EMAIL PROTECTED] ([EMAIL PROTECTED]): > I would like to get a description (organization name) associated with the ip's > in the organization report. > So it currently shows as this in the dat file: > Z lRr 1 1772715 34.026 155.118 > Z lRr 1 68455013.140 143.21 > I would like to see a text string associated with it so if 155.118 is the >accounting dept and 143.21 is the finance dept, > then can I see in the output as: > Z lRr 1 1772715 34.026 155.118 Accounting Dept > Z lRr 1 68455013.140 143.21Finance Dept > Is this possible in Analog and then would ReportMagic recognize it > (an Organization Name column) ? Analog does this with aliases. See http://www.analog.cx/docs/alias.html for details on the ORGALIAS command. Report Magic will pass through whatever information Analog tells it. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] general summary in computer output
Evaldas Imbrasas ([EMAIL PROTECTED]): >> > Is there any way to have 'Average requests per day' and 'Average >> > page requests per day' included in General Summary using computer >> >> No, they are calculated metrics, and can be easily determined from the >> other values in the General Summary. > I understand that. But I'm using RMagic to generate 'pretty' reports > from analog computer-readable output. It just displays values that it > finds in analog-generated General Summary. Report Magic includes averages of traffic for each time period in the Time Reports. > Anyway, it's not a big problem. It would be nice to have this generated > by analog, but I guess I can just make my wrapper script insert those > average values to analog computer-readable report before it calls > RMagic. This has become a FAQ for Report Magic so it will be added in a forthcoming version. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] general summary in computer output
Evaldas Imbrasas ([EMAIL PROTECTED]): > Is there any way to have 'Average requests per day' and 'Average > page requests per day' included in General Summary using computer > output? Those numbers appear in regular (HTML) output, but somehow > are not included when computer-readable output is generated. No, they are calculated metrics, and can be easily determined from the other values in the General Summary. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Host Report Help
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I would like to be able to get my Host report to produce counts at particular > levels.So instead of this in my report.dat > S Rr150 0.35000 111.222.333.44 > S Rr125 0.35000 111.222.333.55 > S Rr125 0.35000 111.222.333.66 > I would like to see it as > S Rr 400 0.35 111.222.333.* > Or move it to even a higher level at 111.222.* > Is there a way to specify this in the config file. The Organisation Report does this in version 5 and later. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Multiple Server Input Files
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I am running version 5.1 on HP-UX 11.x and feeding to it 3 input files. These > files are each from a different web-server that are attached to > a front end load balancer (cisco local director). But when I do I keep getting > warnings about duplicate times.Is there a way to get analog > to read multiple input files from the same date/time frame and compile these > into a single aggregate output (high level view) of all the data > combined?I have seen other references to this in the archive but never an > answer about it. Analog is already doing this. It's just a warning (not an error) in case you expect the log files to not overlap. You can use the WARNINGS command in your Analog configuration file to disable this message. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] User Report Generation
Funmi Beckley ([EMAIL PROTECTED]): > I am having a lot of difficulties generating a user report. Is is > possible to view an example of an Analog generated User Report. The user report is very simple. For each user listed in the log files, it tells the number of request, page request, bytes, etc (depending on columns selected) that the user has. If your log file does not have users (your website does not require logins) then you will have no data in the report and Analog won't generate it. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Analysing Vicomsoft Internet Gateway W3C Extended Format
Peter Reid ([EMAIL PROTECTED]): >> Re: [analog-help] Analysing Vicomsoft Internet >> GatewayOn Thu, 7 Feb 2002, Peter Reid wrote: >>> > >>> >You're on a Mac? Try running it by double-clicking the app >> instead of >>> >dragging and dropping. >>> >>> Tried this and get the same result! >>> > >> In that case, you must have declared it as a configuration file >> somehow. >> It's the only way you could get "unknown configuration command" >> messages. >> Check your settings by using the command SETTINGS ON. > I don't buy this explanation! It doesn't matter what file name the > log file has, and I've tried both my current analog.cfg and the > original one as downloaded. I ALWAYS get the same warnings! > The following is an extract from the settings displayed by Analog > after having turned on the option you mention: > Compile-time variables: > Default configuration file: analog.cfg > Madatory configuration file: manconf.cfg > Configuration files read: > analog.cfg > manconf.cfg Check to make sure that analog.cfg and manconf.cfg did not inadvertently get overwritten with logfile data. They should both look like Analog configuration files. > With SETTINGS ON, I just get all the settings listed, no warning > messages, but no report either! That's what SETTINGS ON is supposed to do. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Webstar V Log Problems
Patrick Gormley ([EMAIL PROTECTED]): > I have been working on this for the past few days and no matter what I do > Analog (5.1c) will not produce any logs from my > Webstar V logs. Someone gave me the following two lines to place in my > Analog cfg file: > LOGFORMAT (%m/%d/%Z\t%W%h:%n:%j\t%c\t%s\t%r\t%j\t%f\t%b\t%B\t%S\t%q\n) > DEFAULTLOGFORMAT > (%m/%d/%Z\t%W%h:%n:%j\t%c\t%s\t%r\t%j\t%f\t%b\t%B\t%S\t%q\n) You only need on or the other of these. The second overrides the first making the format available to logs defined in other files (or the command line) > However, everytime I run Analog I get a message saying that some 60,000 > lines in my file (almost all) were unwanted. I have disabled every EXCLUDE > command I can think of with no results. Unwanted lines mean they were excluded with an INCLUDE or EXCLUDE command. Corrupt lines mean they didn't for the format. Your log format is correct if most of the lines in your file are not corrupt. One of your INCLUDE or EXCLUDE commands must be filtering all those entries. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] How to get rid of %1 in report
Stephen Turner ([EMAIL PROTECTED]): > On Thu, 7 Feb 2002, Jacob-Steen Madsen wrote: >> Help.. >> >> Several of my repport is having this comment append to it: >> "This report shows the first %1 results by number of requests. This report >> is sorted by number of requests." >> why? >> I use Report magic to "polish" my reports. >> >> As far as i know i have not asked, neither analog nor report magic only to >> use the first %1 number of requests. >> >> you can look at http://www.sdu.dk/dou/webstatistik/ >> >> Any Suggestions? >> > Does this happen if you use analog on its on without Report Magic? If not, > it's probably a RM problem and should go to the reportmagic-help mailing > list. This was posted to both lists. I responded to the reportmagic-help list. An upgrade to the latest version fixes this. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Counting redirects
D. Alvarado ([EMAIL PROTECTED]): > OK, I think my previous question was rather out there, so I'd like > to redeem myself by asking a more reasonable question. > I haven't found an immense amount of literature on "REDIRINCLUDE" or > "REDIRHOSTINCLUDE", but do those commands only include files that > have been redirected to by other files? That is, given a page on > my server "a.html", I would like to count the number of times that > page has been requested via a redirect. Could I do this through > the use of "REDIR" commands? What would be a situation where I'd > use "REDIRINCLUDE"? These commands, like the other commands in that section of the manual only affect what is listed in the specific reports. REDIR(IN|EX)LCUDE affects what items are included in the Redirection Report. REDIRHOST(IN|EX)CLUDE affects what items are included in the Redirected Host Report. There are no basic data types of 'redirection' in Analog. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Counting pairs of requests
D. Alvarado ([EMAIL PROTECTED]): >I was wondering if its possible to do the following using analog. >I want to count the number of times users request 2 consecutive > files.That is, if the 2 files in question were "a.html" and > "b.html" and if IP 16.16.16.16 requested a.html and then requested > b.html (at an arbitrary length of time after, but with no requests > from 16.16.16.16 in between), I would like that to count as 1 > request. I would thus like to count the number of times this > happens for all users. >So, is this possible without rewriting analog? I understand that > this is an ambitious question so there may be no easy answers to > this. Short answer: no. See http://www.analog.cx/docs/webworks.html for reasons why you can't really be certain that two consecutive requests in your log files are really consecutive or that a given user maintains the same IP number between requests. Longer answer: Why would you want to count two requests as one? You realize, you are asking to change the definitions of the data that Analog analyzes. The level of detail that you are considering is really something that's essentially a log file. To do this kind of investigation, I think it would be more efficient to use grep or some other tool to collect all the request from a given host and look at them manually. It's possible that you could code up a script in Perl to perform this kind of logic, preprocessing the files before you have Analog analyze them. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Request Report Pie Chart Data Doesn't Match Request Report Listing
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I am running Analog 5.1(Windows) reports for an intranet, one report > covering the entire root directory, and separate reports for each of some > of the subordinate directory trees. In each of the reports, the Request > Report listing seems accurate, but the Request Report pie chart shows only > files for one of the subordinate directories. It is always the same > subdirectory, not only for the report on the entire site but even for the > those which are configured to report on only a single, different > subdirectory. > Why would the pie chart data be different than the listing data for the > Request Report? I have looked at my configuration files, but don't see > anything there that seems likely to have caused this. I know I can turn > off these pie charts, but I would rather be able to correct them. Are you telling Analog to write the pie charts all to the same folder? Then it will overwrite the image files each time it runs and you will be left with the last images created (likely one of the subdirectories). If you are creating several reports with separate images, you should put them into their own directories. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] User Report Generation Problems
Funmi Beckley ([EMAIL PROTECTED]): > I am unable to produce a User Report even though I have the > following line 'USER ON' included in my config file. I have > recorded users in the logfile and configured the server to > authenticate all users. I have also included the following lines in > the analog.cfg file: > APACHELOGFORMAT (%S - %j [%d/%M/%Y:%h:%n:%j %j] "%j %r %j" %c %b > "%f" "%B" "%u") This isn't a valid APACHELOGFORMAT string (Analog would tell you this). This is a standard LOGFORAMT string. (And I can't decipher from your quoting which '>' are supposed to be in there and which aren't so I'm not going to provide an example that may be wrong.) > USERALIAS REGEXPI:user=(.*?) (;|$) $1 There are too many commands on this line (again, Analog should have told you that.) I'm not quite sure, what you want, but I think it's this: USERALIAS REGEXPI:user=(.*?)\s*(;|$) $1 > I have a cookie called 'user' to record all users. Could you please let me > know what else might have to be done or inform me of any mistakes I might be Also check the USERFLOOR settings to make sure that all the users aren't bundled into [not listed] (unless that's what you want). -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] user tracking
François Dagorn ([EMAIL PROTECTED]): > is there a way to get details from the user report i.e. to get the > files requested by each user instead of just the number of requests > per user ? No, see http://www.analog.cx/docs/faq.html#faq128. But you can produce a report of the requests made by a single given user at a time, using USERINCLUDE command. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Further Defining Traffic To Site
Robert Thiel ([EMAIL PROTECTED]): > I'm looking to display not just the equivalent of "att.net" > [below] I would like to display the complete source > "6.chicago-19rh15rt.il.dial-access.att.net" [below]. HOST ON The host report tells you the complete host names > Better yet, please tell me if the http address [linked below] is just the > page the user accessed... and/or did he or she have this page "bookmarked" > so as to go directly to it? The http:://... address in your logfile is a referring page. This will be listed in the Referrer Report and the sites will be summarized in the Referring Sites Report. You can't tell how the user got to THAT page unless you have that site's logs. You normally don't get referrer data for bookmarks (what would the referrer be?) so this tells you that the user clicked on a link on that page to get to yours. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Stupid exclusion but ...
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > Probably asked many times but I can't succeed to generate request report > excluding those hosts : > 101: 39,96%: 31/Jan/02 19:07: http://msn02.cableandwireless.be/donnahi > 69: 22,86%: 31/Jan/02 17:45: http://msn02.cableandwireless.be/stubruhi > 24: 21,38%: 31/Jan/02 11:33: http://msn01.cableandwireless.be/ra2ovlhi > 13: 7,37%: 30/Jan/02 15:31: http://mss.streampower.be/topahi > 16: 5,17%: 31/Jan/02 16:41: http://mss.streampower.be/topalo > 33: 2,36%: 31/Jan/02 19:08: http://msn02.cableandwireless.be/donnalo >7: 0,35%: 31/Jan/02 23:50: http://msn02.cableandwireless.be/stubrulo >9: 0,34%: 31/Jan/02 13:06: http://msn02.cableandwireless.be/donnavideo >6: 0,23%: 30/Jan/02 13:29: http://mss.streampower.be/topavhi > <http://mss.streampower.be/topavhi> > Can someone tell my which exclude I have to put in my configuration Use the REQEXCLUDE command to remove items from the Request Report. This is covered in http://www.analog.cx/docs/include.html. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Newbie Question
Silk Stockings Ltd (Lawrence Roe) ([EMAIL PROTECTED]):
> I have again checked the log file format as you suggest, but all
> CustomLog lines are commented our as shown below, except for the
> original lines I sent.
> # The following directives define some format nicknames for use with
> # a CustomLog directive (see below).
> LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
> combined
> LogFormat "%h %l %u %t \"%r\" %>s %b" common
> LogFormat "%{Referer}i -> %U" referer
> LogFormat "%{User-agent}i" agent
> # The location of the access logfile (Common Logfile Format).
> # If this does not start with /, ServerRoot is prepended to it.
> CustomLog /var/log/httpd/access combined
Uhh... This isn't commented out and wasn't in the original post.
But anyway, I already determined that the logfile line you sent was in
this format.
> I will download all the individual files later today and try and run
> it on each file separately and see if this improves anything.
> I have tried the suggestions you made with regards to APACHELOGFORMAT (%v %h
>%l %u %t \"%r\" %>>s %b \"%{Referer}i\" \"%{User-Agent}i\")
> And
> DEFAULTAPACHELOGFORMAT
I assume you tried this (as one line):
DEFAULTAPACHELOGFORMAT (%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
\"%{User-Agent}i\")
> But I still receive the original errors, I am starting to think that the log
> file may be corrupt?
Turn on
DEBUG C
It will tell you which lines are corrupt and follow it with a line
including a '*' telling you where Analog failed to match the format to
the line.
This should help a lot in determining where the problems are.
But since this is the format that Apache is logging in, I don't see
any reason why Analog would not be able to parse that. I think your
problem is with other logfiles.
You can also use
SETTINGS ON
to have Analog report about what it's doing, which may give insight
about which files it's trying to load.
--
Jeremy Wadsack
Wadsack-Allen Digital Group
+
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
|http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
|http://www.mail-archive.com/analog-help@lists.isite.net/
|http://lists.isite.net/listgate/analog-help/archives/
|http://www.tallylist.com/archives/index.cfm/mlist.7
+
Re: [analog-help] Debug Help
Rodney Knott ([EMAIL PROTECTED]): >> "Rodney Knott" <[EMAIL PROTECTED]> >> >>> I am attempting to run analog on the ISA logs we are using. I allowed >>> analog to auto select a compatible format and it came up with W3 >> extended, >>> but that only processed a very small number of log entries. I ran it >> again >>> with debug C on and it gave me errors like the following for almost >>> all >> of >>> our log entries: >>> >>> C: >>> C:10.X.X.X anonymous Mozilla/4.0 (Compatible;MSIE 6.0; >>> Windows >> NT >>> 5.0;Q312461)2002-01-12 00:00:35 FIREWALL02 - >>> www.streamingfaith.com 10.X.X.X 80 733 140 >>> http Get http://10.X.X.X/images/radiotab.gif inet 304 >>> >> >> The next line in the Debug output puts a * under the first field that >> Analog can't make sense of. >> >> But even without that, a very brief look at >> http://www.analog.cx/docs/logfmt.html#fmtstrings suggests that you want >> a LOGFORMAT something like this: >> >> %S\t%u\t%B\t%Y-%m%d\t%h:%n:%j\t%j\t%j\t%v\t%j\t%j\t%b\t%T\t%j\t%j\t%r\t%j\ >> t%c >> >> Note that I'm guessing that http://10.X.X.X/images/radiotab.gif is >> supposed to be a request (even though requests don't start with >> http://), and that www.streamingfaith.com is a virtual host name. >> >> If ISA has the option of logging in W3 Extended format, then use that, >> so that you won't have to mess around with logformats that nobody >> understands. >> >> Aengus > I started using the following format string with no results: > (%s %B %Y-%m-%d %h:%n:%j %j %j %v %j %j %j %j %t %j %b %j %j %r %j %c) > And on the debug the * appears after the first C: Well, the format you show is very different from the one Aengus suggested. First, his (and your logfile line) use tabs to separate, not spaces. Without quotes around the browser, space-delimited log file lines would be useless. Seconds, there's a username between the host and browser fields that you are missing. Third, you have a lot more fields that you are throwing out (%j) than there are fields in that line. There *is* a type in Aengus's format string. Try this instead: %S\t%u\t%B\t%Y-%m-%d\t%h:%n:%j\t%j\t%j\t%v\t%j\t%j\t%b\t%T\t%j\t%j\t%r\t%j\t%c -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] debug question
j.bottoms ([EMAIL PROTECTED]): > two full lines of the error file > C: www.vicky-web.com cr025r01.sac2.fastsearch.net - - > [20/Jan/2002:09:36:53 -0500] "GET /news/news3-1-2000.html HTTP/1.0" 200 - 0 > 12744 "-" "FAST-WebCrawler/3.3 ([EMAIL PROTECTED]; > http://fast.no/support.php?c=faqs/crawler)" - - > C: > * > the end of the lines look like this though > - - > * > my question is why is the error showing up past the actual line of the log > what could be causing the error? That means that Analog is not expecting and end-of-line there. Your logformat has told Analog to expect at least one more field. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Newbie Question
Silk Stockings Ltd (Lawrence Roe) ([EMAIL PROTECTED]):
> My httpd.conf file advises my log file is written in the following
> formats
> LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
> combined
> LogFormat "%h %l %u %t \"%r\" %>s %b" common
> LogFormat "%{Referer}i -> %U" referer
> LogFormat "%{User-agent}i" agent
These are LogFormat definitions for Apache. To find out what format
the log file is actually being written in you also need to look at the
CustomLog line. It could be one of these named types or an explicit
format.
> But no matter how I enter this data in the analog.cfg file I receive an
> error when I run analogue advising the logfile format is incorrect.
> I have also tried the standard apache formats listed in the help and
> errors files also and I have changed the " to a ( etc
Make sure you are only looking at the access log. Analog won't parse
the error logs and you don't need it too -- all the information is
also in access logs in a more computer-readable format.
If you have older logs that were created in other formats, Analog may
not understand those immediately. Check the names of the files that it
says "incorrect format" for and make sure they are the same format as
your current logs.
> This is a single line from my log file, just in case it helps at all?
> www.silk-stockings.co.uk 195.92.168.163 - - [31/Jan/2001:11:28:37 +]
> "GET / HTTP/1.1" 500 723 "http://www.netcomuk.co.uk/index.htmll";
> "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; Circle0701)"
This is the 'combined' definition above. This format command will work
for this line (make sure the command is all on one line):
APACHELOGFORMAT (%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
\"%{User-Agent}i\")
If you are specifying the log file from a different configuration file
or from the command line then you need to use the
DEFAULTAPACHELOGFORMAT command with the format string above.
--
Jeremy Wadsack
Wadsack-Allen Digital Group
+
| This is the analog-help mailing list. To unsubscribe from this
| mailing list, go to
|http://lists.isite.net/listgate/analog-help/unsubscribe.html
|
| List archives are available at
|http://www.mail-archive.com/analog-help@lists.isite.net/
|http://lists.isite.net/listgate/analog-help/archives/
|http://www.tallylist.com/archives/index.cfm/mlist.7
+
Re: [analog-help] Two URLS same report???
Darryl Koster ([EMAIL PROTECTED]): > I am running a website (www.ontcurl.com) and I also have (ontcurl.com) going > to the same log file. They are getting around 30,000 page views a day right > now but I know it should be slightly higher by the log file size. > How do I get analog to read those lines of the log file with just > ontcurl.com in it as well as www.ontcurl.com? > It probably has these in the docs, it is late, and I am getting frustrated > and need help. Any clues would help immensly. Unless you explicitly excluded the lines with VHOSTEXCLUDE or VHOSTINCLUDE, analog will include all lines from the log file in the report. If you have excluded lines from that server, just add a VHOSTINCLUDE line to get them back. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
[analog-help] Re:
Neubert Anja-CAN111 ([EMAIL PROTECTED]): > All my .cfg-files show analogously up to HOSTURL > and the OUTFILE and now to my question. Can I > summarize the files to one file and how can I do > this? Use the CONFIGFILE command. http://www.analog.cx/docs/syntax.html#CONFIGFILE > In the reports some data are unimportant for our > customers. For example: In the browser-report you > can see the whole browsers, but for our customers > are only a few of them important. How can I filter > this data? You can use *INCLUDE and *EXCLUDE command on the report level to filter items in each report. http://www.analog.cx/docs/include.html Also, you might want to use the Browser Summary, rather than the Browser Report. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] IIS: Lines with no bytes: byte counts may be low
[EMAIL PROTECTED] ([EMAIL PROTECTED]): > I have the following problem: In my old logfiles there is no > information about the number of bytes transferred. Therefore Analog > shows me the corresponding CATEGORY M-warning and stops processing: > Logfile contains lines with no bytes: byte counts may be low > Is there any chance to process logfiles, that lack information about bytes > tranferred / any chance to bypass the error message? Analog is processing it. It's just warning you that the byte counts may be low because half (or whatever) of the requests don't have bytes. You can change the warnings with the WARNINGS command. See http://www.analog.cx/docs/debug.html#warnings for details. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] unknown log format
Peter ([EMAIL PROTECTED]): > With reference to one of my earlier posts i am including one of the lines as it >comes to me through email. I understand the lines are being broken up. I have tried >to manually edit some lines so > that they are all in one and this seems to work. Unfortunatly i am reading 850+ >corrupt lines with about 5 days of files still to open! > I have tried inserting logfile common, plus about five others with no success. I >have looked at the tutorial for creating a log string but to be honest i dont really >understand how the various > strings relate to the code. > 194.117.133.196 - - [27/Jan/2002:13:37:13 +] "GET > /images/diagnosis.gif HTTP/1.1" 200 891 "http://www.paains.org.uk/"; > "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; T312461)" > Can anyone help? I am getting stuck. If the lines are coming through broken up in your email, then you need to talk to your provider about getting the log files a different way. Either they should be sent as an attachment, so you can save it to disk directly, or the provider may offer some way for you to download them. Analog can't recombine or understand the lines if they are broken up. It's possible that you could write some script to rebuild the files, but it would need to know a lot about the data to do that. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Cache file inclusions/exclusions
D. Alvarado ([EMAIL PROTECTED]): > Assuming I create cache files with "STATUS ON" and "REQUEST ON" > settings, and then run analog using the generated cache files as > input and with the settings > "STATUS ON" > "STATUSINCLUDE 204" > "REQUEST ON" > "FILEINCLUDE Main_Page", > will the subsequent status and request reports be accurate (i.e. -- > identical to running analog on the log files that created the cache > files)? No. Cache files do not maintain the relationships that are necessary to apply the data-level inclusions. This is mentioned at the top of the page discussing cache files in the documentation. (http://www.analog.cx/docs/cache.html, paragraph 5) > Also, what is the difference between browser summaries and browser reports? Browser Report is the actual agent string sent by the browser. Browser Summary is a summary of that data in the format we usually expect (MSIE, Netscape, Opera, ... each broken down into version and subversion). [Stephen.. isn't this becoming a FAQ?] -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] help needed for new user
Peter ([EMAIL PROTECTED]): > i have tried a couple of trial runs but without success. To cut a > long story short, i am recieving my log files from the server via > e-mail from a third party. How do create a log file for analog to > read from the e mails?.At the moment i am copy/pasting the content > of the email (the log files), putting them into "my documents" in a > file called Stats and trying to get analog to read and report from > there. What am i doing wrong. Any help appreciated. Well, you're probably breaking up data that should be in one line into several lines. If the files are sent as email attachments, simply save them to disk. If they are the only content in the email, ask your provider to send them as attachments so that the lines aren't broken. Once you have the data properly formatted, Analog will often automatically detect the format. If not, follow the directions in the documentation (http://www.analog.cx/docs/logfmt.html) to build your own LOGFORMAT string. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] analog help
Peter ([EMAIL PROTECTED]): > i have downloaded analog 5.11 and the magic thing for viewing. how > do i set up my config files? i am a complete newbie and need a step > by step guide.. Is there one please. Analog comes with a full set of documentation files in the docs/ folder in the distribution. Start with the Readme.html file. Or you can get them from the websites here: http://www.analog.cx/docs/Readme.html. In particular, look at the 'Starting to use analog' page and select a start page for your platform. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Is there a way to completely exclude items from the user report
Keith Fetterman ([EMAIL PROTECTED]): > Is there a way to completely exclude users from the user report if they > fall below a threshold in the number of requests. USERFLOOR 20r See http://www.analog.cx/docs/othreps.html#FLOOR for details. > Let me explain what I mean by the previous statement. Right now, the > user report shows a list of the top 50 users and then says at the bottom: > 55971: 5426: 67.52%: [not listed: 641 users] > (where 55971 is requests and 5426 is pages) > For me to get the total number of users that visited, I add the 50 to 641. > What I would like to do is exclude those users from this report that > made 5 requests or less. I don't want them included in the count of 641 > of not listed users. That's not really possible. If they are not included in the report, then they are listed in the 'not listed' line. I suppose you could run a report of all users with 5 or fewer requests, then go and run the regular report excluding requests from those users, but that seems overly complicated. Alternately, you could list all users with more than 5 requests and use the count of entries in the report to get the number users. > I also have a suggestion for the next version of analog. Could the > total user count be displayed in the General Summary Section? I agree that this would be useful. > Or better than that, how about displaying a summary line above (or > below) the list of users in the User Report that gives a total > number of users in the report. That way if there is a filter that > only applies to the User Report, it will be applied to the summary > line as well. Just a thought. You can add the 'N' column to your USERCOLS command so that it prints the number of the row for each user. That will give you a count of users listed in the report (but not those excluded or below the floor.) -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] PAGEINCLUDE
Juergen Breitenfelder ([EMAIL PROTECTED]): > Hi, > I try to recognize ?OpenView Files as pages. > The Log looks like this: > 10.161.171.166 gcm.gtb.cib.intranet.db.com - [23/Jan/2002:00:25:21 -0100] "GET >/intranet/gcm/main/gcmprd1.ns4/va_AllSubCat1?OpenView&Expand=3 HTTP/1.1" 200 30070 > Even if I define a Pageinclude command like > PAGEINCLUDE ?OpenView or > PAGEINCLUDE *?OpenView or > PAGEINCLUDE *OpenView* or any other combination this command seems to be ignored. > Does anybody has a suggetion?? The part of a request after the '?' is the query string and is not looked at for PAGEINCLUDE commands. You will need to alias the requests into something that looks like a definable page and then use the PAGEINCLUDE. Something like this should work: FILEALIAS *?OpenView&* $1.OpenView?$2 PAGEINCLUDE *.OpenView This will make requests like /intranet/gcm/main/gcmprd1.ns4/va_AllSubCat1?OpenView&Expand=3 look this this in the reports /intranet/gcm/main/gcmprd1.ns4/va_AllSubCat1.OpenView?Expand=3 You can probably use a REQALIAS to convert them back for the Request Report if you want. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] pulling the logformat from apache
Stephen Turner ([EMAIL PROTECTED]): > On Tue, 22 Jan 2002 [EMAIL PROTECTED] wrote: >> That format is bringing me the same results as before >> (warnings on every line i assume an incorrect >> logformat). My error log is showing Warning F to >> closing the config file and lang file and then the C >> warning the the actual log files. Also there are three >> logs i'm working with error, ssl_engine, and access. I >> am missing something but I don't understand what. How >> do i troubleshoot this? > Three logs? The httpd.conf snippet you sent us only declared one, I think. > Analog doesn't analyse the error log, by the way, so ignore that one. > If you have two logfiles with different formats, you may need something like > LOGFORMAT format1 > LOGFILE file1 > LOGFORMAT format2 > LOGFILE file2 > If you still can't figure it out, turn debugging on. That will show you > where every line was corrupt. Also, Analog doesn't analyze the error log from Apache. It has mainly human-readable information and all of it is duplicated in the access logs in status codes. (This is in the FAQ) If you take out the error log, that may remove a lot of your corrupt line messages. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Combining analog report results
D. Alvarado ([EMAIL PROTECTED]): >So the short version of my question is, assuming identical > configuration files, if I run analog on logs spanning 1/1/2001 to > 1/1/2002, can I use that "report.dat" file somehow with a > "report.dat" file that spans 1/2/2002 to 1/18/2002 to make a single > file that spans 1/1/2001 to 1/18/2002? No. The any report files generated by Analog (including those in COMPUTER format) do not include complete information. If you try to 'combine' the results you will end up with incorrect reports and invalid numbers, especially near the bottom of reports. The proper way to do this is with Analog cache files (http://www.analog.cx/docs/cache.html). This is mentioned in the documentation on COMPUTER output and in the faq: http://www.analog.cx/docs/faq.html#faq157. -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
Re: [analog-help] Enable browser stats
Ana Paula ([EMAIL PROTECTED]): > What can I do for enable statistics to get users browsers details > (like: netscape 5%, Internet Explorer 33%, etc) ??? > I'm using Analog 5.1-1 and Report Magic 2.12 . Turn on the Browser Summary in your Analog config: BROWSERSUM ON -- Jeremy Wadsack Wadsack-Allen Digital Group + | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to |http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at |http://www.mail-archive.com/analog-help@lists.isite.net/ |http://lists.isite.net/listgate/analog-help/archives/ |http://www.tallylist.com/archives/index.cfm/mlist.7 +
