Bug#845546: sarg: Proposal for lastlog feature in sarg-reports - patch provided
Package: sarg
Version: 2.3.10-2
Severity: wishlist
Dear Maintainer,
The patches below (two of which are diffs from the revised versions I proposed
in bug 845455) include the following proposed changes:
1. Configuration option for number of saved logs per report type
It seems to me that surely I'm not the only one who would like to keep
different numbers of different types of report.
At the same time, different people will have different history requirements.
sarg-reports uses the same sarg.conf for all types of reports, but sarg allows
override of the lastlog setting by using the --lastlog= switch.
The change made in 2.3.10-2 does allow a user (should he/she wish) to pass this
switch to sarg by adjusting the cron jobs which the Debian sarg package
installs for sarg-reports.
However, I reckon that this is a common-enough requirement to justify adding
dedicated settings in sarg-reports.conf.
2. Setting of distinct temporary directory for each run of the script, per
recommendation from Frederic Marchal in case of overlap of cron jobs.
References:
https://sourceforge.net/p/sarg/discussion/363374/thread/efb2f9e9/#3cf7
https://sourceforge.net/p/sarg/discussion/363374/thread/efb2f9e9/#7d4d
I acknowledge that my "rm -rf $TMPDIR" may be a bad idea - I'm not sure how
best to make that safe.
Further, I suspect that my patch in the previous bug may have been faulty as I
had problems applying it - I hope you could work with it and that these ones
are also good enough.
--
--- sarg-reports.proposed
+++ sarg-reports.proposed-with-lastlog
@@ -6,7 +6,7 @@
# - martijn
# - Stas Degteff https://sourceforge.net/users/stas_degteff/
-VER=debian-sarg-2.3.10-2
+VER=debian-sarg-2.3.10-2-with-lastlog
## What is this?
# sarg-reports (this file) is a simple bash script written to automate
@@ -55,6 +55,9 @@
#(SARG) The sarg executable location
# (CONFIG) The sarg main configuration file location
# (HTMLOUT) Location where will be saved the reports
+# (LASTLOGDAILY) Number of previous daily reports to keep (affects "today"
and "manual" logs too)
+# (LASTLOGWEEKLY) Number of previous weekly reports to keep
+# (LASTLOGMONTHLY) Number of previous monthly reports to keep
# (PAGETITLE) The title of main index page
# (LOGOIMG) Image logo to view in main index page
#(LOGOLINK) HTTP web page link of logo
@@ -110,6 +113,7 @@
# TEMP Files
TMPFILE=`/bin/mktemp`
ERRORS="${TMPFILE}.errors"
+TMPDIR=`/bin/mktemp -d`
# Date Calc
MANUALDATE=$2
@@ -210,10 +214,15 @@
{
cat $ERRORS | grep -v "$EXCLUDELOG1" | grep -v "$EXCLUDELOG2"
rm -f $TMPFILE*
+ rm -rf $TMPDIR
}
manual ()
{
+ if [ -n "$LASTLOGDAILY" ]
+then
+ LASTLOG="--lastlog=$LASTLOGDAILY"
+ fi
DAILYOUT=$HTMLOUT/$DAILY
mkdir -p $DAILYOUT
create_index_html
@@ -221,43 +230,59 @@
then
echo "No date given, please specify a valid date (DD/MM/)"
else
- $SARG -f $CONFIG -d $MANUALDATE -o $DAILYOUT ${@} ${ACCESS_LOG}*
+ $SARG -f $CONFIG -w $TMPDIR -d $MANUALDATE -o $DAILYOUT $LASTLOG ${@}
${ACCESS_LOG}*
fi
}
today ()
{
+ if [ -n "$LASTLOGDAILY" ]
+then
+ LASTLOG="--lastlog=$LASTLOGDAILY"
+ fi
DAILYOUT=$HTMLOUT/$DAILY
mkdir -p $DAILYOUT
create_index_html
- $SARG -f $CONFIG -d $TODAY -o $DAILYOUT ${@} ${ACCESS_LOG}* >$ERRORS 2>&1
+ $SARG -f $CONFIG -w $TMPDIR -d $TODAY -o $DAILYOUT $LASTLOG ${@}
${ACCESS_LOG}* >$ERRORS 2>&1
exclude_from_log
}
daily ()
{
+ if [ -n "$LASTLOGDAILY" ]
+then
+ LASTLOG="--lastlog=$LASTLOGDAILY"
+ fi
DAILYOUT=$HTMLOUT/$DAILY
mkdir -p $DAILYOUT
create_index_html
- $SARG -f $CONFIG -d day-1 -o $DAILYOUT ${@} ${ACCESS_LOG}* >$ERRORS 2>&1
+ $SARG -f $CONFIG -w $TMPDIR -d day-1 -o $DAILYOUT $LASTLOG ${@}
${ACCESS_LOG}* >$ERRORS 2>&1
exclude_from_log
}
weekly ()
{
+ if [ -n "$LASTLOGWEEKLY" ]
+then
+ LASTLOG="--lastlog=$LASTLOGWEEKLY"
+ fi
WEEKLYOUT=$HTMLOUT/$WEEKLY
mkdir -p $WEEKLYOUT
create_index_html
- $SARG -f $CONFIG -d week-1 -o $WEEKLYOUT ${@} ${ACCESS_LOG}* >$ERRORS 2>&1
+ $SARG -f $CONFIG -w $TMPDIR -d week-1 -o $WEEKLYOUT $LASTLOG ${@}
${ACCESS_LOG}* >$ERRORS 2>&1
exclude_from_log
}
monthly ()
{
+ if [ -n "$LASTLOGMONTHLY" ]
+then
+ LASTLOG="--lastlog=$LASTLOGMONTHLY"
+ fi
MONTHLYOUT=$HTMLOUT/$MONTHLY
mkdir -p $MONTHLYOUT
create_index_html
- $SARG -f $CONFIG -d month-1 -o $MONTHLYOUT ${@} ${ACCESS_LOG}* >$ERRORS 2>&1
+ $SARG -f $CONFIG -w $TMPDIR -d month-1 -o $MONTHLYOUT $LASTLOG ${@}
${ACCESS_LOG}* >$ERRORS 2>&1
exclude_from_log
}
@@ -302,6 +327,7 @@
## HISTORY:
+# debian-sarg-2.3.10-2-with-lastlog - add lastlog option to keep different
numbers of previous reports per type, private temporary directory
# debian-sarg-2.3.10-2 - use all rotated log files, pass through options from
command line
# debian-sarg-? - added reports_autorm function (not used)
# ... - ?
Bug#845455: sarg: Documentation update needed for sarg-reports
Package: sarg Version: 2.3.10-2 Severity: minor Dear Maintainer, I have been trying to set up sarg and while researching solutions to my problems I came across bug 818429 (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818429) whose proposed fix was merged into the version this report relates to. The fix adds functionality but documentation (both within the script /usr/sbin/sarg-reports and the manpage) has not been updated and I imagine that users may not understand that only minimal configuration changes are required to make things work in a useful way. In fact it seems that the Debian package maintainers have made several changes to the sarg-reports script over time (I haven't looked at the full change log) as can be seen by comparing the distribution version with the openSUSE version of the same file (https://build.opensuse.org/package/view_file/openSUSE:Factory/sarg/sarg-reports?expand=1) which carries the same version number. I would like to propose changes to the internal documentation of the script (a patch file follows) and to the man page (changes to the text alone follow, in patch format - I don't have experience working with man pages). The revised version number is only a suggestion, of course; I assume you will determine an appropriate version number. It seems to me that fixing the documentation would go some way towards solving the problem raised in bug 369693 (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=369693), at least as far as sarg is concerned, and especially for users of sarg-reports. -- --- sarg-reports +++ sarg-reports.proposed @@ -6,7 +6,7 @@ # - martijn # - Stas Degteff https://sourceforge.net/users/stas_degteff/ -VER=20050202 +VER=debian-sarg-2.3.10-2 ## What is this? # sarg-reports (this file) is a simple bash script written to automate @@ -43,11 +43,14 @@ #30 02 1 * * sarg-reports monthly #--- END ROOT CRONTAB --- # -#REMEMBER: if you use logrotate, configure it to rotate the logs within MONTHLY basis, -# AFTER sarg-reports created the monthly html report. +#REMEMBER: if you use logrotate, configure it to retain enough logs to ensure availability +# of data for the longest period for which you need a report. +# See https://sourceforge.net/p/sarg/wiki/Keeping%20enough%20log%20to%20process/ +# This version of sarg-reports will use the log file defined in sarg.conf and also +# prior log files which have been rotated out of that location # 3) Customize the following variables: -#(Please, configure accurately the sarg.conf file before) +#(Please, configure accurately the sarg-reports.conf file before) # #(SARG) The sarg executable location # (CONFIG) The sarg main configuration file location @@ -299,6 +302,9 @@ ## HISTORY: +# debian-sarg-2.3.10-2 - use all rotated log files, pass through options from command line +# debian-sarg-? - added reports_autorm function (not used) +# ... - ? # 20050502 - Stas Degteff added support for non latin Charset and added support for OpenBSD # 20030826 - FreeBSD support (thanks to martijn to let me coding on your FreeBSD server :-)) # 20030715 - Some cleanups @@ -314,6 +320,3 @@ # doesn't like this script do: "7 days ago to yesterday" # - Monthly recognition isn't so elegant (is very ugly, i know) # - Suggestions are welcome :-) -# - If you Rotate the squid logs before sarg-reports will run, -# it will not create any html reports -# (TIPS: Rotate the logs after sarg-reports) -- --- sarg-reports-manpage +++ sarg-reports-manpage.proposed @@ -8,7 +8,7 @@ This manual page documents briefly the sarg-reports command. sarg-reports is a script written to automate the SARG (a powerful squid - log analyzer) reports and log management. Sarg it self, provide to end + log analyzer) reports and log management. Sarg itself, provides to end user a generic interface to create reports based on squid access log (begin of log to current date). sarg-reports is useful because it allow you to easly create and manage Daily, Weekly and Monthly reports. @@ -28,6 +28,8 @@ monthly Create Monthly report + Any further parameters supplied will be passed through to sarg. + CRONTAB If you want the reports can be generated automatically insert the following lines (the today report creation time depend mostly of your @@ -39,8 +41,12 @@ 00 01 * * 1 sarg-reports weekly 30 02 1 * * sarg-reports monthly - REMEMBER: if you use logrotate, configure it to rotate the logs within - MONTHLY basis, AFTER sarg-reports created the monthly html report. + REMEMBER: if you use logrotate, configure it to
