Hi all,
Have you ever heard of https://goaccess.io/?
Soo ... I've been busy. :p
The base-sitestats on BlueOnyx 5211R was a bit of a train-wreck and the
one on BlueOnyx 5210R isn't much better.
I just tackled the base-sitestats on BlueOnyx 5211R and published YUM
updates for it and will also backport the relevant changes to BlueOnyx
5210R.
FTP Usage Statistics:
======================
These stopped working a while ago and the effort to fix them vs. the
actual benefit of *having* them working? Not worth the hassles. So I
removed them from the GUI.
Net Usage Statistics:
======================
These used to use "iptables" accounting rules to record traffic
statistics and as both 5210R and 5211R use Firewalld these "iptables"
rules simply don't cut it for us anymore. So I removed this from the GUI
as well.
Web Usage Statistics:
======================
Now this is something we really want. Both for the server as a whole as
well as on a Vsite level.
The old statistics for this worked this way: The daily logrotate ran
split_logs and split the central Apache access_log up into chunks. Each
Vsite then got a var/log/web.log logfile with the accesses that
pertained to this Vsite since the last logrotate (usually the night before).
Then "Analog" ran over that and condensed the statistics into cache
files that got stored in dated directories.
Like var/log/2022/12/26/web.cache - for example.
The GUI then parsed these cache files and was supposed to display the
web usage statistics stored therein. Long story short: "Analog" stopped
working for us at one point or another and the resulting files were garbage.
Enter GoAccess: It can not only display real time web usage statistics.
It can also parse logfiles and dump out either a static HTML page with
the presentation of the gathered statistics. Or it can dump out a JSON
file with just the data. Which is ideal for us.
So I modified split_logs to run goaccess over the individual server and
vsite web.log files and dump out JSON files (web.json) into the date
directories.
And an entirely new "Usage Information" / "Web" page (both on the server
and Vsite level) shows the GoAccess statistics.
As "admin" or similarly privileged user you get to see the live stats
and can use a date picker to see older (archived and therefore static)
statistics.
On a Vsite level "Usage Information" / "Web" will only show static
GoAccess statistics, as they're all served from the Vsite's statistical
archive under var/log/<Date>
The separate menu entry "Server Management" / "Usage Information" /
"GoAccess" has been removed, as the display of the live data has been
merged into "Server Management" / "Usage Information" / "Web".
ATTENTION:
----------
In 5211R the live stats will apparently only work if AdmServ has a valid
SSL cert. If the cert is self signed or expired, then the GoAccess HTML
page fails to maintain a permanent connection with the GoAccess web socket.
All in all? This should serve us nicely. Many thanks to Dirk Estenfeld
for the recommendation.
Monitorix:
==========
This is new for us. To replace the removed FTP and Net statistics on the
server level (not Vsite!) I added the menu entry "Monitorix" to "Usage
Information". In there you'll find the most detailed statistics about
anything that makes sense to be displayed as graphics.
From System Load averages, Kernel and I/O stuff, Network traffic stats
to mail related graphs and much more. Monitorix is custom tailored for
RHEL and clones and still under active development, so it'll do. Even if
its look is a bit dated.
By default it shows the last 24h of activity, but if you click on the
bent arrow icon in the upper right corner of the Monitorix page and open
it up in a separate window you'll get the see a selector that allows you
to specify different date ranges as well as types of statistics to show.
Please note that both GoAccess and Monitorix only will show data that
was aggregated since the services came aboard via a YUM update (or on a
fresh BlueOnyx install).
"Active Monitor" has also been extended with something called "Usage
Information Services", which will monitor the two new daemons related to
the statistics gathering and presentation: "monitorix" and "goaccess".
So if these services fail, then "Active Monitor" will restart them
automatically.
GDPR/DSGVO conformity (for the EU folks):
==========================================
It's the same as before: The real time statistics aren't anonymized, but
anything that is archived will have the IP address partially anonymized
and the stored data will be expired at the chosen expiry date that you
have configured under "System Settings" / "Data Retention".
Further plans:
===============
As said: I'll backport this to BlueOnyx 5210R as well. Gimme a few days
to sort this out, as the changes were extensive and backporting from the
new GUI to the old one feels like potty training a baby-monkey. :p
--
With best regards
Michael Stauber
_______________________________________________
Blueonyx mailing list
Blueonyx@mail.blueonyx.it
http://mail.blueonyx.it/mailman/listinfo/blueonyx
