On Wed, 9 Feb 2022, Matus UHLAR - fantomas wrote:
I have clamav 0.103.5 installed on debian 11 and I'm getting too often
errors when reloading database.
looking back this problem started appearing on:
Mon May 10 11:51:15 2021 -> Database correctly reloaded (12721518 signatures)
Mon May 10 12:48:11 2021 -> ERROR: reload_th: Database load failed: Malformed
database
[...]
this machine has 4G of RAM and some swap, clamd currently eats ~1.5 GB ...
I wonder if this problem may be caused by i386 architecture with 3GB limit ...
Does clamd reload signature database in the same process?
On 09.02.22 09:44, G.W. Haywood via clamav-users wrote:
It's a very long time since I ran ClamAV on i386 so I've no experience
to offer. If your suspicion is correct it might be a problem specific
to the machine:
https://en.wikipedia.org/wiki/3_GB_barrier
On 10.02.22 09:58, Matus UHLAR - fantomas wrote:
yes, this is what I'm guessing.
I'm just curious if someone can confirm this or I have to try.
so far I was lazy to convert this machine (or at least part of it) to
64-bit. 64-bit kernel should help to move the barrier to 4G.
I have rebooted into 64-bit kernel, without changing any installed software.
looks like database updates work flawlessly since:
Fri Feb 11 19:52:38 2022 -> SelfCheck: Database modification detected. Forcing
reload.
Fri Feb 11 19:53:03 2022 -> ERROR: reload_th: Database load failed: Can't
allocate memory
Fri Feb 11 19:53:04 2022 -> WARNING: Database reload failed, keeping the
previous instance
Fri Feb 11 20:42:57 2022 -> +++ Started at Fri Feb 11 20:42:57 2022
Fri Feb 11 20:42:57 2022 -> Not loading PUA signatures.
Fri Feb 11 20:43:28 2022 -> Loaded 12726414 signatures.
Fri Feb 11 20:49:16 2022 -> Database correctly reloaded (12726430 signatures)
Fri Feb 11 20:49:16 2022 -> Activating the newly loaded database...
Fri Feb 11 21:54:23 2022 -> Database correctly reloaded (12726435 signatures)
Fri Feb 11 21:54:23 2022 -> Activating the newly loaded database...
Fri Feb 11 22:49:08 2022 -> SelfCheck: Database modification detected. Forcing
reload.
Fri Feb 11 22:49:45 2022 -> Database correctly reloaded (12726434 signatures)
Fri Feb 11 22:49:45 2022 -> Activating the newly loaded database...
So the 3GB barrier applies to clamav (no wonder) when reloading signatures.
- unlike other SW, no new clamd instance after reload.
There's a configuration option to avoid the doubled memory usage on a
database reload, look in the configuration file for clamd for the
'ConcurrentDatabaseReload' directive. Be aware of the issues, you
might not want to pause scanning during reloads.
I know of this feature, just wanted to avoid it.
even my swap usage is lower, which is a good thing.
I'm going to activate zswap again. Before this change, my machine was
running quite slowly, apparently because of excessive swapping due to
repeated attempts to reload signature.
I have learnt something...
What a lot of signatures! I'm at around 8.8 million at the moment,
with about 45 additional third-party databases and yara rule sets.
On Thu, 10 Feb 2022, Matus UHLAR - fantomas wrote:
I think most of it comes from securiteinfo.com feed, which I have
subscribed into. I have this machine for personal use.
it seems their signatures are the most commonly catched:
% zgrep -Fih FOUND `ls -1tr clamav.log*` | awk ...
84 SecuriteInfo
62 Porcupine
32 Sanesecurity
[...]
(there may be duplicates so the real difference may be smaller)
On 10.02.22 09:38, G.W. Haywood via clamav-users wrote:
That's a bit odd. You seem to be getting roughly twice the hits from
Porcupine that you get from Sansecurity, and over here it's the other
way around although the difference is smaller. We see about 50%-60%
more from Sanesecurity than from Porcupine, 85 and 55 respectively to
date in February. In fact my Yara rules catch many more than that, I
wonder if they catch more of what Porcupine would have caught and your
SecuriteInfo sigs catch more of what Sanesecurity would have caught.
that's what I meant by duplicates.
I've looked into telling ClamAV to report all the matches it can find
instead of just the first, but actually doing that hasn't yet reached
the top of this 'in' tray. I'll stop. A fellow could go nuts.
this could eliminate many duplicates, which could help us quite a bit.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
42.7 percent of all statistics are made up on the spot.
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
