Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On 13.05.19 16:40, Matus UHLAR - fantomas wrote: but much longer time: # time clamscan /tmp/hwinfo /tmp/hwinfo: OK --- SCAN SUMMARY --- Known viruses: 9157095 Engine version: 0.100.3 Scanned directories: 0 Scanned files: 1 Infected files: 0 Data scanned: 0.57 MB Data read: 0.29 MB (ratio 1.95:1) Time: 39.043 sec (0 m 39 s) 38.208u 0.652s 0:39.11 99.3%0+0k 78984+0io 13pf+0w I should add that this is Xeon X3440 @2.53GHz so you can try to compare... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Chernobyl was an Windows 95 beta test site. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
Hi there, On Mon, 13 May 2019, Avinash Sonawane wrote: e.g. I am expecting an email at 6 PM. I don't mind clamd taking that much of a memory *at* 6 PM and then release it. I find it absolutely inconvenient to having to forgo ~1GB memory since the morning. As I said, a poor bargain. The bargain is the one that you made when you installed ClamAV. If you now feel that it is a poor one, you can of course uninstall it at no extra charge. Also consider that the email that you receive at 6PM might conceivably contain something which could completely destroy _all_ the software in your computer system. Perhaps not such a poor bargain then, if ClamAV manages to prevent this malicious message from doing its nasty work? You will probably agree that your use case is unusual (even I get more mail than you do... :). Unfortunately it is difficult to accommodate the needs of every user within a single package. It is unlikely that the development team will schedule big changes to ClamAV for a single user who receives one single email per day. The same install is used by some people on this list to scan more than one message every single second of every single day; the design of ClamAV appears to suit those people better than it suits you. There is still some hope, however. The ClamAV source code is published. If you want to contribute code which reduces the memory consumption of clamd without making serious compromises in performance, I'm sure that people here will be pleased to take a look at it. Incidentally I normally run three copies of clamd on the a single mail server. Each copy uses 1GB RAM. On a typical day, the server sees a few thousand to a couple of tens of thousands of attempts to send mail to it; thankfully most of the time it's at the lower end of the range. The last time any of them found anything was on 26 September 2018, and speaking personally I'm more than happy with that. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
Avinash Sonawane via clamav-users wrote: On Mon, 13 May 2019 16:21:15 +0200 Matus UHLAR - fantomas wrote: loading takes time, much time. How much time are we talking about here? I suppose by 'time' we mean loading time (load binary and signatures) + processing time (comparing signatures). Now, for loading time, when I start firefox within 5-6 seconds it immediately fills up 250+ Mb memory so for 950+ Mb (clamd) loading time shouldn't be that of an issue. ClamAV isn't just pushing bits from disk to RAM; it does some active processing to convert the signatures from their plaintext format on disk into data structures for its pattern matching engine(s) to work with. On lightly-loaded higher-end modern hardware, it should run about 15 seconds IME to load the signatures. On older or less capable hardware, or systems with lots of other processing going on, it can easily hit 30s to load the signatures. On RAM-limited VPSes, you may be hitting swap, in which case load time may well be several minutes at least. (And scanning isn't going to be very fast either.) Of course, at scanning time those signs/dbs need to be in memory. At scanning time not *all the time*. e.g. I am expecting an email at 6 PM. I don't mind clamd taking that much of a memory *at* 6 PM and then release it. I find it absolutely inconvenient to having to forgo ~1GB memory since the morning. As I said, a poor bargain. For your use case it sounds like you could do without ClamAV entirely. -kgd ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019 16:21:15 +0200 Matus UHLAR - fantomas wrote: > loading takes time, much time. How much time are we talking about here? I suppose by 'time' we mean loading time (load binary and signatures) + processing time (comparing signatures). Now, for loading time, when I start firefox within 5-6 seconds it immediately fills up 250+ Mb memory so for 950+ Mb (clamd) loading time shouldn't be that of an issue. Please note that processing time will be the same doesn't matter whether you keep clamd and signatures loaded *all the time* or load on demand. > And, they still would take about the same memory. Yes. The difference is hogging memory *all the time* and loading *on demand* > there are many signatures, they must be parsed and understood by > clamav. The only place they can be stored at scanning time is the > memory. Of course, at scanning time those signs/dbs need to be in memory. At scanning time not *all the time*. e.g. I am expecting an email at 6 PM. I don't mind clamd taking that much of a memory *at* 6 PM and then release it. I find it absolutely inconvenient to having to forgo ~1GB memory since the morning. As I said, a poor bargain. Regards, Avinash Sonawane (rootKea) PICT, Pune https://rootkea.wordpress.com ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019, Matus UHLAR - fantomas wrote: > >> On Mon, 13 May 2019 19:30:12 +0530 > >> Avinash Sonawane wrote: > >> > >> > Single email account here. On average, I receive one email a day. > >> > Devoting 1Gb memory all the time for that seems a poor bargain. > > >On Mon, 13 May 2019, Avinash Sonawane via clamav-users wrote: > >> Why can't clamd let databases/signatures stay in secondary memory > >> itself. Just load them when you actually receive message (or performing > >> the scan explicitly asked by user). Process and then again unload. > >> Waiting for next message. > >> > >> Why clamd needs to have signatures/databases loaded in primary memory > >> all the time? Even when there is no active scan or incoming email? This > >> doesn't make sense. > > On 13.05.19 10:34, Alan Stern wrote: > >What you're asking for is clamscan (as opposed to clamd and clamdscan). > >It loads the signatures when it runs, and after scanning all the memory > >is released. > > however, it uses about the same memory: > > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND > 2634 clamav20 0 999856 866284 12656 S 0.0 21.0 265:55.79 clamd > 24906 root 20 0 967288 875404 22844 R 98.3 21.2 0:38.71 clamscan > > but much longer time: > > # time clamscan /tmp/hwinfo > /tmp/hwinfo: OK > > --- SCAN SUMMARY --- > Known viruses: 9157095 > Engine version: 0.100.3 > Scanned directories: 0 > Scanned files: 1 > Infected files: 0 > Data scanned: 0.57 MB > Data read: 0.29 MB (ratio 1.95:1) > Time: 39.043 sec (0 m 39 s) > 38.208u 0.652s 0:39.11 99.3%0+0k 78984+0io 13pf+0w > > > # time clamdscan /tmp/hwinfo > /tmp/hwinfo: OK > > --- SCAN SUMMARY --- > Infected files: 0 > Time: 0.161 sec (0 m 0 s) > 0.004u 0.000s 0:00.17 0.0% 0+0k 8+0io 0pf+0w True, but it has the behavior that Avinash asked for: It doesn't use up 1 GB of memory when it's not busy loading or scanning. For someone who only receives about one email per day, trading off 39 seconds execution time for 1 GB of permanently occupied memory might be worthwhile. Alan Stern ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019 19:30:12 +0530 Avinash Sonawane wrote: > Single email account here. On average, I receive one email a day. > Devoting 1Gb memory all the time for that seems a poor bargain. On Mon, 13 May 2019, Avinash Sonawane via clamav-users wrote: Why can't clamd let databases/signatures stay in secondary memory itself. Just load them when you actually receive message (or performing the scan explicitly asked by user). Process and then again unload. Waiting for next message. Why clamd needs to have signatures/databases loaded in primary memory all the time? Even when there is no active scan or incoming email? This doesn't make sense. On 13.05.19 10:34, Alan Stern wrote: What you're asking for is clamscan (as opposed to clamd and clamdscan). It loads the signatures when it runs, and after scanning all the memory is released. however, it uses about the same memory: PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 2634 clamav20 0 999856 866284 12656 S 0.0 21.0 265:55.79 clamd 24906 root 20 0 967288 875404 22844 R 98.3 21.2 0:38.71 clamscan but much longer time: # time clamscan /tmp/hwinfo /tmp/hwinfo: OK --- SCAN SUMMARY --- Known viruses: 9157095 Engine version: 0.100.3 Scanned directories: 0 Scanned files: 1 Infected files: 0 Data scanned: 0.57 MB Data read: 0.29 MB (ratio 1.95:1) Time: 39.043 sec (0 m 39 s) 38.208u 0.652s 0:39.11 99.3%0+0k 78984+0io 13pf+0w # time clamdscan /tmp/hwinfo /tmp/hwinfo: OK --- SCAN SUMMARY --- Infected files: 0 Time: 0.161 sec (0 m 0 s) 0.004u 0.000s 0:00.17 0.0% 0+0k 8+0io 0pf+0w -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Enter any 12-digit prime number to continue. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019, Avinash Sonawane via clamav-users wrote: > On Mon, 13 May 2019 19:30:12 +0530 > Avinash Sonawane wrote: > > > Single email account here. On average, I receive one email a day. > > Devoting 1Gb memory all the time for that seems a poor bargain. > > Why can't clamd let databases/signatures stay in secondary memory > itself. Just load them when you actually receive message (or performing > the scan explicitly asked by user). Process and then again unload. > Waiting for next message. > > Why clamd needs to have signatures/databases loaded in primary memory > all the time? Even when there is no active scan or incoming email? This > doesn't make sense. What you're asking for is clamscan (as opposed to clamd and clamdscan). It loads the signatures when it runs, and after scanning all the memory is released. Alan Stern ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019 19:30:12 +0530 Avinash Sonawane wrote: Single email account here. On average, I receive one email a day. Devoting 1Gb memory all the time for that seems a poor bargain. On 13.05.19 19:46, Avinash Sonawane via clamav-users wrote: Why can't clamd let databases/signatures stay in secondary memory itself. Just load them when you actually receive message (or performing the scan explicitly asked by user). Process and then again unload. Waiting for next message. loading takes time, much time. And, they still would take about the same memory. Why clamd needs to have signatures/databases loaded in primary memory all the time? Even when there is no active scan or incoming email? This doesn't make sense. there are many signatures, they must be parsed and understood by clamav. The only place they can be stored at scanning time is the memory. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "Where do you want to go to die?" [Microsoft] ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019 19:30:12 +0530 Avinash Sonawane wrote: > Single email account here. On average, I receive one email a day. > Devoting 1Gb memory all the time for that seems a poor bargain. Why can't clamd let databases/signatures stay in secondary memory itself. Just load them when you actually receive message (or performing the scan explicitly asked by user). Process and then again unload. Waiting for next message. Why clamd needs to have signatures/databases loaded in primary memory all the time? Even when there is no active scan or incoming email? This doesn't make sense. Regards, Avinash Sonawane (rootKea) PICT, Pune https://rootkea.wordpress.com ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019 15:46:42 +0200 Matus UHLAR - fantomas wrote: redusing the number of signatures is the only way I know of. On 13.05.19 19:30, Avinash Sonawane via clamav-users wrote: Actually, I was thinking if I could tweak some clamd conf without removing AV databases/signatures thereby not reducing clamAv's functionality/effectiveness. I'm afraid that the virus database is the only thing that uses memory. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "The box said 'Requires Windows 95 or better', so I bought a Macintosh". ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On Mon, 13 May 2019 15:46:42 +0200 Matus UHLAR - fantomas wrote: > On 13.05.19 18:53, Avinash Sonawane via clamav-users wrote: > redusing the number of signatures is the only way I know of. Actually, I was thinking if I could tweak some clamd conf without removing AV databases/signatures thereby not reducing clamAv's functionality/effectiveness. > but I apparently have lesser traffic. Single email account here. On average, I receive one email a day. Devoting 1Gb memory all the time for that seems a poor bargain. Regards, Avinash Sonawane (rootKea) PICT, Pune https://rootkea.wordpress.com ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] clamd using ~1GB memory on Debian Stretch
On 13.05.19 18:53, Avinash Sonawane via clamav-users wrote: I'm using ClamAV 0.100.3/25448 on Debian Stretch. It's a default install without any tweaks (as far as I remember). As seen from the below top o/p, without any active scan clamd is using ~1Gb memory. That's unacceptable. May I know how do I reduce clamd memory usage? redusing the number of signatures is the only way I know of. PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 8164 clamav 20 0 1255316 988.1m 31296 S 0.0 25.9 0:50.44 clamd PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 2634 clamav20 0 999856 866568 12912 S 0.0 21.0 265:55.12 clamd but I apparently have lesser traffic. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. REALITY.SYS corrupted. Press any key to reboot Universe. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
[clamav-users] clamd using ~1GB memory on Debian Stretch
Hello! I'm using ClamAV 0.100.3/25448 on Debian Stretch. It's a default install without any tweaks (as far as I remember). As seen from the below top o/p, without any active scan clamd is using ~1Gb memory. That's unacceptable. May I know how do I reduce clamd memory usage? PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND 8164 clamav 20 0 1255316 988.1m 31296 S 0.0 25.9 0:50.44 clamd Thanks! Regards, Avinash Sonawane (rootKea) PICT, Pune https://rootkea.wordpress.com ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
