Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-11-27 Thread Sergey
On Tuesday 26 November 2019, Micah Snyder (micasnyd) via clamav-users wrote: > In addition to the improvements in 0.101.5, 0.102.1, we shipped an update > to main & daily yesterday I known and my test was with new main & daily for 0.101.4 and 0.101.5 both. So it shows improvement of clamd's code

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-11-26 Thread Reio Remma via clamav-users
On 26.11.2019 20:12, Micah Snyder (micasnyd) via clamav-users wrote: In addition to the improvements in 0.101.5, 0.102.1, we shipped an update to main & daily yesterday and this morning that reduced load time by removing ignored signatures (signatures in main that we wished to drop, and thus ig

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-11-26 Thread Micah Snyder (micasnyd) via clamav-users
In addition to the improvements in 0.101.5, 0.102.1, we shipped an update to main & daily yesterday and this morning that reduced load time by removing ignored signatures (signatures in main that we wished to drop, and thus ignored in daily.ign2/daily.ign). On my laptop, I observed 0.102.0, da

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-11-26 Thread Sergey
On Tuesday 26 November 2019, Sergey wrote: > 0.101.5 (22 sec): > Tue Nov 26 21:12:02 2019 -> Bytecode: Security mode set to "TrustSigned". > Tue Nov 26 21:12:24 2019 -> Loaded 6565044 signatures. Hm... It's for big cld files. More compact cvd files loaded about 10 seconds longer: Tue Nov 26 21:

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-11-26 Thread Sergey
On Wednesday 04 September 2019, Sergey wrote: > Since some time there has been a noticeable increase a launch time. Good acceleration in the new version! 0.101.4 (124 sec): Tue Nov 26 21:08:26 2019 -> Bytecode: Security mode set to "TrustSigned". Tue Nov 26 21:10:30 2019 -> Loaded 6565044 signa

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-18 Thread G.W. Haywood via clamav-users
Hi there, On Fri, 18 Oct 2019, J.R. via clamav-users wrote: ... most email servers don't close the connection automatically until the client side sends a quit (which many spammers don't), it then falls back to the timeout setting to close ... Time was when you could rely on that, but even som

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-18 Thread J.R. via clamav-users
Vladislav, If you are going to put everything on hold while your AV database reloads, be sure you have appropriate timeout settings for your milter or whatever else is handling things so the email program doesn't timeout waiting for a response from it. While the *default* timeouts for email chat

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-18 Thread G.W. Haywood via clamav-users
Hi there, On Fri, 18 Oct 2019, Vladislav Kurz via clamav-users wrote: On 17/10/2019 17:44, G.W. Haywood via clamav-users wrote: There other ways of dealing with this, as I'm sure you're aware, but using the patched daemon you only have to worry about the increased memory consumption during dat

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-18 Thread Vladislav Kurz via clamav-users
On 17/10/2019 17:44, G.W. Haywood via clamav-users wrote: > Hello again, > > On Thu, 17 Oct 2019, Vladislav Kurz via clamav-users wrote: > >> Is there anything blocking this patch from being accepted ? > > As far as I know, only the (significant) pressures and (AFAICT equally > significant) limi

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-17 Thread Reio Remma via clamav-users
On 17.10.2019 19:04, Micah Snyder (micasnyd) via clamav-users wrote: Vladislav, Ged: Reloading select databases is not feasible at this time, because signatures are loaded into the same structures in memory and that entire thing is recreated on reload. Regarding the threaded reload feature (

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-17 Thread Micah Snyder (micasnyd) via clamav-users
Vladislav, Ged: Reloading select databases is not feasible at this time, because signatures are loaded into the same structures in memory and that entire thing is recreated on reload. Regarding the threaded reload feature ( ticket: https://bugzilla.clamav.net/show_bug.cgi?id=10979 )... The ma

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-17 Thread G.W. Haywood via clamav-users
Hello again, On Thu, 17 Oct 2019, Vladislav Kurz via clamav-users wrote: Is there anything blocking this patch from being accepted ? As far as I know, only the (significant) pressures and (AFAICT equally significant) limitations on developer time at Cisco/Talos/SourceFire. I'm noticing clam

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-17 Thread Vladislav Kurz via clamav-users
On 17/10/2019 16:51, G.W. Haywood via clamav-users wrote: > Hi there, > > On Thu, 17 Oct 2019, Vladislav Kurz via clamav-users wrote: > >> So the question is - what would be easier to code? >> - reloading in background thread >> - reloading limited to new files > > It is not clear to me that the

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-17 Thread G.W. Haywood via clamav-users
Hi there, On Thu, 17 Oct 2019, Vladislav Kurz via clamav-users wrote: So the question is - what would be easier to code? - reloading in background thread - reloading limited to new files It is not clear to me that the latter suggestion is feasible, but... 1. Reload in a separate thread was f

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-17 Thread Vladislav Kurz via clamav-users
On 17/10/2019 15:40, Markus Kolb via clamav-users wrote: > Am 07.10.2019 08:57, schrieb Sergey: >> On Friday 13 September 2019, Markus Kolb via clamav-users wrote: >> >>> I've opened an enhacement bug for this: >>> https://bugzilla.clamav.net/show_bug.cgi?id=12389 >> >> Thanks. But I have one more

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-17 Thread Markus Kolb via clamav-users
Am 07.10.2019 08:57, schrieb Sergey: On Friday 13 September 2019, Markus Kolb via clamav-users wrote: I've opened an enhacement bug for this: https://bugzilla.clamav.net/show_bug.cgi?id=12389 Thanks. But I have one more question. Do I understand correctly that when loading main.cvd base rules

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-10 Thread Mark Fortescue via clamav-users
Hi, You are forgetting things like embedded systems in hospitals that can't reasonably be updated. The NHS got stung by this with XP and Microsoft had to produce a post EOL fix. Outside of the computer industry, software and hardware move forward at a snails pace. Many systems still use Wi

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-08 Thread G.W. Haywood via clamav-users
Hi there, On Mon, 7 Oct 2019, Joel Esler (jesler) wrote: Gotta keep detection around for “old” stuff. First of all, who defines old? ... You don't see many DOS packages nowadays, but they are still around. Over thirty years ago, I wrote a package for product warehousing and distribution in

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-08 Thread Al Varnell via clamav-users
There are some occasion where malware becomes officially obsolete due to things like their C&C servers being seized by authorities. That would require a more detailed knowledge of such threats which I’m sure you don’t have the resources for. And eventually these threats could be resurrected by s

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread Joel Esler (jesler) via clamav-users
Gotta keep detection around for “old” stuff. First of all, who defines old? Second of all, when ClamAV is tested in third party analysis, we aren’t tested against “just new stuff” Sent from my  iPad > On Oct 7, 2019, at 16:11, G.W. Haywood via clamav-users > wrote: > > Hi there, > >> On M

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread G.W. Haywood via clamav-users
Hi there, On Mon, 7 Oct 2019, J.R. via clamav-users wrote: G.W. Haywood wrote: Well I only run Linux systems and I'd _still_ want to scan for Windows and Office 2003 malware. Call it social responsibility. ... ... scan for DOS viruses? Windows 3.1? 95? 98? 2K? mail6:/var/lib/clamav/datab

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread J.R. via clamav-users
> Steve Basford: > So, is the above hash still relevant or should it moved into archived.hsb, > which by default doesn't load ? I would *guess* the ClamAV team would have a *little* more detailed of a back-end system tracking viruses (though I could be wrong)... > G.W. Haywood: > Well I only run

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread Joel Esler (jesler) via clamav-users
On Oct 7, 2019, at 6:39 AM, Vladislav Kurz via clamav-users mailto:clamav-users@lists.clamav.net>> wrote: On 07/10/2019 08:57, Sergey wrote: On Friday 13 September 2019, Markus Kolb via clamav-users wrote: I've opened an enhacement bug for this: https://bugzilla.clamav.net/show_bug.cgi?id=1238

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread G.W. Haywood via clamav-users
Hi there, On Mon, 7 Oct 2019, Steve Basford wrote: 36,49,543 main.hdb 23,657,708 daily.hdb 248,06,499 main.hsb 905,00,729 daily.hsb 36,49,543 main.hdb 23,657,708 daily.hdb 24,806,499 main.hsb 905,00,729 daily.hsb Have you finally worn out your keyboard Steve? :O On 7 October 2019 15:25:41 "

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread Steve Basford
On 7 October 2019 15:25:41 "J.R. via clamav-users" wrote: I don't know how the viruses are tracked, but maybe to reduce size (if applicable) some of the more ancient viruses that only affect EOL operating systems (or programs that should have long since been patched) could be spun-off into a s

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread J.R. via clamav-users
> > Maybe it's time to update main.cvd and reduce daily.* while > > bug 12389 is being processed? > > > > I support this idea. Daily.cvd is at the moment bigger than main.cvd and > main.cvd has not beeen updated at least two years (maybe even more). I don't know how the viruses are tracked, but ma

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-07 Thread Vladislav Kurz via clamav-users
On 07/10/2019 08:57, Sergey wrote: > On Friday 13 September 2019, Markus Kolb via clamav-users wrote: > >> I've opened an enhacement bug for this: >> https://bugzilla.clamav.net/show_bug.cgi?id=12389 > > Thanks. But I have one more question. Do I understand correctly > that when loading main.cvd

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-10-06 Thread Sergey
On Friday 13 September 2019, Markus Kolb via clamav-users wrote: > I've opened an enhacement bug for this: > https://bugzilla.clamav.net/show_bug.cgi?id=12389 Thanks. But I have one more question. Do I understand correctly that when loading main.cvd base rules are created quickly and the proble

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-09-13 Thread Markus Kolb via clamav-users
I've opened an enhacement bug for this: https://bugzilla.clamav.net/show_bug.cgi?id=12389 ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide:

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-09-13 Thread Markus Kolb via clamav-users
Am 13.09.2019 12:09, schrieb Sergey: On Tuesday 10 September 2019, Markus Kolb wrote: Maybe these signatures got added in the last months. My logfiles tell me that the startup time of clamd was ok until February/March 2019. Yes, I didn't notice any problems at the beginning of the year either

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-09-13 Thread Sergey
On Tuesday 10 September 2019, Markus Kolb wrote: > Maybe these signatures got added in the last months. My logfiles tell > me that the startup time of clamd was ok until February/March 2019. Yes, I didn't notice any problems at the beginning of the year either. Meanwhile, the time increased by a

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-09-09 Thread Al Varnell via clamav-users
On Mon, Sep 09, 2019 at 13:46 PM, Markus Kolb wrote: > Sergey wrote: > >> Since some time there has been a noticeable increase a launch time. >> Startup with all databases takes about 220 seconds now. Startup >> without daily.cld takes 12 seconds. What's happened with daily.cld? > > Hey, > > sor

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-09-09 Thread Markus Kolb
Sergey wrote: Since some time there has been a noticeable increase a launch time. Startup with all databases takes about 220 seconds now. Startup without daily.cld takes 12 seconds. What's happened with daily.cld? Hey, sorry for breaking the thread, I've just subscribed and have found your p

Re: [clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-09-04 Thread Sergey
On Wednesday 04 September 2019, Sergey wrote: > Since some time there has been a noticeable increase a launch time. > Startup with all databases takes about 220 seconds now. Startup > without daily.cld takes 12 seconds. What's happened with daily.cld? freshclam downloaded "daily.cvd" in next i

[clamav-users] Continuous increase of startup time (is daily.cld broken?)

2019-09-04 Thread Sergey
Hello. Since some time there has been a noticeable increase a launch time. Startup with all databases takes about 220 seconds now. Startup without daily.cld takes 12 seconds. What's happened with daily.cld? Thu Jul 18 12:36:51 2019 -> clamd daemon 0.101.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86