Cant those be adopted / managed by Sanesecurity?
For all you know, those are already in Sanesecurity.
Regards
Brent Clark
On 2019/04/09 12:25, Mark Allan via clamav-users wrote:
The scan times are definitely better than they were - in fact, they're
back to how they were before last week's inclusion of the Phishtank
signatures. They're still almost double what they used to be though, and
as far as I can see, there are still almost 4000 Phishtank signatures in
the DB:
$ sigtool --find Phishtank | wc -l
3968
Can I request that those ones also be removed please?
Best regards
Mark
On Sun, 7 Apr 2019 at 14:43, Micah Snyder (micasnyd) <micas...@cisco.com
<mailto:micas...@cisco.com>> wrote:
Tim,____
__ __
There are a couple of ways for users to drop specific categories of
signatures at this time. Sadly, they wouldn’t have helped this last
week. These include bytecode signatures, PUA (potentially unwanted
applications) signatures, Email.Phishing and HTML.Phishing
signatures, and the Safebrowsing database. ____
__ __
If we had named the Phishtank.Phishing sigs to
HTML.Phishing.Phishtank or Email.Phishing.Phishtank then they could
have been disabled with the clamscan option `--phishing-sigs=no`
(clamd.conf: `PhishingSignatures no`).____
__ __
Maybe a better option would be for us to create a new optional
database for phishing signatures. However, the names for the
databases are hardcoded into freshclam, so it is non-trivial to add
a new database and would require a few changes to ClamAV’s code. We
have talked about making the databases easier to add/remove in the
future so users can have more categories to enable/disable. In this
light, it ties in well with existing plans.____
__ __
Of note the Phishtank sigs from Friday’s daily were removed
yesterday and scan times should be back to normal. ____
__ __
Regards,____
Micah____
__ __
*From: *Tim Hawkins <tim.hawk...@redflaggroup.com
<mailto:tim.hawk...@redflaggroup.com>>
*Date: *Friday, April 5, 2019 at 6:06 PM
*To: *ClamAV users ML <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>, Mark Allan
<markjal...@gmail.com <mailto:markjal...@gmail.com>>
*Cc: *"Micah Snyder (micasnyd)" <micas...@cisco.com
<mailto:micas...@cisco.com>>
*Subject: *Re: [External] Re: [clamav-users] Scan very slow____
__ __
Hi Micah____
Does clamav partition the database so that signatures that are
mainly associated with email scanning can be dropped out for folks
only needing filesystems scans, none of our systems use email, and
we dont make use of the mailer extension.
Having to load all the email focused signatures could as you have
observed impact performance. ____
Sent from Nine <http://www.9folders.com/>____
------------------------------------------------------------------------
*From:* "Micah Snyder (micasnyd) via clamav-users"
<clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
*Sent:* Saturday, April 6, 2019 03:18
*To:* ClamAV users ML; Mark Allan
*Cc:* Micah Snyder (micasnyd)
*Subject:* [External] Re: [clamav-users] Scan very slow____
__ __
Regarding slow scan times today (and slow scan times in general), it
appears that the signatures we generate based on PhishTank’s feed
for phishing URLs are resulting in very slow load and scan times.____
____
Today’s daily update saw 7448 new Phishtank signatures (much higher
than usual) coinciding with the immediate performance drop for load
time and scan time. One user reported that the load time today on
some of his slower machines was slow enough to exceed the timeout
for service startup
(https://bugzilla.clamav.net/show_bug.cgi?id=12317).____
____
In limited testing on my own machine I saw the following change
after dropping the Phishtank.Phishing signatures from daily.cvd’s
daily.ldb file:____
* Database load time on my laptop went from 75.43203997612 seconds
down to 14.859203100204468 seconds ____
* Scan time (for an arbitrary pdf) went from 1.798 sec to 0.644
sec.____
____
After some discussion between the teams that work on ClamAV and
ClamAV signature content and deployment, we’ve agreed to drop
PhishTank signatures from the database until we can determine a way
to craft Phishtank signatures without incurring such a significant
performance hit. ____
____
The daily update tomorrow will have the change.____
____
-Micah____
____
Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.____
____
____
____
*From: *clamav-users <clamav-users-boun...@lists.clamav.net
<mailto:clamav-users-boun...@lists.clamav.net>> on behalf of "Micah
Snyder (micasnyd) via clamav-users" <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>
*Reply-To: *ClamAV users ML <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>
*Date: *Friday, April 5, 2019 at 1:08 PM
*To: *Mark Allan <markjal...@gmail.com
<mailto:markjal...@gmail.com>>, ClamAV users ML
<clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
*Cc: *"Micah Snyder (micasnyd)" <micas...@cisco.com
<mailto:micas...@cisco.com>>
*Subject: *Re: [clamav-users] Scan very slow____
____
Hi Mark,____
____
Sorry about the delay in responding. I hadn’t looked at my
clamav-users filter this morning. Just investigating now. Will
respond when I know more. ____
____
-Micah____
____
*From: *Mark Allan <markjal...@gmail.com <mailto:markjal...@gmail.com>>
*Date: *Friday, April 5, 2019 at 9:12 AM
*To: *ClamAV users ML <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>, "Micah Snyder (micasnyd)"
<micas...@cisco.com <mailto:micas...@cisco.com>>
*Subject: *Re: [clamav-users] Scan very slow____
____
Also CC'ing Micah directly as the mailing list would appear to be
offline (at least lists.clamav.net <http://lists.clamav.net> isn't
responding to http requests anyway) ____
____
It looks like scan times have gone through the roof. As Oya said,
they're still considerably higher than they were a couple of months
ago, but today's scan time is insane.____
____
Yesterday's scan using____
0.101.2:58:25409:1554370140:1:63:48554:328____
took 7m 3s____
____
On the same hardware, scanning the same read-only disk image, with
today's scan using____
0.101.2:58:25410:1554452941:1:63:48557:328____
the scan time has jumped to 26m 15s____
____
This is the longest it has ever taken to scan this volume (cf my
previous email of 25th March)____
____
Is there anything that can be excluded?____
____
Best regards____
Mark____
____
On Mon, 1 Apr 2019 at 17:11, Micah Snyder (micasnyd) via
clamav-users <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>> wrote:____
Thanks Oya for the update. We will continue to investigate the
signature performance issue.
Regards,
Micah
On 3/28/19, 9:50 AM, "clamav-users on behalf of Tsutomu Oyamada"
<clamav-users-boun...@lists.clamav.net
<mailto:clamav-users-boun...@lists.clamav.net> on behalf of
oyam...@promark-inc.com <mailto:oyam...@promark-inc.com>> wrote:
Hi Micah
It seems that the scanning slow down issue of this time
has been solved
at some level with CVD Update of the other day.
However, there is still big discrepancy in between the
current condition and
the last condition in one month ago.
Date Files Scan time
2019/02/15 2550338 08:53:57
2019/03/15 2612792 19:22:54
2019/03/26 2634489 18:13:56
2019/03/27 2637201 18:10:05
We know the improvement of this time is due to the details
of CVD, because
we did not make any change on the user's system.
We are going to try some tuning for scanning.
We like to know if you still have some room to make further
improvement
for this slow down issue.
Thank you for your help, in advance.
Best regards,
Oya
On Mon, 25 Mar 2019 15:45:02 +0000
"Micah Snyder \(micasnyd\) via clamav-users"
<clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>> wrote:
> Hi Mark, all:
>
> I’m disappointed to hear that it is still slow for you.
>
> We found that the target-type of signatures used for
PhishTank.Phishing signatures were causing a significant
slowdown. We have dropped them as of this past Saturday (
https://lists.gt.net/clamav/virusdb/75279 ) and in the last two
updates have been re-adding them with more specific scan target
types. We’re now investigating some other optimizations we can
make for the next major ClamAV release to improve scan times but
at present we don’t have any other leads for signatures that may
be slowing down scans.
>
> Regards,
> Micah
>
>
> From: clamav-users <clamav-users-boun...@lists.clamav.net
<mailto:clamav-users-boun...@lists.clamav.net>> on behalf of
Mark Allan via clamav-users <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>
> Reply-To: ClamAV users ML <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>
> Date: Monday, March 25, 2019 at 9:37 AM
> To: ClamAV users ML <clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>
> Cc: Mark Allan <markjal...@gmail.com
<mailto:markjal...@gmail.com>>
> Subject: Re: [clamav-users] Scan very slow
>
> Cheers Steve,
>
> In the interest of completeness, here's the scan from
today (TXT from DNS: 0.101.1:58:25399:1553509741:1:63:48528:328)
showing a marked improvement in scan time, although at 6m 7s
it's still almost twice what it used to be.
>
> Mark
>
> On Mon, 25 Mar 2019 at 12:56, Steve Basford
<steveb_cla...@sanesecurity.com
<mailto:steveb_cla...@sanesecurity.com><mailto:steveb_cla...@sanesecurity.com
<mailto:steveb_cla...@sanesecurity.com>>> wrote:
> On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> > Hi all,
> >
> te.
> >
> > Hopefully this helps someone to narrow things down a bit.
> >
> > Mark
> >
>
> 18/3/19 10m 49s TXT from DNS:
> 0.101.1:58:25392:1552904941:1:63:48507:328 ***
>
> Here's the changes for the above update:
>
> https://lists.gt.net/clamav/virusdb/75154
>
> You can also check sigs quickly per update:
>
> https://lists.gt.net/clamav/virusdb/
>
>
>
> --
> Cheers,
>
> Steve
> Twitter: @sanesecurity
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net><mailto:clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml____
*DISCLAIMER*
____
The information contained in this email and any attachments are
confidential. It is intended solely for the individual or entity to
whom they are addressed. Access to this email by anyone else is
unauthorized.____
If you are not the intended recipient, any disclosure, copying,
distribution or any action taken or omitted to be taken in reliance
on it, is prohibited and may be unlawful. If you have received this
communication in error, please notify us immediately by responding
to this email and then delete it from your system.____
The Red Flag Group is neither liable for the proper and complete
transmission of the information contained in this communication nor
for any delay in its receipt.____
Any advice, recommendations or opinion contained within this email
or its attachments are not to be construed as legal advice.____
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
