Re: [Clamav-users] Creating your own Signatures: Bound Offset
On Fri, 2008-01-25 at 18:41 -0800, Dennis Peterson wrote:
> Karsten Bräckelmann wrote:
> > On Fri, 2008-01-25 at 17:54 -0800, Dennis Peterson wrote:
> >> The sigs are full of unbound RE's. That's why scanning mbox mail files is
> >> pointless.
> >
> > Yes, I know. I contributed that fact to the thread a while ago...
> >
> > I do realize the ambiguity here -- there is no plural for 'mail'. :)
> > However, I am talking about a *single* mail. If I would have been
> > talking about mbox files, I'd have used that term.
> I've been out of town and haven't got caught up on all the world's history.
Dennis, now you're confusing me. :)
Nothing to catch up with, I've been referring to the thread "Getting
line numbers" back in Oct. Both of us have been discussing that topic.
> ClamAV's archives on on the list. Bounded (and anchored) RE's always
> run faster and they're more accurate. What's to lose?
I know about the archives, I've been a long time subscriber. Anyway...
What's to lose? Well, as per my OP, it just doesn't work. ClamAV freaks
out, when you start a hex signature with a (bounded) wildcard.
Besides, I'm not convinced bounded wildcards [1] actually do run faster
in clam. Haven't looked at the engines code, but given the rather
limited set of wildcards, I doubt it uses backtracking. And the bound
does impose another constraint while scanning the stream, no?
Good point about running faster when anchored, though. :)
guenther
[1] The doc talks about wildcards -- rightly so. They are no REs. The
only thing that at least comes close is the alternation.
--
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
___
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Creating your own Signatures: Bound Offset
Karsten Bräckelmann wrote: > On Fri, 2008-01-25 at 17:54 -0800, Dennis Peterson wrote: >> Karsten Bräckelmann wrote: >> >>> The main purpose was, to keep ClamAV from scanning the entire, possibly >>> large file (err, mail). And maybe even speed it up. It's good practice >>> to bound your REs or wildcards anyway. >>> >>> I wonder, if this indeed would speed up scanning, however small, of >>> large-ish files. Or would the additional constraint actually impose more >>> CPU cycles spent? >> The sigs are full of unbound RE's. That's why scanning mbox mail files is >> pointless. > > Yes, I know. I contributed that fact to the thread a while ago... > > I do realize the ambiguity here -- there is no plural for 'mail'. :) > However, I am talking about a *single* mail. If I would have been > talking about mbox files, I'd have used that term. > > Dennis, thanks for your reply. Just doesn't answer the question, > unfortunately... ;) > > guenther > > I've been out of town and haven't got caught up on all the world's history. ClamAV's archives on on the list. Bounded (and anchored) RE's always run faster and they're more accurate. What's to lose? dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Creating your own Signatures: Bound Offset
On Fri, 2008-01-25 at 17:54 -0800, Dennis Peterson wrote:
> Karsten Bräckelmann wrote:
>
> > The main purpose was, to keep ClamAV from scanning the entire, possibly
> > large file (err, mail). And maybe even speed it up. It's good practice
> > to bound your REs or wildcards anyway.
> >
> > I wonder, if this indeed would speed up scanning, however small, of
> > large-ish files. Or would the additional constraint actually impose more
> > CPU cycles spent?
>
> The sigs are full of unbound RE's. That's why scanning mbox mail files is
> pointless.
Yes, I know. I contributed that fact to the thread a while ago...
I do realize the ambiguity here -- there is no plural for 'mail'. :)
However, I am talking about a *single* mail. If I would have been
talking about mbox files, I'd have used that term.
Dennis, thanks for your reply. Just doesn't answer the question,
unfortunately... ;)
guenther
--
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
___
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Creating your own Signatures: Bound Offset
Karsten Bräckelmann wrote: > > The main purpose was, to keep ClamAV from scanning the entire, possibly > large file (err, mail). And maybe even speed it up. It's good practice > to bound your REs or wildcards anyway. > > I wonder, if this indeed would speed up scanning, however small, of > large-ish files. Or would the additional constraint actually impose more > CPU cycles spent? The sigs are full of unbound RE's. That's why scanning mbox mail files is pointless. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
[Clamav-users] Creating your own Signatures: Bound Offset
So I finally got around to writing some (well, one for now ;) custom
signatures. There's currently a highly annoying, lame phishing attempt I
want to swat early.
Anyway, while playing with the sigs and trying some optimization, the
sig broke horribly for some weird reason. Please see below for a
stripped down test case. What's so bad about it?
Instead of using the "any" offset, I tried to bound it, by setting the
offset to 0, and starting the hex signature with a limited wildcard.
Also, I noticed the parser isn't happy, if there is such a wildcard with
less than 2 bytes either at the beginning or end of the string.
Well, I could just start the sig with "From " and then anchor it at
offset 0. :) But the question remains -- why?
Another question: Does this actually make sense?
The main purpose was, to keep ClamAV from scanning the entire, possibly
large file (err, mail). And maybe even speed it up. It's good practice
to bound your REs or wildcards anyway.
I wonder, if this indeed would speed up scanning, however small, of
large-ish files. Or would the additional constraint actually impose more
CPU cycles spent?
Thanks for any insight. :)
guenther
$ cat test.ndb
local.test:4:0:{-4096}74657374
$ clamscan --quiet -d test.ndb msg
LibClamAV Error: cli_parse_add(): Problem adding signature (1).
LibClamAV Error: Problem parsing signature at line 1
LibClamAV Error: Problem parsing database at line 1
LibClamAV Error: Can't load test.ndb: Malformed database
ERROR: Malformed database
$ clamscan --version
ClamAV 0.92/5553/Fri Jan 25 22:14:29 2008
--
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
___
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] help-about regular expressions in signatures
xue wen wrote:
> To whom it may concern,
>
> I have tried to understand the signatures in the ClamAV's database. I have
> succeeded to add a string signature into .db file. And when I tried to add a
> regular expression into signature, there were some errors. I have referred
> to the document of signatures.pdf and followed the instructions to add *, ?
> and | etc, into the hex signatures. But when I used these signature as rule,
> the error was printed like this:
>
> LibClamAV Error: cli_parse_add(): Problem adding signatures (2).
> Problem parsing signature at line 1
> Problem parsing database at line 1
> Can't load daily.db: Malformed database
> ERROR: Malformed database
There seems to be a complexity limit on wildcard signatures; for a
while I was automating part of the process of generating signatures for
image-based spam. The automated process regularly produced signatures
which were structured properly, but which were rejected by Clam as
"malformed". Trimming them down (usually just trimming the end off
until it worked) was the only way I could get them functioning.
Nobody really answered my confusion when I asked about it at the time
(late October 2006, check the list archives for "Complexity limits on
(custom) signatures?"), although there was some interesting discussion
that came out of it.
If you post examples, and what you're hoping to match on, several people
on this list can probably point out what you're doing wrong.
> Are there regular expressions in ClamAV's virus signatures? If so, why can't
> I add some into them?
Mmmh... Clamav signatures include a *very* small subset of most regex
syntax - (aa|bb), ?? as "anything", and {nn} to compress a long string
of ?'s down. It's been a while since I looked at creating signatures
myself so I don't recall what other bits there are.
-kgd
___
Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net
http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Password Protected ZIP Files
Dennis Peterson wrote: > In my environment I use a milter to call clamav and it allows me to skip av > testing based on several criteria including To: and From: addresses. > Perhaps you have something similar. We are using Zimbra, I am not sure if there is a way to specify to skip av scan from a specific address... If anybody else is using Zimbra and has run into a simular situation, I would greaty appreciate any assistance on how to "tell" Zimbra to skip av scan if the email is being sent from a specific address. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Password Protected ZIP Files
Donald Johnson wrote: > I have a process which generates a ZIP file and emails it... This file is > REQUIRED to be Password protected. > > The password is the same every time it is generated, and it goes to the same > recipients each time... > > I really don't want to turn off the feature to block Encrypted ZIP files... > > Is there a way to tell ClamAV what password to try on the ZIP file? > If not, could there be a consideration of adding the feature? In my environment I use a milter to call clamav and it allows me to skip av testing based on several criteria including To: and From: addresses. Perhaps you have something similar. dp ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Password Protected ZIP Files
>I have a process which generates a ZIP file and emails it... This file is >REQUIRED to be Password protected. >The password is the same every time it is generated, and it goes to the same >recipients each time... >I really don't want to turn off the feature to block Encrypted ZIP files... >Is there a way to tell ClamAV what password to try on the ZIP file? >If not, could there be a consideration of adding the feature? Would it not just be easier to instruct your mail server not to pass the email through Clamav? I guess from your reference to "Block Encrypted ZIP files" that you might be using MailScanner. If this is the case, post on the MailScanner list, and someone will help you create a ruleset if you're having problems. HTH, Best Regards, Richard Garner (A+, N+, AMBCS, MOS-O) All E-Mail communications are monitored in addition to being content checked for malicious codes or viruses. The success of scanning products is not guaranteed, therefore the recipient(s) should carry out any checks that they believe to be appropriate in this respect. This message (including any attachments and/or related materials) is confidential to and is the property of Computer Service Centre, unless otherwise noted. If you are not the intended recipient, you should delete this message and are hereby notified that any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited. Any views or opinions presented are solely those of the author and do not necessarily represent those of Computer Service Centre. ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
[Clamav-users] Password Protected ZIP Files
I have a process which generates a ZIP file and emails it... This file is REQUIRED to be Password protected. The password is the same every time it is generated, and it goes to the same recipients each time... I really don't want to turn off the feature to block Encrypted ZIP files... Is there a way to tell ClamAV what password to try on the ZIP file? If not, could there be a consideration of adding the feature? ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] help-about regular expressions in signatures
xue wen wrote: > LibClamAV Error: cli_parse_add(): Problem adding signatures (2). > Problem parsing signature at line 1 > Problem parsing database at line 1 > Can't load daily.db: Malformed database > Wildcard signatures go into a .ndb file. > ERROR: Malformed database > > Are there regular expressions in ClamAV's virus signatures? If so, why can't > I add some into them? > > They are not full regular expressions, signatures.pdf calls them wildcards. $ sigtool --unpack main.cvd Have a look at main.ndb Best regards, --Edwin ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
[Clamav-users] help-about regular expressions in signatures
To whom it may concern, I have tried to understand the signatures in the ClamAV's database. I have succeeded to add a string signature into .db file. And when I tried to add a regular expression into signature, there were some errors. I have referred to the document of signatures.pdf and followed the instructions to add *, ? and | etc, into the hex signatures. But when I used these signature as rule, the error was printed like this: LibClamAV Error: cli_parse_add(): Problem adding signatures (2). Problem parsing signature at line 1 Problem parsing database at line 1 Can't load daily.db: Malformed database ERROR: Malformed database Are there regular expressions in ClamAV's virus signatures? If so, why can't I add some into them? Thank you very much. Regards, Xue Wen ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Clamav Update Database
On Jan 25, 2008 12:53 PM, Clovis Tristao <[EMAIL PROTECTED]> wrote: > Hi All, > > I'm using Clamav in Server Fedora Core. > Please, How I up to date clamav databases automatically and I receive > e-mails saying that the system was brought up to date? Take a look at OnUpdateExecute in freshclam.conf -- Please keep list traffic on the list. Rob MacGregor Whoever fights monsters should see to it that in the process he doesn't become a monster. Friedrich Nietzsche ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Clamav Update Database
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/25/2008 06:53 AM, Clovis Tristao wrote: | Hi All, | | I'm using Clamav in Server Fedora Core. | Please, How I up to date clamav databases automatically and I receive | e-mails saying that the system was brought up to date? | Thanks a lot, The database status is reported every morning from logwatch. Do you need to monitor every update or update attempt? - -- ~ Steve -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFHmePIeERILVgMyvARAqUAAJsF7KSK0zVWkXHRGaxrvqULQJpfNgCeKKZr mZGILIDqcA+mqXNt5RddRWg= =LqGJ -END PGP SIGNATURE- ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
[Clamav-users] Clamav Update Database
Hi All, I'm using Clamav in Server Fedora Core. Please, How I up to date clamav databases automatically and I receive e-mails saying that the system was brought up to date? Thanks a lot, Clóvis -- Clovis Tristao - UNICAMP/Faculdade de Engenharia Agricola Administrador de Redes - Secao de Informatica (SINFO) E-mail: [EMAIL PROTECTED] http://www.feagri.unicamp.br Fone(0xx19) 35211031-35211038 ou FAX(55xx19) 35211005/35211010 ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://lurker.clamav.net/list/clamav-users.html
