Re: [Clamav-users] .ppt files take a long time to scan
On 3/16/06, Christopher X. Candreva <[EMAIL PROTECTED]> wrote: > I'm running into issues where (so far as I can tell) .ppt files can take a > long time to scan. As an exmaple, I have a 2.8 meg 5 slide .ppt file that > takes 90 seconds to scan on an otherwise-quiet 1.5ghz Athlon. > > For camparison, a random 3 meg .pdf file scanned in under a second. > > Is this normal, expected, a known issue, or should I be looking for a > mistake I've made ? Known issue. :( I posted on it last year with no particular resolution. As well as Office formats it appears to afflict large XML files too. e.g. http://lurker.clamav.net/message/20051217.152437.bcf5.en.html http://lurker.clamav.net/message/20050922.133756.641817a2.en.html "Your disk is slow" or "don't scan large files" is a common response. If you can provide a sample file to Trog to help find out what the real issue is that would be great. -- des -- http://frommars.org/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] RE: File Attachment Size Problem
On 1/30/06, Bill King <[EMAIL PROTECTED]> wrote: > Thanks! This is working. However I am thinking of trying to skip the scan > of large messages as I am not sure if it is worth the CPU ticks. Does > anyone have ideas about whether or not this is a good plan? There are two schools of thought on this: 1. Your target for mail scanning should be restricted to blocking viral worms which are almost all sending relatively small attachments in order to spread quickly and efficiently. 2. Your target should be any incoming infected file. If you believe in (1) and that your desktop AV software will protect you from (2) then put in an attachment size restriction. The standard mimedefang-filter has an example of this in place for SpamAssassin scanning, restricted to 100K for the same reasons as (1) above. If you believe in (2) then you have to throw hardware at it. IMHO ClamAV isn't terribly efficient at scanning large files and appears to have particular issues with documents that it parses such as XML and MS Office filetypes. Throwing CPUs at it and increasing your timeout limits works ok though. You could also consider prioritising smaller messages if you have limited resources. -- des -- http://frommars.org/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] File Attachment Size Problem
On 1/27/06, Bill King <[EMAIL PROTECTED]> wrote: > I am running ClamAV on a Solaris host, with MIMEDefang. Versions and log > examples are posted below. I am trying to modify the file attachment size > limit for clamd which defaults to 10Mb. I have modified the > "StreamMaxLength" in clamd.conf with no love. This setting seems to apply > only if clamav-milter is configured, which I do not have running because > I'm already using MIMEDefang. > > Jan 26 12:05:31 MTA_Daemon[4795]: Milter (mimedefang): timeout before > data read This sounds like a milter timeout rather than clamd. Check your milter configuration in sendmail.mc, if it says something like S:1m;R:1m it's too low for scanning large messages. Try something like: INPUT_MAIL_FILTER(`mimedefang', `S=unix:/var/spool/MIMEDefang/mimedefang.sock, F=T, T=S:5m;R:5m;E:10m') You might need even higher timeouts depending on your server load and the type/size of messages you want to allow. You also need to start the mimdefang-multiplexor with "-b 280" or similar - a number of seconds a little lower than the sendmail timeout you have configured, otherwise you'll get "Filter timed out" messages from MIMEdefang instead. -- des -- http://frommars.org/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] XML and large file scan performance
On 12/17/05, GiM <[EMAIL PROTECTED]> wrote: > des in message '[Clamav-users] XML and large file scan performance' wrote: > > In investigating heavy load during clamd scanning on an email server, > > I noticed that scanning XML files appears to take longer than similar > > sized binary files. I'm also seeing a big drop off in performance when > > scanning certain large files, e.g. PowerPoint, Word. Tests with > > clamdscan on a dual PIII 1.2GHz: > > > > 1. MS Office files are treated differently then normal files. Order of magnitude slower though? > 2. Are you scanning under windows or *nix system? Linux. > 3. How much tests have you made on each file? The results were an average of 5 runs per file. > 4. Your Scanning rates seems to be below 2MB/s, >this mean, you probably haven't compiled DNA into the kernel This was with SCSI disk and 2GB RAM, running clamdscan against files on the filesystem (eliminating any email part of the equation). The other scanner was running from command line against the same files on disk and wasn't daemonised. The second scanner isn't so much the point though, I'm interested in improving clamd performance. :) The 7MB file took 1.5s with the other scanner. Looking at disk transfer speeds doesn't come close to explaining the PowerPoint scan time (66s). So you are seeing much lower scan times with similar sized files of these types? You don't see the big difference between file types? If so, that's great, I can investigate the OS/hardware setup. I did some brief tests on 3.2 GHz Xeon (same OS) and saw the same the same pattern, but didn't look in detail. Thanks, -- des -- http://frommars.org/ ___ http://lurker.clamav.net/list/clamav-users.html
[Clamav-users] XML and large file scan performance
In investigating heavy load during clamd scanning on an email server, I noticed that scanning XML files appears to take longer than similar sized binary files. I'm also seeing a big drop off in performance when scanning certain large files, e.g. PowerPoint, Word. Tests with clamdscan on a dual PIII 1.2GHz: 750KB Excel file: 0.36s 2MB Word doc: 5.0s 3MB binary file: 0.75s 3MB XML file: 1.28s 6MB SO library: 1.51s 7MB XML file: 3.60s 7.5MB PowerPoint file: 66s A different scanner tested on the same hardware stayed below 1.5s for all the tests (below 0.3s for most of them). Are there any options for tuning clamd, short of buying oodles more and faster cpus? Thanks, -- des -- http://frommars.org/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Password protected ZIP's---howto?
On 20/06/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Johnny Stork wrote: > >>>Is there any way to get clamav to handle password protected > >>> zip files? We > >>>receive and send many files as pw protected zips and since > >>deploying clamav, they have all been flagged as viruses? > >> > >>ArchiveBlockEncrypted > >> Mark encrypted archives as viruses > >> (Encrypted.Zip, Encrypted.RAR). > >> Default: disabled > > Thanks kindly, but I guess this means that they pass through without > > being scanned/checked? > > ClamAV can't scan encrypted archives, because there's no way to tell it the > password. Unless the encrypted archive matches a signature in it's encrypted > form, there's no virus detection here. > > It can either uniformly let through, or uniformly block, all encrypted > archives. > > If you want sophisticated zip file handling, consider MIMEDefang [1] and > Archive::Zip > > [1] www.mimedefang.com Right. I've used this approach to block encrypted zips containing filetypes that are "suspicious" (exe, pif, etc.), but haven't matched a virus signature. You can only scan one level deep. But that way you can let through encrypted zips containing xls files or whatever you consider possibly legimitate traffic. There's an example filter on the MIMEdefang site with some details of using Archive::Zip IIRC. -- des -- http://frommars.org/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Memory limit per process hit
On 27/05/05, Pablo Alsina <[EMAIL PROTECTED]> wrote: > So what we did was to increment the number of childers to an even > bigger value. But then we started to hit with other problems: > > clamav-milter[1932]: ClamAv: thread_create() failed: 12, try again > > We did an strace to that process, only to find out that we are running > out of memory: I had a similar problem using MIMEdefang rather than clamav-milter. See what default stack size is (ulimit -s). Reducing this in your sendmail init script, e.g. "ulimit -s 2048" can help. Worked for me. See earlier thread on this one: http://www.mail-archive.com/clamav-users@lists.sourceforge.net/msg08540.html And then you might be able to inconvenience 10 spammers instead of 1 before they DoS your mail service. But have fun! :) des -- des -- http://frommars.org/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] EICAR signature update: second attempt
On Fri, 18 Feb 2005 15:45:54 -0600 (CST), Damian Menscher <[EMAIL PROTECTED]> wrote: > Latest version of clamav, or clamdwatch? Latest version of clamdwatch (0.7.1, as distributed with clamav 0.83). > Why is the most recent version required (I'm assuming some new > functionality is required, but when was that functionality introduced)? > Can you give us a preview of what the new signature will be so we can > prepare ourselves? AFAIK, the updated Eicar signature used by clamav will enforce the rule that it may be no more than 128 bytes long and only contains whitespace after the signature itself. Older versions of clamdwatch contain the Eicar signature within the perl script itself. They feed the entire perl script to clamd to check if detects the Eicar signature. Until now this would work fine. From Monday the >128 bytes and lots of non-whitespace nature of the script will mean that clamd will not detect the Eicar signature in the clamdwatch script. Newer versions (use 0.7.1) of clamdwatch write a temporary file containing only the Eicar virus. Equals < 128 bytes and nothing but Eicar goodness in the file. This is fed to clamd, which detects the Eicar signature now and after Monday, resulting in happy sysadmins. This functionality was first introduced in clamdwatch v0.7rc1 (12/09/2004), I believe, but you want the current 0.7.1 version. The Eicar string itself hasn't changed, AFAIK, but clamav was forgiving in interpreting the string itself anywhere in a file as sufficient to make a positive match, whereas the "standard" specifies "It may be optionally appended by any combination of whitespace characters with the total file length not exceeding 128 characters." (http://www.eicar.org/anti_virus_test_file.htm) Hope this helps, -- des -- http://frommars.org/ ___ http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
