RE: [Declude.Virus] F-Prot missing viruses and is slow (renamed)
Title: Message Yes, during the entire interval I measured the CPU time was 98-100% for the fpcmd.exe process only. On LOGLEVEL MED, there is a line that shows the errorlevel returned by the scanner, plus the error line indicating that the search string wasn't found in the resulting text file, e.g. this is what is returned on my v2.0.6 system when a "suspicious file" is returned: 04/27/2005 07:48:33 QA63CBF0600647AB8 Could not find parse string Infection: in report.txt04/27/2005 07:48:33 QA63CBF0600647AB8 File(s) are INFECTED [: 8]04/27/2005 07:48:33 QA63CBF0600647AB8 Scanned: CONTAINS A VIRUS [MIME: 3 23729]04/27/2005 07:48:33 QA63CBF0600647AB8 From:munged To:munged [outgoing from 70.187.178.183]04/27/2005 07:48:33 QA63CBF0600647AB8 Subject: Forum notify The resulting virus name is [Unknown File] butadding such a line to my FORGINGVIRUS strings doesn't stop the notification email (but they only go to postmaster, so no big deal for me). I don't know if it made it into the support database, but on testing Declude Virus, I immediately requested a feature enhancement to extend the virus matching string "REPORT" parallel with the "VIRUSCODE" lines for this reason. Otherwise, Matt, I agree on both of your conclusions regarding how F-Prot falls short. Andrew 8) -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MattSent: Thursday, April 28, 2005 9:16 PMTo: Declude.Virus@declude.comSubject: Re: [Declude.Virus] F-Prot missing viruses and is slow (renamed)Ok, follow-up time. It appears that Declude is detecting this with VIRUSCODE 8 and I was just merely confused by the logs. I set things to Debug and found the following: 04/29/2005 00:06:48.652 QB2D6AB7001342A79 [6224] Virus Scanner Started: C:\Progra~1\FSI\F-Prot\fpcmd.exe -SILENT -NOBOOT -NOMEM -ARCHIVE=5 -PACKED -SERVER -DUMB -REPORT=report.txt F:\DB2D6A~1.VIR\04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] Scanning Time: 4812ms [kernel=78 user=4734]04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] Virus scanner 1 reports exit code of 804/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] F:\DB2D6AB7001342A79.vir\04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] F:\DB2D6AB7001342A79.vir\report.txt04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] report.txt len=722 rflen=35 cs=004/29/2005 00:06:53 QB2D6AB7001342A79 Could not find parse string Infection: in report.txtSo I would assume that on other log levels and with other scanners detecting the viruses, there just isn't a clear indication of the virus being found with F-Prot, but it is in fact being detected. Maybe Declude should change the logging to indicate the exit code in other log levels when it matches a VIRUSCODE value.That leaves two real issues; 1) Time/CPU utilization with F-Prot, and 2) F-Prot continuing to report viruses with an exit code of 8.MattMatt wrote: Colbeck, Andrew wrote: F-Prot is indeed returning an errorlevel of 8 on this, and it's definitely way out of line with the scanning time on this file.Your script no doubt shows that F-Prot returns an error level of 8 when run on this file, however there is one big issue here...I have declude now set for VIRUSCODE 8 and it isn't detecting it. I just tested this by sending it to myself and it still didn't detect it as a virus. Here's my config: SCANFILE1 C:\Progra~1\FSI\F-Prot\fpcmd.exe /TYPE /SILENT /NOBOOT /NOMEM /ARCHIVE=5 /PACKED /DUMB /REPORT=report.txtVIRUSCODE1 3VIRUSCODE1 6VIRUSCODE1 8REPORT1 Infection: I used this same command line with your script, making obvious edits for the path and it returned an 8. I'm confused why either Declude isn't picking this up, or why F-Prot isn't somehow reporting it to Declude properly...The time issue is also a big deal of course, but probably not as big as Declude with F-Prot missing it. Can anyone confirm with this sample file whether or not Declude with F-Prot and VIRUSCODE 8 is catching this? I did get a reply on my previous report to them (after 6 days); they brought my request to the attention of the developers, but then reminded me that any non-zero return code is "undesirable". The request was to re-classify Mitglieder from "suspicious" to "virus" so that I could get the correct return code and thus the correct handling in my Declude Virus.I got what was probably the exact same response after a similar amount of time. The person that replied didn't understand the question or used something that was canned. I replied back again nevertheless. I haven't sent anything concerning this issue, although it seems related, but there also seems to be a different bug here with at least F-Prot but possibly also Declude.Matt--
Re: [Declude.Virus] F-Prot missing viruses and is slow (renamed)
Title: Message Andrew, I'm still up doing maintenance... While you are correct about what happens with the error code when only one virus scanner is used, when two are configured like on my system, there is no indication that F-Prot detected a virus unless a REPORT line is matched, which won't happen with a VIRUSCODE 8. In the samples that I previously provided, the only affirmative indication that F-Prot detected a virus is the line "Could not find parse string Infection: in report.txt". 04/28/2005 17:40:57 Q58666795008E87C7 MIME file: [text/html][7bit; Length=695 Checksum=54365] 04/28/2005 17:40:57 Q58666795008E87C7 MIME file: doc.zip [base64; Length=56432 Checksum=6987426] --- 10 second gap while F-Prot scans --- 04/28/2005 17:41:07 Q58666795008E87C7 Could not find parse string Infection: in report.txt 04/28/2005 17:41:08 Q58666795008E87C7 Scanner 2: Virus=the W32/[EMAIL PROTECTED] Attachment= [0] I 04/28/2005 17:41:08 Q58666795008E87C7 File(s) are INFECTED [the W32/[EMAIL PROTECTED]: 13] 04/28/2005 17:41:08 Q58666795008E87C7 Scanned: CONTAINS A VIRUS [Prescan OK][MIME: 3 57490] 04/28/2005 17:41:08 Q58666795008E87C7 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] [incoming from 192.168.100.100] 04/28/2005 17:41:08 Q58666795008E87C7 Subject: [Fwd: Mail Delivery System] Definitely there should be an allowance for multiple REPORT lines to match, but also, it seems to make sense to provide a different indicator showing that a virus was detected and the error code for each scanner. Some scanners don't have parseable reports so when they are run in a multiple scanner config the new logging mechanism would be the only way to properly identify the result for that particular scanner. Matt Colbeck, Andrew wrote: Yes, during the entire interval I measured the CPU time was 98-100% for the fpcmd.exe process only. On LOGLEVEL MED, there is a line that shows the errorlevel returned by the scanner, plus the error line indicating that the search string wasn't found in the resulting text file, e.g. this is what is returned on my v2.0.6 system when a "suspicious file" is returned: 04/27/2005 07:48:33 QA63CBF0600647AB8 Could not find parse string Infection: in report.txt 04/27/2005 07:48:33 QA63CBF0600647AB8 File(s) are INFECTED [: 8] 04/27/2005 07:48:33 QA63CBF0600647AB8 Scanned: CONTAINS A VIRUS [MIME: 3 23729] 04/27/2005 07:48:33 QA63CBF0600647AB8 From:munged To:munged [outgoing from 70.187.178.183] 04/27/2005 07:48:33 QA63CBF0600647AB8 Subject: Forum notify The resulting virus name is [Unknown File] butadding such a line to my FORGINGVIRUS strings doesn't stop the notification email (but they only go to postmaster, so no big deal for me). I don't know if it made it into the support database, but on testing Declude Virus, I immediately requested a feature enhancement to extend the virus matching string "REPORT" parallel with the "VIRUSCODE" lines for this reason. Otherwise, Matt, I agree on both of your conclusions regarding how F-Prot falls short. Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Matt Sent: Thursday, April 28, 2005 9:16 PM To: Declude.Virus@declude.com Subject: Re: [Declude.Virus] F-Prot missing viruses and is slow (renamed) Ok, follow-up time. It appears that Declude is detecting this with VIRUSCODE 8 and I was just merely confused by the logs. I set things to Debug and found the following: 04/29/2005 00:06:48.652 QB2D6AB7001342A79 [6224] Virus Scanner Started: C:\Progra~1\FSI\F-Prot\fpcmd.exe -SILENT -NOBOOT -NOMEM -ARCHIVE=5 -PACKED -SERVER -DUMB -REPORT=report.txt F:\DB2D6A~1.VIR\ 04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] Scanning Time: 4812ms [kernel=78 user=4734] 04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] Virus scanner 1 reports exit code of 8 04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] F:\DB2D6AB7001342A79.vir\ 04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] F:\DB2D6AB7001342A79.vir\report.txt 04/29/2005 00:06:53.667 QB2D6AB7001342A79 [6224] report.txt len=722 rflen=35 cs=0 04/29/2005 00:06:53 QB2D6AB7001342A79 Could not find parse string Infection: in report.txt So I would assume that on other log levels and with other scanners detecting the viruses, there just isn't a clear indication of the virus being found with F-Prot, but it is in fact being detected. Maybe Declude should change the logging to indicate the exit code in other log levels when it matches a VIRUSCODE value. That leaves two real issues; 1) Time/CPU utilization with F-Prot, and 2) F-Prot continuing to report viruses with an exit code of 8. Matt Matt wrote: Colbeck, Andrew wrote: F-Prot is indeed returning an errorlevel of 8 on this, and it's definitely way out of line with the scanning time on this file. Your script no doubt shows that F-Prot returns an error level of 8 when run on this file, however there is one big
RE: [Declude.Virus] High CPU F-Prot
"apparently could add another virus code to Declude for these situations (not yet verified), " Oh, it's verified. As I said, I have been running VIRUSCODE 3,6,8,9 and 10 for at least two years now and not a single report from any customer that ANYthing caught as a virus was needed, meaning no false positives. We run close to a hundred client domains (all businesses)andsee about 20,000 emails a day (the ones that get past our postfix gateway). There has never been a report of a VIRUSCODE 8 catching someone's Word document because of a macro or anything such. The recent rash of new viruses that were getting through other's Declude/Fprot configs never got a single one through mine. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MattSent: Thursday, April 28, 2005 5:24 PMTo: Declude.Virus@declude.comSubject: Re: [Declude.Virus] High CPU F-Prot You should be fine with a second scanner. That's why we use them anyway. McAfee has caught every one of these that I have seen, and I've looked at about 40 examples so far. Many would fail banned extensions otherwise anyway.While you apparently could add another virus code to Declude for these situations (not yet verified), I'm worried that this is more of a general error and it could cause false positives. A corrupted file isn't what I would consider to be uncommon in legit E-mail, although the primary issue is that we only have once sentence with which to evaluate this exit code from F-Prot.Most Declude users that use only F-Prot are probably experiencing significant leakage of otherwise detectable viruses, and are also probably creating extra backscatter for banned extensions where no virus was detected.Besides that there's the fact that F-Prot is taking so long. It appears to also coincide with increased CPU utilization which might explain Darrell's experience, and in a different respect, mine yesterday with all of the F-Prot timeouts. This has been going on for at least a month. I assume that the increased time corresponds to not only keeping more Declude processes open, but also increased CPU utilization. Such a condition is ripe for exploiting, and I'm concerned that it has existed for so long without resolution, and maybe even detection...MattNick wrote: On 28 Apr 2005 at 16:44, Matt wrote: Hi Matt, I assume that this is probably resulting in an exit code of 9 or 10 then because I'm not using either at the moment, and you are the first that I definitively know has them configured. I do not use these codes either - I had 4 "Could not find parse string Infection" in my logs today. The average delay was 4 seconds. Is the answer to add the additl exit codes or is there a downside to that? -Nick 9 - At least one object was not scanned (encrypted file, unsupported/unknown compression method, unsupported/unknown file format, corrupted or invalid file). 10 - At lest one archive object was not scanned (contains more then N levels of nested archives, as specified with -archive switch). Since some of these are not zip files on my system, I am going to assume that it is an exit code of 9 that is being spit out. A file corruption might also explain the issues with F-Prot taking longer on my system. Anyway, I just started to not delete viruses so I should catch one of these soon and then I can work at processing it manually to see what I find. Thanks for sharing. This was helpful. Matt Bill Landry wrote: Matt, I searched 2 weeks of logs on both of my servers (both of which run F-Prot and TrendMicro) and could only find 4 instances of "Could not find parse string Infection", and they were found on the server that is very heavily loaded. I use the following F-Prot strings in my virus.cfg: # F-Prot SCANFILE1 C:\Progra~1\FSI\F-Prot\fpcmd.exe -AI -ARCHIVE=5 -DUMB -NOBOOT -NOBREAK -NOMEM -PACKED -SAFEREMOVE -SERVER -SILENT - REPORT=report.txt VIRUSCODE1 3 VIRUSCODE1 6 VIRUSCODE1 8 VIRUSCODE1 9 VIRUSCODE1 10 REPORT1 Infection: Here is a sample of what I find if I parse for 5 lines before and after the target Q-ID: 04/20/2005 11:53:22 Qa51de08d00e25919 Scanned: Virus Free [MIME: 3 36875] 04/20/2005 11:53:25 Qa523e08f00e25924 MIME file: [text/html][quoted- printable; Length=10177 Checksum=774898] 04/20/2005 11:53:26 Qa523e08f00e25924 Scanned: Virus Free [MIME: 2 11904] 04/20/2005 11:53:27 Qa510a96d00c4590a MIME file: [text/html][quoted- printable; Length=11036 Checksum=792412] 04/20/2005 11:53:28 Qa510a96d00c4590a Scanned: Virus Free [MIME: 2 14609] 04/20/2005 11:53:29 Qa51fa9a300ec591e MIME file: [text/html][7bit; Length=52 Checksum=3520] 04/20/2005 11:53:29 Qa51fa9a300ec591e MIME file: 5.zip [base64; Length=19404 Checksum=2507990] 04/20/2005 11:53:29 Qa51fa9a300ec591e Could not find parse string Infection: in report.txt 04/20/2005 11:53:30 Qa51fa9a300ec591e File(s) are INFECTED [: 0]
RE: [Declude.Virus] F-Prot missing viruses and is slow (renamed)
Title: Message Ding! ... and that's why we've spent so much time on this. The log will show that F-Prot returned an errorlevel, and also the status line that the message contains an infected file. However, when there is more than one scanner, the status line that the message contains an infected file is only logged after both scanners have run? So, Matt, would you agree that what you would want Declude Virus to do is: * Log a status line if the message is infected for each scanner (trivial change?) * Also, let us match, per scanner, multiple errorlevel codes to specific text matches (would this benefit F-Prot users only?) * Also, give us a directive like SKIPIFVIRAL to short-circuit out of the nextscanner if a virus is found. Given the SKIPIFVIRAL directive, we'd have to consider whether a SKIPIFVULN to short-circuit out of any scanning if a vulnerability has been found.Given the other two SKIPs, is a SKIPBAN useful? I just realized that I'm not sure what happens when you ban a file, like an .EXE that is also viral. Andrew 8) -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MattSent: Friday, April 29, 2005 12:20 AMTo: Declude.Virus@declude.comSubject: Re: [Declude.Virus] F-Prot missing viruses and is slow (renamed)Andrew,I'm still up doing maintenance...While you are correct about what happens with the error code when only one virus scanner is used, when two are configured like on my system, there is no indication that F-Prot detected a virus unless a REPORT line is matched, which won't happen with a VIRUSCODE 8. In the samples that I previously provided, the only affirmative indication that F-Prot detected a virus is the line "Could not find parse string Infection: in report.txt". 04/28/2005 17:40:57 Q58666795008E87C7 MIME file: [text/html][7bit; Length=695 Checksum=54365]04/28/2005 17:40:57 Q58666795008E87C7 MIME file: doc.zip [base64; Length=56432 Checksum=6987426]--- 10 second gap while F-Prot scans ---04/28/2005 17:41:07 Q58666795008E87C7 Could not find parse string Infection: in report.txt04/28/2005 17:41:08 Q58666795008E87C7 Scanner 2: Virus=the W32/[EMAIL PROTECTED] Attachment= [0] I04/28/2005 17:41:08 Q58666795008E87C7 File(s) are INFECTED [the W32/[EMAIL PROTECTED]: 13]04/28/2005 17:41:08 Q58666795008E87C7 Scanned: CONTAINS A VIRUS [Prescan OK][MIME: 3 57490]04/28/2005 17:41:08 Q58666795008E87C7 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] [incoming from 192.168.100.100]04/28/2005 17:41:08 Q58666795008E87C7 Subject: [Fwd: Mail Delivery System]Definitely there should be an allowance for multiple REPORT lines to match, but also, it seems to make sense to provide a different indicator showing that a virus was detected and the error code for each scanner. Some scanners don't have parseable reports so when they are run in a multiple scanner config the new logging mechanism would be the only way to properly identify the result for that particular scanner.MattColbeck, Andrew wrote: Yes, during the entire interval I measured the CPU time was 98-100% for the fpcmd.exe process only. On LOGLEVEL MED, there is a line that shows the errorlevel returned by the scanner, plus the error line indicating that the search string wasn't found in the resulting text file, e.g. this is what is returned on my v2.0.6 system when a "suspicious file" is returned: 04/27/2005 07:48:33 QA63CBF0600647AB8 Could not find parse string Infection: in report.txt04/27/2005 07:48:33 QA63CBF0600647AB8 File(s) are INFECTED [: 8]04/27/2005 07:48:33 QA63CBF0600647AB8 Scanned: CONTAINS A VIRUS [MIME: 3 23729]04/27/2005 07:48:33 QA63CBF0600647AB8 From:munged To:munged [outgoing from 70.187.178.183]04/27/2005 07:48:33 QA63CBF0600647AB8 Subject: Forum notify The resulting virus name is [Unknown File] butadding such a line to my FORGINGVIRUS strings doesn't stop the notification email (but they only go to postmaster, so no big deal for me). I don't know if it made it into the support database, but on testing Declude Virus, I immediately requested a feature enhancement to extend the virus matching string "REPORT" parallel with the "VIRUSCODE" lines for this reason. Otherwise, Matt, I agree on both of your conclusions regarding how F-Prot falls short. Andrew 8) -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of MattSent: Thursday, April 28, 2005 9:16 PMTo: Declude.Virus@declude.comSubject: Re: [Declude.Virus] F-Prot missing viruses and is slow (renamed)Ok, follow-up time. It appears that Declude is detecting this with VIRUSCODE 8 and I was just merely confused by the logs. I set things to Debug and found the
Re: [Declude.Virus] F-Prot missing viruses and is slow (renamed)
Title: Message Andrew, Being anal about things when I get down to business (and I believe justifiably so), I would add a few others to your list of recommendations along with an edit of #2 and #3: 1) Log a status line if the message is infected for each scanner (trivial change?). 2) Let us match, per scanner, multiple text matches using additional REPORT lines (not exclusive to F-Prot, and some errors can have multiple text strings). 3) Also, give us a directive like STOPSCANINGONVIRUS ON/OFF to short-circuit out of the next scanner if a virus is found (I would make this an ON/OFF configuration for consistency with other Virus.cfg directives). 4) Help resolve the issues with F-Prot by having Declude attempt to get involved at a higher level with them. 5) Change the recommended VIRUSCODE's in the manual for F-Prot to include VIRUSCODE 8. 6) Change the recommended McAfee command line arguments in the manual to include /NOBOOT and /PROGRAM. Now some notes on your other notes and related items. Although there have been no indications of problems using some of the non-standard configs in both F-Prot and McAfee, I'm not sure that there is enough evidence to support adding VIRUSCODE 9 and 10 to F-Prot. Personally I feel that given the issues with known viruses suddenly popping up as VIRUSCODE 8 along with the CPU/time issue when a suspicious file is detected, it suggests that there could be other issues down the line that might trip VIRUSCODE 9 in F-Prot due to nothing more than a programming error. I of course have no evidence of that, but there also isn't any evidence that it won't happen. VIRUSCODE 10 only detects things that are zipped over and over again, typically these would be 'decompression bombs', but I have seen no evidence of these spreading and I have never heard of this being triggered. Multiple-archiving would be a terrible way to spread a virus since people won't likely dig deep into them to extract the executable and therefore such viruses wouldn't achieve sufficient scale to spread widely. I don't however believe this to be likely to cause issues if used...but you of course never know. VIRUSCODE 3 and 6 are the only purposeful codes returned for known viruses in F-Prot. I won't personally recommend changing the default config that Declude shares, though maybe adding these things and alternative switches to the command line would be warranted if noted properly. The same general thinking also goes for the /ANALYZE, /PANALYZE, /MAILBOX and /MIME switches in McAfee. The additional Declude Virus switches that you mentioned aren't necessarily wise or useful for most installations, though I could see the need in some cases where it would be appropriate, but if misconfigured, they could also produce significant backscatter. SKIPIFBAN would cause issues with bannotify.eml because Declude only sends that if a virus or vulnerability isn't detected and this would disable that detection. It would only be practical in situations where bannotify.eml wasn't being used. SKIPIFVULN could cause more bannotify.eml notifications to be sent as well. Many of the vulnerabilities that Declude has added in the past year are for invalid file types, and when viruses hit before the definitions do, the vulnerability detection will stop a great many of the bannotify.eml notifications from being sent. By in large neither switch would save a great deal of processing as it is legitimate E-mail that causes the most load in most systems, and with the caveats added regarding bannotify.eml and backscatter, it might make a strong case against them. Maybe with a major rewrite of Declude Virus many of these things could be better handled though. I don't wish to be the arbiter of fact around here, so if people want to add, subtract, dispute, etc., please do, but please don't flame me for speaking my mind :) I just want to compel methodical progress that benefits more than just myself. Matt Colbeck, Andrew wrote: Ding! ... and that's why we've spent so much time on this. The log will show that F-Prot returned an errorlevel, and also the status line that the message contains an infected file. However, when there is more than one scanner, the status line that the message contains an infected file is only logged after both scanners have run? So, Matt, would you agree that what you would want Declude Virus to do is: * Log a status line if the message is infected for each scanner (trivial change?) * Also, let us match, per scanner, multiple errorlevel codes to specific text matches (would this benefit F-Prot users only?) * Also, give us a directive like SKIPIFVIRAL to short-circuit out of the nextscanner if a virus is found. Given the SKIPIFVIRAL directive, we'd have to consider whether a SKIPIFVULN to short-circuit out of any scanning if a vulnerability has been found.Given the other two SKIPs, is a SKIPBAN useful? I just realized that I'm not sure what happens when you ban a file,
