[Clamav-users] Re: virus passing through clamav-milter, but not?through clamdscan!
> I have found a certain kind of mime structure and headers, that > causes clamd to produce false negatives errors. The debugging outpout of > clamd reports "LibClamAV debug: getline: buffer overflow stopped" and the > viral attachment is not opened at all. (See > http://users.auth.gr/~apap/clamav/viral-mail.raw and > http://users.auth.gr/~apap/clamav/CLAMD-DEBUG-cannot-detect-attached-virus-in-viral-raw-mail.log) > If the same mail is in mbox format (the only difference is in the > first line "From "), the attachements are opened normally, and Worm.Bagz.D > is found. (See http://users.auth.gr/~apap/clamav/viral-mail.mbox and > http://users.auth.gr/~apap/clamav/CLAMD-DEBUG-detects-attached-virus-in-mbox-mail.log) The problem seems to have been fixed sometime after clamav0.85.1, but no later than CVS version clamav-devel-20050518. I had originally observed the problem with clamav0.84. Many thanks go to Nigel for his immediate support, Andrey Melnikoff for his patch, and all the others that responded. Apostolis Papayanakis ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Fri, May 20, 2005 at 01:14:34AM +0300, Apostolos Papayanakis wrote:
> I have found a certain kind of mime structure and headers, that
> causes clamd to produce false negatives errors. The debugging outpout of
> clamd reports "LibClamAV debug: getline: buffer overflow stopped" and the
> viral attachment is not opened at all. (See
[...]
> If the same mail is in mbox format (the only difference is in the
> first line "From "), the attachements are opened normally, and Worm.Bagz.D
> is found.
>
> A small collection of the viral mails I have received, can be found
> at: http://users.auth.gr/~apap/clamav/viruses-that-bypass-clamav-0.85.1.mbox.
> I receive tens of them every day. They have all been sent to
> [EMAIL PROTECTED] (this is forwarded to my INBOX) and
> originate from unqualified addresses from a specific network. The attacments
> are BASE64 encoded in very long lines (2048 bytes each). No other user on my
> servers (17000 of them active) has reported to get these viruses. All this
> is very puzzling.
For what it's worth, I have a sample of Bagz.C, from nov 2004, that also
shows the same layout, and behaviour of clamav. If I remove the initial
"From " line, the virus is not recognised and --debug output shows the
"buffer overflow stopped".
I suppose that this is a bug? Is clam supposed to recognise emails even
without the leading "From " line?
The reason I ask is: in MIMEDefang, there is this entry in the manpage:
md_copy_orig_msg_to_work_dir_as_mbox_file()
Normally, virus-scanners are passed only the unpacked, decoded
parts of a MIME message. If you want to pass the original,
undecoded message in as a UNIX-style "mbox" file, call
md_copy_orig_msg_to_work_dir_as_mbox_file prior to calling mes
sage_contains_virus. The only difference between this function
and md_copy_orig_msg_to_work_dir is that this function prepends
a "From_" line to make the message look like a UNIX-style mbox
file. This is required for some virus scanners (such as Clam
AntiVirus) to recognize the file as an e-mail message.
The md_copy_orig_msg_to_work_dir() is however a lot more efficient, and
if it's the same to ClamAV (or, well, if it should be treated the same),
then this documentation is not correct?
(MIMEDefang also extracts all attachments, so the virus is found anyway,
albeit in the extracted part).
--
#!perl -wpl # mmfppfmpmmpp mmpffm <[EMAIL PROTECTED]>
$p=3-2*/[^\W\dmpf_]/i;s.[a-z]{$p}.vec($f=join('',$p-1?chr(sub{$_[0]*9+$_[1]*3+
$_[2]}->(map{/p|f/i+/f/i}split//,$&)+97):qw(m p f)[map{((ord$&)%32-1)/$_%3}(9,
3,1)]),5,1)='`'lt$&;$f.eig;# Jan-Pieter Cornet
___
http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>ps. Despite the subject of this mail, clamav-milter now seems to be >unrelated to the problem. That is correct, once you figured out that the "From " line was key, the bug became reproducable using any clamav interface. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
Hello again Nigel, I have found a certain kind of mime structure and headers, that causes clamd to produce false negatives errors. The debugging outpout of clamd reports "LibClamAV debug: getline: buffer overflow stopped" and the viral attachment is not opened at all. (See http://users.auth.gr/~apap/clamav/viral-mail.raw and http://users.auth.gr/~apap/clamav/CLAMD-DEBUG-cannot-detect-attached-virus-in-viral-raw-mail.log) If the same mail is in mbox format (the only difference is in the first line "From "), the attachements are opened normally, and Worm.Bagz.D is found. (See http://users.auth.gr/~apap/clamav/viral-mail.mbox and http://users.auth.gr/~apap/clamav/CLAMD-DEBUG-detects-attached-virus-in-mbox-mail.log) A small collection of the viral mails I have received, can be found at: http://users.auth.gr/~apap/clamav/viruses-that-bypass-clamav-0.85.1.mbox. I receive tens of them every day. They have all been sent to [EMAIL PROTECTED] (this is forwarded to my INBOX) and originate from unqualified addresses from a specific network. The attacments are BASE64 encoded in very long lines (2048 bytes each). No other user on my servers (17000 of them active) has reported to get these viruses. All this is very puzzling. I assume that your "yes" in your previous mail, means that the test-virus you sent me, *did* pass through your mailserver, which *did* attempt to scan for viruses but *did* fail to recognize the attached virus, probably due to mangled mime structure. I suppose that your server silently fixed the mangled structure and as a result the virus was detectable on my mail server. If all the above are correct, then this should be fixed in clamd. I hope the data in http://users.auth.gr/~apap/clamav are enough to verify the problem. Apostolis Papayanakis ps. Despite the subject of this mail, clamav-milter now seems to be unrelated to the problem. On 2005-05-18 08:45 +300 Nigel Horne wrote: > On Wednesday 18 May 2005 00:57, Apostolos Papayanakis wrote: > > Nigel, > > > > Did the viral mail you sent me as a test > > (http://users.auth.gr/~apap/spurious-viral-mbox), passed through your > local > > clamav-milter before reaching my clamav-milter that finally rejected it? > > Yes - I don't (usually) have outgoing scanning on. > > > > A plain yes or no would suffice, at least for now. There seems to be > > a problem with the initial "From " line in the viral mbox-style mailbox > > (removing it hides the virus from clamdscan). I will investigate further > > and will write back. > > Hmm. OK - let me know if you find anything. > > > > Apostolis Papayanakis > > -Nigel > ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
Apostolos Papayanakis: >There seems to be >a problem with the initial "From " line in the viral mbox-style mailbox >(removing it hides the virus from clamdscan). I can confirm this. If I remove the "From " line from my sample files, clamscan says they are OK. With the "From " line, they show as infected with Bagz. I think you are on to something. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Wednesday 18 May 2005 00:57, Apostolos Papayanakis wrote: > Nigel, > > thanks for your immediate response, clamav and clamav-milter user > support is unprecedented! > > Did the viral mail you sent me as a test > (http://users.auth.gr/~apap/spurious-viral-mbox), passed through your local > clamav-milter before reaching my clamav-milter that finally rejected it? Yes - I don't (usually) have outgoing scanning on. > > A plain yes or no would suffice, at least for now. There seems to be > a problem with the initial "From " line in the viral mbox-style mailbox > (removing it hides the virus from clamdscan). I will investigate further > and will write back. Hmm. OK - let me know if you find anything. > > Apostolis Papayanakis -Nigel ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>It certainly doesn't appear to. I am not sure why, though. Attached is >a diff of the outputs of your run and a run here of clamscan (0.85, >though). Maybe somebody else can spot the problem. > > LibClamAV debug: fileblobDestroy: textpart > LibClamAV debug: cli_mbox returning 0 >+LibClamAV debug: Recognized ZIP file >+LibClamAV debug: in scanzip() >+LibClamAV debug: Zip: help.doc.exe= >, crc32: 0x3fcc001f, encrypted: 0, compressed: 150514, normal: 155156, meth= >od: 8, ratio: 1 (max: 250) >+LibClamAV debug: Recognized DOS/W32 executable/library/driver file >+LibClamAV debug: Worm.Bagz.D found in descriptor 7. >+LibClamAV debug: Zip: Infected with Worm.Bagz.D Yeah, I get the same extra log entries when I check the false positive file using clamdscan. The first extra message comes from the routine cli_filetype() in libclamav/filetypes.c, a fairly simple routine that just checks a buffer against magic numbers in a table. No idea why this would fail sometimes. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
Stephen Gran said: > On Tue, May 17, 2005 at 07:45:27PM -0700, Jef Poskanzer said: >> >> Hmm, ScanArchive is not set. It's commented out in both my >> clamd.conf >> >> and in clamd.conf.default. Should I try uncommenting it? >> > >> >Well, there is your problem, presumably. >> >> Good guess, but after uncommenting ScanArchive and restarting >> everything, >> I am still getting false positives. I captured another one and saved it >> to http://www.acme.com/jef/tmp/cl/ This time the log entries start >> around >> line 4164. Can you verify that it is looking inside the ZIP file >> this time? > > It certainly doesn't appear to. I am not sure why, though. Attached is > a diff of the outputs of your run and a run here of clamscan (0.85, > though). Maybe somebody else can spot the problem. > -- > Looks like the missing entry is "Recognized Zip..." for the particular virus. Wonder what magic number it's looking for in the zip to make that determination. It isn't a universal problem as it did recognize another zip file. dp ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Tue, May 17, 2005 at 07:45:27PM -0700, Jef Poskanzer said: > >> Hmm, ScanArchive is not set. It's commented out in both my clamd.conf > >> and in clamd.conf.default. Should I try uncommenting it? > > > >Well, there is your problem, presumably. > > Good guess, but after uncommenting ScanArchive and restarting everything, > I am still getting false positives. I captured another one and saved it > to http://www.acme.com/jef/tmp/cl/ This time the log entries start around > line 4164. Can you verify that it is looking inside the ZIP file > this time? It certainly doesn't appear to. I am not sure why, though. Attached is a diff of the outputs of your run and a run here of clamscan (0.85, though). Maybe somebody else can spot the problem. -- -- | Stephen Gran | I was gratified to be able to answer| | [EMAIL PROTECTED] | promptly, and I did. I said I didn't| | http://www.lobefin.net/~steve | know. -- Mark Twain | -- --- logfile2.txt2005-05-17 23:03:51.0 -0400 +++ logfile3.txt2005-05-17 23:02:01.0 -0400 @@ -1,26 +1,31 @@ -LibClamAV debug: Recognized Raw mail file +LibClamAV debug: Recognized MBox file LibClamAV debug: Starting cli_scanmail(), mrec == 1, arec == 0 LibClamAV debug: in mbox() -LibClamAV debug: parseEmailFile -LibClamAV debug: parseEmailFile: check 'Received: by clamav-milter' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'From: [EMAIL PROTECTED]' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'To: [EMAIL PROTECTED]' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'Received-SPF: pass (gate.acme.com: domain of [EMAIL PROTECTED] designates 210.83.203.71 as permitted sender) receiver=gate.acme.com; client-ip=210.83.203.71; helo=127.0.0.1; [EMAIL PROTECTED]; x-software=spfmilter 0.96 http://www.acme.com/software/spfmilter/ with libspf-unknown;' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'SUBJECT: re: please' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'FROM: [EMAIL PROTECTED]' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'TO: [EMAIL PROTECTED]' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'DATE: [[ ÐÇÆÚÈý, 18 ÎåÔÂ 2005 10:43:13 ]]' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'MIME-Version: 1.0' contMarker 0 fullline 0x0x0 -LibClamAV debug: parseEmailFile: check 'Content-Type: multipart/mixed; boundary="bound--"' contMarker 0 fullline 0x0x0 +LibClamAV debug: Recognized Raw mail file +LibClamAV debug: Extract attachments from email 1 +LibClamAV debug: parseEmailHeaders +LibClamAV debug: parseEmailHeaders: check 'From [EMAIL PROTECTED] Tue May 17 19:40:31 2005' +LibClamAV debug: parseEmailHeaders: check 'Received-SPF: pass (gate.acme.com: domain of [EMAIL PROTECTED] designates 210.83.203.71 as permitted sender) receiver=gate.acme.com; client-ip=210.83.203.71; helo=127.0.0.1; [EMAIL PROTECTED]; x-software=spfmilter 0.96 http://www.acme.com/software/spfmilter/ with libspf-unknown;' +LibClamAV debug: parseEmailHeaders: check 'Received: from 127.0.0.1 ([210.83.203.71])' +LibClamAV debug: parseEmailHeaders: check 'by gate.acme.com (8.13.4/8.13.4) with ESMTP id j4I2eI7g093539' +LibClamAV debug: parseEmailHeaders: check 'for [EMAIL PROTECTED]; Tue, 17 May 2005 19:40:25 -0700 (PDT)' +LibClamAV debug: parseEmailHeaders: check 'Message-Id: <[EMAIL PROTECTED]>' +LibClamAV debug: parseEmailHeaders: check 'SUBJECT: re: please' +LibClamAV debug: parseEmailHeaders: check 'FROM: [EMAIL PROTECTED]' +LibClamAV debug: parseEmailHeaders: check 'TO: [EMAIL PROTECTED]' +LibClamAV debug: parseEmailHeaders: check 'DATE: [[ ÐÇÆÚÈý, 18 ÎåÔÂ 2005 10:43:13 ]]' +LibClamAV debug: parseEmailHeaders: check 'MIME-Version: 1.0' +LibClamAV debug: parseEmailHeaders: check 'Content-Type: multipart/mixed; boundary="bound--"' LibClamAV debug: parseEmailHeader 'Content-Type: multipart/mixed; boundary="bound--"' LibClamAV debug: parseMimeHeader: cmd='Content-Type', arg=' multipart/mixed; boundary="bound--"' LibClamAV debug: messageSetMimeType: 'multipart' LibClamAV debug: mimeArgs = ' boundary="bound--"' LibClamAV debug: Add arguments ' boundary="bound--"' -LibClamAV debug: parseEmailFile: check '' contMarker 0 fullline 0x0x0 +LibClamAV debug: parseEmailHeaders: check 'X-Virus-Scanned: ClamAV version 0.85.1, clamav-milter version 0.85 on gate.acme.com' +LibClamAV debug: parseEmailHeaders: check 'X-Virus-Status: Clean' +LibClamAV debug: parseEmailHeaders: check '' LibClamAV debug: End of header information -LibClamAV debug: getline: buffer overflow stopped -LibClamAV debug: parseEmailFile: return +LibClamAV debug: parseEmailHeaders: ret
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Tue, May 17, 2005 at 10:30:13PM -0400, Stephen Gran said: > On Tue, May 17, 2005 at 07:27:39PM -0700, Jef Poskanzer said: > > Hmm, ScanArchive is not set. It's commented out in both my clamd.conf > > and in clamd.conf.default. Should I try uncommenting it? > > Well, there is your problem, presumably. No, I eat my words. The manpage says it is default. Let me look at the code, though. Ah, it appears to be default. So that should not be the problem. -- -- | Stephen Gran | Canada Bill Jones's Motto: It's| | [EMAIL PROTECTED] | morally wrong to allow suckers to keep | | http://www.lobefin.net/~steve | their money. Canada Bill Jones's | || Supplement: A Smith and Wesson beats | || four aces. | -- pgp9kUwUBmJ8k.pgp Description: PGP signature ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>> Hmm, ScanArchive is not set. It's commented out in both my clamd.conf >> and in clamd.conf.default. Should I try uncommenting it? > >Well, there is your problem, presumably. Good guess, but after uncommenting ScanArchive and restarting everything, I am still getting false positives. I captured another one and saved it to http://www.acme.com/jef/tmp/cl/ This time the log entries start around line 4164. Can you verify that it is looking inside the ZIP file this time? --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Tue, May 17, 2005 at 07:27:39PM -0700, Jef Poskanzer said: > >Well, the first wierd thing I see off the top of my head is that the > >attached zip file never gets scanned. that would be why it's making it > >through the milter, so that part is covered. > > > >Now, of course, the question is why? For that, i'll really need to see > >the entire config file and all the start up options passed to the > >milter. > > > >egrep -v '^[[:space:]]*(#|$)' /path/to/clamd.conf, please - that will > >cut out all the junk and just leave me with what you're actually using. > > By your command, results are appended. > > Hmm, ScanArchive is not set. It's commented out in both my clamd.conf > and in clamd.conf.default. Should I try uncommenting it? Well, there is your problem, presumably. > Is it possible that an external program that clamd wants to use is > not being found? Seems like that would show up in the logfile tho. clamscan can use external unpackers if you specify them on the command line. clamd does not. -- -- | Stephen Gran | Many people resent being treated like | | [EMAIL PROTECTED] | the person they really are. | | http://www.lobefin.net/~steve | | -- pgpqzeen2jskF.pgp Description: PGP signature ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>and all the start up options passed to the milter. Oh yeah: /usr/local/sbin/clamav-milter --quiet --external unix:/var/run/filter/clmilter.sock ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>Well, the first wierd thing I see off the top of my head is that the >attached zip file never gets scanned. that would be why it's making it >through the milter, so that part is covered. > >Now, of course, the question is why? For that, i'll really need to see >the entire config file and all the start up options passed to the >milter. > >egrep -v '^[[:space:]]*(#|$)' /path/to/clamd.conf, please - that will >cut out all the junk and just leave me with what you're actually using. By your command, results are appended. Hmm, ScanArchive is not set. It's commented out in both my clamd.conf and in clamd.conf.default. Should I try uncommenting it? Is it possible that an external program that clamd wants to use is not being found? Seems like that would show up in the logfile tho. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ LogSyslog LogFacility LOG_MAIL PidFile /var/run/clamav/clamd.pid DatabaseDirectory /var/db/clamav LocalSocket /var/run/clamav/clamd FixStaleSocket TCPAddr 127.0.0.1 MaxThreads 500 User filter AllowSupplementaryGroups Foreground Debug ScanMail ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Tue, May 17, 2005 at 07:01:01PM -0700, Jef Poskanzer said: > >OK, let me be completely explicit. You want both > >Debug > >Foreground > >in clamd.conf. Start a single instance of the milter (by hand - no init > >script that may daemonize the milter or otherwise hide output from you)=20 > >on an otherwise quiescent machine. Send an email with the false negative. > > Ok. With both Debug and Foreground set I am now getting debugging output. > Yay! I don't have a quiescent machine, but I was able to catch a > sample pretty quickly (since I get so many of them). The false positive > mail and the full log file are here: http://www.acme.com/jef/tmp/cl/ > The relevant log entries start around line 819, or you can just search > for the from-address of the sample, "[EMAIL PROTECTED]". > > I don't see anything weird in the debug output, but maybe you will. Well, the first wierd thing I see off the top of my head is that the attached zip file never gets scanned. that would be why it's making it through the milter, so that part is covered. Now, of course, the question is why? For that, i'll really need to see the entire config file and all the start up options passed to the milter. egrep -v '^[[:space:]]*(#|$)' /path/to/clamd.conf, please - that will cut out all the junk and just leave me with what you're actually using. -- -- | Stephen Gran | Real programmers can write assembly | | [EMAIL PROTECTED] | code in any language. :-) | | http://www.lobefin.net/~steve | -- Larry Wall in <[EMAIL PROTECTED] | || devvax.JPL.NASA.GOV>| -- pgpkJ80jd4re4.pgp Description: PGP signature ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>OK, let me be completely explicit. You want both >Debug >Foreground >in clamd.conf. Start a single instance of the milter (by hand - no init >script that may daemonize the milter or otherwise hide output from you)=20 >on an otherwise quiescent machine. Send an email with the false negative. Ok. With both Debug and Foreground set I am now getting debugging output. Yay! I don't have a quiescent machine, but I was able to catch a sample pretty quickly (since I get so many of them). The false positive mail and the full log file are here: http://www.acme.com/jef/tmp/cl/ The relevant log entries start around line 819, or you can just search for the from-address of the sample, "[EMAIL PROTECTED]". I don't see anything weird in the debug output, but maybe you will. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Tue, May 17, 2005 at 06:12:31PM -0700, Jef Poskanzer said: > >Also, Debug in the conf file helps quite a bit, and was actually what I > >was referring to. > > Ok, I uncommented that option and stopped/started clamav-milter. > I don't see any new syslog messages, or anything on stdout. Where > should I be looking? Do I also have to re-enable the regular > logfile, in addition to LogSyslog? I just tried that, and all I'm > seeing in the log file is worm-found messages. OK, let me be completely explicit. You want both Debug Foreground in clamd.conf. Start a single instance of the milter (by hand - no init script that may daemonize the milter or otherwise hide output from you) on an otherwise quiescent machine. Send an email with the false negative. You will get a ton of debugging output, most of it from the library. Presumably something is hiding some of the output from you, as I get the 'Foreground is a good idea with Debug' message if I only use Debug. > >Of course, the --debug option to configure is for testing and debugging, > >not production use. 0.85 wouldn't even compile if you passed that > >argument. > > I used --enable-debug, and in 0.85.1 it did compile. Not using that > now tho. You do not need it for what I am asking for. We may get that far, but doing that actually changes some of the internal logic, so I would rather get to the bottom of it without using that option. -- -- | Stephen Gran | Gravity is a myth, the Earth sucks. | | [EMAIL PROTECTED] | | | http://www.lobefin.net/~steve | | -- pgpOKdlcAZwrL.pgp Description: PGP signature ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
Jef Poskanzer wrote: > >Also, Debug in the conf file helps quite a bit, and was actually what I > >was referring to. > > Ok, I uncommented that option and stopped/started clamav-milter. > I don't see any new syslog messages, or anything on stdout. Where > should I be looking? Do I also have to re-enable the regular > logfile, in addition to LogSyslog? I just tried that, and all I'm > seeing in the log file is worm-found messages. Not sure about the milter specifics, but I assume you would need to reload|restart clamd also? Matt ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>Also, Debug in the conf file helps quite a bit, and was actually what I >was referring to. Ok, I uncommented that option and stopped/started clamav-milter. I don't see any new syslog messages, or anything on stdout. Where should I be looking? Do I also have to re-enable the regular logfile, in addition to LogSyslog? I just tried that, and all I'm seeing in the log file is worm-found messages. >Of course, the --debug option to configure is for testing and debugging, >not production use. 0.85 wouldn't even compile if you passed that >argument. I used --enable-debug, and in 0.85.1 it did compile. Not using that now tho. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Tue, May 17, 2005 at 05:13:32PM -0700, Jef Poskanzer said:
> >Run with debugging on, send the problem emails through again, and see
> >if something shows up.
>
> Ok, this is a good suggestion.
[ ... ]
What I was hoping for would be a single run of the email through the
milter, and also debug output from clam{,d}scan so I could learn
something about the file itself beyond what the milter debug outputs.
It is of course difficult to do much with a regular volume mail server
still processing mail at full tilt.
> Looking at the clamav-milter man page I see two different debug flag,
> --debug and --debug-level. I randomly picked the latter, and tried
> running with --debug-level=9. This flag was not recognized, of
> course, since the FreeBSD port isn't compiled with debugging enabled.
> I hacked the port's Makefile to add this, rebuilt & reinstalled, and
> then ran.
Also, Debug in the conf file helps quite a bit, and was actually what I
was referring to.
> Afterwards I stopped and restarted clamav-milter without the
> --debug-level flag. Guess what, it still generates debug info on
> stdout. I guess that is why the FreeBSD port does not enable
> debugging.
Of course, the --debug option to configure is for testing and debugging,
not production use. 0.85 wouldn't even compile if you passed that
argument.
--
--
| Stephen Gran | Good night, Austin, Texas, wherever you |
| [EMAIL PROTECTED] | are!|
| http://www.lobefin.net/~steve | |
--
pgpKSHfrhxTU1.pgp
Description: PGP signature
___
http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
I should add that my /etc/syslog.conf says 'mail.* /var/log/maillog', and /usr/local/etc/clamd.conf says 'LogFacility LOG_MAIL' --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>Run with debugging on, send the problem emails through again, and see if >something shows up. Ok, this is a good suggestion. Looking at the clamav-milter man page I see two different debug flag, --debug and --debug-level. I randomly picked the latter, and tried running with --debug-level=9. This flag was not recognized, of course, since the FreeBSD port isn't compiled with debugging enabled. I hacked the port's Makefile to add this, rebuilt & reinstalled, and then ran. What I find in the syslog is a whole bunch of debug log entries, from every time a milter routine gets called. However what I don't see is any way to correlate which log entries belong to which mail sessions. That makes them less useful than they should be. What I do when I'm debugging my milters is give each connection a unique i.d. number, and put that in all the syslogs. Anyway, I picked a false-positive from my non-clamavs folder and looked for it in the mail log. I was able to find the initial connection, and the final disposition, with an elapsed time of 30 seconds between them. During those 30 seconds there were 3283 other mail syslogs. If I eliminate all but clamav-related logs, there were only 805. About 740 of those were clamfi_connect entries for other connections, so I could get rid of those. Most of the rest were also from other connections, and I could get rid of them one by one. After this winnowing process, the only syslogs left between the initial connection and the final disposition were 16 connect2clamd lines. Presumably one of the 16 was for the connection I was interested in and the others were for other connections. There were no syslogs for any other clamfi calls whatsoever. So that was a big bust, unless you can infer something from negative evidence. Anyway, the results are below. Afterwards I stopped and restarted clamav-milter without the --debug-level flag. Guess what, it still generates debug info on stdout. I guess that is why the FreeBSD port does not enable debugging. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ May 17 16:34:34 gate sm-mta[49779]: NOQUEUE: connect from [220.248.104.130] May 17 16:34:34 gate clamav-milter[11373]: clamfi_connect: connection from 220.248.104.130 May 17 16:34:40 gate clamav-milter[11373]: connect2clamd May 17 16:34:40 gate clamav-milter[11373]: connect2clamd May 17 16:34:42 gate clamav-milter[11373]: connect2clamd May 17 16:34:46 gate clamav-milter[11373]: connect2clamd May 17 16:34:46 gate clamav-milter[11373]: connect2clamd May 17 16:34:49 gate clamav-milter[11373]: connect2clamd May 17 16:34:50 gate clamav-milter[11373]: connect2clamd May 17 16:34:52 gate clamav-milter[11373]: connect2clamd May 17 16:34:52 gate clamav-milter[11373]: connect2clamd May 17 16:34:53 gate clamav-milter[11373]: connect2clamd May 17 16:34:54 gate clamav-milter[11373]: connect2clamd May 17 16:34:56 gate clamav-milter[11373]: connect2clamd May 17 16:35:01 gate clamav-milter[11373]: connect2clamd May 17 16:35:02 gate clamav-milter[11373]: connect2clamd May 17 16:35:03 gate clamav-milter[11373]: connect2clamd May 17 16:35:04 gate clamav-milter[11373]: connect2clamd May 17 16:35:04 gate sm-mta[49779]: j4HNYYJY049779: [EMAIL PROTECTED], size=204868, class=0, nrcpts=1, msgid=<[EMAIL PROTECTED]>, bodytype=8BITMIME, proto=ESMTP, daemon=MTA, relay=[220.248.104.130] May 17 16:35:04 gate clamav-milter[11373]: j4HNYYJY049779: clean message from [EMAIL PROTECTED] May 17 16:35:04 gate sm-mta[53666]: j4HNYYJY049779: forward [EMAIL PROTECTED] => "|/usr/local/bin/procmail #jef" May 17 16:35:04 gate sm-mta[53666]: j4HNYYJY049779: to="|/usr/local/bin/procmail #jef", [EMAIL PROTECTED] (1544/1002), delay=00:00:16, xdelay=00:00:00, mailer=prog, pri=235496, dsn=2.0.0, stat=Sent May 17 16:35:04 gate sm-mta[53666]: j4HNYYJY049779: done; delay=00:00:16, ntries=1 May 17 16:35:04 gate jef: filtered: non-ClamAV worm ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
Nigel, thanks for your immediate response, clamav and clamav-milter user support is unprecedented! Did the viral mail you sent me as a test (http://users.auth.gr/~apap/spurious-viral-mbox), passed through your local clamav-milter before reaching my clamav-milter that finally rejected it? A plain yes or no would suffice, at least for now. There seems to be a problem with the initial "From " line in the viral mbox-style mailbox (removing it hides the virus from clamdscan). I will investigate further and will write back. Apostolis Papayanakis [EMAIL PROTECTED] On Tue, 17 May 2005 Nigel Horne wrote: > Date: Tue, 17 May 2005 16:44:55 +0100 > From: Nigel Horne <[EMAIL PROTECTED]> > Subject: Re: [Clamav-users] Re: virus passing through clamav-milter, > but not through clamdscan! > To: ClamAV users ML > Message-ID: <[EMAIL PROTECTED]> > Content-Type: text/plain; charset="iso-8859-1" > > I tried your test and got this, so your end is NOT passing this virus through > clamav-milter: > > The original message was received at Tue, 17 May 2005 16:41:57 +0100 > from bandsman.co.uk [127.0.0.1] > >- The following addresses had permanent fatal errors - > <[EMAIL PROTECTED]> > (reason: 554 5.7.1 virus Worm.Bagz.D detected by ClamAV - > http://www.clamav.net) > >- Transcript of session follows - > ... while talking to olympos.ccf.auth.gr.: > >>> DATA > <<< 554 5.7.1 virus Worm.Bagz.D detected by ClamAV - http://www.clamav.net > 554 5.0.0 Service unavailable > > > > -- > > ___ > clamav-users mailing list > ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
On Tue, May 17, 2005 at 10:27:00AM -0700, Jef Poskanzer said: > >I tried your test and got this, so your end is NOT passing this virus > >through clamav-milter: > > I.e. "clamav-milter works for me, therefore it works for you, > therefore you are doing something else wrong." No, more like "when I tried to send you a test virus, it got a proper 550". These two statements are not the same. > This may be true but it's far from proven. Furthermore, if Apostolos' > problem is like mine, then the false-negatives have ClamAV headers > added, showing that they *do* pass through clamav-milter. Here are > the headers off the latest of the many thousands of examples in my > non-clamav virus folder: > > X-Virus-Scanned: ClamAV 0.84/882/Mon May 16 23:48:03 2005 on gate.acme.com > X-Virus-Status: Clean > > Running this file through clamscan or clamdscan shows: Worm.Bagz.E > FOUND. I have yet to see enough of your config or startup options to see why this would be. I have seen a diff of your conf files (some time ago, so it is hazy), but I have no other information about startup parameters or any other system peculiaritites, and you have not done any of the debugging work that would be helpful to people trying to help you fix your system, or at least I don't remember seeing it. Run with debugging on, send the problem emails through again, and see if something shows up. If you don't see anything odd, by all means send the whole output. I understand you are frustrated that nobody has answered, but I personally haven't seen enough in one place to be able to help you. Very few are willing to do the work of eliciting the information to fix your problem if it hasn't already been supplied. -- -- | Stephen Gran | Most people will listen to your | | [EMAIL PROTECTED] | unreasonable demands, if you'll | | http://www.lobefin.net/~steve | consider their unacceptable offer. | -- pgpnz6dHS5hAd.pgp Description: PGP signature ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
>I tried your test and got this, so your end is NOT passing this >virus through clamav-milter: I.e. "clamav-milter works for me, therefore it works for you, therefore you are doing something else wrong." This may be true but it's far from proven. Furthermore, if Apostolos' problem is like mine, then the false-negatives have ClamAV headers added, showing that they *do* pass through clamav-milter. Here are the headers off the latest of the many thousands of examples in my non-clamav virus folder: X-Virus-Scanned: ClamAV 0.84/882/Mon May 16 23:48:03 2005 on gate.acme.com X-Virus-Status: Clean Running this file through clamscan or clamdscan shows: Worm.Bagz.E FOUND. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
Re: [Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
I tried your test and got this, so your end is NOT passing this virus through clamav-milter: The original message was received at Tue, 17 May 2005 16:41:57 +0100 from bandsman.co.uk [127.0.0.1] - The following addresses had permanent fatal errors - <[EMAIL PROTECTED]> (reason: 554 5.7.1 virus Worm.Bagz.D detected by ClamAV - http://www.clamav.net) - Transcript of session follows - ... while talking to olympos.ccf.auth.gr.: >>> DATA <<< 554 5.7.1 virus Worm.Bagz.D detected by ClamAV - http://www.clamav.net 554 5.0.0 Service unavailable ___ http://lurker.clamav.net/list/clamav-users.html
[Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
> I first posted this a week ago, but I still have not found a >solution. > > Since v0.84, I've been receiving various obviously crafted mails >that contain viruses, but pass through clamav-milter ok. However, when I >save the mail and scan the mbox file with clamdscan (not clamscan) >"Worm.Bagz.D" is found. Yeah. I too have been posting about this issue for weeks, and have been almost completely ignored. I'm happy (sort of) to see that other folks have the same problem, anyway. For me it started when I was running version 0.83, on 01May. I'm still getting 3000 to 5000 of these false negatives per day, where my usual rate is more like a tenth of that. --- Jef Jef Poskanzer [EMAIL PROTECTED] http://www.acme.com/jef/ ___ http://lurker.clamav.net/list/clamav-users.html
[Clamav-users] Re: virus passing through clamav-milter, but not through clamdscan!
Hi evrybody
I first posted this a week ago, but I still have not found a
solution.
Since v0.84, I've been receiving various obviously crafted mails
that contain viruses, but pass through clamav-milter ok. However, when I
save the mail and scan the mbox file with clamdscan (not clamscan)
"Worm.Bagz.D" is found.
When I submit the contaminated mailbox
(http://users.auth.gr/~apap/spurious-viral-mbox) to www.clamav.net, I get
the expected response "clamav already recognizes the content you submitted",
"there is no reason to resubmit it. "
It seems that when the crafted mail is sent directly to my mail
server (now sendmail 8.13.4, clamav-milter 0.85, ClamAV 0.85.1/882/Tue May
17 09:48:03 2005), the mail passes through. As I have found out, if it gets
relayed to another mail server with clamav, some how the virus is then
detected, but if the recipient is local, the viral mail gets through. It
seems that there is something strange in the original headers, that gets
cleared when passing through a mail server.
Here is the raw evidence that a mail that gets detected as viral by
clamdscan, passes through clamav-milter that uses the very same clamd, at
least at the first mail server in the path. Both clamdscan, and the mail
server clamav-milter use the very same clamd.
$ wget -q http://users.auth.gr/~apap/spurious-viral-mbox # Fetch a copy of my
viral mail
$ clamdscan spurious-viral-mbox # Checkit your self
/home/apap/spurious-viral-mbox: Worm.Bagz.D FOUND
$ /usr/sbin/sendmail -v [EMAIL PROTECTED] >> EHLO helios.ccf.auth.gr
250-olympos.ccf.auth.gr Hello helios.ccf.auth.gr [155.207.1.6], pleased to meet
you
>>> MAIL From:<[EMAIL PROTECTED]> SIZE=202598 BODY=8BITMIME
250 2.1.0 <[EMAIL PROTECTED]>... Sender ok
>>> RCPT To:<[EMAIL PROTECTED]>
>>> DATA
250 2.1.5 <[EMAIL PROTECTED]>... Recipient ok
354 Enter mail, end with "." on a line by itself
>>> .
250 2.0.0 j4HDrlkc007312 Message accepted for delivery
[EMAIL PROTECTED] Sent (j4HDrlkc007312 Message accepted for delivery)
Closing connection to smtp.ccf.auth.gr
>>> QUIT
221 2.0.0 olympos.ccf.auth.gr closing connection
--
Apostolis Papayanakis
[EMAIL PROTECTED], 2310-998416
On Wed, 11 May 2005, Apostolos Papayanakis wrote:
> Hi everybody,
>
> I've received more than twenty profoundly viral mails since last night.
> They passed without being stopped, through our sendmail Clamav (ClamAV
> 0.84/875/Tue May 10 14:27:59 2005+clamav-milter 0.84e). However if I save
> each of these viral mails in a seperate mbox, "clamdscan" with the same
> definitions can suddenly detect "Worm.Bagz.D" in them.
>
> It seems that clamav-milter cannot handle these mails correctly, and
> misses something while communicating (externally) with clamd. I should
> mention that the mbox contains an attachment BASE64 encoded in long lines
> o 2048 bytes(!), a mangled date header and a crafted filename with lots
> of spaces, eg: "help.doc .exe"
>
> I cannot submit the viral mbox on www.clamav.net, because it says that
> "the virus is already detected".
>
> Is this a wide-spread problem?
>
> Apostolis Papayanakis
>
> p.s. Here follows a part of the mailbox that passes through our mail server=
> ,
> and detected as "Worm.Bagz.D" from clamdscan:
> (">" is added at the start of each line to avoid being detected as "broken =
> executable" by clamd)
> ---=
>
> >From [EMAIL PROTECTED] Wed May 11 03:02:23 2005
> >Received: from 127.0.0.1 ([211.191.198.7])
> >by olympos.ccf.auth.gr (8.13.3/8.13.3) with ESMTP id j4B02EsG01374=
> 5
> >for [EMAIL PROTECTED]; Wed, 11 May 2005 03:02:1=
> 5 +0300 (EEST)
> >Message-Id: <[EMAIL PROTECTED]>
> >SUBJECT: text
> >FROM: [EMAIL PROTECTED]
> >TO: [EMAIL PROTECTED]
> >DATE: [[ =BC=F6, 11 5 2005 =BF=C0=C0=FC 9:02:24 ]]
> >MIME-Version: 1.0
> >Content-Type: multipart/mixed; boundary=3D"bound--"
> >X-Virus-Scanned: ClamAV version 0.84, clamav-milter version 0.84e on antiv=
> irus1.ccf.auth.gr
> >X-Virus-Status: Clean
> >X-Spam-Checker-Version: SpamAssassin 3.0.2-gr1 (2004-11-16) on
> >helios.ccf.auth.gr
> >X-Spam-Level: *
> >X-Spam-Status: No, score=3D5.7 required=3D7.0 tests=3DBAYES_50,FORGED_HOTM=
> AIL_RCVD2,
> >HEAD_ILLEGAL_CHARS,INVALID_DATE,MSGID_FROM_MTA_ID,NO_REAL_NAME,
> >RCVD_IN_NJABL_DUL,RCVD_IN_SORBS_DUL autolearn=3Dno version=3D3.0.2=
> -gr1
> >Status: R
> >Content-Length: 207546
> >X-Keywords:
> >
> >--bound--
> >Content-Type: text/plain; charset=3Dus-ascii
> >Content-Transfer-Encoding: 7bit
> >
> >Hello,
> >What version of windows you are using?
> >This last document I received from you came out weird.
> >Please see the attached word file and resend the file to me.
> >Many thanks,
> >User
> >
> >--bound--
> >Content-Type: application/x-msdownload; name=3D"help.doc =
> .exe"
> >Content-Transfer-Encoding: base64
> >Conte
