Re: [clamav-users] [Clamav-devel] ClamAV(R) blog: ClamAV 0.102.0 Release Candidate is now available

[Dennis Peterson]

This particular hard requirement (libcurl) affects the communication channel 
which is different than causing the code to fail to run at all. So the question 
is do the new libcurl requirements immediately break existing systems that are 
not yet updated with new libcurl functionality. It is kind of a big deal to 
update a widely used library and creates knock-on problems from ripple effect 
for production systems subject to strong configuration management policies.


dp

On 10/4/19 7:22 AM, G.W. Haywood via clamav-users wrote:

Hi there,

On Fri, 4 Oct 2019, Matus UHLAR - fantomas wrote:


On 03.10.19 11:05, Dennis Peterson wrote:

Does this obsolete earlier versions of ClamAV?


depending on what you exactly mean by obsoleting.
older versions will work and will be supported for some time.
but freshclam will warn about newer version available.

if you have installed clamav via any OS/distribution, you usually can trust
them to provide newer version soon.


In case it's of any interest:

For several weeks I ran a mix of three versions of clamd, from both
0.101.4, and 0.102rc side-by-side, all on the same machine, with no
problems (other than that obviously they're a little picky about
library versions).  They all used the same set of databases, updated
by a single freshclam instance.  A milter piped the mail to all three
clamd instances concurrently, and in this usage I saw no evidence of
any differences in their performance.




___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] [Clamav-devel] ClamAV(R) blog: ClamAV 0.102.0 Release Candidate is now available

[Dennis Peterson]

Does this obsolete earlier versions of ClamAV?

dp

On 10/2/19 2:20 PM, Joel Esler (jesler) via clamav-users wrote:

Ssl interaction with mirrors and ClamAV.net.

Sent from my  iPhone


On Oct 2, 2019, at 16:42, Rick Cooper  wrote:


Not wanting to appear stuipid but exactly what important security feature 
does the new lincurl include that is so important to moving clamav forward?



___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV: Local Private Mirror

[Dennis Peterson]

Before retiring I had a requirement to place an AV tool on all our Unix systems, 
most of which did not have direct internet access. They were distributed across 
several subnets, as well. A single local mirror was able to handle the load, and 
our load on the ClamAV mirror farm was not impacted. The mirror was a VM, and 
most of the memory was used as file system cache as there was nothing else 
running on the box. It was very effective and provided a single point of logging 
for the updates.


A similar requirement a couple years earlier was solved by integrating a local 
mirror with CFEngine to push signatures to the systems on a schedule that 
ensured redundant systems were not all reloading signatures at the same time.


dp

On 7/30/19 10:13 AM, Joel Esler (jesler) via clamav-users wrote:

I'm interested as to why people want to do private mirrors?  Other than to save bandwidth 
going to "the internet"?




___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Probably something simple but new to ClamAV

[Dennis Peterson]

In the directory where the signatures are stored run the following:

for file in *.cld;do sigtool -i $file;done

for file in *.cvd;do sigtool -i $file;done

Or run freshclam -v.

You can also examine the clamd log files.

dp

On 6/4/19 8:29 AM, Rodney Stratford via clamav-users wrote:

I have installed ClamAV in my PCF environment.  But security team is looking at 
how to display the virus signature level is of the AV.  Is there a command or a 
tool that can display this?  Any help is appreciated.  Thanks




___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] LSD Malwares

[Dennis Peterson]

That domain is hosted on a cloudflare IP block. They're become part of the 
problem.

dp

On 4/25/19 7:52 AM, J.R. via clamav-users wrote:

Perhaps it would also be worthwhile to report dd.heheda.tk to their
hosting provider & domain registrar that they are hosting malware and
get that site shut down...

___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml




___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Txt.Trojan.Kryptik-6887991-0 FOUND

[Dennis Peterson]

That does not appear to be a well anchored regex.

dp

On 3/12/19 9:15 PM, Al Varnell via clamav-users wrote:
All I can add is some technical information about the signature. I have no 
idea what kind of infection it causes and on what platform.


The signature was added to the database by daily - 25386 earlier today as an 
.ldb. Looking for a single ascii string in any type of file:



sigtool -fTxt.Trojan.Kryptik-6887991-0|sigtool --decode-sigs
VIRUS NAME: Txt.Trojan.Kryptik-6887991-0
TDB: Engine:51-255,FileSize:262144-1048576,Target:0
LOGICAL EXPRESSION: 0
 * SUBSIG ID 0
 +-> OFFSET: ANY
 +-> SIGMOD: NONE
 +-> DECODED SUBSIGNATURE:
1/g,"");if(!/^[-_a-zA-Z0-9#.:* ,>+~[\]()=^$|]+$/.test(c))throw  E



___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] after installation in an RHEL7, clamd not there

[Dennis Peterson]

https://fedoraproject.org/wiki/EPEL

On 2/22/19 9:38 PM, Sunhux G via clamav-users wrote:
Heard from an ex-colleague that using latest Clam packages from the latest 
epel will solve this.

Anyone know the link/url for this latest epel ?

Sun




___

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Using clamav to test for bad links in incoming emails

[Dennis Peterson]

Does SA scan attachments now?

dp

On 2/14/19 8:07 AM, Alessandro Vesely wrote:

On Sat 09/Feb/2019 00:07:28 +0100 Gene Heskett wrote:

Has anyone rigged clamd to check what looks like questionable links
contained in incoming emails? It seems over the last 2 weeks my spam has
tripled, and I suspect the real payload is in the urls in the message.

Shouldn't that be done with SA?
http://uribl.com/usage.shtml

Best
Ale



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Using clamav to test for bad links in incoming emails

[Dennis Peterson]

Best practice has always been least-expensive first and incrementally more 
expensive to follow. This begins with iptables (essential regardless of 
expense), tcpwrappers, DenyHosts, Fail2Ban, grey listing, country-code tables, 
access tables (sendmail and Postfix), multilayer milters, finally, AV scanning. 
The first three are also very effective defense for ftp, ssh, rsync, imap, pop, 
etc.


My ipset table has just a few blocks: afrinic, apnic, arin, lacnic, ripe. There 
are thousands of x.0.0.0/8 - x.0.0.0/24 drop all entries found in there.


Expense here refers to resource load (memory, cpu, network, disk io).

dp

On 2/9/19 9:47 AM, G.W. Haywood wrote:

Hi there,

On Sat, 9 Feb 2019, Gene Heskett wrote:


Has anyone rigged clamd to check what looks like questionable links
contained in incoming emails? It seems over the last 2 weeks my spam has
tripled, and I suspect the real payload is in the urls in the message.


Trawl the logs to see where it comes from.  I find blocking incoming
mail by country code to be far more effective than almost anything else.
I'll hazard the guess that Asia and Eastern Europe will figure large in
the results.


Or is this so time consuming and bandwidth wasting its not worth it?


ClamAV is pretty resource intensive, so more or less anything that
will reduce the number of calls to ClamAV processes will be well worth
doing.  Here, at the moment, clamd sees about 1.3% of attempts to send
mail to us.  That is, in February, 98.7% of incoming mail connections
were rejected before clamav-milter ever got to see any data.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] How to use clamav-unofficial-sigs with clamd

[Dennis Peterson]

Highly configurable scripts exist to handle the third-party signatures and it is 
all very well documented at the Sane Security web site ( 
https://sanesecurity.com ). These same scripts are available at multiple repos 
as installable packages for many operating systems as well.


dp

On 2/10/19 6:08 AM, Andrew Watkins wrote:


Hello,

May sound like a stupid question but what is the best way to use the 
clamav-unofficial-sigs which we download.


So, clamav-unofficial-sigs are copied to 
"clamav-unofficial-sigs/unofficial-dbs/*/*"

and

clamav are downloaded to  "share/clamav"

So, do we copy clamav-unofficial-sigs to clamav location so that clamd can 
access them are is the away to add several locations to


Thanks,

Andrew



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Input Stream Scanning for very large files

[Dennis Peterson]

Should have been file type as reported by the file command. Any usage of ClamAV 
outside its design objectives is vulnerable to failure, but the method I pointed 
out works, period. But if asked if I thought it was worth it I would say no, of 
course not. The OP seems determined though. ClamAV is first and foremost an 
acceptable real-time email scanner with limited ability to do file system and 
stream scanning.


dp


On 2/3/19 2:37 PM, Ángel wrote:

On 2019-01-25 at 18:43 -0800, Dennis Peterson wrote:

You can easily use the unix split command and cat to scan files of any size. Or
use perl to break stream file segments to the stream. The first file in a split
or segment contains the file time and will need to be concatenated to the
beginning of each split or segment so clamav knows what it is. It doesn't matter
if the file makes no sense just so long as no malware is found. You will need
two split sizes in order to ensure a signature doesn't span splits which means
at least two runs of each large file, but that is trivial when scripted. SSD
drives would be useful.

dp

Sorry, but I think ClamAV is smarter than what you seem to think. While
this will allow clamav to still detect some signatures, your approach
will trivially fail for:
* Extended signatures that specify an offset (can create both False
Positives and Negatives)
* Logical signatures using eg. FileSize or NumberOfSections.
* Container signatures, as the container will be corrupted
* Hash signatures


Kind regards


PS: I assume you meat 'file mime', not 'file time'
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Input Stream Scanning for very large files

[Dennis Peterson]

Sometimes it is a management or compliance requirement.

dp

On 1/25/19 11:38 AM, G.W. Haywood wrote:

Hi there,

On Fri, 25 Jan 2019, Kushal Kumar wrote:

Re: Input Stream Scanning for very large files


... how do you propose I should scan an archive of 100GB ( let's say) size.


I wouldn't propose anything like that, because I'd call it madness.

If you think there's a problem, why not deal with it before it gets
into your archives?



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Input Stream Scanning for very large files

[Dennis Peterson]

You can easily use the unix split command and cat to scan files of any size. Or 
use perl to break stream file segments to the stream. The first file in a split 
or segment contains the file time and will need to be concatenated to the 
beginning of each split or segment so clamav knows what it is. It doesn't matter 
if the file makes no sense just so long as no malware is found. You will need 
two split sizes in order to ensure a signature doesn't span splits which means 
at least two runs of each large file, but that is trivial when scripted. SSD 
drives would be useful.


dp

On 1/25/19 10:12 AM, Paul Kosinski wrote:

I understand that it's impractical for ClamAV to scan exceedingly large
files, as it could fill up RAM and/or page forever. But the current 4GB
hard limit is overly restrictive, especially since 32-bit addresses and
numbers are ancient history in current OSes.

In particular, scanning big archives immediately after downloading is
desirable, since there can be malware that attacks the de-compressing or
de-archiving mechanism itself. Thus simply scanning the individual
contained files isn't completely adequate.

Is there any plan to allow scanning bigger files? There still are,
after all, size limits specifiable in the config file(s), and warnings
about the consequences of setting limits too big can be documented.


On Fri, 25 Jan 2019 14:32:55 +
"Micah Snyder (micasnyd)"  wrote:


Regarding specific limits, I'm sorry to say that ClamAV is presently
limited to max file size of 4GB on most systems (and, I think
unintentionally, 2GB on some systems).

-Micah


On Jan 24, 2019, at 4:23 PM, J.R.
mailto:themadbea...@gmail.com>> wrote:

I think I framed my problem statement differently.
So, our requirement is similar the one asked by John in the
below link. I do not know if the solution proposed is a correct one..
Also, how do you propose I should scan an archive of 100GB ( let's
say) size. Does clamav have any limitations on scanning a single file
of such huge size ??

Without knowing more about this "archive" it's hard to say if ClamAV
will even pick up anything, due to the reason Micah gave in his reply.
But another issue is if this is just one humongous file you are trying
to shove through and say it *does* trigger some virus... How are you
going to know what / where the virus is? All you know is its somewhere
in your massive archive file...

You would be much better off scanning the individual files as you
assemble said archive, and obviously only need to scan files where an
infection would make sense (i.e. a text file isn't going to contain a
virus)...

There are stream settings in the clamd.conf, but I don't know what the
hard upper-limits are.

In cases like this, it's probably best to assemble you own sample
archives, one clean & one infected, and run through your proposed
process. If it works as intended, then create a few more samples and
re-test... If it doesn't work as intended then you'll need to re-think
your process...

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Can't reached server update

[Dennis Peterson]

Try it without the space after the "-".

host -t txt current.cvd.clamav.net 

dp

On 12/25/18 1:22 AM, Dorian ROSSE wrote:

Hello clamav worker,


I still have this error when I launch "host - t txt current.cvd.clamav.net 
" without the quotes


=> ;; connection timed out; no servers could be reached


I haven't update, upgrade and installing since one of my Linux server I put 
machine learning scripts!


Do I need to do a fresh install of the e-mail server?

Do my IPV6 internet blacklisted by my ISP administrator?

My clam av could be works by itself which cronjob!

Thank you in advance to bring help,

Regards.


Dorian Rosse.

Dorian Rosse.


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

On 12/20/18 10:56 AM, Dennis Peterson wrote:
This can be calculated by counting the number of ClamAV hits in the clamd log 
using ClamAV signatures and the time period between the first and last hits. 
In my case I have clamd logs back to April (252 days) and 58 hits on ClamAV 
signatures or about 4 per day. Total hits from all signature vendors over the 
same period is 5921 or roughly 100/day.


I'm in not much of a hurry to get the next daily update file within seconds of 
it becoming available. That was not the case most recently when I was 
responsible for production email systems with message rates in excess of 
1M/day. And compared to some I've run over the years that is not a lot.


dp

By way of comparison, since early November I've rejected 1300 messages using 
only tcpwrappers from a total of 4958 rejections from all MTA options employed. 
No way to know how effective firewall blocks are but there are whole regions in 
the world I never hear from directly. Proxy providers complicate that.



dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

This can be calculated by counting the number of ClamAV hits in the clamd log 
using ClamAV signatures and the time period between the first and last hits. In 
my case I have clamd logs back to April (252 days) and 58 hits on ClamAV 
signatures or about 4 per day. Total hits from all signature vendors over the 
same period is 5921 or roughly 100/day.


I'm in not much of a hurry to get the next daily update file within seconds of 
it becoming available. That was not the case most recently when I was 
responsible for production email systems with message rates in excess of 1M/day. 
And compared to some I've run over the years that is not a lot.


dp


On 12/20/18 10:37 AM, G.W. Haywood wrote:

Hi there,

Attempting to bring some sort of perspective to all this...

The number of updates per day (or hour or minute), and the currency or
otherwise of the updated data are not, I think, the things that matter.

Isn't what matters most the probability that some malicious payload
will get past your scanner?


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

The TTL of the TXT record is 30 minutes so unless you are directly polling one 
of the clamav.net dns servers you are going to get what ever is in your local 
NSCD cache.


dp

On 12/19/18 12:26 PM, Paul Kosinski wrote:


snip

They all do DNS TXT queries 3-5 times per hour, and *only* if that says
there are new CDIFFs do they invoke freshclam. As before, this is all
based on cron, and the times are staggered to avoid peaking.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

On 12/17/18 11:57 AM, Joel Esler (jesler) wrote:

Inline:


On Dec 15, 2018, at 6:23 PM, Paul Kosinski  wrote:

I don't know if flushing the daily.cvd cache would be adequate, since
there are probably some downstream caches that wouldn't follow suit.

Actually I had someone correct me after I wrote this email, we already have 
been doing that the whole time.


Thanks for that clarification - your original statement  fooled at least me :)


dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

Ignoring latency which is probably no where near the problem it was with the 
volunteer network of mirrors.


dp

On 12/15/18 2:43 PM, Alain Zidouemba wrote:

When a new

cdiff is released, is a new daily.cvd also released at the same time?

Yes.

-Alain


On Dec 15, 2018, at 4:26 PM, J.R.  wrote:

When a new
cdiff is released, is a new daily.cvd also released at the same time?

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

This raises another point which is and has been the DNS version does not and has 
not meant there was an update to the daily CVD file - just that the cdiffs exist 
to update the users' local copy of the CLD to the current version using a 
reliable and efficient signed process. This only ever mattered to people with 
private local mirrors but there have always been private local work-arounds to 
what is largely not a problem so much as an inconvenience. Security and 
convenience have an inverse relationship.


dp

On 12/15/18 11:55 AM, Joel Esler (jesler) wrote:

When Sourcefire acquired ClamAV "back in the day", we stopped accepting 
donations, as accounting for them on a corporate revenue side is more of a hassle than it 
is worth, so we just support it out of pocket.

That being said, this thread is long and I wanted to reply to is.

What if I flushed the daily.cvd cache every time we publish?  Hm...
Pointing everyone at cloudflare is an interesting idea, may be expensive for me 
though (since I pay for cloudflare from my budget).

Interesting discussions points here...



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

Things have changed a lot since Thomasz and Lucia were bearing the brunt of 
support, but other things change slowly.


https://lists.gt.net/clamav/users/115

dp

On 12/15/18 10:32 AM, Gene Heskett wrote:

On Saturday 15 December 2018 10:58:12 Micah Snyder (micasnyd) wrote:


I was actually wondering about this part too.  You would need quite a
few machines downstream of your local mirror to make up the difference
switching from cdiffs for each machine to CVD's, at least given the
current size of daily.cvd.  It probably is about time for us to fold
daily into main, and start fresh with a smaller daily.

I do want to say, since I'm not sure I've said it before, thank-you to
everyone who is making an effort to reduce bandwidth usage.  Despite
being a part of a huge corporation - we are an open source project
that doesn't have a subscription service or anything to make money for
the company.  As a result, we have very limited funds year to year and
your efforts do make a difference.  Thanks!

-Micah

NP Micah. I am a firm believer in TANSTAAFL, and have wondered why you
haven't gone to small annual fee to help pay for the bandwidth, but
since A, its working flawlessly here, and B, its free, I only have my
freshclam looking for updates 4x a day. So I am a very light load
compared to some I've read saying they are updating at 30 minute
intervals. Since it appears my ISP is also blocking stuff, I could go
down to a daily check. Clamscan of incoming mail, my main usage here,
has only resulted in a .25 megabyte viri/quarantine file in around 90
days. Thats more than good enough for the girls I go with.

Anyone, corporate or private, that is tapping your servers 48x a day, is
flat out abusing the system IMNSHO. Thank you Micah and Cisco, for this
service, I appreciate it.

On Dec 15, 2018, at 10:14 AM, J.R.
mailto:themadbea...@gmail.com>> wrote:

Third... Have you done a cost-benefit analysis? I know you said you
wanted to help reduce bandwidth, but when you are downloading the
entire daily.cvd file each time there is an update, that's currently a
little over 50MB each update. I downloaded the last 10 cdiff files and
they look to average about 15k... So by that math (I'm still drinking
my coffee this morning, so I could be wildly wrong)... You would need
to have over 3,333 machines to be saving any bandwidth...





___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] No good deed goes unpunished, or, why CVD files don't work

[Dennis Peterson]

From a best practices perspective it is best to use freshclam when talking to 
ClamAV resources. Once you have what you need from them you can do anything you 
like internally. You don't have to be nice to them at this point. I had a couple 
hundred RedHat servers to manage and they all required scanning software because 
of the industry I was in and because of HIPPA, credit card, social security, 
phone numbers and other personal information rules we were bound to. I created a 
lot of locally generated signatures to look for this information. This was 
before smart file systems that would do this for us.


When I built the local private mirror I used the cdiff files (scripted downloads 
were permitted) to create local patched .cld files. These had to be distributed 
to the hundreds of other machines and for that I initially used rsync because it 
is just bullet proof, and later I moved it all to CFengine (predecessor to 
puppet, chef).


The CFengine master server received the cld files from a snapshot file system 
(freshclam triggered the snapshot before and after an update) so new updates 
would not corrupt existing signature files, and it then immediately informed all 
the clients they had work to do to become conformal (in CFengine  terms). 
CFengine is smart enough to know to transmit differences between local and 
remote files on the fly (rsync) so net traffic is minimized as the daily and 
bytecode files don't change much. And because of the way the process works 
(creating hidden files until the differences are resolved), the hidden files are 
renamed to the original names which is very close to an atomic operation, so 
problems working with files in transport were prevented. The CFengine client 
would notify the local clamd instance when the files were ready. Clamd has to be 
told not to reload when it detects signature change. All very clean, fast, and 
secure owing to using secure processes at each step and hands-free on my part. 
It also passed federal government security audits which was the best part.


Short answer - don't use freshclam to get the signature files from your mirror 
to your clients and it won't matter if they are cld, cvd, cud, etc., and it 
doesn't burden the ClamAV servers by pulling full copies of CVD files.


As for the cdiff files not changing, that is by design because each cdiff file 
brings the local cld file to the cdiff version, and because it can't be known 
how many cdiffs have been created between user updates, they are retained for a 
period of time and freshclam applies them in order until the final cdiff matches 
the current DNS TXT record.


dp


On 12/14/18 6:58 PM, Paul Kosinski wrote:

The Good Deed

When we started using ClamAV, we wanted to distribute the database
to the several machines on our LAN in order to reduce the load on the
volunteer servers and minimize the load on our old DSL (now gone). The
best way to do this, it seemed, was to set up a trivial HTTP server to
mirror and deliver the new files. And, of course, they had to be cvd
files which, according to the FAQ, precluded "Scripted Updates" and the
much smaller cdiff files.


The Punishment

This all worked quite well until ClamAV switched to distributing the
updates via Cloudflare: then The Delays started. The Delays initially
exhibited themselves when freshclam itself(!) found that the DNS TXT
record said that a new daily.cvd was available but upon trying to
retrieve it freshclam failed, complaining about network problems. This
eventually would cause all the mirrors to be disabled.

After much investigation (documented at length in previous posts) I
noticed that the daily.cvd from the BOS Cloudflare server was often far
behind that from the IAD Cloudflare server (which always seemed to
match the DNS TXT advertisement). I began to suspect that this was
perhaps caused by a caching web proxy, probably a transparent one
"helpfully" interposed by Comcast.

While all this was going on, Joel stated that nobody else was having
(or at least reporting) these Delay problems.

Now I think I know why.


The Explanation

Most everybody (I would guess) uses the Scripted Update feature, which
is enabled by default. So, I ran an experiment. On one machine I
bypassed local mirroring, enabled Scripted Update *and* captured the
HTTP traffic to/from Cloudflare via dumpcap. What I found was that
Scripted Update does HTTP GETs for one or more daily-12345.cdiff
files in sequence, each, presumably, updating "daily" from the
numerically previous version.

Now it became clear! Each daily-12345.cdiff *always* has the same
content, no matter when it is retrieved. The content of daily.cvd, on
the other hand varies over time. That makes *any* caching of daily.cvd
files susceptible to cause versioning problems, whereas the cdiff files
(such as daily-12345.cdiff) are totally invulnerable to any caching
whatsoever: web caches work according to file *name*, not file content.

This problem is exacerbated by the fact that the 

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

[Dennis Peterson]

I wonder if the file size changed when Joel regenerated the daily.cvd file  (or 
I had in unexplainable file size error). I still use all the technology but no 
longer for big dot coms. The patched files are larger because they have a lot of 
unneeded bits in them.


dp

On 12/12/18 7:43 AM, Paul Kosinski wrote:

The daily.cvd is still less than half as big as main.cvd:

   -rw-r--r-- 1 clamav clamav 117892267 Jun  7  2017 main.cvd
   -rw-r--r-- 1 clamav clamav  53147013 Dec 11 14:03 daily.cvd

but indeed using the cdiffs could save bandwidth.

I never tried using cdiffs since the FAQ said "Let freshclam download
the *.cvd files", and I wasn't sure if "scripted update" would actually
create a proper cvd for both local mirroring *and* HAVP. Also, I
figured that we were already saving lots of bandwidth by doing local
mirroring instead of N separate freshclam external downloads.

P.S. After retirement there is less pressure, but the technology I deal
with daily (for my own purposes, rather than for pay) doesn't seem to
get any simpler.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

[Dennis Peterson]

You know the daily.cvd file is now larger than the main.cvd file, so you are 
burning up a lot of bandwidth if your world-facing ClamAV mirror is ignoring 
cdiff files. If it is using freshclam then it is using cdiffs and merging them 
as part of the process of mirroring. In that case your clients won't see the 
cdiff files which is perfectly acceptable. I used to use a proxy when many 
systems were co-located and it was very effective and was also being used for 
other purposes. Life is much simpler now that I'm retired.


dp

On 12/11/18 11:45 AM, Paul Kosinski wrote:

Ever since we set up a local mirror on our LAN, we have not been using
cdiffs. The reason for this is that I followed the procedure outlined
on the ClamAV website (about 2/3 down the page) at:

   http://www.clamav.net/documents/clamav-virus-database-faq

where it says:

[Q] I’m running ClamAV on a lot of clients on my local network.  Can I serve 
the cvd files from a local server
 so that each client doesn’t have to download them from your servers?
   
[A] Sure, you can find more details on our Mirror page.
   
If you want to take advantage of incremental updates, install a proxy server and then

 configure your freshclam clients to use it (watch for the HTTPProxyServer 
parameter in man freshclam.conf).
   
The second possible solution is to:
   
   Configure a local webserver on one of your machines (say machine1.mylan)
   
   Let freshclam download the *.cvd files from http://database.clamav.net to the webserver’s DocumentRoot.
   
   Finally, change freshclam.conf on your clients so that it includes:
   
   DatabaseMirror machine1.mylan
   
   ScriptedUpdates off
   
   First the database will be downloaded to the local webserver and then the other clients

 on the network will update their copy of the database from it.
   
   Important: For this to work, you have to add ScriptedUpdates off on all of your machines!


Since I didn't want to set up a proxy server for this purpose, I used
the 2nd solution (and a very trivial web server). Thus, cvd files only.

P.S. I am now thinking about trying the BOS vs IAD test for cdiff
files. But, even if cdiff files always work without any delays, doesn't
"scripted update" on occasion have to back off to downloading full cvds?

P.P.S. Thanks for the curl help!



On Mon, 10 Dec 2018 20:34:45 -0800
Dennis Peterson  wrote:


You were using curl (I did remember that after I posted as I'd helped
you sort out curl options to do what you wanted) to explore what was
available on the servers compared to what was on the DNS TXT record,
and that was outside process. It also ignored cdiff files that may
have been available in a version that matched the TXT record. The
purpose of the cdiff files is to cut down on bandwidth.

dp

On 12/10/18 6:34 PM, Paul Kosinski wrote:

We ARE using freshclam to perform the actual update. And always have
been!

We've only been using curl (not wget, if that matters) to pull the
first few bytes of the cvd to see if its version number matches
what the DNS TXT query said.

We do this because, after the conversion to Cloudflare, we were
getting lots of FAILURES where *freshclam* said things were out of
sync (and eventually disabled all the mirrors).

And we have recently seen that our Web server sometimes can get the
new updates (from IAD) *hours* before our main LAN does (from BOS).

P.S. It's been quite frustrating getting some replies seemingly
based on assumptions that we are doing things we shouldn't, when we
aren't in fact doing those things. (Like not using freshclam.)



On Mon, 10 Dec 2018 16:46:42 -0800
Dennis Peterson  wrote:


Exactly right. We can't be blaming the ClamAV process when we don't
use the ClamAV process. People that don't use freshclam should have
no expectation of high reliability. In fact any expectations are
baseless when the wrong tools are employed.

dp

On 12/9/18 5:44 AM, Joel Esler (jesler) wrote:

As it should be.  No one should be downloading the daily and main,
(although thousands are), cdiffs were created for a reason.

Sent from my  iPhone


On Dec 9, 2018, at 06:58, Eric Tykwinski 
wrote:

   From back in archives, I think he’s using wget to just pull the
files, but freshclam would just pull the cdiffs and keep you up
to date on the next check.

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Can't detect deceptive URL's as infected !!

[Dennis Peterson]

Yes - the extension can be one or the other. The other thing to check is the 
file ownership and permissions, and finally to search your clamd.log file (or 
what ever it is called on your system) for "FOUND". If it is a useful signature 
source your logs should indicate clamd is finding targets from the safebrowsing 
signature file. In your freshclam log you should see the safebrowing file is 
being updated from time to time. My own system, with rare exception, only ever 
finds Sane Security signatures, and most http links are caught by my milter via 
dns-based URLBL blacklists before it sends the messages to Clamd.


dp

On 12/11/18 3:54 AM, Sunny Marwah wrote:

I can see below files in /var/lib/clamav/ directory :

main.cvd
bytecode.cvd
safebrowsing.cld
daily.cld
mirrors.dat

But it is 'safebrowsing.cld', not 'safebrowsing.cvd'.

Is it Ok ??



On Tue, Dec 11, 2018 at 1:47 PM Dennis Peterson <mailto:denni...@inetnw.com>> wrote:


In your ClamAV signature folder does there exist a safebrowsing.cvd file?

dp

On 12/10/18 9:46 PM, Sunny Marwah wrote:
>
> Same question again : Chrome don't open malicious links due to labeling
them
> dangerous as per "Safebrowsing". Then why ClamAV is not able to identify
such
> malicious links when "Safebrowsing" option is already enabled ??

___
clamav-users mailing list
clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



--
Regards
Sunny
System Engineer
Mob : +91 9711155549


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Can't detect deceptive URL's as infected !!

[Dennis Peterson]

In your ClamAV signature folder does there exist a safebrowsing.cvd file?

dp

On 12/10/18 9:46 PM, Sunny Marwah wrote:


Same question again : Chrome don't open malicious links due to labeling them 
dangerous as per "Safebrowsing". Then why ClamAV is not able to identify such 
malicious links when "Safebrowsing" option is already enabled ??


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

[Dennis Peterson]

You were using curl (I did remember that after I posted as I'd helped you sort 
out curl options to do what you wanted) to explore what was available on the 
servers compared to what was on the DNS TXT record, and that was outside 
process. It also ignored cdiff files that may have been available in a version 
that matched the TXT record. The purpose of the cdiff files is to cut down on 
bandwidth.


dp

On 12/10/18 6:34 PM, Paul Kosinski wrote:

We ARE using freshclam to perform the actual update. And always have
been!

We've only been using curl (not wget, if that matters) to pull the first
few bytes of the cvd to see if its version number matches what the DNS
TXT query said.

We do this because, after the conversion to Cloudflare, we were getting
lots of FAILURES where *freshclam* said things were out of sync (and
eventually disabled all the mirrors).

And we have recently seen that our Web server sometimes can get the new
updates (from IAD) *hours* before our main LAN does (from BOS).

P.S. It's been quite frustrating getting some replies seemingly based on
assumptions that we are doing things we shouldn't, when we aren't in
fact doing those things. (Like not using freshclam.)



On Mon, 10 Dec 2018 16:46:42 -0800
Dennis Peterson  wrote:


Exactly right. We can't be blaming the ClamAV process when we don't
use the ClamAV process. People that don't use freshclam should have
no expectation of high reliability. In fact any expectations are
baseless when the wrong tools are employed.

dp

On 12/9/18 5:44 AM, Joel Esler (jesler) wrote:

As it should be.  No one should be downloading the daily and main,
(although thousands are), cdiffs were created for a reason.

Sent from my  iPhone


On Dec 9, 2018, at 06:58, Eric Tykwinski 
wrote:

  From back in archives, I think he’s using wget to just pull the
files, but freshclam would just pull the cdiffs and keep you up to
date on the next check.

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

[Dennis Peterson]

Helps too to read the entire thread and the thread that preceded this one. The 
OP has used combinations of dig and wget in diagnosing his problems.


dp

On 12/10/18 5:22 PM, Gary R. Schmidt wrote:

On 11/12/2018 11:46, Dennis Peterson wrote:
Exactly right. We can't be blaming the ClamAV process when we don't use the 
ClamAV process. People that don't use freshclam should have no expectation of 
high reliability. In fact any expectations are baseless when the wrong tools 
are employed.




Sigh.

Does no one actually READ THE MESSAGES???

The OP's problem is:

FRESHCLAM FAILS, REPEATEDLY, UNTIL ALL MIRRORS ARE MARKED AS BAD
AND NO UPDATES CAN OCCUR.

Pissing up a rope about "you shouldn't do various work-arounds" is a waste of 
time and bandwidth.


The OP has shown that different Cloudflare nodes give (him) different results, 
someone should be asking CLoudflare about how this can be addressed, not 
dismissing the very valid and basic problem.


This sort of behaviour just proves that Dunning-Kruger is alive and involved 
in far too many OSS projects.


Cheers,
    Gary    B-)
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] A workaround for the major ClamAV DB update delays we have been experiencing

[Dennis Peterson]

Exactly right. We can't be blaming the ClamAV process when we don't use the 
ClamAV process. People that don't use freshclam should have no expectation of 
high reliability. In fact any expectations are baseless when the wrong tools are 
employed.


dp

On 12/9/18 5:44 AM, Joel Esler (jesler) wrote:

As it should be.  No one should be downloading the daily and main, (although 
thousands are), cdiffs were created for a reason.

Sent from my  iPhone


On Dec 9, 2018, at 06:58, Eric Tykwinski  wrote:

 From back in archives, I think he’s using wget to just pull the files, but 
freshclam would just pull the cdiffs and keep you up to date on the next check.

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Installation problem.

[Dennis Peterson]

The missing tools are either not in your path or not installed. You could run 
yum info */g++ to see if it is installed, and if it is run locate g++ and 
compare locations to your path with echo $PATH.


dp

On 12/6/18 11:28 PM, nikos wrote:

Hello list.

I'm trying to install the now version of clam and it seems to be compilation 
problems.


I run ./configure --sysconfdir=/etc --enable-milter in the programs folder and 
I get the error:


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Can't detect deceptive URL's as infected !!

[Dennis Peterson]

My most effective blocks are tcpwrappers and DNS-based IP blacklists and URI 
blacklists. Low returns on effort go to pattern matching regular expressions in 
message bodies. It isn't possible to measure the effectiveness of ipset 
blocklists when using NNN.0.0.0/8 IP blocks but there are a lot of them in my 
firewall and hosts.deny files.


dp

On 12/6/18 12:27 AM, Al Varnell wrote:
Frankly, I'm surprised that ClamAV finds any such URL's. They are way to 
dynamic (blacklisted one day and removed the next). ClamAV does malware 
detection over the long haul and trying to keep up with fraudulent web sites 
would be a full time job and better done by other means (e.g. Google Safe 
Browsing).


-Al-

On Wed, Dec 05, 2018 at 11:33 PM, Sunny Marwah wrote:

Hello Team,

We are using clamav-0.100.2 to scan few HTML email templates.

Sometimes, there are deceptive URL's mentioned in those templates and that 
template should be detected as infected via ClamAV scan process.


I can see weird output of ClamAV scan process. Sometimes it detect such 
templates as infected and sometimes, it does not detect them as infected. And 
the URL's i am talking about, are so deceptive that even Google chrome 
browser don't let us open these URL's and show us clear warning as 
"Dangerous" about deceptive website.


Can you put your views behind such unpredictable behavior ?

If you want then i can report such URL's on your malware link for reporting.

Regards
Sunny


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Can't detect deceptive URL's as infected !!

[Dennis Peterson]

You should probably look at http://uribl.com/ for this problem. ClamAV is 
targeted toward viruses and malware in email. The uribl process uses DNS just 
like DNS blacklists, is fairly light weight, and well maintained.


dp

On 12/5/18 11:33 PM, Sunny Marwah wrote:

Hello Team,

We are using clamav-0.100.2 to scan few HTML email templates.

Sometimes, there are deceptive URL's mentioned in those templates and that 
template should be detected as infected via ClamAV scan process.


I can see weird output of ClamAV scan process. Sometimes it detect such 
templates as infected and sometimes, it does not detect them as infected. And 
the URL's i am talking about, are so deceptive that even Google chrome browser 
don't let us open these URL's and show us clear warning as "Dangerous" about 
deceptive website.


Can you put your views behind such unpredictable behavior ?

If you want then i can report such URL's on your malware link for reporting.

Regards
Sunny




___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] [OT] is clamav.securiteinfo.com no more?

[Dennis Peterson]

It is implemented here as a DNS URLBL and used by a milter.

dp

On 12/5/18 9:21 AM, Benny Pedersen wrote:

G.W. Haywood skrev den 2018-12-05 18:16:


On Wed, 5 Dec 2018, Dennis Peterson wrote:

All the "tiny" url hosts are blacklisted here ...

A list of them could be useful.  Do you have such a thing, or a pointer?


https://github.com/rspamd/rspamd/blob/master/conf/redirectors.inc
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] is clamav.securiteinfo.com no more?

[Dennis Peterson]

All the "tiny" url hosts are blacklisted here because I don't need the grief 
they disguise. But he did answer my question. I haven't subscribed to those BL's 
in a very long time and was surprised to see them pop up in my log file.


dp

On 12/4/18 9:38 PM, Al Varnell wrote:

Not official, but it's a pretty standard response from those of us in the 
computer security business when we see it. I'm surprised that you haven't 
observed it before, but I posted it publicly as a PSA to anybody else who might 
be subscribed to this list. Sorry if you were offended by my doing so.

Sent from my iPad

-Al-

On Dec 4, 2018, at 21:08, Arnaud Jacques  wrote:

Did you speak the official voice of Cisco/Sourcefire/ClamAV ? Is it official 
rule of this mailing list ?
If not, then your personal point of view could be sent directly to my email.

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] is clamav.securiteinfo.com no more?

[Dennis Peterson]

I think it must have gotten re-activated when I upgraded ClamAV to 0.100.2 
recently. I haven't seen those log entries until today.


Thanks.

dp

On 12/4/18 7:17 PM, Arnaud Jacques wrote:

Hello Dennis,

Yes it is dead since years.
It has been replaced by this : http://ow.ly/LqfdL


Le 05/12/2018 à 04:09, Dennis Peterson a écrit :

I don't see a dns response for that site and logs show no recent connection.

dp




___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] is clamav.securiteinfo.com no more?

[Dennis Peterson]

I don't see a dns response for that site and logs show no recent connection.

dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Disable MaxFileSize and MaxFileSize to scan the whole system

[Dennis Peterson]

If it is a big concern you can use the split command to create  "splits" of the 
suspect file. Split accepts various size arguments (bytes, lines...) and will 
create as many files as it takes to split the entire large file. These can be 
scanned individually and discarded when done. There is a risk of a split 
happening in the middle of a section that might match a signature but that is 
small. A work around is to split a file, scan it, delete the splits, then split 
it a second time using a different split size and repeat the scan.


This is obviously tedious and works best on static files. There's always a way 
if you don't mind the effort. It is easily scriptable.


dp

On 12/3/18 8:23 AM, Albert o wrote:
Well I just want to be sure that the big files which can't be scanned don't 
contain viruses...

BTW thanks everyone for helping me out

On Mon, Dec 3, 2018, 17:21 Noel Jones <mailto:njo...@megan.vbhcs.org> wrote:


What kind of giant files are you scanning?  Many big files, such as
hard drive/DVD images or "raw" database files, are likely to
generate random false positives.




  -- Noel Jones


On 12/3/2018 3:59 AM, Albert o wrote:
> Alright thank you. Is there a way to make clamscan do the same?
>
> On Mon, Dec 3, 2018, 09:18 Al Varnell mailto:alvarn...@mac.com>
> <mailto:alvarn...@mac.com <mailto:alvarn...@mac.com>> wrote:
>
>     MaxFileSize 0 disables limiting, but that only applies to
>     clamdscan scanning.
>
>     Sent from my iPad
>
>     -Al-
>
>     On Dec 2, 2018, at 23:18, Albert o rote:
>
>>     What do I need to use in clamd.conf to scan the maximum
>>     possible size?
>>     MaxFileSize 39999M
    >>     MaxFileSize 3999M
>>     Is this syntax correct?
>>
>>     On Mon, Dec 3, 2018, 00:06 Dennis Peterson
>>     mailto:denni...@inetnw.com>
<mailto:denni...@inetnw.com <mailto:denni...@inetnw.com>> wrote:
>>
>>         I wonder how many signature writers bother to match
>>         content at the end of files. Hopefully, none, in which
>>         case full file scanning is pointless.
>>
>>         dp
>>
>>         On 12/2/18 3:02 PM, Al Varnell wrote:
>>>         Trial and error, depending on your setup.
>>>
>>>         Must not exceed the amount of RAM you have installed less
>>>         what is needed to run your system and whatever else you
>>>         have running at the time.
>>>
>>>         Best advice would be to set it to the size of the largest
>>>         file you need to scan.
>>>
>>>         -Al-
>>>
>>>         On Sun, Dec 02, 2018 at 09:35 AM, Albert o wrote:
>>>>         I removed that option.
>>>>         So what is the right way to make clamAV scan the maximum
>>>>         possible size?
>>>>         On Wed, Nov 28, 2018 at 7:31 AM Henrik K mailto:h...@hege.li>
>>>>         <mailto:h...@hege.li <mailto:h...@hege.li>>> wrote:
>>>>>
>>>>>         On Tue, Nov 27, 2018 at 05:01:40PM -0500, Albert o wrote:
>>>>>>         "sudo clamscan -r --remove=yes /"
>>>>>
>>>>>         ClamAV doesn't exactly have a perfect track record
>>>>>         regarding false positives
>>>>>         (not that any scanner would have).  Are you sure you'd
>>>>>         want --remove=yes to
>>>>>         remove some critical system files/libraries?
>>>
>>>  ___
>>>         clamav-users mailing list
>>> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
<mailto:clamav-users@lists.clamav.net 
<mailto:clamav-users@lists.clamav.net>>
>>> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users
>>>
>>>
>>>         Help us build a comprehensive ClamAV guide:
>>> https://github.com/vrtadmin/clamav-faq
>>>
>>> http://www.clamav.net/contact.html#ml
>>
>>
>>  ___
>>         clamav-users mailing list
>> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
>>         <mailto:clamav-users@lists.clamav.net
<mailto:clamav-users@lists.clamav.net>>
  

Re: [clamav-users] Disable MaxFileSize and MaxFileSize to scan the whole system

[Dennis Peterson]

I wonder how many signature writers bother to match content at the end of files. 
Hopefully, none, in which case full file scanning is pointless.


dp

On 12/2/18 3:02 PM, Al Varnell wrote:

Trial and error, depending on your setup.

Must not exceed the amount of RAM you have installed less what is needed to 
run your system and whatever else you have running at the time.


Best advice would be to set it to the size of the largest file you need to scan.

-Al-

On Sun, Dec 02, 2018 at 09:35 AM, Albert o wrote:

I removed that option.
So what is the right way to make clamAV scan the maximum possible size?
On Wed, Nov 28, 2018 at 7:31 AM Henrik K mailto:h...@hege.li>> 
wrote:


On Tue, Nov 27, 2018 at 05:01:40PM -0500, Albert o wrote:

"sudo clamscan -r --remove=yes /"


ClamAV doesn't exactly have a perfect track record regarding false positives
(not that any scanner would have).  Are you sure you'd want --remove=yes to
remove some critical system files/libraries?


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV mirrors have gotten worse!

[Dennis Peterson]

I think these reports don't tell you what you think they mean. In fact they're 
pretty much meaningless. The two different servers have different versions of 
the signature. That is perfectly normal - there is simply zero chance and it is 
naive to think they will always be fully synced in the same second of time of 
day. You can infer nothing when this occurs.


In any event these signature serial numbers are associated with the DNS txt 
record. The designed process is entirely serial - freshclam knows your installed 
signature file serial number, it knows the DNS txt record, and it requests 
updates from any of the signature servers if the local version is different from 
the DNS txt record. It will try all the mirrors until success or the list of 
mirrors is exhausted. Other things that mess with the fully synchronized state 
is that DNS caching, TTL, local system clock differences, and policies of 
various name service admins to ignore authoritative TTL suggestions.


The database.clamav.net dns is a round robin of 5 different servers and you 
cannot predict what you will receive. In fact in the best case the list be 
reordered each time you request the A record. And the chances of two different 
clients getting the same A record is very low.


Your own local resolver looks in its own cache to see if it has expired. The TTL 
record for the TXT record is 1800 seconds. If you use the dig command retrieve 
the TXT record you can watch the TTL count down:


    dig  txt current.cvd.clamav.net |grep TXT

To eliminate this as a problem source you can always use host table entries 
rather than dns for your tests. The round robin records ensure reliability for 
the client and crude load balancing for the server farm.


So worst case is the record you see can be 1800 seconds behind an updated TXT 
record. Obviously polling the current.cvd.clamav.net server directly will return 
an uncached record at the expense of recursing queries (use the IP instead of 
the hostname to avoid this).


Because these variables exist, freshclam is somewhat fault tolerant and will 
retry 3 times per mirror (default and is configurable), and if a mirror is in a 
failed state freshclam will map it out of the servers to try next time 
(mirrors.dat). The other variable is some of the sync process is demand-driven. 
In very busy systems (which these are) stale files should not exist very long. 
Your request just might be a trigger to refresh a stale file, and the next 
person to hit that server will retrieve the updated file and your system will 
move to another mirror. This scenario presumes files are pulled to the mirrors, 
not pushed.


I do believe your angst over not having complete system synchronization is 
unwarranted as there are too many uncontrollable variables and it's really not 
critical if the first mirror doesn't respond.


Finally - the current cloudflare process is pretty solid - it is a vast 
improvement over the historical mirror collaboration


On 11/26/18 4:19 PM, Paul Kosinski wrote:

I believe that the delays we have been observing are due to some
problem with the Boston Cloudflare servers, or, perhaps, Comcast has a
"transparent" caching proxy which is causing us trouble.

I recently installed the same build and configuration of ClamAV 0.100.2
on our Web server, a virtual machine hosted in NYC. It runs the same
extra code (curl etc.) to check the cvd version number that we have
locally. Since Friday, there have been no delays there, although there
have been several significant delays locally. They check at exactly
the same time as each other (i.e., via NTP synced cron jobs).

I also am now running, at each location, simple curls to read the first
few bytes of the cvd files (to get the version number), *and* to log
all the headers sent and received. These are also run at exactly the
same time (as each other) via cron.

The headers show that our local system uses the 'BOS' Cloudflare server,
while the remote one uses the 'IAD' server:

   CF-RAY: 47fd0b7af79dae32-BOS
   CF-RAY: 47fd0b8064d9c1b8-IAD

Interestingly, these two cron jobs sometimes show that the BOS server
is out of date relative to the IAD server. For example, the following
curls show that one cvd file served by the BOS server is one version
behind that served by the IAD server at the *same* time. The files'
"Last-modified" lines are of particular interest. The BOS server says
the file was last modified on Mon, 26 Nov 2018 at 06:19:22 GMT, while
the IAD server says the file was last modified on Mon, 26 Nov 2018 at
14:15:24 GMT.

In particular, the BOS "Date:" header says it's already about 14 mins
*later* than the IAD "Last-modified:" timestamp indicates. In other
words, the file delivered by the BOS server is, at time of *delivery*,
already about 14 minutes out of date.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us bui

Re: [clamav-users] ClamAV mirrors have gotten worse!

[Dennis Peterson]

On 11/22/18 8:51 PM, Paul Kosinski wrote:

I wonder how many users of ClamAV actually log their freshclam updates.
Those who don't likely won't notice freshclam temporary failures due
to an out-of-sync condition.


I just checked logs on two systems dating from July 1 and see no failures. I 
isolated the signature serial numbers and time tags and all were received with 
clock like precision. Freshclam is launched every three hours from cron.d and 
incorporates a randomizer to create a delay to help avoid pileups on common 
cardinal clock positions. No serial numbers were missed within the time slot.


dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: The ClamAV 0.101.0 release candidate is here!

[Dennis Peterson]

Does this change how socket-connected clients (milters, for example) 
communicate?

On 11/19/18 11:40 AM, Joel Esler (jesler) wrote:

# Changes to the libclamav API:

  * Those who build applications around our shared library will need to change
how they declare and pass scanning options to libclamav. Please take a
look at the change to our example code for details.
  * Scanning functions now have a filename argument. The argument is optional,
but improves the efficiency when parsing certain types that require a file
on disk to open and read, and will allow for additional improvements in
the future.
  * Many of the scanning option #defines have changed. These can be found in
our clamav.h header.
  * The libclamav version number has changed.

# Some of the clamd config and clamscan command line option names have changed. 
The original versions will still work for a time, but eventually they will be 
deprecated. These options in question are detailed in the NEWS document.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV mirrors have gotten worse!

[Dennis Peterson]

On 11/13/18 12:04 PM, Paul Kosinski wrote:

"Why are you looking at October reports?"

It was the first one. And it also shows that the problem began *before*
0.100.1 was deemed OUTDATED.

So, here's one from this morning.

I also have 4 from yesterday, 3 from Sunday Nov 11 etc. Posting them
all would be a bit tedious.


What does this line mean - that is, what is fetching from that IP? Local mirror?

Using ip '10.11.14.160' for fetching.

And we're having a completely different experience here with reliability over 
the same time span:

Mirror #1
IP: 104.16.189.138
Successes: 19
Failures: 0
Last access: Thu Nov 15 07:01:02 2018
Ignore: No
-
Mirror #2
IP: 104.16.186.138
Successes: 19
Failures: 0
Last access: Wed Nov 14 23:01:03 2018
Ignore: No
-
Mirror #3
IP: 104.16.185.138
Successes: 18
Failures: 0
Last access: Mon Nov 12 21:05:32 2018
Ignore: No
-
Mirror #4
IP: 104.16.187.138
Successes: 18
Failures: 0
Last access: Sun Nov 11 01:07:46 2018
Ignore: No
-
Mirror #5
IP: 104.16.188.138
Successes: 19
Failures: 0
Last access: Mon Nov 12 14:03:05 2018
Ignore: No

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV mirrors have gotten worse!

[Dennis Peterson]

On 11/12/18 6:28 PM, Paul Kosinski wrote:

As some of you may remember, I "solved" the problems of the Cloudflare
mirrors being out of sync by not relying on what version the DNS TXT
record reports, but double checking it by retrieving the head of the
CVD file via curl.


Why are you looking at October reports?


dp


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] request of support for flagging fraud domain

[Dennis Peterson]

If you have no reason for accepting mail from the .su top level domain then just 
block that and be done with it. Sometimes it's reasonable to take a broad brush 
response to these problematic domains.


dp

On 10/21/18 6:09 AM, Darius Baumann wrote:
I want to submit the following fraud domain for flagging in ClamAV - 
"servicemarket.su":


General Abuse details:

This domain is a fraud phishing pharmacy store and gets forwarded over spam 
and domains advertised over spam.


Evidence why malicious - That domain is flagged phishing/spam/malicious on the 
following resources:
1) 
hybrid-analysis.com/sample/d53b1767676e2397598d66ad868101674fa00947ff53b611004333d7567f22fa/5bcc38b67ca3e1682c7d469d


Flagged Spamhaus, Quttera, Bitdefender
2) 
virustotal.com/#/url/6f4b1668d3e06b174b3d1ec50d254380a6299701d8b87cd1077d5fa9f451e210/detection


Gets forwarded to by the following network of urls - collected with the 
following online tracing url:

urlscan.io/result/82a515d3-c468-42b5-91cc-e1a4172b546d#transactions
---
1) gruzvn . ru/repartitionv.html

2) dietlines4health 
.world/all/myww/cpc?bhu=CWpYzpXJ6ChgL7PL2g1c3bVeLd5Wu6aVRx2Wk

Which is also rated malicious:
hybrid-analysis.com/sample/f686717f7eaadcd9b9189c69c358eecae931186c2242f32f100a188e23c113b9/5bcba1707ca3e1789b753573

3) servicemarket . su - the complaint url

Thanks, Darius Baumann


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Latest report on update "delays"

[Dennis Peterson]

You should abandon the notion of first time perfect with these kinds of things. 
There is a false sense of urgency that is imposing a workload on a team that is 
providing a free produce and service. The tools for correcting a moment zero 
malware exists in the tool for the operator to use. The real problem is the 
discovery and validation and that is why moment zero solutions will never be 
possible.


There is a finite time required to receive a malware instance, discover it is a 
malware, discover what it applies to, and to create a signature that reasonably 
avoids false positives. I'm convinced that interval exceeds the delay due to 
sync problems by such a margin that the first interval needs as much focus as 
can be committed while the distribution issues are handled at a lower priority.


There are other probabilities - as an example, the probability that a new 
malware is sufficiently in the wild to pose a threat to an important number of 
recipients and which can be very low. Those can be queued for release cutting 
down on the number of low-value updates. And somebody has to decide what is an 
important number.


Evidence of self-replication is recognizable by the rate of increase of 
infestations and is data that can be used in setting priorities. How to collect 
that? How to collect any metrics? So far it is largely buzz generated by 
responders and which is largely anecdotal.


To be honest, many problems would be solved if all outbound mail were scanned in 
real time.


dp

On 10/20/18 8:10 AM, Paul Kosinski wrote:

Yes, file synchronization is difficult. But we *started out* using the
provided (i.e., standard) freshclam tool to update our daily.cvd (etc.).
I only built our current non-standard tool (reading the file header)
when the Cloudflare mirrors started serving out-of-date file versions
which caused freshclam to fail and blacklist the mirror (which
eventually resulted in all mirrors being blacklisted).

This says to me that the old, "standard", DNS TXT approach built in to
freshclam doesn't play well with Cloudflare (or similar mirrors?).



On Sat, 20 Oct 2018 06:57:55 -0700
Dennis Peterson  wrote:


Caching file systems do validate the requested file against a master
file to see if there has been a change. De-dupe caches do the same.
It isn't instantaneous but they also don't have to wait for the cache
to refresh as they can deliver a pass through request at the same
time they're updating the cache. This is more expensive than
scheduled sync methods, but those necessarily have a delay. These
systems should reject requests for files they don't have but that is
difficult if the updated file has the same name as the one it
replaces. I know it was always a big deal for the dot com I worked
for to update Akamai because of sync problems around the world.
Atomic synchronized file updates are pretty much impossible when you
have a million page requests/minute.

I agree with Joel about using non-standard tools to request
signatures and people that do so should have no expectation of
consistent high reliability, and support requests should go in the
bit bucket. The risk associated with self-service falls on the
operator, not the vendor.

dp

On 10/19/18 2:19 PM, Paul Kosinski wrote:

I'm glad modern multi-core / multi-thread CPU's don't operate this
way.

Imagine if, when your code on CPU1 tried to access memory location
M, your code got what CPU1 happened to have in its cache, instead
of what CPU2 stored into M a few microseconds ago. Fortunately,
with real CPUs, CPU2 invalidates the other CPUs' caches, and CPU1
takes  the extra time to fetch the new and correct data from memory.

Thus, what Cloudflare *should* have (if you can't explicitly upload
a file), is a mechanism to tell it that a file is out of date. This
mechanism could operate very quickly. Then, what Cloudflare would
do is either to stall the HTTP response -- I doubt it would have to
stall for long -- or reply with the appropriate HTTP status code
warning the requester that something is amiss. (Codes 503, 504 or
409 might be applicable.)


On Thu, 18 Oct 2018 22:34:03 +
"Joel Esler (jesler)"  wrote:


Cloudflare will grab the file from our infrastructure once it's
been requested.  (Otherwise it wouldn't know it was there, we
can't push into Cloudflare.). But we have discussed a few ideas
internally that I think will fix this, let us try a couple things
and see if it cuts down on this.

On Oct 18, 2018, at 1:55 PM, Eric Tykwinski
mailto:eric-l...@truenet.com>> wrote:

As far as I know you don't upload to cloudflare, it's more of how
often does cloudflare check to see if the files have changed.
So you setup a TTL on the check frequency on the cloudflare
website.

Since updates are new they should just be pulled when you ask from
the main clam server.
So you ask for daily-25048.cdiff, and Cloudflare will

Re: [clamav-users] Latest report on update "delays"

[Dennis Peterson]

Caching file systems do validate the requested file against a master file to see 
if there has been a change. De-dupe caches do the same. It isn't instantaneous 
but they also don't have to wait for the cache to refresh as they can deliver a 
pass through request at the same time they're updating the cache. This is more 
expensive than scheduled sync methods, but those necessarily have a delay. These 
systems should reject requests for files they don't have but that is difficult 
if the updated file has the same name as the one it replaces. I know it was 
always a big deal for the dot com I worked for to update Akamai because of sync 
problems around the world. Atomic synchronized file updates are pretty much 
impossible when you have a million page requests/minute.


I agree with Joel about using non-standard tools to request signatures and 
people that do so should have no expectation of consistent high reliability, and 
support requests should go in the bit bucket. The risk associated with 
self-service falls on the operator, not the vendor.


dp

On 10/19/18 2:19 PM, Paul Kosinski wrote:

I'm glad modern multi-core / multi-thread CPU's don't operate this way.

Imagine if, when your code on CPU1 tried to access memory location M,
your code got what CPU1 happened to have in its cache, instead of what
CPU2 stored into M a few microseconds ago. Fortunately, with real CPUs,
CPU2 invalidates the other CPUs' caches, and CPU1 takes  the extra time
to fetch the new and correct data from memory.

Thus, what Cloudflare *should* have (if you can't explicitly upload a
file), is a mechanism to tell it that a file is out of date. This
mechanism could operate very quickly. Then, what Cloudflare would do is
either to stall the HTTP response -- I doubt it would have to stall for
long -- or reply with the appropriate HTTP status code warning the
requester that something is amiss. (Codes 503, 504 or 409 might be
applicable.)


On Thu, 18 Oct 2018 22:34:03 +
"Joel Esler (jesler)"  wrote:


Cloudflare will grab the file from our infrastructure once it's been
requested.  (Otherwise it wouldn't know it was there, we can't push
into Cloudflare.). But we have discussed a few ideas internally that
I think will fix this, let us try a couple things and see if it cuts
down on this.

On Oct 18, 2018, at 1:55 PM, Eric Tykwinski
mailto:eric-l...@truenet.com>> wrote:

As far as I know you don't upload to cloudflare, it's more of how
often does cloudflare check to see if the files have changed.
So you setup a TTL on the check frequency on the cloudflare website.

Since updates are new they should just be pulled when you ask from
the main clam server.
So you ask for daily-25048.cdiff, and Cloudflare will ask Clam's main
server for that file and cache it.

So my guess would be same as the TTL on the DNS check:
current.cvd.clamav.net. 1800
IN  TXT "0.100.2:58:25048:1539883740:1:63:48006:327"
I.E. 30 minutes for older files, and new ones are when they come in.

Sound about right Joel, Micah?

Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300

-Original Message-
From: clamav-users [mailto:clamav-users-boun...@lists.clamav.net] On
Behalf Of Paul Kosinski
Sent: Thursday, October 18, 2018 1:23 PM
To:
clamav-users@lists.clamav.net
Subject: Re: [clamav-users] Latest report on update "delays"

How can it take 10, 20 30 or more minutes (and I've seen well over an
hour at times) to upload the ClamAV database to Cloudflare? Does it
have to be uploaded separately (and maybe sequentially) from Cisco to
each Cloudflare mirror? Or is Cloudflare's automatic propagation slow?


On Thu, 18 Oct 2018 16:07:38 +
"Micah Snyder (micasnyd)"
mailto:micas...@cisco.com>> wrote:

Hi Paul,

I realize it may look misleading to state that you're up to date when
a newer database has been announced.  However, if the newer database
is still being uploaded to the CDN, it is more accurate to say that
the DNS announcement is premature.

The change to freshclam is an effort to ignore potentially premature
database version numbers listed via DNS.

Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV 0.100.2 has been released!

[Dennis Peterson]

On 10/3/18 10:37 AM, Joel Esler (jesler) wrote:

https://blog.clamav.net/2018/10/clamav-01002-has-been-released.html

Are you sure 1.0 is going to happen in my lifetime? I'm not a kid anymore.

dp
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

It would be a mistake to think everyone is using freshclam to dl signatures. The 
system needs to accommodate that.


dp

On 7/4/18 10:08 AM, G.W. Haywood wrote:

Hi Joel,

FWIW I believe we've had no problems at all with mirrors since March
2018, when I responded to a post on 23rd March by Orion Poplawski, who
saw a few timeouts.  We also saw a very few timeouts in mid-late March.

On Wed, 4 Jul 2018, Joel Esler wrote:


... It's the people that are downloading the *same* diff 1000x an
hour that are the problem.


That sounds like probable cause.  I'd drop 'em in the TARPIT.

Could freshclam not be made to respect e.g. "304 NOT MODIFIED"?
(That doesn't mean I wouldn't still drop abusers in the pit.:)



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

What do you see if you run freshclam --list-mirrors, and are you running 
freshclam in daemon mode? The reason I ask is if you deleted mirrors.dat then 
freshclam should have no knowledge of any previous errors.


dp

On 7/4/18 1:18 AM, Michael Da Cova wrote:

Hi

still getting issues, (I have removed the mirror file)

the setup we have has been in place for years except for minor hiccup, but 
never this bad


main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: 
sigmgr)
daily.cvd version from DNS: 24721
Retrieving http://database.clamav.net/daily.cvd
Ignoring mirror 104.16.187.138 (due to previous errors)
Ignoring mirror 104.16.189.138 (due to previous errors)
Ignoring mirror 104.16.186.138 (due to previous errors)
Ignoring mirror 104.16.185.138 (due to previous errors)
Ignoring mirror 104.16.188.138 (due to previous errors)
Ignoring mirror 2400:cb00:2048:1::6810:bd8a (due to previous errors)
Ignoring mirror 2400:cb00:2048:1::6810:bc8a (due to previous errors)
Ignoring mirror 2400:cb00:2048:1::6810:b98a (due to previous errors)
Ignoring mirror 2400:cb00:2048:1::6810:bb8a (due to previous errors)
Ignoring mirror 2400:cb00:2048:1::6810:ba8a (due to previous errors)
ERROR: Can't download daily.cvd from database.clamav.net
Giving up on database.clamav.net...
Update failed. Your network may be down or none of the mirrors listed in 
/etc/clamd.d/freshclam.conf is working. Check 
http://www.clamav.net/documentation.html for possible reasons.



any help pointers gratefully excepted

Michael
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

Your proxy is not passing the request to the server. But never give up - try:

curl -H "Range: bytes=35-39" -s --proxy http://proxy:3128 
http://db.us.clamav.net/daily.cvd |strings


On 7/3/18 1:29 PM, SCOTT PACKARD wrote:

Hmm, I went to recreate both cases before replying, and I can get both to work, 
sort of.
I still can't resolve DNS TXT records, but I can it seems throw the URI
http://db.us.clamav.net/daily.cvd to the proxy server and it can handle it.
Beats me what IP db.us.clamav.net resolves to.
I get the whole daily.cvd, with either wget or curl.

curl's -r 35-39 isn't honored though, when fetching externally.  I get the 
whole daily.cvd.

(I swear  this doesn't work at 6am Monday morning though. :) )

Thanks, Scott


-Original Message-
From: clamav-users [mailto:clamav-users-boun...@lists.clamav.net] On Behalf Of 
Dennis Peterson
Sent: Tuesday, July 03, 2018 12:53 PM
To: clamav-users@lists.clamav.net
Subject: [External] Re: [clamav-users] We STILL cannot reliably get virus 
updates (since new mirrors)

Does your wget not support the -e args to access a proxy?

Example:
wget http://someurl.com/filename.html -e use_proxy=yes -e
http_proxy=xxx.xxx.xxx.xxx:3128

The proxy IP or hostname can be used.

dp

On 7/3/18 11:11 AM, SCOTT PACKARD wrote:

The current DNS TXT does not work within my company, as a firewall fully blocks 
things, including DNS.
(as an aside, curl works, with sufficient massaging, but wget cannot, as it 
does not have an option to work with a proxy).

I rely on someone in Arizona to pull definitions from, but sometimes their 
server goes out, other times clamav's content system

breaks,

and it's a pain to figure out which one is the culprit.

Regards, Scott


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

Does your wget not support the -e args to access a proxy?

Example:
wget http://someurl.com/filename.html -e use_proxy=yes -e 
http_proxy=xxx.xxx.xxx.xxx:3128


The proxy IP or hostname can be used.

dp

On 7/3/18 11:11 AM, SCOTT PACKARD wrote:

The current DNS TXT does not work within my company, as a firewall fully blocks 
things, including DNS.
(as an aside, curl works, with sufficient massaging, but wget cannot, as it 
does not have an option to work with a proxy).

I rely on someone in Arizona to pull definitions from, but sometimes their 
server goes out, other times clamav's content system breaks,
and it's a pain to figure out which one is the culprit.

Regards, Scott



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

Well damn - they say memory is the first thing to go...

curl -s -r 35-39 http://db.us.clamav.net/daily.cvd |strings

The -s (silent) inhibits stats.

dp

On 7/3/18 12:02 AM, Dennis Peterson wrote:
I had completely forgotten about freshclam grabbing the entire file to 
determine currency. I recall knocking off a quick script to avoid that which 
included:


curl -q -r 35-39 http://db.us.clamav.net/daily.cvd |strings

It returns the ID of what ever version is on the mirror. I've added strings to 
the end as a safety valve in case someone wants to try it with different 
arguments to the -r.


Being retired I no longer sweat the small schtuff, but when I was responsible 
for hundreds of servers I used every trick in the book to avoid wasting time 
(CFengine was involved and freshclam was not). Because the filename daily.xxx 
is overloaded (version agnostic) this kind of trick was needed.


dp

On 7/2/18 6:37 PM, Paul Kosinski wrote:

Any system whereby new versions of files are announced before they are
actually available to automated downloads is awkward (to say the least).

If, in addition, a server which doesn't have the announced version is
blacklisted by the automated downloader, the whole mechanism can grind
to a halt (as it has for us).

Even if a server which is out of sync (i.e., behind) is not
blacklisted, but merely temporarily skipped, it uses extra bandwidth in
the current scheme. In the case of daily.cvd, the only way freshclam
detects that the server is out of sync is by downloading the whole file
(currently about 47 MB) -- the waste of bandwidth is enormous. For
example, our logs this afternoon show 15 complete downloads of
daily.cvd over about 1 hour. Of these, all but the last failed due to
out of sync. This is why we have recently taken to deleting mirrors.dat
before each freshclam run -- to compensate for the blacklisting -- and
running freshclam 3 times an hour hoping for sync.

This behavior is both unreasonable and inefficient.

P.S. Just before I sent this mail, I sent some proposals for how ClamAV
might possibly avoid this behavior.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

I had completely forgotten about freshclam grabbing the entire file to determine 
currency. I recall knocking off a quick script to avoid that which included:


curl -q -r 35-39 http://db.us.clamav.net/daily.cvd |strings

It returns the ID of what ever version is on the mirror. I've added strings to 
the end as a safety valve in case someone wants to try it with different 
arguments to the -r.


Being retired I no longer sweat the small schtuff, but when I was responsible 
for hundreds of servers I used every trick in the book to avoid wasting time 
(CFengine was involved and freshclam was not). Because the filename daily.xxx is 
overloaded (version agnostic) this kind of trick was needed.


dp

On 7/2/18 6:37 PM, Paul Kosinski wrote:

Any system whereby new versions of files are announced before they are
actually available to automated downloads is awkward (to say the least).

If, in addition, a server which doesn't have the announced version is
blacklisted by the automated downloader, the whole mechanism can grind
to a halt (as it has for us).

Even if a server which is out of sync (i.e., behind) is not
blacklisted, but merely temporarily skipped, it uses extra bandwidth in
the current scheme. In the case of daily.cvd, the only way freshclam
detects that the server is out of sync is by downloading the whole file
(currently about 47 MB) -- the waste of bandwidth is enormous. For
example, our logs this afternoon show 15 complete downloads of
daily.cvd over about 1 hour. Of these, all but the last failed due to
out of sync. This is why we have recently taken to deleting mirrors.dat
before each freshclam run -- to compensate for the blacklisting -- and
running freshclam 3 times an hour hoping for sync.

This behavior is both unreasonable and inefficient.

P.S. Just before I sent this mail, I sent some proposals for how ClamAV
might possibly avoid this behavior.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

On 7/2/18 3:39 PM, Joel Esler (jesler) wrote:

I’m not at a large keyboard right now.   But with Cloudflare currently acting 
as our mirror network, none of the current assumptions about how the mirror 
network works is accurate.

We have not changed the donated mirror network, as our discussions with 
cloudflare are on going.

Sent from my iPhone

I've been out of town so just had a look at the current structure and see what 
you mean. Since June 25 my systems are using only cloudflare mirrors. There is 
now 5 physical mirrors total in a DNS round robin on different subnets but in 
the same data center. If they're using a clustered/NFS/shared/cached file system 
if any client is out of sync every client will be out of sync at the same time 
even if they use an atomic file transfer (somewhat like what rsync does). If 
they're not using any kind of shared file system and also not using some kind of 
atomic file transfer then there will still be issues.


So far, though, it has been working perfectly here. With only 5 servers though I 
don't think that will last if they start showing up in mirrors.dat.


dp
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

The current system announces a new version of the signatures is available before 
all the mirrors have received the update.  Another design option is for ClamAV 
to upload the updates to all the mirrors and then announce the new version. That 
is not what we have and there are good reasons for it.



Freshclam 101 as I understand it:

The site that announces the version ID via DNS is not the mirror that 
distributes the new version. In the current system there is lag between the 
announcement in DNS and the availability at the mirrors. In alternate design 
described above the mirrors are guaranteed to have the update when the 
announcement (DNS result) is updated. It also means that many mirrors have the 
update long before the last mirror does, but the freshclam clients don't know to 
look for it until the DNS record is changed.


The as-built process:

Freshclam is aware of the currently installed signature version and if the next 
time it runs it learns there is a new version from the DNS server it will 
attempt to retrieve it from a mirror. Freshclam has no means of knowing if any 
of the mirrors it is configured to retrieve from is synched until it asks a 
mirror. Being told it does not exist at the first mirror it polls, it gracefully 
adds a log entry and tries the next. It is entirely possible that none of the 
configured mirrors has the update because there is an unavoidable lag in the 
system. It will gracefully stop trying when the last mirror is found out of 
sync. So far, so good. It will try again at the next scheduled poll.



Things will go to hell in a hurry though if the mirrors.dat algorithm 
disqualifies a mirror for being out of sync. I've never looked but I hope this 
is not the case because a mirror that is out of sync is not evidence of a broken 
mirror.


Not having run a mirror I don't know of the updates are pushed out from the 
repository of if they are pulled from the mirror. If they are pushed there 
should be little lag updating the first mirror. If they are pushed out in 
parallel to every mirror there will be a big burst of bandwidth and lag will be 
determined by bandwidth. If they are pushed out serially there will be lag 
between the first and last mirror update.


A different push method would push the new version ID to the mirrors via a 
lightweight process (DNS, for example) and each mirror would respond by pulling 
the new signature. This would also create a burst of bandwidth, but it would be 
more gentle than a parallel push.


If a push is not employed then the repository would have to be polled from the 
mirror and there is lag in the polling process. How often does the polling run?


An intelligent design would create mirror tiers where a subset of mirrors are 
synched (push) quickly from the repository and the next tier of mirrors can now 
update from this block of mirrors rather than the repository alone, and this 
will distribute the load and minimize bandwidth induced lag. NIS works in this 
fashion.


Another option is to build a tuple space server which can service these requests 
in a massively parallel way.


dp

On 7/2/18 7:20 AM, Paul Kosinski wrote:

I don't understand your reply. Exactly *how* do we "wait until every
mirror is synchonized, become notified, then try".

Freshclam is run periodically, automatically (via cron, in our case).
Shouldn't it be freshclam's job to do things at the right time. And how
would *it* know when all mirrors are synced? Is it Talos that populates
the mirrors? Then Talos shouldn't update the DNS TXT records until *all*
mirrors are ready.

P.S. The client's mirrors.dat file is updated in 18 different places in
manager.c, which is in the freshclam subsystem.


On Sun, 1 Jul 2018 21:11:29 -0700
Dennis Peterson  wrote:


What makes it a problem? You can never dl it until it is available,
so the problem is you become aware of it too soon. But think about
what that means. Your choices are to know immediately when an update
is available and try to get it, or wait until every mirror is
synchonized, become notified, then try. The first choice is a
crapshoot you might win. The second choice isn't a crapshoot but it
also doesn't save time. Remembering all this is automated the result
is actually some uninteresting log entries.

It would be interesting to know if an update notice is sent to all
mirrors in the fashion of a DNS notification to slaves which would
cause a parallel pull, or if the update itself is pushed, and what
the process is for updating the client mirrors.dat file.

dp

On 7/1/18 9:01 PM, Al Varnell wrote:

Seems to me that it's only a problem if it takes a significant
amount of time between the DNS update and the mirror updates. I
don't have a good feel for how long that is from the postings so
far, but it does sound like it may have increased as a result of
the move from ClamAV mirrors to the ClamAV CDN.

Sent from my iPad

-Al-

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

My interest is if a non-synched mirror would trigger an entry in which case many 
false entries are possible. That is a cascading  error that would be complicated 
by close-in-time updates. Just noodling out of the box a bit, here.


dp

On 7/1/18 9:28 PM, Al Varnell wrote:

As far as the client mirrors.dat file, it's updated locally by freshclam to 
indicate either success or failure for a specific IP. After a specific number 
of failures (I've forgotten what that is) the IP is given a “time-out” which 
precludes it's use until some amount of time passes. Under normal 
circumstances, it's self-correcting over time, but what seems to be happening 
now is involves multiple failures over an extended time resulting in all 
mirrors being locked out, requiring manual intervention to delete the file 
which restarts the process.

Sent from my iPad

-Al-


On Jul 1, 2018, at 21:11, Dennis Peterson  wrote:

What makes it a problem? You can never dl it until it is available, so the 
problem is you become aware of it too soon. But think about what that means. 
Your choices are to know immediately when an update is available and try to get 
it, or wait until every mirror is synchonized, become notified, then try. The 
first choice is a crapshoot you might win. The second choice isn't a crapshoot 
but it also doesn't save time. Remembering all this is automated the result is 
actually some uninteresting log entries.

It would be interesting to know if an update notice is sent to all mirrors in 
the fashion of a DNS notification to slaves which would cause a parallel pull, 
or if the update itself is pushed, and what the process is for updating the 
client mirrors.dat file.

dp


On 7/1/18 9:01 PM, Al Varnell wrote:
Seems to me that it's only a problem if it takes a significant amount of time 
between the DNS update and the mirror updates. I don't have a good feel for how 
long that is from the postings so far, but it does sound like it may have 
increased as a result of the move from ClamAV mirrors to the ClamAV CDN.

Sent from my iPad

-Al-


On Jul 1, 2018, at 20:38, Dennis Peterson  wrote:

On 7/1/18 8:24 PM, Paul Kosinski wrote:
My conclusion is that the cause of this is a typical race condition:
the DNS TXT record is updated before Cloudflare has propagated the new
cvd file to all the mirrors.



Is this a problem?

dp
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

What makes it a problem? You can never dl it until it is available, so the 
problem is you become aware of it too soon. But think about what that means. 
Your choices are to know immediately when an update is available and try to get 
it, or wait until every mirror is synchonized, become notified, then try. The 
first choice is a crapshoot you might win. The second choice isn't a crapshoot 
but it also doesn't save time. Remembering all this is automated the result is 
actually some uninteresting log entries.


It would be interesting to know if an update notice is sent to all mirrors in 
the fashion of a DNS notification to slaves which would cause a parallel pull, 
or if the update itself is pushed, and what the process is for updating the 
client mirrors.dat file.


dp

On 7/1/18 9:01 PM, Al Varnell wrote:

Seems to me that it's only a problem if it takes a significant amount of time 
between the DNS update and the mirror updates. I don't have a good feel for how 
long that is from the postings so far, but it does sound like it may have 
increased as a result of the move from ClamAV mirrors to the ClamAV CDN.

Sent from my iPad

-Al-


On Jul 1, 2018, at 20:38, Dennis Peterson  wrote:


On 7/1/18 8:24 PM, Paul Kosinski wrote:
My conclusion is that the cause of this is a typical race condition:
the DNS TXT record is updated before Cloudflare has propagated the new
cvd file to all the mirrors.



Is this a problem?

dp
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] We STILL cannot reliably get virus updates (since new mirrors)

[Dennis Peterson]

On 7/1/18 8:24 PM, Paul Kosinski wrote:

My conclusion is that the cause of this is a typical race condition:
the DNS TXT record is updated before Cloudflare has propagated the new
cvd file to all the mirrors.



Is this a problem?

dp
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Errors connecting to mirrors

[Dennis Peterson]

Since db.us.clamav.net is a round robin resolving to db.us.big.clamav.net, 
another round robin, try the actual server hostname to dl a known file. The 
specific diff files come and go and may not be on a particular mirror server. 
The following worked for me - I send the output to /dev/null to save time.


curl --resolve db.us.big.clamav.net:80:72.21.91.8
http://db.us.big.clamav.net/bytecode.cvd 2>&1 >/dev/null

dp


On 4/5/18 2:56 PM, Orion Poplawski wrote:

On 03/30/2018 09:48 AM, Orion Poplawski wrote:

And still having persistent problems with 72.21.91.8 as reported here:
https://bugzilla.clamav.net/show_bug.cgi?id=12068


And it is still not there:

# curl --resolve db.us.clamav.net:80:72.21.91.8
http://db.us.clamav.net/daily-24447.cdiff

http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";>
http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en">
 
 404 - Not Found
 
 
 404 - Not Found
 


Nor any other db files...

Feel like I'm shouting into the void with this


Here's a little test script:

host db.us.clamav.net |
awk '/address/ { print $4 }' |
while read ip;
do echo Trying $ip;
curl --resolve db.us.clamav.net:80:$ip -w 'result=%{http_code}\n\n' -o
/dev/null http://db.us.clamav.net/daily-24447.cdiff;
done

Output:


Trying 74.115.25.14
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
   0 00 00 0  0  0 --:--:--  0:02:07 --:--:--
0result=000

curl: (7) Failed connect to db.us.clamav.net:80; Connection timed out
Trying 200.236.31.1
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0  37220  0 --:--:-- --:--:-- --:--:-- 37300
result=200

Trying 72.21.91.8
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100   345  100   3450 0   6873  0 --:--:-- --:--:-- --:--:--  6900
result=404

Trying 146.112.59.53
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0  43418  0 --:--:-- --:--:-- --:--:-- 43494
result=200

Trying 198.148.78.4
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0  91546  0 --:--:-- --:--:-- --:--:-- 91858
result=200

Trying 150.214.142.197
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0  18416  0 --:--:-- --:--:-- --:--:-- 18399
result=200

Trying 204.130.133.50
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0   230k  0 --:--:-- --:--:-- --:--:--  231k
result=200

Trying 12.167.151.1
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0  88046  0 --:--:-- --:--:-- --:--:-- 88553
result=200

Trying 155.98.64.87
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0  67394  0 --:--:-- --:--:-- --:--:-- 67262
result=200

Trying 12.167.151.2
   % Total% Received % Xferd  Average Speed   TimeTime Time  Current
  Dload  Upload   Total   SpentLeft  Speed
100 12309  100 123090 0  85108  0 --:--:-- --:--:-- --:--:-- 85479
result=200

So looks like 74.115.25.14 is bad too.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ping database.clamav.net

[Dennis Peterson]

Ping is not a good test of DNS. You should use dig, nslookup, host, or other DNS 
tool.


dp

On 3/29/18 5:10 AM, Régis Houssin wrote:

yes but for this IP this not a clamav website !

dev.lepartidegauche.fr (178.33.105.132)


thank you




___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Errors connecting to mirrors

[Dennis Peterson]

If your proxy ignores the TTL for the mirrors then quite likely things will 
grind to a halt for you. All the mirrors are in round-robin dns pools.


dp

On 3/27/18 4:32 PM, Orion Poplawski wrote:

On 03/27/2018 05:21 PM, Al Varnell wrote:

Using the same IP each time with failure will also cause mirrors.dat to 
temporarily block that IP's use for some period of time. That will require you 
to trash mirrors.dat and allow it to be rebuilt at the next check.

-Al-

I don't think mirrors.dat comes into play here as the proxy is doing the dns
lookup, not freshclam.


On Tue, Mar 27, 2018 at 03:40 PM, Orion Poplawski wrote:

On 03/27/2018 03:13 PM, Orion Poplawski wrote:

Thanks for the response.

I ended up switching freshclam to use our proxy servers and increasing the
ConnectTimeout to 60 seconds.  This has helped a bit, but I still get the
occasional issue.  Latest was trying to get daily-24426.cdiff from 72.21.91.8
around Tue Mar 27 13:31:14 2018 PDT.  These are annoying because they generate
emails.

This was exacerbated by squid continuing to use the same IP address for the
connection each time freshclam retried the download.  I'm trying enabling
http://www.squid-cache.org/Doc/config/balance_on_multiple_ip/ 
 to see if that
helps.




___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml





___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about the clamdscan

[Dennis Peterson]

Tripwire presumes a golden fileset at the outset, that is, scanned to the degree 
possible before enabling Tripwire. The fear of zero-day loop is infinite.


dp

On 3/21/18 6:41 PM, Paul Kosinski wrote:

A few years ago, when Tripwire was no longer free, I set up a "scan
once" environment for ClamAV, identifying files using SHA1 hashing
(with a few 'stat' results like inode and timestamp for good measure).

I gave up when I realized that even if a file had already been scanned,
it might have contained "0-day" malware when it was scanned. This could
make it quite nasty, especially if ClamAV is behind in 0-day detection.


On Wed, 21 Mar 2018 16:56:06 -0700
Dennis Peterson  wrote:


It is possible to integrate ClamAV and Tripwire to get to a scan-once
environment. Include puppet or CFEngine for a more complete tool.

dp



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about the clamdscan

[Dennis Peterson]

It is possible to integrate ClamAV and Tripwire to get to a scan-once 
environment. Include puppet or CFEngine for a more complete tool.


dp

On 3/20/18 5:01 AM, Micah Snyder (micasnyd) wrote:

Good morning Tsutomu,

Al is quite correct.  clamd and clamdscan maintain no memory of what has been 
scanned before.

In your ordinary use case, you simply run clamdscan over whatever you want to 
scan.  You can exclude specific directories in your configuration if you want 
to point clamdscan at a high level directory to scan many items.

In truth, I've never tried accessing the files as they were scanned, but I do 
not believe that there any reason why the files would be locked by ClamAV 
except in the following case.

On newer versions of Linux that have been built with CONFIG_FANOTIFY=y enabled, you can 
configure clamd to monitor directories.  An additional option may be enabled that we call 
"OnAccessPrevention" can intentionally block access to the file until it has 
been scanned and will deny access if the file is flagged.  OnAccessPrevention requires 
your kernel has been built with CONFIG_FANOTIFY_ACCESS_PERMISSION=y.   If you're 
interested in trying this out, please read 
http://blog.clamav.net/2016/03/configuring-on-access-scanning-in-clamav.html

Sadly, OnAccess scanning and prevention only exist for Linux at this time.


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV performance overhead on RHEL & Solaris

[Dennis Peterson]

I ran it on dozens of enterprise systems, real and virtual, under RHEL and 
Oracle Linux. As a mail scanner running on demand it was never a great issue 
regarding performance as they were dedicated servers. But we found that when 
scanning file systems for compliance it would thrash the disk cache and Oracle 
performance would suffer. The thrashing would happen because the scans would 
crawl seldom and never accessed files which bumped active files out of cache. 
There are moderating options available in Oracle to mediate this.


I used to run it on older Sparc systems and noticed that compared to x86 Solaris 
and Linux the Sparc systems to several minutes longer to refresh the signatures 
following an update. This was moderated the only way possible by lowering the 
update rate and running it off hours when off hours existed. I've not run it on 
contemporary Sparc systems. It would rail a bound proc during the refresh.


dp

On 3/17/18 11:04 AM, Len Sanschargrin wrote:

OMG, Why are you guys trying to make this so difficult?

Ok, Here's where I am. We are planning to implement ClamAV on all Solaris & RHEL servers 
(medium sized with 1500 total servers - 4 core, 36gb mem).  The keyword being 
"PLANNING". We have not implemented it anywhere yet. So I'm looking for GENERAL 
guidance about what kind of overhead has been observed on a single server with ClamAV running. 
It's no more difficult than that. So if you have some observations worth sharing, thanks. 
Otherwise we'll get our own metrics when we install and test it in a month or so... The idea of a 
user group is for users to share experiences. I'm not trying to get free support, just user 
experience/observations and no I'm not addressing a performance problem and yes I understand that 
we won't have the exact same configuration. Think of it more as an expectation-setting & 
capacity-planning exercise

Thanks very much, Len



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ERROR: NotifyClamd: Can't connect to clamd on 127.0.0.1:3310: Connection refused

[Dennis Peterson]

If you can successfully run nc -l 3310 then clamd is not using the port. Check 
lsof -i |grep clam and examine the clamd.conf file. Something you're sure of is 
wrong.


dp

On 2/1/18 9:23 AM, Chris wrote:

On Thu, 2018-02-01 at 07:51 -0800, Dennis Peterson wrote:

Use the nc tool to connect to that port. If you get a connection then
type PING.
It should return PONG and disconnect. If that doesn't happen you have
a config
misunderstanding.

dp

Thanks Dennis, I used nc -zv to try and connect to port 3310 with
127.0.0.1 as per my settings:

nc -zv 127.0.0.1 3300-3400
nc: connect to 127.0.0.1 port 3300 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3301 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3302 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3303 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3304 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3305 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3306 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3307 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3308 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3309 (tcp) failed: Connection refused
nc: connect to 127.0.0.1 port 3310 (tcp) failed: Connection refused

Odd that in all the years I've run ClamAV with the same settings I've
not had this problem.

Using nc -l 3310 in one terminal and nc 127.0.0.1 3310 I get:

nc -l 3310
test
this is a test

  nc 127.0.0.1 3310
test
this is a test

So, IIUC I can talk to port 3310 with 127.0.0.1 or am I incorrect?


On 2/1/18 6:49 AM, Chris wrote:

First of all regarding my previous post - "Cannot connect to unix
socket '/var/lib/clamav/clamd.socket': connect: No such file or
directory" on Tuesday, I at least have that working. However, now
whenever an update is done to a database I'm seeing - ERROR:
NotifyClamd: Can't connect to clamd on 127.0.0.1:3310: Connection
refused. This is:


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ERROR: NotifyClamd: Can't connect to clamd on 127.0.0.1:3310: Connection refused

[Dennis Peterson]

Use the nc tool to connect to that port. If you get a connection then type PING. 
It should return PONG and disconnect. If that doesn't happen you have a config 
misunderstanding.


dp

On 2/1/18 6:49 AM, Chris wrote:

First of all regarding my previous post - "Cannot connect to unix
socket '/var/lib/clamav/clamd.socket': connect: No such file or
directory" on Tuesday, I at least have that working. However, now
whenever an update is done to a database I'm seeing - ERROR:
NotifyClamd: Can't connect to clamd on 127.0.0.1:3310: Connection
refused. This is:



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.99.3 has been released!

[Dennis Peterson]

On 1/26/18 2:39 PM, Scott Kitterman wrote:

Couldn't (old) 0.99.3 beta users just have ignored (new) 0.99.3? As far as I 
can tell, the beta had all the fixes.

Assuming that is correct, I think better advice for beta users would be to do 
nothing now and update to 0.100 beta when it is available.

Scott K
___


Many businesses correctly disallow production use of beta software. Because it 
is policy and not necessarily logical even beta software that is byte-identical 
with the golden release is discouraged and the reason is a version query could 
report beta and set off a flag. That is not a fun thing to experience in large 
data centers.


dp
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] mirrors, again

[Dennis Peterson]

While working the problems this morning I note that freshclam --list-mirrors 
shows 7 mirrors for db.us.clamav.net and 6 of them are being ignored. And that 
is after I removed mirrors.dat. In your spare time...


dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] High CPU load during startup/reload of sigs for a long time.

[Dennis Peterson]

If I were debugging this I'd want to know if all the vm's run on the same or 
different hosts, what the allocation of resources to each vm is, if different 
hosts then what each host's base loads are for cpu, memory, and disk caching. If 
you don't own the hosts this can be difficult. Then I'd compare the output of 
sysctl -a on each vm to see if something jumps out. Check sar reports, lsof, and 
other tools to check ram usage and disk iowaits, and how much free memory is 
available for caching. There's more to it, of course, but this provides a good 
foundation for comparison.


dp

On 12/28/17 5:03 AM, Thorsten Schöning wrote:

Hi all,

I have some problem with ClamAV for some months now and would like to
get some attention on a question I already asked on superuser.com[1]
and ask some additional ones to try to better understand the problem.

In the end, my problem breaks down to the fact that ClamAV startup or
reload because of new signatures takes different time and CPU load on
the same physical host, but in different VMs. The VMs are Ubuntu 14.04
and 16.04 LTS Servers and in only one of those I have the problem,
while the version of ClamAV is all the same 0.99.2 and all use the
same version b2f0b9ba2019d6293c0fefe142d7265592842157 of unofficial
sigs with the the same sigs.

In all but one VMs startup/reload is pretty fast and takes less than a
minute always, in the one exception it never takes less than a minute,
but instead 2-5 or in very bad cases it even takes 7-10 minutes.
Additionally, in those very bad cases an enormous load is created in
the VM with very high CPU load on all cores and everything is pretty
slow. Even a simple SSH connection and using "mc" in the terminal with
the cursor keys. In htop it looks like all actively running processes
accumulate, regardless how CPU intensive they really are "normally".
In those cases I have a lot of context switches in the physical host,
~500'000, far less in the VM, ~10'000, and practically no I/O in the
VM or host.

So here are my questions:

1. Does clamd scan memory during startup and/or restart?[1] The
problem seems to occur less with less committed memory in the VM.

2. If memory is scanned, which? Does that depend on the user ClamAV is
running or the users other services are running under? I couldn't
reproduce the problem with only e.g. cached file content or large
open logs as root.

3. Does ClamAV use more than one CPU core during startup/reload?
Because if my problem occurs, htop shows a load of more than 100%
for the ClamAV process, sometimes up to 500.

4. Is there any situation in which more CPU cores are known to lower
performance of startup/reload?

5. What should be most likely the bottleneck during startup/reload,
available time on one CPU core or I/O to read sigs? I don't seem to
have any reasonable I/O when the high CPU load occurs.

6. Are there any "benchmarks" available how long startup/reload takes
on other CPUs, so I could compare my times?

Thanks for your answers!

[1]: 
https://superuser.com/questions/1208220/does-clamd-scan-memory-during-startup-and-or-restart

Mit freundlichen Grüßen,

Thorsten Schöning



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] How to download and update main.cvd and daily.cvd manually AND update mirrors

[Dennis Peterson]

Did you make sure permissions are set so that the clam user can read them?


On 12/14/17 8:49 AM, George wrote:

Hi,

I mistakingly copied this twice in the email. But I did it as in your
reply. that's not the problem.

Thanks,
George

2017-12-14 18:39 GMT+02:00 Dennis Peterson :


you are downloading main.cvd twice. Change one of the wget commands to
download daily.cvd.

Example:

wget database.clamav.net/main.cvd
sudo cp main.cvd /var/lib/clamav
wget database.clamav.net/daily.cvd
sudo cp daily.cvd /var/lib/clamav


dp

On 12/14/17 8:28 AM, George wrote:


Dear All,

I am still getting the message that my database is more than 7 days old. I
successfully downloaded and updated main.cvd and daily.cvd manually, as
follows:

wget database.clamav.net/main.cvd
sudo cp main.cvd /var/lib/clamav
wget database.clamav.net/main.cvd
sudo cp daily.cvd /var/lib/clamav



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] How to download and update main.cvd and daily.cvd manually AND update mirrors

[Dennis Peterson]

you are downloading main.cvd twice. Change one of the wget commands to download 
daily.cvd.


Example:

wget database.clamav.net/main.cvd
sudo cp main.cvd /var/lib/clamav
wget database.clamav.net/daily.cvd
sudo cp daily.cvd /var/lib/clamav


dp

On 12/14/17 8:28 AM, George wrote:

Dear All,

I am still getting the message that my database is more than 7 days old. I
successfully downloaded and updated main.cvd and daily.cvd manually, as
follows:

wget database.clamav.net/main.cvd
sudo cp main.cvd /var/lib/clamav
wget database.clamav.net/main.cvd
sudo cp daily.cvd /var/lib/clamav



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Trouble getting cvd files from private local mirror

[Dennis Peterson]

Consider using tcpdump or the network sniffer of your choice on the server to 
see what the connection dialog is between your freshclam client and the httpd 
server. Or to learn if there is even a connection attempted.


dp

On 12/8/17 9:16 PM, John Kennedy wrote:

Were you to read my original email - I can download the file with curl and
wget (even supplied the output) so there is a successful connection to port
80 by other means, just NOT with freshclam. That is why I am having a
difficult time with this.


John Kennedy  (_8(|)

If I'm a sarcastic asshole when I talk to you it's either because I really
like you and feel comfortable teasing you or I really hate you and don't
care if you know it. Good luck figuring out which one...

Sometimes it happens, sometimes it doesn't - Pedro Catacora

The Dunning-Kruger effect occurs when incompetent people not only fail to
realize their incompetence, but consider themselves much more competent
than everyone else. Basically - they're too stupid to know that they're
stupid.

On Fri, Dec 8, 2017 at 9:21 PM, Reindl Harald 
wrote:



Am 08.12.2017 um 19:34 schrieb John Kennedy:


connect_error: getsockopt(SO_ERROR): fd=4 error=110: Connection timed out
Can't connect to port 80 of host clamav.trustx.com (IP: 10.10.10.10)
WARNING: Can't download main.cvd from clamav.trustx.com


and what is difficult to understand that on 10.10.10.10 port 80 does not
respond for whatever reason far oustide of freshclam and clamav at all?

why in the world don't you dig that much around until you can make sure
that a) the hostname resolves from the client and b) "telnet ip 80" results
in a succesful connection?

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Trouble getting cvd files from private local mirror

[Dennis Peterson]

The client is ignoring your servers because they are listen in mirrors.dat as 
broken. Remove the mirrors.dat file and try again.


You have not mentioned DNS or host tables but the natural assumption is all your 
clients and servers have the host tables or dns information needed to find each 
other and that router tables and net masks are not an issue.


dp

On 12/8/17 8:23 AM, John Kennedy wrote:

I have set up a private local mirror at clamav.trustx.com. Our environment
is AWS based with many VPC's. We have an "admin" VPC that is reachable from
all other VPCs.

I have tried both the second (Serve CVD files from a local web server - my
preferred method) and third (Serve CVD and CDIFF files from a local web
server) options from the Private Mirror FAQ with no luck. I can use both
curl and wget from the client machine to pull down the cvd files but when I
try and run freshclam I get the following error:
-

# freshclam -v
Current working dir is /var/lib/clamav
Max retries == 3
ClamAV update process started at Fri Dec  8 16:14:06 2017
Using IPv6 aware code
Querying current.cvd.clamav.net
TTL: 60
Software version from DNS: 0.99.2
main.cvd version from DNS: 58
Retrieving http://clamav.trustx.com/main-58.cdiff
Ignoring mirror 10.10.10.10 (due to previous errors)
Ignoring mirror 10.10.10.10 (due to previous errors)
WARNING: getpatch: Can't download main-58.cdiff from clamav.trustx.com
Retrieving http://clamav.trustx.com/main-58.cdiff
Ignoring mirror 10.10.10.10 (due to previous errors)
WARNING: getpatch: Can't download main-58.cdiff from clamav.trustx.com
Retrieving http://clamav.trustx.com/main-58.cdiff
Ignoring mirror 10.10.10.10 (due to previous errors)
WARNING: getpatch: Can't download main-58.cdiff from clamav.trustx.com
WARNING: Incremental update failed, trying to download main.cvd
Whitelisting short-term blacklisted mirrors
Retrieving http://clamav.trustx.com/main.cvd
connect_error: getsockopt(SO_ERROR): fd=4 error=110: Connection timed out
Can't connect to port 80 of host clamav.trustx.com (IP: 10.10.10.10)
Ignoring mirror 10.10.10.10 (due to previous errors)
WARNING: Can't download main.cvd from clamav.trustx.com
Trying again in 5 secs...
-

# curl -q http://clamav.trustx.com/main.cvd --output main.cvd
   % Total% Received % Xferd  Average Speed   TimeTime Time
Current
  Dload  Upload   Total   SpentLeft
Speed
100  112M  100  112M0 0  8856k  0  0:00:13  0:00:13 --:--:--
11.5M
-

>From the web server:
-

# cat 100-clamav.conf
server {
   listen  80;
   server_name  clamav.trustx.com;
   sendfile on;

   add_header Strict-Transport-Security "max-age=31536000;
includeSubDomains";


   root  /var/data/clamav;

   location /simple {
 allow 10.0.0.0/8;
 allow 77.75.100.144/28;
 deny all;
 autoindex on;
   }

   access_log  /var/log/nginx/clamav_access.log;
   error_log   /var/log/nginx/clamav_error.log;
} # End server clamav.trustx.com
-

Both clamav_access.log and clamav_error.log are empty
-

# pwd
/var/data/clamav
[root@DevOps clamav]# ls -l
total 208800
-rw-r--r-- 1 nginx root   770 Dec  7 02:17 bytecode-319.cdiff
-rw-r--r-- 1 nginx root153228 Dec  7 02:17 bytecode.cvd
-rw-r--r-- 1 nginx root  6437 Nov 28 01:09 daily-24080.cdiff
-rw-r--r-- 1 nginx root  7802 Nov 28 09:07 daily-24081.cdiff
-rw-r--r-- 1 nginx root  9705 Nov 28 17:09 daily-24082.cdiff
-rw-r--r-- 1 nginx root 10406 Nov 29 01:08 daily-24083.cdiff
-rw-r--r-- 1 nginx root  7508 Nov 29 09:03 daily-24084.cdiff
-rw-r--r-- 1 nginx root  6990 Nov 29 17:08 daily-24085.cdiff
-rw-r--r-- 1 nginx root 12340 Nov 30 01:10 daily-24086.cdiff
-rw-r--r-- 1 nginx root  7461 Nov 30 09:09 daily-24087.cdiff
-rw-r--r-- 1 nginx root  6331 Nov 30 17:10 daily-24088.cdiff
-rw-r--r-- 1 nginx root  8811 Dec  1 01:12 daily-24089.cdiff
-rw-r--r-- 1 nginx root  9504 Dec  1 09:11 daily-24090.cdiff
-rw-r--r-- 1 nginx root  6476 Dec  1 17:09 daily-24091.cdiff
-rw-r--r-- 1 nginx root  8647 Dec  2 01:09 daily-24092.cdiff
-rw-r--r-- 1 nginx root  6714 Dec  2 09:12 daily-24093.cdiff
-rw-r--r-- 1 nginx root  4034 Dec  2 17:08 daily-24094.cdiff
-rw-r--r-- 1 nginx root  3766 Dec  3 01:11 daily-24095.cdiff
-rw-r--r-- 1 nginx root  3609 Dec  3 09:10 daily-24096.cdiff
-rw-r--r-- 1 nginx root  5718 Dec  3 17:09 daily-24097.cdiff
-rw-r--r-- 1 nginx root  4577 Dec  4 01:10 daily-24098.cdiff
-rw-r--r-- 1 nginx root  3616 Dec  4 09:09 daily-24099.cdiff
-rw-r--r-- 1 nginx root  6595 Dec  4 17:12 daily-24100.cdiff
-rw-r--r-- 1 nginx root 10800 Dec  5 01:12 daily-24101.cdiff
-rw-r--r-- 1 nginx root  9302 Dec  5 09:11 daily-24102.cdiff
-rw-r--r-- 1 nginx root 11367 Dec  5 17:12 daily-24103.cdiff
-rw-r--r-- 1 nginx root 40675 Dec  6 01:15 daily-24104.cdiff
-rw-r--r-- 1 nginx root 30876 Dec  6 09:10 daily-24105.cdiff
-rw-r--r-- 1 nginx root  9570 Dec  6 17:13 daily-24106.cdiff
-rw-r--r-

Re: [clamav-users] Local Mirror error "Can't download daily.cvd"

[Dennis Peterson]

Do you have a host table entry for clamav.clamavsrv.tk ?

On 12/7/17 3:27 AM, Emanuel wrote:

Hello,

Here the config:

# client server

DatabaseDirectory /var/lib/clamav

snip
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam Fails

[Dennis Peterson]

I'm wondering why it is trying to dl main-58.cdiff.

dp

On 11/9/17 9:32 PM, Krishnakumar Nair wrote:

Is there any possible cause from clamav end ?? it was working fine.

Thanks & Regards,
kk



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] FreshClam - DNS issues since October 31st

[Dennis Peterson]

I've never had a successful download from that ip.

dp

On 11/9/17 11:36 PM, Al Varnell wrote:

As you probably already know, in past discussions of the US round robin it was 
revealed that there weren't enough US mirrors to support the demand and that 
was the primary reason for including low demand off-shore servers as 
supplements to handle the over-flow. I don't know whether that situation still 
exists now that Cisco has assumed responsibility for the network.

Certainly true that Singapore is a long way from Kansas and a quick traceroute 
revealed a lot of bouncing around ending in considerable latency:




___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] FreshClam - DNS issues since October 31st

[Dennis Peterson]

Any chance you can remove 128.199.133.36  from the US round robin? It's a long 
way from Kansas.


dp


On 11/8/17 7:50 AM, Joel Esler (jesler) wrote:

The team working on these issues is seeing these emails, so it’s good that you 
are writing in, if you are still experiencing issues.



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] fail updates

[Dennis Peterson]

Come to think of it, 130.59.10.36 shouldn't even still be in mirrors.dat and 
that is part of the systemic problems in the system. Nothing cleans up stale 
entries in mirrors.dat except rm -f mirrors.dat.


dp

On 11/6/17 9:02 AM, Benny Pedersen wrote:

freshclam --list-mirrors

Mirror #1
IP: 130.59.10.36
Successes: 391
Failures: 97
Last access: Mon Dec 19 00:46:43 2016
Ignore: No
- 


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] fail updates

[Dennis Peterson]

Your report includes mirrors that should be ignored based on last access. I 
built a list of current mirrors from freshclam logs that go back only to August.


grep -h Ignoring freshclam* |grep -v Reading |awk '{print $9}' |sort |uniq -c 
|sort -rn


The result is an easy to understand (if not jaw dropping) summary - number of 
times seen and the IP:

    387 128.199.133.36
    372 104.131.196.175
    292 12.167.151.1
    288 74.115.25.14
    282 204.130.133.50
    282 194.8.197.22
    268 155.98.64.87
    245 69.12.162.28
    233 72.21.91.8
    220 198.148.78.4

Even if these mirrors are healthy there is still a serious underlying systemic 
problem.


dp

On 11/6/17 9:02 AM, Benny Pedersen wrote:

freshclam --list-mirrors

Mirror #1
IP: 130.59.10.36
Successes: 391
Failures: 97
Last access: Mon Dec 19 00:46:43 2016
Ignore: No
-
Mirror #2
IP: 193.1.193.64
Successes: 2122
Failures: 208
Last access: Mon Nov  6 16:44:43 2017
Ignore: Yes
-
Mirror #3
IP: 81.91.100.173
Successes: 2079
Failures: 101
Last access: Sat Nov  4 01:06:08 2017
Ignore: Yes
-
Mirror #4
IP: 129.67.1.218
Successes: 2374
Failures: 59
Last access: Sat Nov  4 00:03:02 2017
Ignore: Yes
-
Mirror #5
IP: 172.110.204.67
Successes: 160
Failures: 364
Last access: Tue May  9 14:47:24 2017
Ignore: No
-
Mirror #6
IP: 130.59.113.36
Successes: 393
Failures: 0
Last access: Thu Feb 16 21:45:53 2017
Ignore: No
-
Mirror #7
IP: 178.79.177.182
Successes: 302
Failures: 112
Last access: Sun Nov  5 05:04:18 2017
Ignore: Yes
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] update mirror trouble?

[Dennis Peterson]

There are still a lot of broken mirrors out there aside from this problem.

dp

On 11/6/17 8:05 AM, Joel Esler (jesler) wrote:

This should be resolving itself as we speak.

--
Joel Esler | Talos: Manager | jes...@cisco.com








___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Mirror issues and what we are doing to fix it

[Dennis Peterson]

Awesome, Joel. Everything is greatly appreciated.

dp

On 8/30/17 3:28 PM, Joel Esler (jesler) wrote:

Dennis,

The team has been cleaning this up almost all day.  Expect the work to continue 
for awhile.

--
Joel Esler | Talos: Manager | jes...@cisco.com<mailto:jes...@cisco.com>






On Aug 30, 2017, at 1:11 PM, Dennis Peterson 
mailto:denni...@inetnw.com>> wrote:

I had the same thing happen and I also got successful dl's of the daily.cld 
file multiple times and I'm sure it would have continued looping forever if I'd 
not stopped it after observing it was stuck in a loop. Same symptoms on two 
separate systems. Couldn't find the cdiff file and the corresponding daily.cld 
file was not available.

Several times my client would start a daily.cld only to have the connection 
terminated by the server. The data speed was very low - after 10 minutes a 
daily.cld file would be at 40%. Because the daily.cld file was not current the 
attempt to dl a diff file would begin immediately.

I think it unwise to have mirrors in Germany and Spain included in the 
db.us.clamav.net<http://db.us.clamav.net> RR, and the cdiff files should not be 
available until the corresponding daily.cld file is already available else these 
loops will happen.

dp

On 8/30/17 6:15 AM, Gene Heskett wrote:
On Wednesday 30 August 2017 08:48:42 Joel Esler (jesler) wrote:

Gene,

Thanks.  I’ll give this to the ops team.
I had a total failure at 18:00 EDT last night:
=
Tue Aug 29 18:02:04 2017 -> Received signal: wake up
Tue Aug 29 18:02:04 2017 -> ClamAV update process started at Tue Aug 29 
18:02:04 2017
Tue Aug 29 18:02:04 2017 -> main.cld is up to date (version: 58, sigs: 4566249, 
f-level: 60, builder: sigmgr)
Tue Aug 29 18:02:35 2017 -> nonblock_recv: recv timing out (30 secs)
Tue Aug 29 18:02:35 2017 -> WARNING: getfile: Error while reading database from 
db.us.clamav.net<http://db.us.clamav.net> (IP: 150.214.142.197): Operation
now in progress
Tue Aug 29 18:02:35 2017 -> WARNING: getpatch: Can't download daily-23735.cdiff from 
db.us.clamav.net<http://db.us.clamav.net>
Tue Aug 29 18:03:08 2017 -> Downloading daily-23735.cdiff [100%]
Tue Aug 29 18:03:39 2017 -> nonblock_recv: recv timing out (30 secs)
Tue Aug 29 18:03:39 2017 -> WARNING: getfile: Error while reading database from 
db.us.clamav.net<http://db.us.clamav.net> (IP: 200.236.31.1): Operation now
in progress
snippage
___
clamav-users mailing list
clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Mirror issues and what we are doing to fix it

[Dennis Peterson]

I had the same thing happen and I also got successful dl's of the daily.cld file 
multiple times and I'm sure it would have continued looping forever if I'd not 
stopped it after observing it was stuck in a loop. Same symptoms on two separate 
systems. Couldn't find the cdiff file and the corresponding daily.cld file was 
not available.


Several times my client would start a daily.cld only to have the connection 
terminated by the server. The data speed was very low - after 10 minutes a 
daily.cld file would be at 40%. Because the daily.cld file was not current the 
attempt to dl a diff file would begin immediately.


I think it unwise to have mirrors in Germany and Spain included in the 
db.us.clamav.net RR, and the cdiff files should not be available until the 
corresponding daily.cld file is already available else these loops will happen.


dp

On 8/30/17 6:15 AM, Gene Heskett wrote:

On Wednesday 30 August 2017 08:48:42 Joel Esler (jesler) wrote:


Gene,

Thanks.  I’ll give this to the ops team.

I had a total failure at 18:00 EDT last night:
=
Tue Aug 29 18:02:04 2017 -> Received signal: wake up
Tue Aug 29 18:02:04 2017 -> ClamAV update process started at Tue Aug 29 
18:02:04 2017
Tue Aug 29 18:02:04 2017 -> main.cld is up to date (version: 58, sigs: 4566249, 
f-level: 60, builder: sigmgr)
Tue Aug 29 18:02:35 2017 -> nonblock_recv: recv timing out (30 secs)
Tue Aug 29 18:02:35 2017 -> WARNING: getfile: Error while reading database from 
db.us.clamav.net (IP: 150.214.142.197): Operation
now in progress
Tue Aug 29 18:02:35 2017 -> WARNING: getpatch: Can't download daily-23735.cdiff 
from db.us.clamav.net
Tue Aug 29 18:03:08 2017 -> Downloading daily-23735.cdiff [100%]
Tue Aug 29 18:03:39 2017 -> nonblock_recv: recv timing out (30 secs)
Tue Aug 29 18:03:39 2017 -> WARNING: getfile: Error while reading database from 
db.us.clamav.net (IP: 200.236.31.1): Operation now
in progress

snippage
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam failure - Still ongoing???

[Dennis Peterson]

It will fall through to db.local.clamav.net.

dp

On 8/27/17 1:07 AM, Andreas Schulze wrote:

Am 25.08.2017 um 22:44 schrieb Joel Esler (jesler):
We are working on ways to not only fix the on going mirror issues, but 
prevent them in the future, as well as bring back the Mirror page on 
ClamAV.net at some point soon.

Joel,

expect a mirror monitoring would make visible that many mirrors are no longer 
up to date.

I could imagine, some db.XY.clamav.net don't have current mirrors at all.

Andreas
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam failure - Still ongoing???

[Dennis Peterson]

On 8/26/17 10:49 AM, Dennis Peterson wrote:
I grabbed a tld file to use to locate (best effort) all ClamAV mirrors using a 
couple patterns I've discovered. Surely there is a better way but I'm old and 
time is precious.


db.TLD.clamav.net
db.TLD.rr.clamav.net 


Snippage happened.


I should add that just because a site is pingable is no guarantee that it will 
respond to freshclam requests, or have the current version of the signature 
files. It takes time to distribute these files to the mirrors.


dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam failure - Still ongoing???

[Dennis Peterson]

I grabbed a tld file to use to locate (best effort) all ClamAV mirrors using a 
couple patterns I've discovered. Surely there is a better way but I'm old and 
time is precious.


db.TLD.clamav.net
db.TLD.rr.clamav.net

I used the host command to find every mirror available to this method. That 
returned over 500 hosts but with many IP duplicates. After dedup'ing the list I 
got 144 uniq servers. I pinged that list and got 33 pingable servers world wide. 
I ran that list through geoiplookup and got this list of healthy pingable 
servers sorted by world region.


Many of the unreachable IP's are the result of firewall decisions and don't 
necessarily reflect a failed system.


My intention now is to create a list of fairly local systems and create a DNS RR 
table for my own use. I will track performance using the mirrors.dat file 
(freshclam --list-mirrors). I suggest before anyone removed this file that you 
gather a report using freshclam so that you can monitor your own site 
interactions as the new mirrors.dat file collects data. A little forensics goes 
a long way.


The following list are the only mirrors world wide that I could find and which 
also responded to a ping. It isn't much of a list.


212.71.0.66
GeoIP Country Edition: BE, Belgium
--
46.29.125.16
GeoIP Country Edition: FR, France
--
212.180.1.29
GeoIP Country Edition: FR, France
--
130.133.110.67
GeoIP Country Edition: DE, Germany
--
144.76.28.11
GeoIP Country Edition: DE, Germany
--
212.227.138.145
GeoIP Country Edition: DE, Germany
--
213.133.110.235
GeoIP Country Edition: DE, Germany
--
194.8.197.22
GeoIP Country Edition: DE, Germany
--
195.30.97.3
GeoIP Country Edition: DE, Germany
--
62.27.56.14
GeoIP Country Edition: DE, Germany
--
62.245.181.53
GeoIP Country Edition: DE, Germany
--
5.9.253.237
GeoIP Country Edition: DE, Germany
--
218.189.210.14
GeoIP Country Edition: HK, Hong Kong
--
180.92.182.5
GeoIP Country Edition: HK, Hong Kong
--
90.147.160.69
GeoIP Country Edition: IT, Italy
--
85.254.217.235
GeoIP Country Edition: LV, Latvia
--
196.192.32.38
GeoIP Country Edition: MG, Madagascar
--
128.199.133.36
GeoIP Country Edition: SG, Singapore
--
92.240.244.203
GeoIP Country Edition: SK, Slovakia
--
158.197.16.70
GeoIP Country Edition: SK, Slovakia
--
84.255.209.87
GeoIP Country Edition: SI, Slovenia
--
155.232.191.239
GeoIP Country Edition: ZA, South Africa
--
196.4.160.79
GeoIP Country Edition: ZA, South Africa
--
82.159.137.16
GeoIP Country Edition: ES, Spain
--
82.195.224.39
GeoIP Country Edition: CH, Switzerland
--
130.59.113.36
GeoIP Country Edition: CH, Switzerland
--
178.79.177.182
GeoIP Country Edition: GB, United Kingdom
--
81.91.100.173
GeoIP Country Edition: GB, United Kingdom
--
129.67.1.218
GeoIP Country Edition: GB, United Kingdom
--
204.130.133.50
GeoIP Country Edition: US, United States
--
104.131.196.175
GeoIP Country Edition: US, United States
--
69.12.162.28
GeoIP Country Edition: US, United States
--
69.163.100.14
GeoIP Country Edition: US, United States
--

dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam failure - Still ongoing???

[Dennis Peterson]

Are you not able to create your own mirror?

dp

On 8/25/17 8:48 AM, Paul Dean wrote:

Hi,

Thanks for the info, but doing this across several groups of servers is not 
really suitable. But that said, if there is no viable fix in the next day or 
so, then this might be the only solution.

Does anyone have a list of confirmed working mirrors?

Thanks Joel for getting onto this, let me know if I can help somehow.

--


Thanks

Paul Dean.

"Life is not WHAT you make it, it's WHO you have in it..."


On Fri, 25 Aug 2017 07:43:08 -0700
Dennis Peterson  wrote:


You don't need ClamAV ppl to help - you have complete control over this 
process. Try this:

Find a healthy mirror
Put that healthy mirror's IP address in your freshclam.conf file as the first 
definition of DatabaseMirror
Run freshclam manually.

grep ^DatabaseMirror freshclam.conf

You should see a host name such as db.de.clamav.net. Use the host or nslookup 
command to expand that to reveal all the round-robin IP addresses. It looks 
like this (as of this moment):

host db.de.clamav.net

db.de.clamav.net has address 195.30.97.3
db.de.clamav.net has address 212.227.138.145
db.de.clamav.net has address 213.174.32.130
db.de.clamav.net has address 5.9.253.237
db.de.clamav.net has address 62.27.56.14
db.de.clamav.net has address 62.201.161.84
db.de.clamav.net has address 62.245.181.53
db.de.clamav.net has address 84.39.110.99
db.de.clamav.net has address 88.198.17.100
db.de.clamav.net has address 130.133.110.67
db.de.clamav.net has address 144.76.28.11
db.de.clamav.net has address 178.63.73.246
db.de.clamav.net has address 193.27.49.165

Ping each of these hosts to see which ones respond. Use geoiplookup or similar 
tool to find which healthy sites are located near you. No point trying to poll 
a site around the world from you. If you don't have a geoip tool then notice 
the response time of each mirror's successful ping. The lower the time the 
better suited it could be for you.

Choose an IP that looks promising. Put that in your freshclam.conf file and run 
freshclam. For example:

DatabaseMirror 195.30.97.3

Remember this is a temporary fix - don't depend on this configuration to work 
in the long run. Undo it when the ClamAV ppl fix the mirrors problem.

As an aside try some diagnostics - run freshclam manually:
freshclam --list-mirrors

This will give you a list of mirrors and their health. On the report I see some 
of the mirrors have never responded, others respond about 50% of the time. If 
you see a mirror that is 100% successful you can use that in your 
freshclam.conf file. It would be helpful if the ClamAV ppl would remove the bad 
actors from the DNS records.

If none of this makes sense then do nothing and wait for the ClamAV ppl. Better 
safe than sorry.

dp




On 8/25/17 2:14 AM, Paul Dean wrote:

Oh shoot ClamAV ppl, help please...

--


Thanks

Paul Dean.

"Life is not WHAT you make it, it's WHO you have in it..."


On Fri, 25 Aug 2017 10:47:23 +0200
maxal  wrote:
  

hi,

yes, this is ongoing as there are numerous broken mirrors in different
country zones out there, eg german zone db.de.clamav.net:

db.de.clamav.net has address 62.201.161.84 -> OK
db.de.clamav.net has address 195.30.97.3 -> OK
db.de.clamav.net has address 130.133.110.67 -> OK
db.de.clamav.net has address 212.227.138.145 -> OK
db.de.clamav.net has
address 62.27.56.14 -> OK
db.de.clamav.net has address 62.245.181.53 ->
OK
db.de.clamav.net has address 193.27.49.165 -> OK
db.de.clamav.net has address 88.198.17.100 -> FAIL
db.de.clamav.net has address 84.39.110.99 -> FAIL
db.de.clamav.net has address 144.76.28.11 -> FAIL
db.de.clamav.net has address 213.174.32.130 -> FAIL
db.de.clamav.net has address 5.9.253.237 -> FAIL
db.de.clamav.net has address 178.63.73.246 -> FAIL

regards
max


On Fri, 2017-08-25 at 16:24 +0800, Paul Dean wrote:

Hi,

I've checked the lists and nuked the mirror.dat file as suggested,
but still getting failure on dling daily-23699.cdiff via freshclam.
Also tried via wget, and got a 404 error. So currently I'm stuck on
23698.

Also nuked all .cld files and still failed.

I've got a few servers/machines that use ClamAV, so hoping a overall
fix instead of each machine would be preferable.

All machines are based in AU and failures happen with
db.local.clamav.net and database.clamav.net.

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.

Re: [clamav-users] Freshclam failure - Still ongoing???

[Dennis Peterson]

This is abysmal.

# freshclam --list-mirrors |grep Success |sort -n -k2
Successes: 0
Successes: 0
Successes: 0
Successes: 0
Successes: 0
Successes: 0
Successes: 0
Successes: 4
Successes: 7
Successes: 8
Successes: 11
Successes: 11
Successes: 19
Successes: 46
Successes: 79
Successes: 81
Successes: 85
Successes: 90
Successes: 176
Successes: 178
Successes: 188
Successes: 215

# freshclam --list-mirrors |grep Fail |sort -n -k2
Failures: 0
Failures: 0
Failures: 2
Failures: 4
Failures: 12
Failures: 19
Failures: 21
Failures: 23
Failures: 55
Failures: 90
Failures: 102
Failures: 109
Failures: 110
Failures: 148
Failures: 148
Failures: 160
Failures: 163
Failures: 183
Failures: 274
Failures: 274
Failures: 275
Failures: 275

# freshclam --list-mirrors |grep -B2 Fail
IP: 208.72.56.53
Successes: 0
Failures: 275
--
IP: 200.236.31.1
Successes: 81
Failures: 160
--
IP: 64.6.100.177
Successes: 0
Failures: 274
--
...

dp

On 8/25/17 4:00 AM, Joel Esler (jesler) wrote:

On it

Sent from my iPhone



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Freshclam failure - Still ongoing???

[Dennis Peterson]

You don't need ClamAV ppl to help - you have complete control over this process. 
Try this:


Find a healthy mirror
Put that healthy mirror's IP address in your freshclam.conf file as the first 
definition of DatabaseMirror

Run freshclam manually.

grep ^DatabaseMirror freshclam.conf

You should see a host name such as db.de.clamav.net. Use the host or nslookup 
command to expand that to reveal all the round-robin IP addresses. It looks like 
this (as of this moment):


host db.de.clamav.net

db.de.clamav.net has address 195.30.97.3
db.de.clamav.net has address 212.227.138.145
db.de.clamav.net has address 213.174.32.130
db.de.clamav.net has address 5.9.253.237
db.de.clamav.net has address 62.27.56.14
db.de.clamav.net has address 62.201.161.84
db.de.clamav.net has address 62.245.181.53
db.de.clamav.net has address 84.39.110.99
db.de.clamav.net has address 88.198.17.100
db.de.clamav.net has address 130.133.110.67
db.de.clamav.net has address 144.76.28.11
db.de.clamav.net has address 178.63.73.246
db.de.clamav.net has address 193.27.49.165

Ping each of these hosts to see which ones respond. Use geoiplookup or similar 
tool to find which healthy sites are located near you. No point trying to poll a 
site around the world from you. If you don't have a geoip tool then notice the 
response time of each mirror's successful ping. The lower the time the better 
suited it could be for you.


Choose an IP that looks promising. Put that in your freshclam.conf file and run 
freshclam. For example:


DatabaseMirror 195.30.97.3

Remember this is a temporary fix - don't depend on this configuration to work in 
the long run. Undo it when the ClamAV ppl fix the mirrors problem.


As an aside try some diagnostics - run freshclam manually:
freshclam --list-mirrors

This will give you a list of mirrors and their health. On the report I see some 
of the mirrors have never responded, others respond about 50% of the time. If 
you see a mirror that is 100% successful you can use that in your freshclam.conf 
file. It would be helpful if the ClamAV ppl would remove the bad actors from the 
DNS records.


If none of this makes sense then do nothing and wait for the ClamAV ppl. Better 
safe than sorry.


dp




On 8/25/17 2:14 AM, Paul Dean wrote:

Oh shoot ClamAV ppl, help please...

--


Thanks

Paul Dean.

"Life is not WHAT you make it, it's WHO you have in it..."


On Fri, 25 Aug 2017 10:47:23 +0200
maxal  wrote:


hi,

yes, this is ongoing as there are numerous broken mirrors in different
country zones out there, eg german zone db.de.clamav.net:

db.de.clamav.net has address 62.201.161.84 -> OK
db.de.clamav.net has address 195.30.97.3 -> OK
db.de.clamav.net has address 130.133.110.67 -> OK
db.de.clamav.net has address 212.227.138.145 -> OK
db.de.clamav.net has
address 62.27.56.14 -> OK
db.de.clamav.net has address 62.245.181.53 ->
OK
db.de.clamav.net has address 193.27.49.165 -> OK
db.de.clamav.net has address 88.198.17.100 -> FAIL
db.de.clamav.net has address 84.39.110.99 -> FAIL
db.de.clamav.net has address 144.76.28.11 -> FAIL
db.de.clamav.net has address 213.174.32.130 -> FAIL
db.de.clamav.net has address 5.9.253.237 -> FAIL
db.de.clamav.net has address 178.63.73.246 -> FAIL

regards
max


On Fri, 2017-08-25 at 16:24 +0800, Paul Dean wrote:

Hi,

I've checked the lists and nuked the mirror.dat file as suggested,
but still getting failure on dling daily-23699.cdiff via freshclam.
Also tried via wget, and got a 404 error. So currently I'm stuck on
23698.

Also nuked all .cld files and still failed.

I've got a few servers/machines that use ClamAV, so hoping a overall
fix instead of each machine would be preferable.

All machines are based in AU and failures happen with
db.local.clamav.net and database.clamav.net.

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml




___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Unable to download database

[Dennis Peterson]

After testing several of the DNS round robin aliases I found the 
db.ca.clamav.net had the most reliable server set for North America. After 
editing the freshclam.conf file the files updated on the next cron.hourly cycle.


I also found that the number of viable mirror sites is a small portion of the 
total number of mirrors. I also found that a lot of "local" mirrors are not all 
that local.


I think I'll run a health check of every mirror in the western hemisphere and 
use the results in a local DNS round robin running my own servers. It is a form 
of dynamic load balancing using real-time network response time. If nothing else 
it will stop most if not all attempts to missing mirrors which seem to be the 
majority. Obviously it will also ignore mirrors that disallow icmp traffic.


dp

On 8/23/17 9:48 AM, Dennis Peterson wrote:

nslookup db.local.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.us.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.ca.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.ru.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.uk.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1


Nobody home.

dp

On 8/23/17 12:26 AM, lukn555 wrote:

Good Day ClamAV List

Since yesterday at around noon CET I've been having issues downloading
the ClamAV database:

freshclam --version
ClamAV 0.99.2/23696/Tue Aug 22 14:36:14 2017


# /usr/local/bin/freshclam --verbose
Current working dir is /usr/local/share/clamav
Max retries == 3
ClamAV update process started at Wed Aug 23 09:11:52 2017
Using IPv6 aware code
Querying current.cvd.clamav.net
TTL: 609
Software version from DNS: 0.99.2
main.cvd version from DNS: 58
main.cld is up to date (version: 58, sigs: 4566249, f-level: 60,
builder: sigmgr)
daily.cvd version from DNS: 23700
Retrieving http://database.clamav.net/daily-23697.cdiff
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
WARNING: getpatch: Can't download daily-23697.cdiff from database.clamav.net
Retrieving http://database.clamav.net/daily-23697.cdiff
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
WARNING: getpatch: Can't download daily-23697.cdiff from database.clamav.net
Retrieving http://database.clamav.net/daily-23697.cdiff
Ignoring mirror 193.230.240.8 (due to previous errors)
Ignoring mirror 130.59.113.36 (due to previous errors)
WARNING: getpatch: Can't download daily-23697.cdiff from database.clamav.net
WARNING: Incremental update failed, trying to download daily.cvd
Whitelisting short-term blacklisted mirrors
Retrieving http://database.clamav.net/daily.cvd
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
WARNING: Can't download daily.cvd from database.clamav.net
Trying again in 5 secs...


# dig database.clamav.net +short
db.local.clamav.net.
db.centraleu.clamav.net.
130.59.113.36
193.230.240.8


# wget http://database.clamav.net/daily-23697.cdiff
--2017-08-23 09:14:16-- http://database.clamav.net/daily-23697.cdiff
Resolving database.clamav.net (database.clamav.net)... 193.230.240.8,
130.59.113.36
Connecting to database.clamav.net
(database.clamav.net)|193.230.240.8|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2017-08-23 09:14:16 ERROR 403: Forbidden.


Is this an issue on my side or on ClamAV mirror side?
Any help is appreciated.
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Unable to download database

[Dennis Peterson]

nslookup db.local.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.us.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.ca.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.ru.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1

nslookup db.uk.clamav.net |awk '/Address:/ {print $2}' |xargs -L1 ping -c 1


Nobody home.

dp

On 8/23/17 12:26 AM, lukn555 wrote:

Good Day ClamAV List

Since yesterday at around noon CET I've been having issues downloading
the ClamAV database:

freshclam --version
ClamAV 0.99.2/23696/Tue Aug 22 14:36:14 2017


# /usr/local/bin/freshclam --verbose
Current working dir is /usr/local/share/clamav
Max retries == 3
ClamAV update process started at Wed Aug 23 09:11:52 2017
Using IPv6 aware code
Querying current.cvd.clamav.net
TTL: 609
Software version from DNS: 0.99.2
main.cvd version from DNS: 58
main.cld is up to date (version: 58, sigs: 4566249, f-level: 60,
builder: sigmgr)
daily.cvd version from DNS: 23700
Retrieving http://database.clamav.net/daily-23697.cdiff
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
WARNING: getpatch: Can't download daily-23697.cdiff from database.clamav.net
Retrieving http://database.clamav.net/daily-23697.cdiff
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
WARNING: getpatch: Can't download daily-23697.cdiff from database.clamav.net
Retrieving http://database.clamav.net/daily-23697.cdiff
Ignoring mirror 193.230.240.8 (due to previous errors)
Ignoring mirror 130.59.113.36 (due to previous errors)
WARNING: getpatch: Can't download daily-23697.cdiff from database.clamav.net
WARNING: Incremental update failed, trying to download daily.cvd
Whitelisting short-term blacklisted mirrors
Retrieving http://database.clamav.net/daily.cvd
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
Ignoring mirror 130.59.113.36 (due to previous errors)
Ignoring mirror 193.230.240.8 (due to previous errors)
WARNING: Can't download daily.cvd from database.clamav.net
Trying again in 5 secs...


# dig database.clamav.net +short
db.local.clamav.net.
db.centraleu.clamav.net.
130.59.113.36
193.230.240.8


# wget http://database.clamav.net/daily-23697.cdiff
--2017-08-23 09:14:16--  http://database.clamav.net/daily-23697.cdiff
Resolving database.clamav.net (database.clamav.net)... 193.230.240.8,
130.59.113.36
Connecting to database.clamav.net
(database.clamav.net)|193.230.240.8|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2017-08-23 09:14:16 ERROR 403: Forbidden.


Is this an issue on my side or on ClamAV mirror side?
Any help is appreciated.
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Main CVD and Main Cdiff have been published

[Dennis Peterson]

The main.cld is equivalent to main.cvd and the date is correct. The difference 
is one is compressed, the other not.


dp

On 6/8/17 9:30 PM, mlnl wrote:

Hi,


should this be correct?

-rw-r--r--.  1 clam clam654336 Jun  7 03:18 bytecode.cld
-rw-r--r--.  1 clam clam 123921920 Jun  9 03:26 daily.cld
-rw-r--r--.  1 clam clam 307499008 Jun  8 03:18 main.cld

ls -la *.cld
-rw-r--r-- 1 clamav clamav654336 Jun  7 06:06 bytecode.cld
-rw-r--r-- 1 clamav clamav 123921920 Jun  9 05:46 daily.cld
-rw-r--r-- 1 clamav clamav 307499008 Jun  8 07:16 main.cld
-rw-r--r-- 1 clamav clamav 120095744 Jun  9 05:47 safebrowsing.cld

under Debian Jessie.
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] clamav-users Digest, Vol 150, Issue 19

[Dennis Peterson]

If I were to have gotten a suspicious message notice from 
epl.paypal-communication.com and gone through a whois, nslookup, whois (ip 
address), dig txt paypal-communication.com, dig mx paypal-communication.com, dig 
mx epl.paypal-communication.com routine I would have found a very suspicious 
pedigree and I would add the IP and domain name to my blacklist. And that is 
exactly what I did. Businesses that send email that is indistinguishable from 
spam/phishing/obfuscation/cloaking/tracking don't deserve space in my systems. 
And because I'll not remember long that I did all this forensic investigation 
and was dissatisfied with the results, I go with the least-effort option of 
blocking. It is your problem to fix. Be obvious or be blocked. There's too much 
at risk.


And including a link to a one-pixel (spacer1.gif) image, obviously a tracking 
beacon, in already suspect messages always looks more suspicious yet.


dp

On 6/1/17 1:19 AM, outre...@epsilon.com wrote:

Hi Reindl and Al,

Thank you for your feedback.

The domain https://epl.paypal-communication.com is used by Paypal for link 
tracking purposes in their emails. Their sending domains are for example: 
mail.paypal.com, mail.paypal.co.uk, mail.paypal.fr etc.

To clarify, I work for Epsilon which is a major Email Service Provider 
(www.epsilon.com) and Paypal use our platform to deploy their emails, hence me 
contacting you about this delivery issue.

I will pass back your feedback to Paypal so they can make a decision on whether 
or not they will want to make any changes to their emails moving forward.

Best regards,


Anne-Sophie Marsh, Sr Email Deliverability Manager EMEA
  T   +44 2086143219   M +44 7469352383   Epsilon, 67 Broad Street, Teddington 
TW11 8QZ, UK  epsilon.com



-Original Message-
From: clamav-users [mailto:clamav-users-boun...@lists.clamav.net] On Behalf Of 
Reindl Harald
Sent: 01 June 2017 07:24
To: clamav-users@lists.clamav.net
Subject: Re: [clamav-users] clamav-users Digest, Vol 150, Issue 19



Am 01.06.2017 um 03:04 schrieb Al Varnell:

I made an attempt to determine whether epl.paypal-communication.com was a 
legitimate domain owned by PayPal with very mixed results.

No WhoIs service could identify it directly

and here is stop to read - let me guess you entered 
"epl.paypal-communication.com" including the subdomain and/or used some obsucre 
website doing whois requests


[harry@srv-rhsoft:~]$ whois paypal-communication.com

Whois Server Version 2.0

Domain names in the .com and .net domains can now be registered with many 
different competing registrars. Go to http://www.internic.net for detailed 
information.

 Domain Name: PAYPAL-COMMUNICATION.COM
 Registrar: MARKMONITOR INC.
 Sponsoring Registrar IANA ID: 292
 Whois Server: whois.markmonitor.com
 Referral URL: http://www.markmonitor.com
 Name Server: NS1.P57.DYNECT.NET
 Name Server: NS2.P57.DYNECT.NET
 Name Server: PDNS100.ULTRADNS.COM
 Name Server: PDNS100.ULTRADNS.NET
 Status: clientDeleteProhibited
https://icann.org/epp#clientDeleteProhibited
 Status: clientTransferProhibited
https://icann.org/epp#clientTransferProhibited
 Status: clientUpdateProhibited
https://icann.org/epp#clientUpdateProhibited
 Updated Date: 05-mar-2017
 Creation Date: 06-apr-2011
 Expiration Date: 06-apr-2018

  >>> Last update of whois database: Thu, 01 Jun 2017 06:20:04 GMT <<<

For more information on Whois status codes, please visit https://icann.org/epp

NOTICE: The expiration date displayed in this record is the date the 
registrar's sponsorship of the domain name registration in the registry is 
currently set to expire. This date does not necessarily reflect the expiration 
date of the domain name registrant's agreement with the sponsoring registrar.  
Users may consult the sponsoring registrar's Whois database to view the 
registrar's reported date of expiration for this registration.

Domain Name: paypal-communication.com
Registry Domain ID: 1649488607_DOMAIN_COM-VRSN Registrar WHOIS Server: 
whois.markmonitor.com Registrar URL: http://www.markmonitor.com Updated Date: 
2017-03-05T02:14:48-0800 Creation Date: 2011-04-06T05:23:32-0700
  


Registrar Registration Expiration Date: 2018-04-06T00:00:00-0700
  


Registrar: MarkMonitor, Inc.
  


Registrar IANA ID: 292
  


Registrar Abuse Contact Email: abusecomplai...@markmonitor.com
  


Registrar Abuse Contact Phone: +1.2083895740
  


Domain Status: clientUpdateProhibited
(https://www.icann.org/epp#clientUpdateProhibited)
  


Domain Status: clientTransferProhibited
(https://www.icann.org/epp#clientTransferProhibited)
  


Domain Status: clientDeleteProhibited
(https://www.icann.org/epp#clientDeleteProhibited)
Domain Status: serverUpdateProhibited
(https://www.icann.org/epp#serverUpdateProhibited)
Domain Status: serverTransferProhibited
(https://www.icann.org/epp#serverTransferProhibited)
Domain Status: serverDeleteProhibited
(https://www.icann.org/epp#serverDelete

Re: [clamav-users] Malware/ransomware and Yara signatures with clamav

[Dennis Peterson]

If not email what is the vector?

dp

On 5/15/17 5:11 PM, Joel Esler (jesler) wrote:

To be clear let me link to our blog post on the subject:

http://blog.talosintelligence.com/2017/05/wannacry.html

There has been No email vector seen in WannaCry to date.  Almost everyone that 
has claimed this, has retracted it. Please read the above blog post for all the 
facts as we know them.

This is an ongoing threat.

--
Joel Esler | Talos: Manager | jes...@cisco.com



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about ClamScan

[Dennis Peterson]

On 5/12/17 10:19 AM, crazy thinker wrote:

@Maarten

I mailing to both ClamAV Developers and Users.. Hope you unerstand this
.ClamAV  Developers Mailing list  seems inactive.. They are not responding

Given that your crazyplan is to develop a new fork of ClamAV they can hardly be 
blamed for not helping. You should download the source and start your own 
developer/user group mail lists and register your CrazyClam on one of the 
software developer sites. And you should stop bothering non-developers here with 
your developer issues. It is the polite thing to do.


dp
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about ClamAV

[Dennis Peterson]

I would consider a malware author that does not pass his/her new product through 
several file scanners to be incompetent. There is little point in distributing 
such files if it is commonly detectable. Scanners are one of the best quality 
inspection tools a malware author has at their disposal. Conveniently, it can be 
done cheaply at VirusTotal and other sites that do live scans using multiple 
engines.


dp

On 5/11/17 8:21 AM, Matthew Molyett wrote:

Crazy Thinker,


As per my understanding, Signature Based Scanner will never involve in
false postive/false negative results. But Heuristic scanner some times
gives false postive/false negative results.

Signature Based scanning can and will have false positive and false
negative results. In fact, the high rate of False Negatives from Signature
Based is the entire reason Heuristic scanning ( and run-time scanning ) is
performed. A brand new, unknown threat, from a careful author, will be free
of existing signatures. Similarly, a signature on a library only seen
before in malicious software will cause a False Positive when a legitimate
software begins using it.

Large, exact signatures prevent False Positives, but can be trivially
defeated. Flexible signatures with wildcards can identify larger blocks
malicious content, but at the price of potential False Positives.

The response from Maarten Broekman does a great job discussing the issues
we are facing.

Thank you for your choosing Clam AV. Helping protect you and your users is
what keeps me happily getting to work each day.


On Thu, May 11, 2017 at 9:54 AM, Arnaud Jacques / SecuriteInfo.com <
webmas...@securiteinfo.com> wrote:


Hello,


is that a *technical* reason or do you *think* it's recommended for
whatever reason

It is technical : we avoid duplicate signatures in our databases. It means
everyday we remove samples already detected by Clamav.


- as example sanesecurity works just fine without the
official stuff an dthe difference are hundrets of MB useless wasted RAM
while i have not seen any relevant hit on our inbound MX caught by the
official signatures which woul dhave slipped through sanesecurity

In your example you are right. On mail filtering, sanesecurity and
spam_marketing.ndb from SecuriteInfo.com are good enough to protect
mailboxes,
because Win32 malwares are not spreaded by mail nowadays.

In any other case (system protection, HTTP scanning, file hosting, etc...)
you
have to get Clamav official + 3rd party signatures for a maximum detection.

--
Best regards,

Arnaud Jacques
SecuriteInfo.com

Facebook : https://www.facebook.com/pages/SecuriteInfocom/132872523492286
Twitter : @SecuriteInfoCom
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml






___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV UnOfficial Database

[Dennis Peterson]

You make this harder than is necessary. Create a directory for your preferred 
signature files in it (/var/lib/crazyclam, for example), put your preferred 
signature files in it, create a new clamd config file (crazyclamd.conf, for 
example) with that directory defined (DatabaseDirectory /var/lib/crazyclam, for 
example), and use clamd --config-file=/path_to/crazyclamd.conf to start clamd.


dp

On 5/4/17 4:28 AM, crazy thinker wrote:

Hi ClamAV Developers, Users

To my curiosity, i want  to remove ClamAV Official Database and plan to
integrate unofficial database with clamav engine.. i heard that
Sanesecurity signatures increases  ClamAV  performance upto 90%.. so i am
thinking  that  excluding ClamAV Official Database not afffecting ClamAV
performance in this scenario. because. i guess Sanesecurity unofficial
database covers signatures which is covered by ClamAV Official Database..
Am i right?  Th e reason behind to do like this is  i  want to keep
optimized database
i would like to get  some suggestions/advices on my experimental  thought

Thanks,

Crazy Thinker Inc
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Mirror problem

[Dennis Peterson]

Anyone else seeing this?

Sat Apr  1 14:02:39 2017 -> Trying host db.us.clamav.net (209.198.147.20)...
Sat Apr  1 14:03:09 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)
Mon Apr  3 08:02:39 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)

Mon Apr  3 14:02:10 2017 -> Ignoring mirror 209.198.147.20 (due to previous 
errors)
Mon Apr  3 14:02:40 2017 -> Ignoring mirror 209.198.147.20 (due to previous 
errors)
Wed Apr  5 23:03:09 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)
Sat Apr  8 23:02:40 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)
Mon Apr 10 17:02:40 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)

Mon Apr 10 23:02:10 2017 -> Ignoring mirror 209.198.147.20 (due to previous 
errors)
Thu Apr 13 17:03:10 2017 -> Trying host db.us.clamav.net (209.198.147.20)...
Thu Apr 13 17:03:40 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)

Thu Apr 13 23:02:10 2017 -> Ignoring mirror 209.198.147.20 (due to previous 
errors)
Sat Apr 15 08:02:40 2017 -> Trying host db.us.clamav.net (209.198.147.20)...
Sat Apr 15 08:03:10 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)

Sat Apr 15 14:02:10 2017 -> Ignoring mirror 209.198.147.20 (due to previous 
errors)
Wed Apr 19 17:02:39 2017 -> Trying host db.us.clamav.net (209.198.147.20)...
Wed Apr 19 17:03:09 2017 -> Can't connect to port 80 of host db.us.clamav.net 
(IP: 209.198.147.20)

Wed Apr 19 23:02:10 2017 -> Ignoring mirror 209.198.147.20 (due to previous 
errors)

dp

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] error when starting clamd: LibClamAV Warning: Don't know how to create filter for: BC.Win.Exploit.CVE_2017_0060-6099223-0.{}

[Dennis Peterson]

Which version of ClamAV are you running?

dp

On 4/19/17 5:46 PM, Jobst Schmalenbach wrote:

Hi

Upon starting clamd I am receiving following messages:

   Starting clamd: LibClamAV Warning: Don't know how to create filter for: 
BC.Win.Exploit.CVE_2017_0060-6099223-0.{}
   LibClamAV Warning: cli_ac_addpatt: cannot use filter for trie

Searched the net, found nothing.
Any ideas?

thanks






___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV for EnterPrise

[Dennis Peterson]

You should hire an integrator that already knows how to do this.

dp

On 4/18/17 3:28 AM, crazy thinker wrote:

Hi ClamAV Developers, ClamAV Users


I have refered ClamAV Docs but i could find any info to set up clamav in
Business
Environment. i have a small business office  where 50-75 employees are
working

Could anyone of you please help me in this?
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Identify Threat Risk Level with ClamAV

[Dennis Peterson]

This is probably not the best list for this conversation. You make get better 
results by talking with developers, not end-users.


dp

On 4/14/17 9:33 AM, crazy thinker wrote:

Oh.. ok..But  how  Commercial AV  Calculating risk level of  malware and
what is the criteria for that.?

On 14 April 2017 at 22:00, Reindl Harald  wrote:



Am 14.04.2017 um 18:28 schrieb crazy thinker:


All Commericial AV's showign risk status of malware

SanSecurity creating signature database files based and it showing risk
status of malware


sanesecurity shows *risk of false-positives*
don't confuse such basics


On 14 April 2017 at 21:17, SCOTT PACKARD 

wrote:



No. all malware would not be large risk.. for an example,  EICAR  test
file


is sample virus file.it can't make big damage to system.


Can you present another example, other than the EICAR test file?
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


___

clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml