Re: [clamav-users] Specify more servers for clamdscan to pass for scanning

2018-11-05 Thread Brent Clark 

Good day

Thanks all for the the replies.
Much appreciated.

Regards
Brent

On 2018/11/05 17:19, Micah Snyder (micasnyd) wrote:

Hi Brent,

clamdscan and clamd share the same config file.  clamd uses it to set up 
the TCP socket to listen, and clamdscan uses it to know how to talk to 
clamd.  As a consequence, setting multiple TCPAddr's would make no sense 
to clamd as it will only open 1 socket to listen for connections.


I'd be interested to know if someone has come up with a hack for how to 
have clamdscan fail over to a secondary clamd instance - but I'm not 
aware of a way to do that.


I would bare in mind that if your'e using clamd on a machine remote from 
the clamdscan machine, the TCP connection is not encrypted at this time 
- meaning your file contents may be streamed in plaintext across your 
network.  If you need to do this, consider setting up a SSH tunnel to 
facilitate an encrypted connection.


Regards,
Micah


On Nov 5, 2018, at 7:48 AM, Brent Clark > wrote:


Good day Guys

I have setup two clamd servers.

On my Webservers, I need to stream a file to the clamd for scanning.

I would like to ask, how would I specify two TCPAddr.

If I specify just one, server, everything works ok.
Ive tried various options and google does not appears to be of assists.

How does one specify more than one server for scanning?

I would like to use this a poor mans "fail over", so that if one 
server is down, clamscan will move on to the next server.


Thanks in advance.

Regards
Brent






___
clamav-users mailing list
clamav-users@lists.clamav.net 
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Specify more servers for clamdscan to pass for scanning

2018-11-05 Thread Henrik K 
On Mon, Nov 05, 2018 at 03:19:44PM +, Micah Snyder (micasnyd) wrote:
>
> I'd be interested to know if someone has come up with a hack for how to have
> clamdscan fail over to a secondary clamd instance - but I'm not aware of a way
> to do that.  

It's called "writing your own clamdscan".  Connect socket and stream, not
rocket science.  No different than needing to write our own custom clamd,
since after all these years official STILL can't do a basic job of reloading
signatures without hanging the process.  :-D

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Specify more servers for clamdscan to pass for scanning

2018-11-05 Thread Kris Deugau 

Brent Clark wrote:

Good day Guys

I have setup two clamd servers.

On my Webservers, I need to stream a file to the clamd for scanning.

I would like to ask, how would I specify two TCPAddr.

If I specify just one, server, everything works ok.
Ive tried various options and google does not appears to be of assists.

How does one specify more than one server for scanning?

I would like to use this a poor mans "fail over", so that if one server 
is down, clamscan will move on to the next server.


We use Linux LVM load balancing to group "many" processing nodes 
(currently two, although we've had more on older hardware in the past) 
into one logical service.  You can then point your clamdscan (or 
clamav-milter) callers to the load-balanced IP.


-kgd
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Specify more servers for clamdscan to pass for scanning

2018-11-05 Thread G.W. Haywood 

Hi there,

On Mon, 5 Nov 2018, Micah Snyder wrote:
On Nov 5, 2018, at 7:48 AM, Brent Clark wrote:


> How does one specify more than one server for scanning?

I'd be interested to know if someone has come up with a hack for how
to have clamdscan fail over to a secondary clamd instance - but I'm
not aware of a way to do that.


You'd need to modify the existing clamav-milter, which might be quite
an undertaking if you're not familiar with milters, or perhaps write a
separate milter.  I've written a milter which could easily be modified
to do this.  I call it 'eXtensible-Milter', XM.  At present it doesn't
call clamd at all - clamav-milter does that here.  Until now I've had
no particular reason to replace clamav-milter although XM has replaced
half a dozen other milters which were giving me assorted troubles, and
now I use only the two.

The idea of the possibility of a 'clamd farm' is intriguing.  If you'd
be interested I can think about adding such functionality to XM.

Hmmm.  On reflection, 'easily' might have been an overstatement but at
worst in principle it's easy and it's very doable. :)

XM is written in pure Perl, and at the moment it has only ever been
used with Sendmail.  It hasn't been published.

--

73,
Ged.
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] About clamav's requirements for system resources

2018-11-05 Thread Graeme Fowler 
Not milter, but Exim calls ClamAV using the SCAN command when using a UNIX 
socket, or zINSTREAM for TCP sockets.

I've got 3 'clusters' (loosely coupled groups, more accurately) VMs of 
differing roles with slightly differing setups here at Loughborough Uni.


  *   CentOS 6 MX servers with a small number of custom sig files - consuming 
around 2GB RAM per clamd instance, scanning around 25-100k messages each per 
day. ClamAV MaxThreads set to greater than the max permitted number of inbound 
simultaneous SMTP connections, with a short pending queue.



  *   CentOS 7 MX servers with stock ClamAV sigs - consuming around 1.5GB RAM 
per clamd instance, scanning around 15-75k messages each per day. ClamAV 
MaxThreads set to greater than the max permitted number of inbound connections 
with a small, but a short pending queue.


  *   CentOS 6 MTA (outbound) servers with stock ClamAV sigs - consuming around 
2GB RAM per clamd instance, scanning around 25-100k messages each per day. 
ClamAV MaxThreads set to less than the max permitted number of inbound 
simultaneous SMTP connections, with a long pending queue where (pending + 
active) = max inbound SMTP connections.

Each of these groups are the same in 'hardware' terms - 4 cores, 8GB RAM. They 
normally don't break a sweat.

From memory, we had a single instance in the last 12 months where the kernel 
OOM killer was invoked and killed off clamd after an external 3rd party 
attempted to exploit a web form on one of our websites; the form sent several 
hundred thousand messages via one of the MTA servers which got a touch upset. 
We never did work out why.

Is that helpful in any way?

Graeme



From: clamav-users  on behalf of "Micah 
Snyder (micasnyd)" 
Reply-To: ClamAV users ML 
Date: Monday, 5 November 2018 at 15:14
To: ClamAV users ML 
Subject: Re: [clamav-users] About clamav's requirements for system resources

At this time, we don't have recommendations for those using clamav-milter in 
conjunction with a mail server under any amount of load.  I'd be interested to 
hear from the community what your experience has been with real-world milter 
applications.


___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Specify more servers for clamdscan to pass for scanning

2018-11-05 Thread Micah Snyder (micasnyd) 
Hi Brent,

clamdscan and clamd share the same config file.  clamd uses it to set up the 
TCP socket to listen, and clamdscan uses it to know how to talk to clamd.  As a 
consequence, setting multiple TCPAddr's would make no sense to clamd as it will 
only open 1 socket to listen for connections.

I'd be interested to know if someone has come up with a hack for how to have 
clamdscan fail over to a secondary clamd instance - but I'm not aware of a way 
to do that.

I would bare in mind that if your'e using clamd on a machine remote from the 
clamdscan machine, the TCP connection is not encrypted at this time - meaning 
your file contents may be streamed in plaintext across your network.  If you 
need to do this, consider setting up a SSH tunnel to facilitate an encrypted 
connection.

Regards,
Micah


On Nov 5, 2018, at 7:48 AM, Brent Clark 
mailto:brentgclarkl...@gmail.com>> wrote:

Good day Guys

I have setup two clamd servers.

On my Webservers, I need to stream a file to the clamd for scanning.

I would like to ask, how would I specify two TCPAddr.

If I specify just one, server, everything works ok.
Ive tried various options and google does not appears to be of assists.

How does one specify more than one server for scanning?

I would like to use this a poor mans "fail over", so that if one server is 
down, clamscan will move on to the next server.

Thanks in advance.

Regards
Brent






___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] About clamav's requirements for system resources

2018-11-05 Thread Micah Snyder (micasnyd) 
Our QA engineer Joe did some testing a couple of months ago to come up with 
some basic minimum system requirements for ClamAV.  He passed these along to 
the thread author already in a private message, but I wanted to share these 
here as well.

The following minimum recommended system requirements are for using `clamscan` 
or `clamd` and `clamdscan` binaries with the standard ClamAV signature database 
provided by Cisco.

Minimum recommended RAM:

- FreeBSD and Linux server edition: 1 GiB+
- Linux non-server edition: 2 GiB+
- Windows 7 & 10 32-bit: 2 GiB+
- Windows 7 & 10 64-bit: 3 GiB+
- macOS: 3 GiB+

Minimum recommended CPU:

- FreeBSD and Linux systems: 1 CPU 2.0 Ghz+
- Windows 7 & 10: 1 CPU 2.0 Ghz+
- OSX: 2 CPUs at 2.0 Ghz+

Minimum available hard disk space required:

For the ClamAV application we recommend having 5 GB of free space available. 
This recommendation is in addition to the recommended disk space for each OS.

_Please note_: The tests to determine these minimum requirements were performed 
on systems that were not running other applications. If other applications are 
being run on the system, additional resources will be required in addition to 
our recommended minimums.

We'll add the above recommendations to the documentation before 0.101 release.

At this time, we don't have recommendations for those using clamav-milter in 
conjunction with a mail server under any amount of load.  I'd be interested to 
hear from the community what your experience has been with real-world milter 
applications.

If you have any additional recommendations you think we should add to the user 
manual - we would like to hear from you.

Cheers,
Micah

Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.


On Nov 5, 2018, at 8:38 AM, Vladislav Kurz 
mailto:vladislav.k...@webstep.net>> wrote:

On 11/3/18 5:23 PM, Matus UHLAR - fantomas wrote:
zhuangxiaohui wrote:
I have some servers(Centos6/7). Most of them have 1GB memory, 600M
available.
But also servers with low memory. For example 512M memory, 200M
available.
When I install the "clamav" on server which have 600M available
memory and
start the "clamd" service,
I find that clamd's resident memory is about 500M. But on servers
that have
only 200M of available memory,
the resident memory is about 100M. So I doubt if clamd will work
properly on
these servers, although both
scan and database's updates are normally.

Would you please tell me the lowest clamav's requirements for system
resources especially the memory?
I've searched on your website but got nothing about this :(

On 02.11.18 15:43, Kris Deugau wrote:
I wouldn't run ClamAV with stock signatures on anything less than 1G,
and I wouldn't run much else on that machine.  If you're running a
very light workload with a dedicated machine, you might get away with
512M.

I run clamav with 3rd party signatures from Debian package
clamav-unofficial-sigs everywhere.  In this case, clamav eats nearly 1G of
RAM.

I can't tell you how much of it eats clamav without those signatures, but I
wouldn't run clamav on machines with less than 1GB either.


The unofficial signatures do not eat much extra memory. I think it is
not more than 10% extra, virtually for free. I agree that 1 GB is
minimum, and as you would most probably have a mail server as well, I
recommend 2 GB.

--
Best regards
   Vladislav Kurz

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] About clamav's requirements for system resources

2018-11-05 Thread Vladislav Kurz 
On 11/3/18 5:23 PM, Matus UHLAR - fantomas wrote:
>> zhuangxiaohui wrote:
>>> I have some servers(Centos6/7). Most of them have 1GB memory, 600M
>>> available.
>>> But also servers with low memory. For example 512M memory, 200M
>>> available.
>>> When I install the "clamav" on server which have 600M available
>>> memory and
>>> start the "clamd" service,
>>> I find that clamd's resident memory is about 500M. But on servers
>>> that have
>>> only 200M of available memory,
>>> the resident memory is about 100M. So I doubt if clamd will work
>>> properly on
>>> these servers, although both
>>> scan and database's updates are normally.
>>>
>>> Would you please tell me the lowest clamav's requirements for system
>>> resources especially the memory?
>>> I've searched on your website but got nothing about this :(
> 
> On 02.11.18 15:43, Kris Deugau wrote:
>> I wouldn't run ClamAV with stock signatures on anything less than 1G,
>> and I wouldn't run much else on that machine.  If you're running a
>> very light workload with a dedicated machine, you might get away with
>> 512M.
> 
> I run clamav with 3rd party signatures from Debian package
> clamav-unofficial-sigs everywhere.  In this case, clamav eats nearly 1G of
> RAM.
> 
> I can't tell you how much of it eats clamav without those signatures, but I
> wouldn't run clamav on machines with less than 1GB either.
> 

The unofficial signatures do not eat much extra memory. I think it is
not more than 10% extra, virtually for free. I agree that 1 GB is
minimum, and as you would most probably have a mail server as well, I
recommend 2 GB.

-- 
Best regards
Vladislav Kurz

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Specify more servers for clamdscan to pass for scanning

2018-11-05 Thread Brent Clark 

Good day Guys

I have setup two clamd servers.

On my Webservers, I need to stream a file to the clamd for scanning.

I would like to ask, how would I specify two TCPAddr.

If I specify just one, server, everything works ok.
Ive tried various options and google does not appears to be of assists.

How does one specify more than one server for scanning?

I would like to use this a poor mans "fail over", so that if one server 
is down, clamscan will move on to the next server.


Thanks in advance.

Regards
Brent






___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml