Damian Menscher wrote the following on 08/11/2004 04:40 PM :
On Wed, 11 Aug 2004, Lionel Bouton wrote:
Since some time I am thinking of a bittorrent approach too. Bittorrent
is quite efficient at distributing files and there are implementations
allowing multiple trackers to distribute the remaining server-side load.
Please take this as a question rather than a criticism of the approach:
My experience with bittorrent has been with downloading huge things, like Fedora. It tends to start up really slowly (since it has to find peers) and then speed up. But the speedup doesn't occur until several megs have been downloaded. If we're only sending a 1-meg main.cvd, then wouldn't bittorrent lose its advantage to all the overhead of finding peers?
It depends on the torrents you download. Some starts really quickly. From my experience it really depends on how much clients are really available to upload files and the ratio of good/bad clients known by the tracker : when numerous clients stop distributing content after finishing a download, some (and sometimes most of the) clients the tracker send you are now unavailable and you then start having timeouts and asking other clients to the tracker. In a more controlled environnement where the client behaviour will be hard-coded in a hypothetical freshclam-torrent to serve the last two db files and properly warn the trackers when they stop distributing contents, you most probably won't experience such problems.
Remember that bittorrent was designed specifically to handle the very same problem we are discussing, the quality of some popular implementations may damage the efficiency of the protocol, but in the clamav context we don't have to deal with this problem.
With regard to all the other ideas: Please remember to keep this *simple*. Here's where I, IMHO, think we stand:
Opening a new port on a mailserver so updates can be pushed to it is a
BAD idea. As a sysadmin, I would not allow such a thing on my
production machines. It creates a huge security risk, since now you
have one more opening to a remote root vulnerability.
The freshclam process is less risky because tricking freshclam into connecting to an arbitrary system requires injecting DNS entries somewhere, controlling the routing process or even injecting content in TCP conversations which all are more complex techniques than exploiting a buffer overflow on a server for example. But the problem is not really in opening ports or not but in controlling the quality of the code you run. TCP and DNS are not secure so freshclam must add its security layer above, theoritically it doesn't really matter if it is running as a TCP client or server.
The idea of DNS sounds really good, but it doesn't appear we can fit all the data there. And putting just a version number there appears to make things worse, since it will just make everyone hit the mirrors at the same time.
Using DNS doesn't mean the freshclam processes will all be waking up at the same time to query the DNS entry. They may use shorter delays than the current 1 or 2 hours, but even with one second delay, the queries leaving the local DNS servers in the recursive process would be distributed on a TTL-wide period. The TTL being controlled by the authoritative servers.
If there's a problem of bandwidth/load on the mirrors, then move it away from them by designing a new (parallel ?) distribution mechanism or throw more mirrors on it. But as I said I'm not yet convinced there's a real problem. Shorting the delay from db update to db install on each mail server is good, but don't forget that 1 hour is already low compared to the time between the first virus appearance and its signature addition in the database. We will never eliminate that delay and must do with it.
If we can somehow distribute signatures that way it would be nice, but it just doesn't seem practical.
I'm really starting to like the idea of a mailing list that can have dedicated (and random for each site) subscription addresses and pipe the list straight into "sigtool --add". It means we'd have to find someone to host the list, but that's probably no more difficult than finding someone to host a mirror.
Are you kidding ? Our mirrors have seen more than 120000 clamav clients last month and we only see a *part* of the *european* clients. Mirrors handled with round-robin DNS are fairly easy to setup, but sending 120000 mails for each db update can very well end up burying one mail server under the load.
Presumably there could even be multiple
"mirrors" sending the list, to improve speed (taking an idea from
spammers who use open relays to do the hard part).
Using spammers' techniques could help but at quite a cost in complexity and in the end I don't think it would be practical for the end-users.
-- Lionel Bouton - inet6 --------------------------------------------------------------------- o Siege social: 51, rue de Verdun - 92158 Suresnes / _ __ _ Acces Bureaux: 33 rue Benoit Malon - 92150 Suresnes / /\ /_ / /_ France \/ \/_ / /_/ Tel. +33 (0) 1 41 44 85 36 Inetsys S.A. Fax +33 (0) 1 46 97 20 10
------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ Clamav-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/clamav-users
