On Sun, Sep 11, 2011 at 04:11:07PM -0400, Dan wrote: > > At 11:40 PM +0200 9/7/2011, Luca Gibelli wrote: > >Traffic is around 5TB/month on each mirror. > > Short of a paid service, which I doubt any of us want, few have such > bandwidth available to donate.
First of all, I think this whole thread is overreacting. I seriously doubt the mirror capacity is at maximum. Anyways, 5TB comes at 2MB/s average, which is not that much. I can do it with my $15 OVH/Kimsufi box and so do probably thousands of others. > Clam needs to leverage the power of the Internet - as it is now, not > yesterday. The simple, semi-linear "propagate thru a few mirrors" > design has obviously reached a limit... 5 TB *per mirror* per > month!!!??? Just to maintain a tiny 36 MB database? d'oh! It does sound a bit much for all the cdiffs etc, but maybe I'm underestimating the number of ClamAV users.. > It may have worked just fine yesterday, but, seriously, just a model > that's waiting to fall on its face as Clam becomes more popular. I don't think it can suddenly come _that_ much more popular, since it's already quite popular. > So, I'm thinking that leaves two choices: 1) a cloud, a la Amazon S3. 2) p2p. > > Maybe, someday, when the well-cached cloud services are fully > propagated *and* reliable world-wide, using a cloud in leiu of the > traditional mirror set-up might be viable. But IMO that's years > away and too expensive. There's nothing wrong with the current method. It's simple and cheap. You are underestimating the bandwidth available in the world. Either there really is no problem and ClamAV is just lazily "fishing" for more mirrors, or then they are just clueless and/or not having the "substantial financial and engineering resources of a much larger organization" (advertised in faq). Heck, even I could buy few boxes for mirrors, but I'm not going to do that as a private person since there are bazillion commercial entities that have or can get the bandwidth if needed, including Sourcefire itself. > Right now, IMO, a p2p set-up would be the most viable. Continue to > propagate via mirrors. *ADD* the torrent. Together, we clam users > have many times the bandwidth needed! > > Is there a way to make freshclam grab and verify database files from > a local directory? If there is, creating a torrent set-up would be > fairly easy, even on an ad-hoc basis. I think it would be > interesting to get a test going... > > WRT the reputation of p2p/torrents... There are quite a few legit > uses for p2p. A number of open source products are even distributed > via bittorrent. Yes, some ISPs are blocking the protocol -- but > when shown that it's a legit use, they're usually willing to fix > that. I like the idea of some 3rd party offering torrent service for the p2p-minded. What I don't want to see is freshclam bloated with some torrent libraries and stuff. You do realize that torrents actually need to have central servers for the .torrent files themselves? That's just the first step (freshclam would have already downloaded cdiffs at the same step). Then you actually need to have some trackers also, unless you are relying on DHT. Hopefully it's not the main database you end up downloading from some guys slow ADSL link.. _______________________________________________ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
