Gary Smith said: > What about sf.net. Their list group sucks but they seem to have some > bandwidth. > > Gary > > -----Original Message----- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Sent: Monday, July 19, 2004 7:54 PM > To: Dan Kohn > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: Re: [RDJ] Is it broken? > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > Dan Kohn writes: >> Would the Apache folks get upset if you let the SARE people store > their >> scripts on spamassassin.apache.org and then just let everyone have at >> them? Isn't that the purpose of the Apache mirrors? I would think it >> would be fine as long as the SARE rules are released under the Apache >> license. > > As far as I know, as long as it's ASF-licensed (ie. covered by a > CLA), it can go on the SpamAssassin.apache.org website. > > However, that's *not* mirrored, and heavy bandwidth usage is more > acceptable there than on our previous sites -- but still not *that* > acceptable :( I think a frequently-polled set of files would > probably not fly there, not without some negotiating first. ;) > > (The mirrored portion of the ASF site is http://www.apache.org/dist/ , > and that's only used for full releases that have been voted on > and all sorts of procedure along those lines.) > > - --j. > >> Push is a lot more efficient, but it's also harder to implement on the >> client side. I'm not convinced that push optimizes on the scarce >> variable. >> >> - dan >> -- >> Dan Kohn <mailto:[EMAIL PROTECTED]> >> <http://www.dankohn.com/> <tel:+1-650-327-2600> >> -----Original Message----- >> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] >> Sent: Monday, July 19, 2004 16:55 >> To: Gary Smith >> Cc: Fred; Chris Santerre; Spamassassin-Talk (E-mail) >> Subject: Re: [RDJ] Is it broken? >> >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Hey guys -- >> >> suggestion: how's about sending out new versions as ZIP files via a >> mailing list? ie. turn it from "pull" into "push"? >> >> if you provide a script, it should be possible to automate something >> that >> users can put into /etc/aliases, then subscribe that address to the > "RDJ >> list" -- and the updates get auto-installed as they're generated. >> >> It'd also be a handy way to spread the load of HTTP service; each > mirror >> host has an alias that extracts the files and puts them onto its > mirror. >> >> - --j. >> >> Gary Smith writes: >> > Damn, didn't know things were going that bad for you guys... >> > >> > I guess all that talk about letting us help you mirror to reduce the >> load was missed somewhere in the stream of emails. Anyways, I have my >> scripts set for: >> > >> > 0 6 * * * /root/cronjobs/mailstats.sh >> > 45 5 * * * /etc/mail/spamassassin/my_rules_du_jour.sh >> > >> > And I'm still getting the error all weekend. Rate limited is broken >> guys... >> > >> > So, yes I can just dump RDJ and copy the files to a central location >> daily. It's not a problem. I just think you're going about it in a >> complety odd way. Force the wget's to authenticate. Force rate >> limiting based on performance and not denial, have a list of active >> subscribers that aren't rate limited by IP. >> > >> > Gary >> > >> > >> > ________________________________ >> > >> > From: Fred [mailto:[EMAIL PROTECTED] >> > Sent: Mon 7/19/2004 3:13 PM >> > To: Chris Santerre; Spamassassin-Talk (E-mail) >> > Subject: Re: [RDJ] Is it broken? >> > >> > Gary Smith wrote: >> > > Even after the problem has been fixed with RDJ it will just mask > the >> > > fact that we cannot connect to the server. Basically, we will > never >> > > know that it failed because no update occured. >> > >> > True, however, in my cron scripts I am alerted of errors or output >> from any >> > script which is run. If you check the output of those scripts you >> will see >> > when things go bad. >> > >> > I'm a member of SARE and I'm all for the ratelimiter. we spend a >> great >> > deal of time and effort doing our thing and some admins are >> irresponsible >> > enough to set updates for once every 2 minutes. We have to protect >> > ourselves, if people are going to abuse free services (AND YOU KNOW >> THEY >> > WILL) those people providing free services need to find ways to keep >> > themselves in business or limit the abusers so it doesn't affect the >> rest. >> > >> > This rate limiter is a wake up call for anyone doing more than 1 >> update per >> > day. Even if it is 5 seperate servers behind a proxy. This is free >> stuff >> > you are abusing, please stop abusing it and re-implement your method >> of >> > doing things. If you know that each of those 4-5 servers will >> download the >> > same files each day, wouldn't it be best to download to a central >> location >> > and then have your servers get it from there? It seems having all >> servers >> > download straight from us is the lazy approach. (the same lazy >> approach >> > that causes traffic congestion in all parts of my life), if people > car >> > pooled more, we would have more room on the roads, if people buy in >> bulk, >> > you don't have to revisit the same stores so often. If virus > writers >> used >> > IRC more often, they would not have to flood the entire internet > with >> port >> > scans looking for their zombies. In my book (and this is not > directly >> > towards anyone in specific) this is all due to being lazy. (it's >> easier to >> > flood internet with packets looking for zombies than to have the >> zombies >> > phone home?) (it's easier if you jump in your own car and drive >> yourself to >> > work versus waiting for someone else) We often value ease without >> thinking >> > of the consequences of our actions. For once, we have the ability > to >> force >> > people to follow our restrictions and I'll fight to keep it in > effect >> (AS >> > SOON AS WE GET THE BUGS WORKED OUT). >> > >> > RDJ to an central storage, then you could even have your servers run >> RDJ on >> > that location to check for updates. >> > >> > The problem was a few people made such bad mistakes that we have to >> make >> > changes which affect everyone. We don't like putting limits on > things >> like >> > this and if it wasn't to protect our own butts we would not have > done >> it. >> > >> > SARE is run by donation from people and businesses combined. We > have >> a >> > business that donated a shell account and bandwidth. If we abuse > the >> free >> > donation, we'll be looking for a new home. If people setup mirrors, >> then >> > everyone can hammer the mirrors all they want, as long as it doesn't >> > threaten our very existance. >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.2.4 (GNU/Linux) >> Comment: Exmh CVS >> >> iD8DBQFA/F9JQTcbUG5Y7woRAgVaAKDa5l6bvD1OuSVii318yVqjxdULzwCgsGqV >> gwCt2bj49AgHmFet5GGEKho> =8pRW >> -----END PGP SIGNATURE----- > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.4 (GNU/Linux) > Comment: Exmh CVS > > iD8DBQFA/Ik1QTcbUG5Y7woRAvkxAKCQftm8dVrr4WzgQZ/jWAiJeLKA+gCeN9gp > 6wjgLUVjLncPtdajtZlaIUg=G46z > -----END PGP SIGNATURE----- > > >
Hi all, While SARE is discussing the possibility of using mirrors and several folks are offering, I just want to throw a little info out there... One of the more recent examples cited to us by the fine folks hosting rulesemporium.com was the usage for July 14th, over 2.5 GB of data was pulled from the site. As far as I can see, at most there was only one file that may have been updated that day. Thats a whole lot of people pulling the same data over and over and over....... If we decide to mirror the site, thats a sample of the bandwidth we'll need unless we come up with a method of controlling those who setup cron jobs to download the data every few minutes. If you have not already made sure that you are pulling the rulesets once per 24 hour period, please do so. For those that are already configured to only pull once a day, let me say thank you! We're working on fixing the rate limiter and discussing how to handle this long term, so please be patient. :) -matt
