RE: sa-update versus rulesdujour questions
Bret Miller wrote: > > Theo Van Dinter wrote: > > > FWIW, it happens to be the "official" tool since no one ever > > > submitted RDJ to be the official tool, so we had to write our own. > > > > > I would have offered, had I known there was any interest. > > > > Chris T. > > > I'm glad it isn't the official tool since it doesn't run natively on > Windows. Sa_update does. As the official tool, I presume it would have been ported to Perl. Based on what I've seen of the code, it wouldn't be too difficult. But with sa-update available, it's a moot point now. -- Bowie
RE: sa-update versus rulesdujour questions
> Theo Van Dinter wrote: > > FWIW, it happens to be the "official" tool since no one > ever submitted > > RDJ to be the official tool, so we had to write our own. > > > I would have offered, had I known there was any interest. > > Chris T. I'm glad it isn't the official tool since it doesn't run natively on Windows. Sa_update does. Bret
Re: sa-update versus rulesdujour questions
Theo Van Dinter wrote: FWIW, it happens to be the "official" tool since no one ever submitted RDJ to be the official tool, so we had to write our own. I would have offered, had I known there was any interest. Chris T.
Re: sa-update versus rulesdujour questions
Jo Rhett wrote: Is there any difference here that I'm overlooking? Any advantage to RDJ? And leading to my next point, given that sa-update is working fine -- isn't rdj going to be slimmed down to just the part that restarts the process after running sa-update? Hi Jo, I'm the author of RDJ. I wrote it for the purpose of updating the then blossoming field of 3rd party rulesets at a time when there was no official update mechanism in SA. Now that there/ is/ an official SA update mechanism, I have no plans to further enhance RDJ. I will continue to fix bugs and add rulesets to the default list, if people request so. There will be no "slimming-down" since I see no reason to surprise 5000 users by making them change what works fine for them now :). The discussion of pros and cons for implementing sa-update or rdj has already been handled nicely by other people, so I won't go into that. Want my advice as the RDJ author? Use sa-update. If you want to update a ruleset which has no sa-update channel then set up RDJ at that time and use it conjunction with sa-update. Chris T. PS sorry I got into this conversation so late. I tend to track my mailing lists infrequently
Re: sa-update versus rulesdujour questions
Daryl C. W. O'Shea wrote: To start, again, I have *nothing* against RDJ. I just like things to be as efficient as practical (it's how I live and make a living), which is why I like sa-update. I'll explain why sa-update is more efficient... [snip] Thank you very much for the detailed response. That was *EXACTLY* what I was looking for. :-) -- Jo Rhett Network/Software Engineer Net Consonance
RE: sa-update versus rulesdujour questions
Daryl C. W. O'Shea wrote: > To start, again, I have *nothing* against RDJ. I just like things to > be as efficient as practical (it's how I live and make a living), > which is why I like sa-update. I'll explain why sa-update is more > efficient... I wasn't intending to advocate either. I was just listing advantages and disadvantages as I see it. I use RDJ mainly due to inertia. If it works, don't touch it. > Bowie Bailey wrote: > > > I don't know that there is much difference in the configuration > > required. For sa-update, you create a file with a list of > > rulefiles. For RDJ, you create a file with a list of rulefiles and > > a restart command. > > I agree. When it comes to rulesets already supported by RDJ selecting > which rulesets to use are pretty much on an equal config basis. > > RDJ has the benefit of being able to update any plain .cf file > available via HTTP. The downside is, you either need to modify the > script for not-yet-supported rulesets or wait for an update. > > sa-update has the benefit of not needing to be updated for new > channels. You simply add the channel to your config. The downside > is the rulesets have to be available in channel form. However, > channel files are really easy to make and can be trivially scripted > and automated. > > > > They are both good. RDJ was made to deal with third party rulesets > > and it does a good job. sa-update was made to deal with official > > ruleset updates and has been extended to also handle third party > > rulesets. > > Not really important, but sa-update has always supported channels from > anyone... no extending required. :) I guess I was thinking of the adding of the extra channels as the extension since the only channel available was the default rules for quite a while. > > RDJ has the advantage that it can update almost any ruleset that is > > available on the web. > > > > sa-update has the advantage of also updating the official rules. > > The downside is that you have to create channels for new rulesets, > > so it isn't quite as simple as creating the ruleset and making it > > available on the web. > > While true, you need to make a tarball, sign the tarball, and > generate a sha1 hash of the tarball (3 commands total, all > scriptable) and update DNS (also scriptable) the pay-offs are huge > infrastructure wise. You forgot to mention that you have to be able to make changes to your DNS server in order to host a channel. > Since sa-update uses DNS to determine if there are new updates for a > channel, users can check for updates more often without causing a > significant increase in use of the channel providers resources. > > By adjusting TTLs to a value that they can comfortably support (HTTP > server resource wise) the channel provider can provide updates faster > while preventing what could effectively turn into a DDoS if their > clients were using RDJ and a check interval of only a few minutes. > > In the case of the channels I provide for the SARE rulesets, if you > want to run sa-update every few minutes go for it. The current TTL > on those zones is an hour, so you're worst case wait time for new > updates for those channels is an hour (plus the maximum of 21 minutes > for my server to notice that there's been new updates). Compared to > a worse case wait time of 24 hours for SARE rules via RDJ, you'll be > getting updates via the sa-update channel a lot faster. If rules are > updated at random times throughout the day, you're looking at an > approx 40 minute delay via sa-update and a 12 hour delay via RDJ. True, but I only check once a day anyway. > > Not as long as people continue to use it. Quite a few of us (me > > included) see no reason to switch. > > I also expect that RDJ will be in use for quite some time, especially > with all those 2.x and 3.0.x users out there. Although, with engine > software that old, I'm not too sure why they're all too concerned with > automated updates. :) sa-update has quite a few advantages. If you are setting up a new server, I would recommend using it. On the other hand, if you already have RDJ running and don't require fast updates, I don't think there is a major case for switching to sa-update. -- Bowie
Re: sa-update versus rulesdujour questions
To start, again, I have *nothing* against RDJ. I just like things to be as efficient as practical (it's how I live and make a living), which is why I like sa-update. I'll explain why sa-update is more efficient... Bowie Bailey wrote: I don't know that there is much difference in the configuration required. For sa-update, you create a file with a list of rulefiles. For RDJ, you create a file with a list of rulefiles and a restart command. I agree. When it comes to rulesets already supported by RDJ selecting which rulesets to use are pretty much on an equal config basis. RDJ has the benefit of being able to update any plain .cf file available via HTTP. The downside is, you either need to modify the script for not-yet-supported rulesets or wait for an update. sa-update has the benefit of not needing to be updated for new channels. You simply add the channel to your config. The downside is the rulesets have to be available in channel form. However, channel files are really easy to make and can be trivially scripted and automated. They are both good. RDJ was made to deal with third party rulesets and it does a good job. sa-update was made to deal with official ruleset updates and has been extended to also handle third party rulesets. Not really important, but sa-update has always supported channels from anyone... no extending required. :) RDJ has the advantage that it can update almost any ruleset that is available on the web. sa-update has the advantage of also updating the official rules. The downside is that you have to create channels for new rulesets, so it isn't quite as simple as creating the ruleset and making it available on the web. While true, you need to make a tarball, sign the tarball, and generate a sha1 hash of the tarball (3 commands total, all scriptable) and update DNS (also scriptable) the pay-offs are huge infrastructure wise. Since sa-update uses DNS to determine if there are new updates for a channel, users can check for updates more often without causing a significant increase in use of the channel providers resources. By adjusting TTLs to a value that they can comfortably support (HTTP server resource wise) the channel provider can provide updates faster while preventing what could effectively turn into a DDoS if their clients were using RDJ and a check interval of only a few minutes. In the case of the channels I provide for the SARE rulesets, if you want to run sa-update every few minutes go for it. The current TTL on those zones is an hour, so you're worst case wait time for new updates for those channels is an hour (plus the maximum of 21 minutes for my server to notice that there's been new updates). Compared to a worse case wait time of 24 hours for SARE rules via RDJ, you'll be getting updates via the sa-update channel a lot faster. If rules are updated at random times throughout the day, you're looking at an approx 40 minute delay via sa-update and a 12 hour delay via RDJ. Not as long as people continue to use it. Quite a few of us (me included) see no reason to switch. I also expect that RDJ will be in use for quite some time, especially with all those 2.x and 3.0.x users out there. Although, with engine software that old, I'm not too sure why they're all too concerned with automated updates. :) Daryl
Re: sa-update versus rulesdujour questions
On Wed, Oct 18, 2006 at 03:42:26PM -0400, Bowie Bailey wrote: > They are both good. RDJ was made to deal with third party rulesets > and it does a good job. sa-update was made to deal with official > ruleset updates and has been extended to also handle third party > rulesets. That's not exactly true. sa-update was written to deal with rulesets, one of which is the official ruleset. There happen to be a few hardcoded in entries for the official ruleset (default channel, GPG key, etc,) but it's a generic tool. FWIW, it happens to be the "official" tool since no one ever submitted RDJ to be the official tool, so we had to write our own. > sa-update has the advantage of also updating the official rules. The > downside is that you have to create channels for new rulesets, so it > isn't quite as simple as creating the ruleset and making it available > on the web. Channels have several upsides, such as using DNS to specify the latest version. So an "am I up to date" query is a really really cheap operation (DNS query) as opposed to having to connect to a web server and going through the whole thing (even a IMS GET is relatively heavy in comparison to the channel version). So yes, channels make publishing a little more involved than slapping a file on a website. On the flip side, channels give a number of benefits above the "file on website" method (support for multiple cf/pre files, cheap version queries, mirrors for update files, sha1 and gpg checksums, etc, etc). -- Randomly Selected Tagline: "Why is there only one quote? Because the whiteboard doesn't do syntax checking." - Prof. Finkel pgpfYJUp72yxw.pgp Description: PGP signature
RE: sa-update versus rulesdujour questions
Jo Rhett wrote: > Hm. I'm surprised on no answers. Can I persist? This topic is of > real interest to me... > > Jo Rhett wrote: > > Okay, there's no docs on this so I wanted to ask if someone has any > > insights different than what I have observed. > > > > SA-Update seems to require less configuration changes. In short, > > all I did was make a file with a list of rulefiles that SA-Update > > should check, and everything worked without any other changes. > > > > RDJ required lots of local.cf changes for no obvious gain that I > > could see. This is why I chose sa-update. I don't know that there is much difference in the configuration required. For sa-update, you create a file with a list of rulefiles. For RDJ, you create a file with a list of rulefiles and a restart command. > > RDJ does do the job of restarting the daemon for you, but it needed > > to be hacked if you were using amavisd. I wouldn't say it needs to be hacked, you just need to specify the amavisd restart command in the config file. > > Is there any difference here that I'm overlooking? Any advantage > > to RDJ? They are both good. RDJ was made to deal with third party rulesets and it does a good job. sa-update was made to deal with official ruleset updates and has been extended to also handle third party rulesets. RDJ has the advantage that it can update almost any ruleset that is available on the web. sa-update has the advantage of also updating the official rules. The downside is that you have to create channels for new rulesets, so it isn't quite as simple as creating the ruleset and making it available on the web. > > And leading to my next point, given that sa-update is working fine > > -- isn't rdj going to be slimmed down to just the part that > > restarts the process after running sa-update? Not as long as people continue to use it. Quite a few of us (me included) see no reason to switch. > > > > Why not? -- Bowie
Re: sa-update versus rulesdujour questions
Jo Rhett wrote: On Oct 18, 2006, at 11:15 AM, Tim Litwiller wrote: I've never changed anything in local.cf when using RDJ - what did you have to change? Reading RDJ setup, it kept mentioning that I would have to add statements to local.conf for each and every ruleset that I imported. This is why I didn't bother, and used sa-update instead. I'm wondering if there is anything I'm missing... Yes. Its not /etc/spamassassin/local.cf you add lines to. Rather, it is /etc/rulesdujour/config. Mine looks like: ext1:~# cat /etc/rulesdujour/config TRUSTED_RULESETS="TRIPWIRE ANTIDRUG SARE_EVILNUMBERS0 SARE_EVILNUMBERS1 SARE_EVILNUMBERS2 RANDOMVAL BOGUSVIRUS SARE_ADULT SARE_FRAUD SARE_BML SARE_RATWARE SARE_SPOOF SARE_BAYES_POISON_NXM SARE_OEM SARE_RANDOM SARE_HEADER SARE_HEADER0 SARE_HEADER1 SARE_HEADER2 SARE_HEADER3 SARE_HEADER_ENG SARE_HEADER_X264_X30 SARE_HEADER_X30 SARE_HTML SARE_HTML0 SARE_HTML1 SARE_HTML2 SARE_HTML3 SARE_HTML4 SARE_HTML_ENG SARE_SPECIFIC SARE_OBFU SARE_OBFU0 SARE_OBFU1 SARE_OBFU2 SARE_OBFU3 SARE_REDIRECT SARE_REDIRECT_POST300 SARE_SPAMCOP_TOP200 SARE_GENLSUBJ SARE_GENLSUBJ0 SARE_GENLSUBJ1 SARE_GENLSUBJ2 SARE_GENLSUBJ3 SARE_GENLSUBJ_X30 SARE_GENLSUBJ_ENG SARE_HIGHRISK SARE_UNSUB SARE_URI0 SARE_URI1 SARE_URI2 SARE_URI3 SARE_URI_ENG SARE_WHITELIST" MAIL_ADDRESS="root" SINGLE_EMAIL_ONLY="true" ... the idea is that you have to explicitly "turn on" by inclusions the rules you want. Run as lean or as heavy as you like. Mine is damn near everything available, save the PRE300 sets and the two BLACKLIST sets. Works just ducky. -- --Michel Vaillancourt Wolfstar Systems www.wolfstar.ca
Re: sa-update versus rulesdujour questions
Jo Rhett wrote: According to the home page for the script http://www.exit0.us/index.php?pagename=RulesDuJour Add a TRUSTED_RULESETS line to your config file that contains the names of the rulesets you chose. Example below: You missed the previous line: Create a blank configuration file at /etc/rulesdujour/config That's the file you put the changes in. It's the way you tell RDJ which rulesets to download, what the appropriate restart command is for your setup, who to notify if an update is found, etc. You can also create the file as /etc/mail/rulesdujour, /etc/rulesdujour, or /etc/sysconfig/rulesdujour depending on what fits best with your system layout. -- Kelson Vibber SpeedGate Communications
Re: sa-update versus rulesdujour questions
On Wed, 2006-10-18 at 12:14 -0700, Jo Rhett wrote: > According to the home page for the script > http://www.exit0.us/index.php?pagename=RulesDuJour > > > Add a TRUSTED_RULESETS line to your config file that contains the > > names of the rulesets you chose. Example below: > > > > * TRUSTED_RULESETS="TRIPWIRE SARE_ADULT SARE_OBFU0 SARE_OBFU1 > > SARE_URI0 SARE_URI1" This typically goes into /etc/rulesdujour/config. From the rules_du_jour script itself (sorry for the wrapping): ## Note: When running this script interactively, debug mode is enabled to allow you to view the results. # Usage instructions: # 1) Create a configuration file at /etc/rulesdujour/config # 2) Choose rulesets to update (copy TRUSTED_RULESETS from below to your config file, then edit) # 3) Configure Local settings in your config file (SA_DIR, MAIL_ADDRESS, SA_RESTART from below) # 4) Make this script executable (chmod +x rules_du_jour) # 5) Run this script periodically (manually or via crontab) # 5a) To run manually, simply execute (./rules_du_jour) # 5b) To run via cron, edit your cron (crontab -e) and add a line such as this: # 28 1 * * * /root/bin/rules_du_jour # The crontab line above runs /root/bin/rules_du_jour at 1:28AM every day. (choose a different time, please) # Make sure the user who's crontab you are editing has permission to write files to the SA config dir. signature.asc Description: This is a digitally signed message part
Re: sa-update versus rulesdujour questions
... snip ... According to the home page for the script http://www.exit0.us/index.php?pagename=RulesDuJour Add a TRUSTED_RULESETS line to your config file that contains the names of the rulesets you chose. Example below: * TRUSTED_RULESETS="TRIPWIRE SARE_ADULT SARE_OBFU0 SARE_OBFU1 SARE_URI0 SARE_URI1" Well, you copy your rulesdujour script configuration section to /etc/rulesdujour or some other place were you have config files - there are a list of places/file where it looks for a config file in the first few lines of the script. that is where you set the paths for spamassassin and rules you want to check and other options that over ride the defaults that are in the script. then you can upgrade rulesdujour whenever there is an update and you don't loose your list of rules by overwriting the script.
Re: sa-update versus rulesdujour questions
Jake Vickers wrote: Tim Litwiller wrote: I've never changed anything in local.cf when using RDJ - what did you have to change? ... I use RDJ myself, and also did not add anything to my local.cf to make it work. I adjusted some scores, but that was all. On Oct 18, 2006, at 11:31 AM, Kelson wrote: Me three. As many changes as I've made to local.cf, none of them had anything to do with RDJ. According to the home page for the script http://www.exit0.us/index.php?pagename=RulesDuJour Add a TRUSTED_RULESETS line to your config file that contains the names of the rulesets you chose. Example below: * TRUSTED_RULESETS="TRIPWIRE SARE_ADULT SARE_OBFU0 SARE_OBFU1 SARE_URI0 SARE_URI1" -- Jo Rhett Senior Network Engineer Network Consonance
Re: sa-update versus rulesdujour questions
On Oct 18, 2006, at 11:15 AM, Tim Litwiller wrote: I've never changed anything in local.cf when using RDJ - what did you have to change? Reading RDJ setup, it kept mentioning that I would have to add statements to local.conf for each and every ruleset that I imported. This is why I didn't bother, and used sa-update instead. I'm wondering if there is anything I'm missing... Jo Rhett wrote: Okay, there's no docs on this so I wanted to ask if someone has any insights different than what I have observed. SA-Update seems to require less configuration changes. In short, all I did was make a file with a list of rulefiles that SA-Update should check, and everything worked without any other changes. RDJ required lots of local.cf changes for no obvious gain that I could see. This is why I chose sa-update. RDJ does do the job of restarting the daemon for you, but it needed to be hacked if you were using amavisd. Is there any difference here that I'm overlooking? Any advantage to RDJ? And leading to my next point, given that sa-update is working fine -- isn't rdj going to be slimmed down to just the part that restarts the process after running sa-update? Why not? -- Jo Rhett Senior Network Engineer Network Consonance
Re: sa-update versus rulesdujour questions
On Oct 18, 2006, at 11:25 AM, Chris Santerre wrote: Last month this topic blew up into a flame fest. I compltely understand why no on wants in on this again. Oops. I should have checked the archive. Short answer: use what you like. Of course. I was just wondering if there was certain well known advantages or well known failures, etc. -- Jo Rhett Senior Network Engineer Network Consonance
Re: sa-update versus rulesdujour questions
Jake Vickers wrote: Tim Litwiller wrote: I've never changed anything in local.cf when using RDJ - what did you have to change? ... I use RDJ myself, and also did not add anything to my local.cf to make it work. I adjusted some scores, but that was all. Me three. As many changes as I've made to local.cf, none of them had anything to do with RDJ. -- Kelson Vibber SpeedGate Communications
Re: sa-update versus rulesdujour questions
Tim Litwiller wrote: I've never changed anything in local.cf when using RDJ - what did you have to change? Jo Rhett wrote: Hm. I'm surprised on no answers. Can I persist? This topic is of real interest to me... Jo Rhett wrote: Okay, there's no docs on this so I wanted to ask if someone has any insights different than what I have observed. SA-Update seems to require less configuration changes. In short, all I did was make a file with a list of rulefiles that SA-Update should check, and everything worked without any other changes. RDJ required lots of local.cf changes for no obvious gain that I could see. This is why I chose sa-update. RDJ does do the job of restarting the daemon for you, but it needed to be hacked if you were using amavisd. Is there any difference here that I'm overlooking? Any advantage to RDJ? And leading to my next point, given that sa-update is working fine -- isn't rdj going to be slimmed down to just the part that restarts the process after running sa-update? Why not? I use RDJ myself, and also did not add anything to my local.cf to make it work. I adjusted some scores, but that was all.
RE: sa-update versus rulesdujour questions
Title: RE: sa-update versus rulesdujour questions > > > Hm. I'm surprised on no answers. Can I persist? This topic > is of real > interest to me... Last month this topic blew up into a flame fest. I compltely understand why no on wants in on this again. Short answer: use what you like. --Chris
Re: sa-update versus rulesdujour questions
I've never changed anything in local.cf when using RDJ - what did you have to change? Jo Rhett wrote: Hm. I'm surprised on no answers. Can I persist? This topic is of real interest to me... Jo Rhett wrote: Okay, there's no docs on this so I wanted to ask if someone has any insights different than what I have observed. SA-Update seems to require less configuration changes. In short, all I did was make a file with a list of rulefiles that SA-Update should check, and everything worked without any other changes. RDJ required lots of local.cf changes for no obvious gain that I could see. This is why I chose sa-update. RDJ does do the job of restarting the daemon for you, but it needed to be hacked if you were using amavisd. Is there any difference here that I'm overlooking? Any advantage to RDJ? And leading to my next point, given that sa-update is working fine -- isn't rdj going to be slimmed down to just the part that restarts the process after running sa-update? Why not?
Re: sa-update versus rulesdujour questions
Hm. I'm surprised on no answers. Can I persist? This topic is of real interest to me... Jo Rhett wrote: Okay, there's no docs on this so I wanted to ask if someone has any insights different than what I have observed. SA-Update seems to require less configuration changes. In short, all I did was make a file with a list of rulefiles that SA-Update should check, and everything worked without any other changes. RDJ required lots of local.cf changes for no obvious gain that I could see. This is why I chose sa-update. RDJ does do the job of restarting the daemon for you, but it needed to be hacked if you were using amavisd. Is there any difference here that I'm overlooking? Any advantage to RDJ? And leading to my next point, given that sa-update is working fine -- isn't rdj going to be slimmed down to just the part that restarts the process after running sa-update? Why not? -- Jo Rhett Network/Software Engineer Net Consonance
sa-update versus rulesdujour questions
Okay, there's no docs on this so I wanted to ask if someone has any insights different than what I have observed. SA-Update seems to require less configuration changes. In short, all I did was make a file with a list of rulefiles that SA-Update should check, and everything worked without any other changes. RDJ required lots of local.cf changes for no obvious gain that I could see. This is why I chose sa-update. RDJ does do the job of restarting the daemon for you, but it needed to be hacked if you were using amavisd. Is there any difference here that I'm overlooking? Any advantage to RDJ? And leading to my next point, given that sa-update is working fine -- isn't rdj going to be slimmed down to just the part that restarts the process after running sa-update? Why not? -- Jo Rhett Senior Network Engineer Network Consonance
