Re: New expensive Regexps

2014-02-10 Thread Kevin A. McGrail
On 2/7/2014 7:49 AM, Torge Husfeldt wrote: Hi list, I hope I triggered a constructive and useful discussion here. In between I realized my data was skewed and I wanted to apologize for that. The peak in load happened when I rolled out the rule-set which was running fine on one machine to the

Re: New expensive Regexps

2014-02-07 Thread Torge Husfeldt
Am 07.02.2014 07:09, schrieb Axb: On 02/07/2014 03:04 AM, Kevin A. McGrail wrote: On 2/6/2014 8:32 PM, Dave Warren wrote: On 2014-02-06 17:17, John Hardin wrote: On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process

New expensive Regexps

2014-02-06 Thread Torge Husfeldt
Hi List, recently, we're experiencing very high loads on our spamassassin-cluster. What struck us in the search for a possible culprits were the recent addition of the tests named SINGLE_HEADER_\dK All of which haver extremely low scores in our contect (nonet, nobayes). From our point of view

Re: New expensive Regexps

2014-02-06 Thread Axb
On 02/06/2014 12:38 PM, Torge Husfeldt wrote: Hi List, recently, we're experiencing very high loads on our spamassassin-cluster. What struck us in the search for a possible culprits were the recent addition of the tests named SINGLE_HEADER_\dK All of which haver extremely low scores in our

Re: New expensive Regexps

2014-02-06 Thread Kevin A. McGrail
On 2/6/2014 6:38 AM, Torge Husfeldt wrote: recently, we're experiencing very high loads on our spamassassin-cluster. What struck us in the search for a possible culprits were the recent addition of the tests named SINGLE_HEADER_\dK All of which haver extremely low scores in our contect

Re: New expensive Regexps

2014-02-06 Thread Kevin A. McGrail
On 2/6/2014 8:17 PM, John Hardin wrote: On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process is the following: - we measure and report on metrics for the rules that are promoted such as rank (existing), computational

Re: New expensive Regexps

2014-02-06 Thread John Hardin
On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process is the following: - we measure and report on metrics for the rules that are promoted such as rank (existing), computational expense, time spent on rule. I assume

Re: New expensive Regexps

2014-02-06 Thread Kevin A. McGrail
On 2/6/2014 8:51 PM, Daniel Staal wrote: I would probably give the meta-rule no cost - add up the cost of the components if you want it. (With the understanding that all no-cost rules are meta rules.) Meta rules are a scenario that has to be considered for sure. This is good discussion and

Re: New expensive Regexps

2014-02-06 Thread Dave Warren
On 2014-02-06 17:17, John Hardin wrote: On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process is the following: - we measure and report on metrics for the rules that are promoted such as rank (existing), computational

Re: New expensive Regexps

2014-02-06 Thread Kevin A. McGrail
On 2/6/2014 8:32 PM, Dave Warren wrote: On 2014-02-06 17:17, John Hardin wrote: On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process is the following: - we measure and report on metrics for the rules that are

Re: New expensive Regexps

2014-02-06 Thread Daniel Staal
--As of February 6, 2014 5:32:47 PM -0800, Dave Warren is alleged to have said: On 2014-02-06 17:17, John Hardin wrote: On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process is the following: - we measure and report

Re: New expensive Regexps

2014-02-06 Thread Dave Warren
On Feb 6, 2014, at 18:04, Kevin A. McGrail kmcgr...@pccc.com wrote: On 2/6/2014 8:32 PM, Dave Warren wrote: On 2014-02-06 17:17, John Hardin wrote: On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process is the

Re: New expensive Regexps

2014-02-06 Thread Kevin A. McGrail
On 2/6/2014 9:11 PM, Dave Warren wrote: Without triple checking the code, my 99.9% belief is Rules are cached. Calling them multiple times does not trigger a re-check. I believe so too, which is why this matters. If they were re-evaluated, you could just sum up a meta rule and not care.

Re: New expensive Regexps

2014-02-06 Thread Axb
On 02/07/2014 03:04 AM, Kevin A. McGrail wrote: On 2/6/2014 8:32 PM, Dave Warren wrote: On 2014-02-06 17:17, John Hardin wrote: On Thu, 6 Feb 2014, Kevin A. McGrail wrote: I've discussed it with Alex a bit but one of my next ideas for the Rules QA process is the following: - we measure and