On Jan 7, 2008 8:43 AM, Sidney Markowitz <[EMAIL PROTECTED]> wrote: > I haven't done much with rules testing and I seem to be confused. > > I once created a sandbox and experimented with some rules. When I was done I > deleted them. > Recently I created some new rules, putting them into > rules/trunk/sandbox/sidney/72_other.cf and committing it. > > When I look at the Rule QA web page the only rules I find named T_SIDNEY* are > the old ones > which should no longer exist. I don't see them anywhere in my svn working > directory tree.
found it: this is not serious. basically, when the rule-QA backend runs "hit-frequencies", it does so in a checkout containing rule files which were last compiled back in July. hit-frequencies has code to handle listing zero-hit rules, by getting the entire list of all known rules in ../rules; if any of those rules is not hit in the mass-check logs, it'll still be output in the "freqs" listing, but with 0 hits. in this case, that includes your old rules. not sure how to fix this, other than either: - not listing zero-hitting rules. but that's confusing if someone checks in a *new* zero-hitter and wants to see the freqs! - running "svn up" in that hit-frequencies checkout. will investigate this. > The wiki page http://wiki.apache.org/spamassassin/RuleSandboxes says that the > sandbox is > under trunk/rulesrc/sandbox, while > http://wiki.apache.org/spamassassin/RulesProjPromotion > says it is rules/trunk/sandbox. However they seem to contain the same thing, > I guess due > to something that corresponds to a symlink in an svn repository? > > In any case the rules I committed do not show up in Rule QA and the ones I no > longer see > anywhere in my svn tree do show up. ok, I think it was just a time delay thing. when I search for /SIDNEY they show up now: http://ruleqa.spamassassin.org/?daterev=20080107-r609531-n&rule=%2FSIDNEY To explain: here's what can happen. At 0850 UTC the nightly tag happens, then at 0900 UTC the various mass-checks all start. As the day progresses, mass-check results arrive and are put into ruleqa's output. However, it's not until a certain point well into the day, that we can reasonably guess that that nightly's results are all in. Now, in your case, your new rules were added in r609296, 2008-01-06 11:53:49 +0000. That's just *after* the nightly tag for the day -- the worst possible timing! ;) so they didn't show up in a nightly until the next day, today: 20080107-r609531-n . When you took a look earlier on today, that mass-check wasn't considered "last-night" since it was too fresh, and there were not going to be enough results in. (It is now, though.) This is kind of unavoidable, given that we only tag for mass-checks once per day, and it takes several hours to run them :( here's a tip: on the front page, click on 'today's nightly run' for the most recent nightly -- typically this is only partially completed (depending on when you view it) but is more likely to have your up-to-date rule changes. alternatively, every checkin gets an "mc-fast" preflight mass-check, straight away: http://ruleqa.spamassassin.org/?daterev=20080106-r609296-b&rule=&srcpath=sidney&g=Change but those will typically have slightly different (and less reliable) results. I'd like to fix that by improving the mc-fast corpus sometime.... --j. > > Help? > > -- sidney > >
