Robert Menschel writes: > Wednesday, October 5, 2005, 6:11:26 PM, you wrote: > JM> "Daryl C. W. O'Shea" writes: > >> Sounds good, but I think the limited (and relatively static?) corpus may > >> be an issue for rule development aimed at catching new spam signs. > > JM> A static-ish ham corpus isn't a big problem, but we may need to supplement > JM> the spam corpus with fresh feeds of new spam. It should be possible to do > JM> this either from trap feeds, or via submissions from the nightly corpus > JM> submitters (rsync up bits of your corpus as you see fit). Traps is > JM> probably easier. > > Yes, a ham corpus has some privacy concerns, but a spam corpus, > especially spam sent to non-existant addresses captured via a > catch-all account, has almost none. > > I believe it'd be feasible for me to commit 5k or so spam weekly. > > Given the almost instant feedback of the preflight buildbot, I can > easily see a SARE Ninja like (well, no, let's not mention any names) > submitting a rules test at 6:00, modifying the rule and resubmitting > at 7:00, again at 8:00, again at noon, again at 4:00 pm, and again > shortly before the nightly corpus run. > > It'd be good to be able to specify that the nightly corpus run should > test only the last version. it'd also be good to be able to specify > that the nightly corpus run should test more than one version. Any > way to do that?
I think it could be tricky to do that. By default it'll test the "last version", ie nearest to the "tag time" when the nightly-check label is attached to the SVN tree. An easy way to do it, though, is just to copy the rule! Standard practice for us in testing with the nightly mass-checks, in the past, has been to make maybe 10 variants on a rule -- and then once the results are all in the next day, we can compare the results for all 10. > I'm thinking that not only should we be able to submit rules, but also > submit control parameters such as "preflight run 05-10-08-04-44 rule > TEST_STOCK_EXPLODES -- don't bother" to remove replaced/useless rules > from the nightly queue. ...or just comment the rule in the source ;) If the rule isn't in the source when the tag is applied, it won't get run. > Finally, according to > http://wiki.apache.org/spamassassin/RulesProjBuildBot, the BuildBot > provides "Good web UI for "builds in progress"; you can monitor > progress as it happens", but I can't see where to review that > progress. Any pointers? here's the web UI -- http://buildbot.spamassassin.org:8010/ . (damn, tests are failing!! need to fix that) If you trigger a build by clicking on a slave's name (e.g. "t-solaris-10"), then "Force Build", then click on the "log" link of the now-started build in progress, you can see the build's progress, live. Seriously... anything output by the build scripts is viewable in your browser, as it happens, without a reload. It's very nifty! --j.
