On Fri, Oct 27, 2006 at 03:01:45PM -0700, Sammy Anderson wrote: > I manually ran sa-learn --force-expire, and it hammered the box. Here is a > debug and timing information (for just a 5 MB file!): > > [18002] dbg: bayes: token count: 161725, final goal reduction size: 49225
want to get rid of (max) 49225 tokens > [18002] dbg: bayes: can't use estimation method for expiry, unexpected > result, calculating optimal atime delta (first pass) have to do step 1 and can't estimate > [18002] dbg: bayes: expiry max exponent: 9 > ------ about 20 seconds elapsed it's going through every token in your db > [18002] dbg: bayes: atime token reduction > [18002] dbg: bayes: ======== =============== > [18002] dbg: bayes: 43200 144256 > [18002] dbg: bayes: 86400 133029 > [18002] dbg: bayes: 172800 111350 > [18002] dbg: bayes: 345600 72306 > [18002] dbg: bayes: 691200 9457 > [18002] dbg: bayes: 1382400 0 [...] > [18002] dbg: bayes: first pass decided on 691200 for atime delta 691200 wins the Price Is Right (9457 is the closest without going over) > ------ about 40 seconds elapsed [a sort going on here???] It's creating a new DB file, going back through every token in the original DB, and for any that are newer than 9457 seconds ago, it copies the entry to the new DB. > expired old bayes database entries in 60 seconds <= YIKES yep. expiry is relatively resource intensive and slow w/ DBMs, but there's no other good way to do it (or at least, no one has suggested a really better way to do it...) -- Randomly Selected Tagline: I believe it's not butter, I just can't believe it's $1.59!
pgpFcu5EsuOzk.pgp
Description: PGP signature
