Helmut Schneider wrote: > with certain mails on FreeBSD 8.0 and SA 3.3.1 I have a performance > problem:
I might have been able to "catch" a non-confident example mail[1] (bad example because of the size, but an example). While SA 3.2.5 needs ~45 seconds, with SA 3.3.1: Jun 4 03:36:41.029 [56496] dbg: timing: total 103627 ms - init: 1615 (1.6%), parse: 52 (0.0%), extract_message_metadata: 1971 (1.9%), poll_dns_idle: 772 (0.7%), get_uri_detail_list: 348 (0.3%), tests_pri_-1000: 467 (0.5%), compile_gen: 292 (0.3%), compile_eval: 35 (0.0%), tests_pri_-950: 23 (0.0%), tests_pri_-900: 28 (0.0%), tests_pri_-400: 35 (0.0%), tests_pri_0: 99167 (95.7%), dkim_load_modules: 59 (0.1%), check_dkim_signature: 26 (0.0%), check_razor2: 2061 (2.0%), check_pyzor: 1.52 (0.0%), tests_pri_500: 188 (0.2%) Jun 4 03:35:02.226 [56496] dbg: rules: run_generic_tests - compiling eval code: body, priority 0 Jun 4 03:35:02.227 [56496] dbg: rules: compiled body tests Jun 4 03:35:18.067 [56496] dbg: rules: ran body rule __I_INHERIT ======> got hit: "I inherited" Jun 4 03:35:23.007 [56496] dbg: rules: ran body rule __YOU_HAVE_WON ======> got hit: "you won" Jun 4 03:35:24.733 [56496] dbg: rules: ran body rule __FB_MA ======> got hit: "MA" Jun 4 03:35:24.964 [56496] dbg: rules: ran body rule __MASTERS ======> got hit: "Masters" Jun 4 03:35:32.730 [56496] dbg: rules: ran body rule __DOS_BODY_WED ======> got hit: "Wednesday" Jun 4 03:35:33.415 [56496] dbg: rules: ran body rule __FRAUD_IRJ ======> got hit: "holding company" Jun 4 03:35:34.848 [56496] dbg: rules: ran body rule __KAM_LOTTO3 ======> got hit: "claim" Jun 4 03:35:35.281 [56496] dbg: rules: ran body rule __MILLIONS ======> got hit: "millions of dollar" Jun 4 03:35:36.387 [56496] dbg: rules: ran body rule __DEAL ======> got hit: "the deal" Jun 4 03:35:41.524 [56496] dbg: rules: ran body rule __FB_NATIONAL ======> got hit: "National" Jun 4 03:35:46.119 [56496] dbg: rules: ran body rule __F_LARGE_MONEY_2 ======> got hit: "10 million" Jun 4 03:36:04.770 [56496] dbg: rules: ran body rule __MBA ======> got hit: "mba" Jun 4 03:36:06.071 [56496] dbg: rules: ran body rule __DOS_BODY_FRI ======> got hit: "Friday" Jun 4 03:36:07.273 [56496] dbg: rules: ran body rule __DOS_LINK ======> got hit: "link" Jun 4 03:36:09.234 [56496] dbg: rules: ran body rule __DOS_BODY_SAT ======> got hit: "sat" Jun 4 03:36:09.838 [56496] dbg: rules: ran body rule __FILL_THIS_FORM_FRAUD_PHISH ======> got hit: "password. Jun 4 03:36:09.839 [56496] dbg: rules: [...] " Jun 4 03:36:15.269 [56496] dbg: rules: ran body rule __SUBSCRIPTION_INFO ======> got hit: "opt out" Jun 4 03:36:15.521 [56496] dbg: rules: ran body rule __HAS_ANY_EMAIL ======> got hit: "m...@wsj.c" Jun 4 03:36:16.798 [56496] dbg: rules: ran body rule __FB_NUM_PERCNT ======> got hit: "2%" Jun 4 03:36:16.998 [56496] dbg: rules: ran body rule __YOU_WON_01 ======> got hit: "you won" Jun 4 03:36:17.226 [56496] dbg: rules: ran body rule __NONEMPTY_BODY ======> got hit: "A" Jun 4 03:36:21.053 [56496] dbg: rules: ran body rule __FB_PICK ======> got hit: "pick" Jun 4 03:36:23.851 [56496] dbg: rules: ran body rule __FB_GAME ======> got hit: "Game" Jun 4 03:36:30.641 [56496] dbg: rules: ran body rule __FRAUD_DBI ======> got hit: "dollars" Jun 4 03:36:34.278 [56496] dbg: rules: ran body rule __F_LARGE_MONEY ======> got hit: "200,000" Jun 4 03:36:36.247 [56496] dbg: rules: ran body rule __HUSH_HUSH ======> got hit: "private" Jun 4 03:36:36.785 [56496] dbg: rules: ran body rule __LOTSA_MONEY_03 ======> got hit: "$300 million" Jun 4 03:36:38.060 [56496] dbg: rules: ran body rule __FB_S_PRICE ======> got hit: "price" Jun 4 03:36:38.066 [56496] dbg: async: select found 1 responses ready (t.o.=0.0) Jun 4 03:36:38.066 [56496] dbg: async: completed in 96.650 s: URI-DNSBL, DNSBL:zen.spamhaus.org.:2.46.246.72 Jun 4 03:36:38.067 [56496] dbg: dns: harvested completed queries Jun 4 03:36:38.068 [56496] dbg: rules: running uri tests; score so far=1.206 sa-compile didn't make a difference: Jun 4 04:15:10.870 [84689] dbg: timing: total 103477 ms - init: 1573 (1.5%), parse: 40 (0.0%), extract_message_metadata: 1196 (1.2%), poll_dns_idle: 6 (0.0%), get_uri_detail_list: 333 (0.3%), tests_pri_-1000: 468 (0.5%), compile_gen: 262 (0.3%), compile_eval: 28 (0.0%), tests_pri_-950: 29 (0.0%), tests_pri_-900: 40 (0.0%), tests_pri_-400: 25 (0.0%), tests_pri_0: 99833 (96.5%), dkim_load_modules: 58 (0.1%), check_dkim_signature: 20 (0.0%), check_razor2: 266 (0.3%), check_pyzor: 3 (0.0%), tests_pri_500: 196 (0.2%) I also can reproduce this with Ubuntu 10.4. [1] http://www.charlieroot.de/downloads/email.txt -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
