The following bug has been ASSIGNED. ====================================================================== http://www.dbmail.org/mantis/bug_view_advanced_page.php?bug_id=0000149 ====================================================================== Reported By: paul Assigned To: paul ====================================================================== Project: DBMail Bug ID: 149 Category: IMAP daemon Reproducibility: always Severity: feature Priority: normal Status: assigned ====================================================================== Date Submitted: 07-Jan-05 16:04 CET Last Modified: 22-Aug-05 10:34 CEST ====================================================================== Summary: imap search performance Description: Dbmail's imap search performance is well below par. The whole search implementation needs to be redesigned. 1) more search types should be sql-based 2) sub-searches should be limited if earlier sub-searches already restricted the result-set and the searchtype is boolean and.
04 UID SEARCH 1:10 HEADER MESSAGE-ID <[EMAIL PROTECTED]> currently combines three independant result-sets that all three encompass the whole mailbox involved. This means that in this case all messages are parsed to find the message-id, instead of only parsing message-ids 1:10. ====================================================================== ---------------------------------------------------------------------- aaron - 12-Jan-05 19:36 CET ---------------------------------------------------------------------- Paul, is this the work you have ready already? ---------------------------------------------------------------------- paul - 19-Jan-05 10:42 CET ---------------------------------------------------------------------- I'm testing a patch for search only. Search is quite simple because the order of the search keys will generally be from cheap to expensive: SEQ-RANGE/FLAGS/PARSED like A01 UID SEARCH 1:100 RECENT HEADER FROM <[EMAIL PROTECTED]> For sort things are not so simple. Generally sort will be performed using a header field that requires parsing. Currently *all* messages in the mailbox are parsed, before moving on to the next search key, and combining the sets. A02 UID SORT (FROM) US-ASCII RECENT HEADER FROM <[EMAIL PROTECTED]> What I want to do is fix build_imap_search so that the sk->sub_search lists are ordered by the expected cost of the searchkey. That way I can set it up so that matching is always done: SEQ-RANGE/FLAGS/PARSED so only messages that are not already excluded will actually be parsed. ---------------------------------------------------------------------- paul - 20-Jan-05 13:33 CET ---------------------------------------------------------------------- I've just committed a patch to 2.0 and head that fixes this issue. Searchlists are now optimized by using a weighted search cost for reordering the searchlist. The sorting of the searchlist is done brute force. Fancy btree sorting seemed like overkill. Assuming this code is bug-free, there isn't a whole lot more to do to improve search/sort speeds until we fix the recent-flag bug (already done in cvs-head), and implement the header table which will allow more sql-based searches. ---------------------------------------------------------------------- aaron - 18-Mar-05 17:33 CET ---------------------------------------------------------------------- Ready to close this bug? ---------------------------------------------------------------------- paul - 19-Mar-05 19:40 CET ---------------------------------------------------------------------- this bug is ready to close if I finish the header_tables code in trunk and the search code uses those tables. ---------------------------------------------------------------------- paul - 22-Aug-05 10:34 CEST ---------------------------------------------------------------------- work in progress Bug History Date Modified Username Field Change ====================================================================== 07-Jan-05 16:04paul New Bug 07-Jan-05 16:05paul Description Updated 07-Jan-05 16:05paul ETA none => > 1 month 07-Jan-05 16:05paul Projection none => major rework 12-Jan-05 19:36aaron Bugnote Added: 0000523 19-Jan-05 10:42paul Bugnote Added: 0000551 20-Jan-05 13:33paul Bugnote Added: 0000563 18-Mar-05 17:33aaron Bugnote Added: 0000617 19-Mar-05 19:40paul Bugnote Added: 0000621 22-Aug-05 10:34paul Bugnote Added: 0000857 22-Aug-05 10:34paul duplicate_id 0 => 113 22-Aug-05 10:34paul ETA > 1 month => < 1 month 22-Aug-05 10:34paul Assigned To => paul 22-Aug-05 10:34paul Projection major rework => minor fix 22-Aug-05 10:34paul Status new => assigned 22-Aug-05 10:34paul version => SVN Trunk ======================================================================
