The following issue has been RESOLVED. ====================================================================== http://www.dbmail.org/mantis/view.php?id=149 ====================================================================== Reported By: paul Assigned To: paul ====================================================================== Project: DBMail Issue ID: 149 Category: IMAP daemon Reproducibility: always Severity: feature Priority: normal Status: resolved Resolution: fixed Fixed in Version: SVN Trunk ====================================================================== Date Submitted: 07-Jan-05 16:04 CET Last Modified: 14-Feb-06 16:54 CET ====================================================================== Summary: imap search performance Description: Dbmail's imap search performance is well below par. The whole search implementation needs to be redesigned. 1) more search types should be sql-based 2) sub-searches should be limited if earlier sub-searches already restricted the result-set and the searchtype is boolean and.
04 UID SEARCH 1:10 HEADER MESSAGE-ID <[EMAIL PROTECTED]> currently combines three independant result-sets that all three encompass the whole mailbox involved. This means that in this case all messages are parsed to find the message-id, instead of only parsing message-ids 1:10. ====================================================================== Relationships ID Summary ---------------------------------------------------------------------- duplicate of 0000113 IMAP Search should parse only headers a... ====================================================================== ---------------------------------------------------------------------- aaron - 12-Jan-05 19:36 ---------------------------------------------------------------------- Paul, is this the work you have ready already? ---------------------------------------------------------------------- paul - 19-Jan-05 10:42 ---------------------------------------------------------------------- I'm testing a patch for search only. Search is quite simple because the order of the search keys will generally be from cheap to expensive: SEQ-RANGE/FLAGS/PARSED like A01 UID SEARCH 1:100 RECENT HEADER FROM <[EMAIL PROTECTED]> For sort things are not so simple. Generally sort will be performed using a header field that requires parsing. Currently *all* messages in the mailbox are parsed, before moving on to the next search key, and combining the sets. A02 UID SORT (FROM) US-ASCII RECENT HEADER FROM <[EMAIL PROTECTED]> What I want to do is fix build_imap_search so that the sk->sub_search lists are ordered by the expected cost of the searchkey. That way I can set it up so that matching is always done: SEQ-RANGE/FLAGS/PARSED so only messages that are not already excluded will actually be parsed. ---------------------------------------------------------------------- paul - 20-Jan-05 13:33 ---------------------------------------------------------------------- I've just committed a patch to 2.0 and head that fixes this issue. Searchlists are now optimized by using a weighted search cost for reordering the searchlist. The sorting of the searchlist is done brute force. Fancy btree sorting seemed like overkill. Assuming this code is bug-free, there isn't a whole lot more to do to improve search/sort speeds until we fix the recent-flag bug (already done in cvs-head), and implement the header table which will allow more sql-based searches. ---------------------------------------------------------------------- aaron - 18-Mar-05 17:33 ---------------------------------------------------------------------- Ready to close this bug? ---------------------------------------------------------------------- paul - 19-Mar-05 19:40 ---------------------------------------------------------------------- this bug is ready to close if I finish the header_tables code in trunk and the search code uses those tables. ---------------------------------------------------------------------- paul - 22-Aug-05 10:34 ---------------------------------------------------------------------- work in progress Issue History Date Modified Username Field Change ====================================================================== 07-Jan-05 16:04 paul New Issue 07-Jan-05 16:05 paul Projection none => major rework 07-Jan-05 16:05 paul ETA none => > 1 month 07-Jan-05 16:05 paul Description Updated 12-Jan-05 19:36 aaron Note Added: 0000523 19-Jan-05 10:42 paul Note Added: 0000551 20-Jan-05 13:33 paul Note Added: 0000563 18-Mar-05 17:33 aaron Note Added: 0000617 19-Mar-05 19:40 paul Note Added: 0000621 22-Aug-05 10:34 paul Note Added: 0000857 22-Aug-05 10:34 paul Assigned To => paul 22-Aug-05 10:34 paul Relationship added duplicate of 0000113 22-Aug-05 10:34 paul Status new => assigned 22-Aug-05 10:34 paul Projection major rework => minor fix 22-Aug-05 10:34 paul ETA > 1 month => < 1 month 22-Aug-05 10:34 paul version => SVN Trunk 14-Feb-06 16:54 paul Status assigned => resolved 14-Feb-06 16:54 paul Fixed in Version => SVN Trunk 14-Feb-06 16:54 paul Resolution open => fixed ======================================================================
