On Thu, Aug 20, 2009 at 01:20:41PM +0400, Andrey wrote: > > Can anyone help me to find a fastest method to search in exim logs and > get relevant message sessions or errors for particular from, to or both > fields. The problem is that our exim logs are huge (>300Mb) and exigrep > does not fast in that case. > > As I understand, exigrep is constructing 2 hash tables based on log > file. First contains relevant message session messages by id and second > contains flags if it contains searching pattern. Then it prints out only > hash records from first table if flag=1. But the problem is that exigrep > reads exim log file line by line and unfortunately it is not good > solution in case of huge logs. > > Is there faster algorithm to perform search in exim logs based on > from,to or both fields. Also I need to print error messages that are not > relevant on message id, for example too many recipients messages.
I regularly search through bzipped logs which in bzip-format is combined probably 1.2Gb or bigger. The combination of bzip2 and exigrep is working for me. Another tool we are using is exilog which saves the logs from our three mail servers remotely in a postgresql database. Using exilog for basic queries is quite fast if you want to look for spesific emails. However it does not record log entries of aborted connections due to errors. Regards Johann -- Johann Spies Telefoon: 021-808 4599 Informasietegnologie, Universiteit van Stellenbosch "But I would not have you to be ignorant, brethren, concerning them which are asleep, that ye sorrow not, even as others which have no hope. For if we believe that Jesus died and rose again, even so them also which sleep in Jesus will God bring with him." I Thessalonians 4:13,14 -- ## List details at http://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/