On 4/6/2010 1:10 PM, Mark Martinec wrote:
> Peter,
>
>> I recently implemented amavisd-new (amavisd-new-2.6.4_5,1) on a FreeBSD
>> mail server. I am using both postfix and spamassassin with it. I have 2
>> problems happening and I am not sure how to start figuring this out. I
>> apologize if this is trivial.
>>
>> 1. /var/amavis/tmp has 7,000 (and growing) leftover directories in it.
>>
>> I have turned up my log level to 5 and watched my mail log. I can see
>> that it is definitely cleaning up some of the dirs, ie:
>> amavis[8032]: (08032-01) rmdir_recursively:
>> /var/amavis/tmp/amavis-20100406T102829-08032/parts, excl=1
>>
>> When I sample the email inside some of those leftover directories I find
>> what looks like a bulk email. It is unclear to me if it is true spam or
>> not. I also see this occasionally in the maillog:
>>
>> amavis[8540]: (08540-17) (!)PRESERVING EVIDENCE in
>> /var/amavis/tmp/amavis-20100406T104452-08540
>>
>> That particular email.txt looks like a bulk email. It is an ad from
>> Beliefnet - Daily Specials. Not necessarily spam.
>
> Grep the log for '08540-17' and see what went wrong, i.e. what caused
> the later preserving of evidence. The cause should be fixed.
>
>
I followed up on that error and I forgot to make a particular directory
that amavis was trying to use. Thank you for helping me to find that. I
still am accumulating directories in the /var/amavis/tmp directory
though. I am 8,000 now. I no longer have the "PRESERVING EVIDENCE" error
in my logs though. I have this in my amavis.conf:
$max_servers = 5; # number of pre-forked children
$max_requests = 20; # retire a child after that many accepts
I realize that while it is working on something you will see
directories. With 5 max servers how many directories can/will amavis
create simultaneously? Should the 8,000 directories be a concern? What
other reasons are there for these directories being abandoned? From what
I could find it seems like they should be being removed/pruned
automatically.
>> I see references in the documentation to not letting this directory fill
>> up too much. I am not sure what "too much" is. Is this "normal" behavior?
>>
>> 2. The $QUARANTINEDIR = '/var/virusmails' has 1400 (and growing) entries
>> in it. Only about 50 of those are virus-xxx the rest are spam-xxx.
>> Reviewing them they all look legitimate as spam and viruses. My question
>> is again, how much is "normal" or to be expected? Is this a directory
>> that I need to clean out with some cron job?
>
> Depends on a file system in use. For ZFS for example the limit is
> huge: 2^48 — Number of entries in any individual directory .
>
> When quarantining as files, it is common to insert one layer of
> subdirectories below the /var/virusmails, so as to reduce the number
> of files within each individual directory. The settings is:
>
> $quarantine_subdir_levels = 1; # 0 disables, 1 enables
>
> It adds 62 one-char subdirectories below $QUARANTINEDIR,
> and stores each quarantined file in a subdirectory matching the
> first character of a filename.
>
>
> Instead of, or in addition to $quarantine_subdir_levels, it is
> also useful to structure/subdivide the quarantine based on a
> partition tag, such as a week-of-the-year number. This makes
> purging of old quarantine files possibly easier or faster:
>
> amavisd.conf:
>
> $sql_partition_tag =
> sub { my($msginfo)=...@_;
> sprintf("%02d",iso8601_week($msginfo->rx_time)) };
>
> $virus_quarantine_method = 'local:W%P/virus/%m';
> $spam_quarantine_method = 'local:W%P/spam/%m.gz';
>
> The %P in the template will be replaced by a result of executing a
> subroutine stored in $sql_partition_tag. (do not be distracted by
> 'sql' in the name, the feature works even without SQL).
>
> Btw, the directories W01, W02, ... W53 under $QUARANTINEDIR (or wherever
> a template indicates) must be created manually.
>
> Mark
>
I have now sub categorized /var/virusmails/ the with your
sql_partition_tag. That is really handy. Thank you. I take it from your
response that the real problem ("too many" files) comes in with regards
to the maximum possible files in a directory for a file system and not
something in amavisd having some practical limit for processing speed?
Or am I understanding that incorrectly?
Thank you, again.
Peter
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
AMaViS-user mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/
