Re: [AMaViS-user] optimizations if partition_tag indexes

2009-09-14 Thread Thomas Gelf
LWAYS get it by chunk_ind order? I would not bet on this ;-) However, while ORDER is usually a very "bad" command, it should not hurt much in this case, as the resultset is very small. Best regards, Thomas Gelf -- mail: tho...@gelf.net web: http://thomas.gelf.net/ -

[AMaViS-user] Mail log aggregation daemon for Postfix and Amavis

2009-09-08 Thread Thomas Gelf
opinions and to get some feedback! Best regards, Thomas Gelf NB: No, I'm not sorry for spamming and crossposting ;-p -- mail: tho...@gelf.net web: http://thomas.gelf.net/ -- Let Crystal Reports handle the re

Re: [AMaViS-user] applying sql based white/blacklists to more than one recipient

2009-09-06 Thread Thomas Gelf
by defining your very personal variants in your amavisd.conf. Best regards, Thomas Gelf -- mail: tho...@gelf.net web: http://thomas.gelf.net/ -- Let Crystal Reports handle the reporting - Free Crystal Reports

Re: [AMaViS-user] Amavis Directory getting clogged

2009-09-02 Thread Thomas Gelf
Hi Mark! Mark Martinec wrote: > Unless you really need to shave off the few milliseconds, I'd suggest > NOT to move temp directory to a ram disk, which is typically much > smaller than a real disk partition, so can fill up much faster and > block mail flow if something unpredictable happens. Rega

Re: [AMaViS-user] Message reject after invalid or no DKIM

2009-08-19 Thread Thomas Gelf
Thomas Gelf wrote: > Using ADSP will save you a lot of manual work - manually configuring > DKIM behaviour per domain makes no fun. NB: ADSP has been published as RFC a few days ago as RFC5617 http://tools.ietf.org/search/r

Re: [AMaViS-user] Message reject after invalid or no DKIM

2009-08-19 Thread Thomas Gelf
how_bug.cgi?id=6087 Regards, Thomas Gelf -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do b

Re: [AMaViS-user] Mysql 5.1 + Partitioning Storage Schema / Cleanup

2009-08-14 Thread Thomas Gelf
Mark Martinec wrote: >> I'm actually kinda leaning towards 52 >> partitions at this point for maximum speed and flexibility >> while using iso8601 week as the value. > > Not to forget that some years can have 53 weeks (1..53) > according to ISO 8601. >> 52 partitions -> I've also been evaluating

Re: [AMaViS-user] Mysql 5.1 + Partitioning Storage Schema / Cleanup

2009-08-14 Thread Thomas Gelf
TABLE quarantine DROP PARTITION p3; > ALTER TABLE quarantine ADD PARTITION (PARTITION p3 VALUES IN > (4,8,12,16,20,24,28,32,36,40,44,48,52)); Will work as expected. Cleanup took 4 entire days 'til being finished on the last slave before using partitions. Now it takes just some seconds :-

Re: [AMaViS-user] How to identify SPAM and SPAMMY mail in SQL

2009-08-03 Thread Thomas Gelf
ction when comparing values. If running a large DB you should definitively change column types, as using string functions in your WHERE statements is seldom a good idea (performance-wise). Best regards, Thomas Gelf Luis Daniel Lucio Quiroz wrote: > Hi all, > > Having SQL schema, in wha

Re: [AMaViS-user] Amavis should not send DSN if D_REJECT is active

2009-07-09 Thread Thomas Gelf
Peer Heinlein wrote: > Am Donnerstag 09 Juli 2009 schrieb Thomas Gelf: > >> As far as I remember all docs I used long time ago to set up my first >> Amavis boxes clearly state that it is absolutely necessary to avoid >> REJECTs by the postfix instance Amavis is rei

Re: [AMaViS-user] Amavis should not send DSN if D_REJECT is active

2009-07-08 Thread Thomas Gelf
ecks AFTER pre_queue filters. I do not have any experience with such setups, so no idea if this assumption is true. Nonetheless I cannot immagine any special case where this DSNs (in case of REJECT) from Amavis would make sense - so as you did I'd also opt for suppressing them. Best regards,

Re: [AMaViS-user] mysql partitioning setup cheat sheet?

2009-07-07 Thread Thomas Gelf
Michael Scheidell wrote: > I guess you don't need the 'bigint(20) on id if the id is unique only on > each partition, and you need to add partition_tag to primary key (since > a list partition has to use primary keys) Yes and no: primary key is defined as PRIMARY KEY (id, partition_tag), however

Re: [AMaViS-user] How to log the 'helo' domain passed from upstream.

2009-07-07 Thread Thomas Gelf
Xueron Nee wrote: > Is there any macros can be used in log_templ point to the Helo/Ehlo > domain passed from upstream? Not sure how your setup looks like - but usually this is a task that should be accomplished by your front MX server (Postfix of whatever). Regards, Thoma

Re: [AMaViS-user] Amavis-new and dual master replication with MySQL

2009-07-07 Thread Thomas Gelf
tion from 4 days to 4 seconds. Regards, Thomas Gelf -- Enter the BlackBerry Developer Challenge This is your chance to win up to $100,000 in prizes! For a limited time, vendors submitting new applications to BlackBerry Ap

Re: [AMaViS-user] mysql partitioning setup cheat sheet?

2009-07-03 Thread Thomas Gelf
Michael Scheidell wrote: > I looked at the readme's and some of the stuff Thomas Gelf wrote about > mysql partitioning and amavisd partitioning, and tried a couple of them > without much success. > > Did anyone write a 'setup/ quarantine cleanup' cheat sheet

Re: [AMaViS-user] Brainstorm request - scaling out Penpals

2009-06-15 Thread Thomas Gelf
#x27;ve never used it before. To accomplish what you are suggesting it should IMO suffice to "replace" penpals_find() (is it possible to do so?) with some custom code reading from Memcache - plusadding something int before_send(), storing penpal-information. Cheers, Thomas Gelf -

Re: [AMaViS-user] Nested MIME Mail not passed to ClamAV - why?

2009-06-11 Thread Thomas Gelf
Mark Martinec wrote: > Perhaps m...@customer.tld has bypass_virus_checks while o...@mailbox.tld > does not? Elevated log level would tell. (but see further on) No, that's what we immediately verified - as it's the most obvious explaination. > Having qr'^MAIL$' in @keep_decoded_original_maps seems

[AMaViS-user] Brainstorm request - scaling out Penpals

2009-06-11 Thread Thomas Gelf
se) could also be an option. That's all so far. I'm pretty sure I've forgotten something - but I'm confident one of you will for sure find it ;-) Your feedback is more than welcome, and it doesn't need to be positive - feel free to tell me why you consider the proposed approa

[AMaViS-user] Nested MIME Mail not passed to ClamAV - why?

2009-06-11 Thread Thomas Gelf
st seemed reasonable to me. But as run_av does not even be called for the decoded MIME parts that's probably not the issue here. Any suggestions? Best regards, Thomas Gelf -- Crystal Reports - New Free Runtime and

Re: [AMaViS-user] amavisd-nanny - patch for cron usage

2009-05-07 Thread Thomas Gelf
the process to terminate: %s\n", + printf STDERR ("Waiting for the process to terminate: %s\n", join(', ',@killed)) if @killed; } $| = 1; @@ -280,5 +280,5 @@ $cursor->c_close if defined $cursor; # ignoring status $db->db_close==0 o

[AMaViS-user] amavisd-nanny - patch for cron usage

2009-05-07 Thread Thomas Gelf
led as follows: amavisd-nanny -c 1 >/dev/null That's all, feedback / improvements are welcome. Best regards, Thomas Gelf -- The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning e

Re: [AMaViS-user] amavisd-new-2.6.1-rc1 RELEASE CANDIDATE is available

2008-06-28 Thread Thomas Gelf
Wow, thank you!! Cheers, Thomas Gelf - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/ser

Re: [AMaViS-user] amavisd-new-2.6.1-rc1 RELEASE CANDIDATE is available

2008-06-25 Thread Thomas Gelf
t; -> ) ENGINE=InnoDB; > Query OK, 0 rows affected (0.03 sec) You can't create table quarantine referencing table msgs as long as msgs doesn't exist. Could this be the root your problem? Cheers, Thomas Gelf -

Re: [AMaViS-user] Scalable amavisd-new installation (3): log aggregation

2008-06-24 Thread Thomas Gelf
Ed W schrieb: > Thomas Gelf wrote: >> [part 3 of: "Scalable amavisd-new installation"] >> >> There is also a self-written component I'm really proud of: the log >> aggregation system. > > Funnily enough I need to now build something similar to th

Re: [AMaViS-user] amavisd-new-2.6.1-rc1 RELEASE CANDIDATE is available

2008-06-24 Thread Thomas Gelf
even if someone is using aX3C4f6btXgX instead of aX3C4f6btXgX[25] in scenarios where he should use partition_tag. Thanks a lot for the great work!! Thomas Gelf PS: Thanks also for mentioning myself in the release notes! PPS: Most work I spent has not been for those little changes but for

Re: [AMaViS-user] amavisd-new-2.6.1-rc1 RELEASE CANDIDATE is available

2008-06-24 Thread Thomas Gelf
--- apps One could also add quar_type binary(1), even if it is it not really needed right now. Cheers, Thomas Gelf - Check out the new SourceForge.net Marketplace. It's the best place to buy o

Re: [AMaViS-user] amavisd-new-2.6.1-rc1 RELEASE CANDIDATE is available

2008-06-24 Thread Thomas Gelf
ou want to do with your quarantine. So just copy & paste default db scheme would not really help - but such an example could be a hint of how things COULD be done. Regards, Thomas Gelf Leonardo Rodrigues Magalhães schrieb: > Please check this message i sent last week, rega

Re: [AMaViS-user] Choosing the right DB partitioning (MySQL)

2008-05-23 Thread Thomas Gelf
ted MySQL bug - but it could really be something like that - if both threads are getting the same autoincremental id - and BOTH INSERT's are therefore failing. Such behaviour could match your description of amavisd-new's logic. Don't know whether this i

Re: [AMaViS-user] Choosing the right DB partitioning (MySQL)

2008-05-23 Thread Thomas Gelf
Thomas Gelf schrieb: > Thank you for your great work!! I'll switch my mail-traffic away from > one of my filters within the next hours and test some domain on the > patched amavisd-new in production. If everything goes well I'll switch > the routing back and put the patch

Re: [AMaViS-user] Releasing mail, question about AM.PDP

2008-05-23 Thread Thomas Gelf
server putting mail in quarantine has been they don't care as it is not relevant. With file-based quarantine this would not work unless you are using a shared filesystem. Cari saluti, Thomas Gelf Paolo Cravero schrieb: > Anyone? Mark? > > When releasing from full MySQL quarantine

Re: [AMaViS-user] Choosing the right DB partitioning (MySQL)

2008-05-23 Thread Thomas Gelf
tion. If everything goes well I'll switch the routing back and put the patch on all (both) current live filter hosts - even if doing so could waste my weekend ;-) Cheers, Thomas Gelf - This SF.net email is s

Re: [AMaViS-user] Choosing the right DB partitioning (MySQL)

2008-05-21 Thread Thomas Gelf
Mark Martinec schrieb: > I can prepare a small change to amavisd which would use > (partition_tag,mail_id) as a unique key instead of just the (mail_id). > ... > Do you think it would help? That would be great! (and probably also speed up amavisd's lookups without the need to add an additional ke

Re: [AMaViS-user] SMTP response

2008-05-21 Thread Thomas Gelf
Thomas Gelf schrieb: > Mark Martinec schrieb: >>> The full patch is on its way... >> So here it is, adding back the missing 'Ok, id=01234-01-2, from MTA...' >> to SMTP responses (with some code streamlining): > Wow, what a service! I'll test it

Re: [AMaViS-user] Choosing the right DB partitioning (MySQL)

2008-05-21 Thread Thomas Gelf
Thomas Gelf schrieb: > Thomas Gelf schrieb: >> db partitioning caused me some headache last days... > > It's getting even worse. The following SQL will not work... > > > CREATE TABLE IF NOT EXISTS `msgs` ( > > ... > > PRIMARY KEY (`mail_id`,`

Re: [AMaViS-user] Choosing the right DB partitioning (MySQL)

2008-05-21 Thread Thomas Gelf
Thomas Gelf schrieb: > db partitioning caused me some headache last days... It's getting even worse. The following SQL will not work... > CREATE TABLE IF NOT EXISTS `msgs` ( > ... > PRIMARY KEY (`mail_id`,`partition_tag`), > FOREIGN KEY (`sid`, `partition_tag`) &g

[AMaViS-user] Choosing the right DB partitioning (MySQL)

2008-05-21 Thread Thomas Gelf
sting partitions - so IMO the only option left are LIST partitions. Kind regards, Thomas Gelf NB: All statements above are valid for MySQL only. - This SF.net email is sponsored by: Microsoft Defy all challenges. Micros

Re: [AMaViS-user] SMTP response

2008-05-21 Thread Thomas Gelf
ion to my DB structure) and let you know the result. Thanks a lot! Thomas Gelf - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go

Re: [AMaViS-user] SMTP response

2008-05-21 Thread Thomas Gelf
Zdravo Mark, >> After XFORWARD has been received, the information is in: >> $msginfo->client_addr >> $msginfo->client_port > > You mean your own IP address? See 'package Amavis::In::Connection', > such info is stored in $conn->socket_ip and $conn->socket_port. Slowly I'm understanding small

Re: [AMaViS-user] Concerns regarding amavisd-release & partition_tag

2008-05-21 Thread Thomas Gelf
Thomas Gelf schrieb: > The problem is, that there is no way to keep an additional unique key > on the tables. MySQL docs are telling me: > > "...every unique key on the table must use every column in the table's > partitioning expression." (this also includes the t

Re: [AMaViS-user] SMTP response

2008-05-21 Thread Thomas Gelf
Thomas Gelf schrieb: > Can someone give me a little hint where to get this info > from while being in process_smtp_request()? "this info" -> [123.45.67.89]:25 - This SF.net email is sponsored by:

Re: [AMaViS-user] SMTP response

2008-05-21 Thread Thomas Gelf
Thomas Gelf schrieb: > Hi Marc, > >> Looking into it... Looks like an undesired side effect >> of a change in a client-side SMTP pipelining support. > > the following patch fixes the problem at least for my > personal needs Everything seems to be fine, I'm

Re: [AMaViS-user] SMTP response

2008-05-21 Thread Thomas Gelf
MTA([123.45.67.89]:25): ... Cheers, Thomas Gelf --- amavisd 2008-04-23 20:50:05.0 +0200 +++ amavisd.new 2008-05-21 12:19:18.0 +0200 @@ -15668,7 +15668,10 @@ &$check_mail($conn,$msginfo,$lmtp); # do all the contents checking $self->{tempdir}->

Re: [AMaViS-user] SMTP response

2008-05-20 Thread Thomas Gelf
g into it... Looks like an undesired side effect > of a change in a client-side SMTP pipelining support. Thanks! I tried to disable pipelining announcement on the host amavis has to deliver the mails through (forward_method) - but this does not seem to have any effect, so no workar

[AMaViS-user] SMTP response

2008-05-20 Thread Thomas Gelf
ur? Best regards, Thomas Gelf - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/

Re: [AMaViS-user] Concerns regarding amavisd-release & partition_tag

2008-05-20 Thread Thomas Gelf
n that I will get the same mail_id more times (once per week) it should not do any harm if my "3rd-party-apps" are able to handle it - is that correct? Or could amavisd-new somehow get into trouble? Many Thanks, Thomas Gelf ---

Re: [AMaViS-user] Concerns regarding amavisd-release & partition_tag

2008-05-20 Thread Thomas Gelf
4 VALUES LESS THAN (30), PARTITION p5 VALUES LESS THAN (36), PARTITION p6 VALUES LESS THAN (42), PARTITION p7 VALUES LESS THAN (48), PARTITION p8 VALUES LESS THAN (54) ); (and that's why I was confused regarding mail_id). If there is a bett

Re: [AMaViS-user] Received-headers: myproduct_name

2008-05-20 Thread Thomas Gelf
;myhostname'), $conn->socket_ip), +c('myproduct_name'), $conn->socket_port eq '' ? 'unix socket' : "port ".$conn->socket_port); $s .= "\n with $smtp_proto" if $smtp_proto=~/^(ES|S|L)MTPS?A?\z/i; # rfc3848 $s .= "\n

[AMaViS-user] Concerns regarding amavisd-release & partition_tag

2008-05-20 Thread Thomas Gelf
so: how can I avoid duplicate mail_id's (and trouble when someone is going to release such a mail)? Thanks a lot for clarification! Regards, Thomas Gelf - This SF.net email is sponsored by: Microsoft Defy all challenge

Re: [AMaViS-user] HELO name / myheloname

2008-05-16 Thread Thomas Gelf
mouss schrieb: > Thomas Gelf wrote: >> Amavisd-new seems to use localhost_name for it's own EHLO, >> but is using $myheloname for it's own SMTP greeting banner >> (and 250 reply to HELO)... > > if it helos with myhostname and this happens to be the hostname of

Re: [AMaViS-user] Received-headers: myproduct_name

2008-05-16 Thread Thomas Gelf
Thomas Gelf schrieb: ... Attached to this mail there is a little patch using $product_name in the received header lines. In my old installation I've used a hard- coded string, but $myproduct_name should do the job. I'll test it within the next hours to let you know, if it works. As e

[AMaViS-user] HELO name / myheloname

2008-05-15 Thread Thomas Gelf
$myheloname = 'localhost'; # $myheloname = '[127.0.0.1]'; $myheloname = '[' . $conn->socket_ip . ']'; IMO c('myhostname') would be the better choice, isn't it? Cheers, Thomas Gelf

[AMaViS-user] Received-headers: myproduct_name

2008-05-15 Thread Thomas Gelf
believe to be useful. Attached to this mail there is a little patch using $product_name in the received header lines. In my old installation I've used a hard- coded string, but $myproduct_name should do the job. I'll test it within the next hours to let you know, if i

Re: [AMaViS-user] Scalable amavisd-new installation (2): database partitioning

2008-05-13 Thread Thomas Gelf
Thomas Gelf schrieb: > Other questions: > > - Is there some better way to achieve a VERY scalable setup I didn't >think about? Btw: Anyone successfully using MySQL 5.1 with NDB Backend in production?

[AMaViS-user] Scalable amavisd-new installation (3): log aggregation

2008-05-08 Thread Thomas Gelf
at before doing so? - are you (all of you!!) going to clean up and perfectionate my code once opensourced - so that I can lay back and let you do my job? That's all, folks ;o) Cheers, Thomas Gelf - This SF.net email

[AMaViS-user] Scalable amavisd-new installation (2): database partitioning

2008-05-08 Thread Thomas Gelf
this problem (requirement for new packages / "outdated" distros) in your environments?! Other questions: - Is there some better way to achieve a VERY scalable setup I didn't think about? Kind regards, Thomas Gelf --

[AMaViS-user] Scalable amavisd-new installation (1): environment description

2008-05-08 Thread Thomas Gelf
l never be THAT large ;-) So, that's it (more or less). I feel sorry for my broken English, I'm trying to give my best! You'll find technical details and questions in the following two mails. Regards, Thomas Gelf -

Re: [AMaViS-user] Per-day msgs/msgrcpt/quarantine tables

2007-09-04 Thread Thomas Gelf
tick > to manual partitioning over different tablesets. Hmmm... and how do you accomplish this? Cheers, Thomas Gelf - This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems?

Re: [AMaViS-user] Per-day msgs/msgrcpt/quarantine tables

2007-09-03 Thread Thomas Gelf
Mark Martinec schrieb: > I have now a somewhat more advanced variant of my previously posted patch, > which automatically sets partition_tag to a current ISO8601 week number > (1..53) if $sql_partition_tag is undef, which will be a default. > If anyone is interested please drop me a line. /me drop

Re: [AMaViS-user] Long bad whitespace-lines

2007-09-03 Thread Thomas Gelf
Mark Martinec schrieb: > Your understanding is correct. There is not much need to clear a > database of process statuses, unless there are hundreds or thousands > of stale entries left there, but in this case you'd have a more serious > problem to solve anyway. Apart from amavisd-nanny, an amavisd

Re: [AMaViS-user] Long bad whitespace-lines

2007-09-03 Thread Thomas Gelf
Mark Martinec schrieb: > Well, you already said the mail contents is full of spaces, so > the above does not apply. It is still good to know nevertheless, > and the $sa_spawned trick still applies, regardless of a > reason for SpamAssassin segfaults. I'm a little bit sceptic about FuzzyOCR & Co -

Re: [AMaViS-user] zip file causing crashes

2007-09-03 Thread Thomas Gelf
Jerry Durand schrieb: > This is probably just a Clamd problem, but I thought I'd run it by > here, too. I've been busy the past couple of days and haven't kept > up on list messages. > > I've received several e-mails that hang the mail system (Amavisd-new > reports TROUBLE) and the processor i

Re: [AMaViS-user] Long bad whitespace-lines

2007-09-03 Thread Thomas Gelf
Mark Martinec schrieb: > So if you believe the file(1) is not a cause of delays, > please show a TIMING report from the amavisd log for > problematic messages (available at the log level 2 or above). Log level is 2, but as told before the last line there seems to be "providing full original messag

Re: [AMaViS-user] Long bad whitespace-lines

2007-09-03 Thread Thomas Gelf
Thomas Gelf schrieb: > thanks for your answer! file is 4.17-5etch2 - which is not 4.21, but > Debian people should have backported all relevant fixes as of my > /usr/share/doc/file/changelog.Debian: NB: CVE-2007-2026 is also fixed, tests from http://www.amavis.org/security/asa-2007-3.txt

Re: [AMaViS-user] Long bad whitespace-lines

2007-09-03 Thread Thomas Gelf
Mark Martinec schrieb: > My guess is that you are bitten by a Linux regexp library problem, > triggered by the file(1) utility (it doesn't affect for example FreeBSD > with the same version of a file utility). It has been discussed before > on this ML. Also mentioned in: http://www.amavis.org/secur

[AMaViS-user] Long bad whitespace-lines

2007-09-02 Thread Thomas Gelf
quot;clamdscan such_a_strange_mail" succeeds. Amavisd-new version: 2.4.2 from Debian Etch (we'll move away from Debian's outdated amavis package und use bleeding-edge amavis as soon as we put this system into full production). Did anyone encounter similar mails