Re: [Dovecot] Postfix aliases with quota-status service
On Tue, Aug 06, 2013 at 09:20:13PM +0200, Thomas Leuxner wrote: Now everything in between seems to create SMTPD rejections in some cases _or_ queue the mail and let it hit the quota in other cases. That's my whole point... I'm sorry, I don't get your point. Are you saying that quota-status does not eliminate all over-quota bounces? That's to be expected. quota-status does not reserve quota when questioned, it only tells Postfix whether enough space is currently free or not. This is not free of race conditions, and therefore cannot eliminate all late bounces, even when quota-status has all the information, including size. For example, if a mailbox has 3000 bytes of free space, and two 2000 byte messages arrive close enough together so that the first one has not been delivered by the time the second one sends RCPT TO ... SIZE=..., quota-status will allow both messages, even though the second one will be rejected when Postfix actually tries local delivery (LTMP/LDA). Are you saying that rejects depend on SIZE= being sent during the RCPT TO stage (i.e., messages that announce their size correctly are rejected during the SMTP transaction, while those without size inidcation are passed)? If so, follow Rob's suggestion, and run quota-status (again) as part of smtpd_end_of_data_restrictions. By then, Postfix will know the size of a message and pass it to quota-status. (This only works for single-recipient messages; multi-recipient messages will always pass at this stage, because Postfix' policy protocol will not send recipient=... in this case, and thus quota-status replies with DUNNO. In my environment, most messages are single-recipient, so it works well enough for me.) Are you saying something else that I missed? Please tell me, because I'd like to understand your point.
Re: [Dovecot] Postfix aliases with quota-status service
* Ulrich Zehl ulrich-dove...@topfen.net 2013.08.07 08:29: Are you saying that rejects depend on SIZE= being sent during the RCPT TO stage (i.e., messages that announce their size correctly are rejected during the SMTP transaction, while those without size inidcation are passed)? Yes. signature.asc Description: Digital signature
Re: [Dovecot] Postfix aliases with quota-status service
On 7.8.2013, at 9.29, Ulrich Zehl ulrich-dove...@topfen.net wrote: On Tue, Aug 06, 2013 at 09:20:13PM +0200, Thomas Leuxner wrote: Now everything in between seems to create SMTPD rejections in some cases _or_ queue the mail and let it hit the quota in other cases. That's my whole point... I'm sorry, I don't get your point. Are you saying that quota-status does not eliminate all over-quota bounces? That's to be expected. quota-status does not reserve quota when questioned, it only tells Postfix whether enough space is currently free or not. This is not free of race conditions, and therefore cannot eliminate all late bounces, even when quota-status has all the information, including size. I think it would work also to do the check in SMTP RCPT TO stage and do quota bouncing only there. In LMTP/LDA stage use infinite quota to avoid bouncing. Yeah, user might get a little bit over quota (even over quota_grace) but not by much.
Re: [Dovecot] Postfix aliases with quota-status service
* Timo Sirainen t...@iki.fi 2013.08.06 18:15: Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work. quota_grace was meant to solve that. You'll allow the user to become a bit over quota. What I meant is before the mail enters the Postfix queues. If the SIZE extension is not used during MAIL FROM by the remote server, then there's no way to reject an over-quota mail upfront, losing the benefit of the policy service. signature.asc Description: Digital signature
Re: [Dovecot] Postfix aliases with quota-status service
On 6.8.2013, at 19.25, Thomas Leuxner t...@leuxner.net wrote: * Timo Sirainen t...@iki.fi 2013.08.06 18:15: Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work. quota_grace was meant to solve that. You'll allow the user to become a bit over quota. What I meant is before the mail enters the Postfix queues. If the SIZE extension is not used during MAIL FROM by the remote server, then there's no way to reject an over-quota mail upfront, losing the benefit of the policy service. The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed.
Re: [Dovecot] Postfix aliases with quota-status service
* Timo Sirainen t...@iki.fi 2013.08.06 19:42: The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed. I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents... signature.asc Description: Digital signature
Re: [Dovecot] Postfix aliases with quota-status service
On 6.8.2013, at 20.57, Thomas Leuxner t...@leuxner.net wrote: * Timo Sirainen t...@iki.fi 2013.08.06 19:42: The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed. I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents... Either you're still misunderstanding me, or vice versa. The quota rejections can be done complete in SMTP side even without SIZE: 1) quota at 99% : MAIL FROM:sen...@example.com 250 2.1.0 Ok RCPT TO:t...@dovecot.org 250 2.1.0 Ok DATA ... . 250 2.0.0 Ok: queued as 12345 2) quota is now at 103% : MAIL FROM:send...@example.com 250 2.1.0 Ok RCPT TO:t...@dovecot.org 554 5.2.2 User is over quota
Re: [Dovecot] Postfix aliases with quota-status service
On Tue, Aug 06, 2013 at 09:27:20PM +0300, Timo Sirainen wrote: On 6.8.2013, at 20.57, Thomas Leuxner t...@leuxner.net wrote: * Timo Sirainen t...@iki.fi 2013.08.06 19:42: The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed. I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents... Either you're still misunderstanding me, or vice versa. The quota rejections can be done complete in SMTP side even without SIZE: Another way, in Postfix, is to wait for end-of-DATA. Regardless of SIZE being given, at that point, the actual size is known. Of course as Thomas would probably point out, such a rejection is unsafe, because ANY overquota recipient would cause rejection for EVERY recipient; SMTP cannot have per-recipient results except at RCPT TO:. Personally, I'd much rather allow the last overquota mail, even in cases where the user goes far over the quota. Apparently Thomas intends to have a solid, inflexible quota. In that case I'd suggest going for a lower quota and adding quota_grace. Let quota_grace plus quota be the most you can tolerate in your users' mailboxes. 1) quota at 99% : MAIL FROM:sen...@example.com 250 2.1.0 Ok RCPT TO:t...@dovecot.org 250 2.1.0 Ok DATA ... . 250 2.0.0 Ok: queued as 12345 2) quota is now at 103% : MAIL FROM:send...@example.com 250 2.1.0 Ok RCPT TO:t...@dovecot.org 554 5.2.2 User is over quota -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
Re: [Dovecot] Postfix aliases with quota-status service
Am 06.08.2013 20:27, schrieb Timo Sirainen: On 6.8.2013, at 20.57, Thomas Leuxner t...@leuxner.net wrote: * Timo Sirainen t...@iki.fi 2013.08.06 19:42: The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed. I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents... Either you're still misunderstanding me, or vice versa. The quota rejections can be done complete in SMTP side even without SIZE: 1) quota at 99% : MAIL FROM:sen...@example.com 250 2.1.0 Ok RCPT TO:t...@dovecot.org 250 2.1.0 Ok DATA ... . 250 2.0.0 Ok: queued as 12345 2) quota is now at 103% : MAIL FROM:send...@example.com 250 2.1.0 Ok RCPT TO:t...@dovecot.org 554 5.2.2 User is over quota Thomas is right in general, thats a general problem with mail quota, that was the reason why there wasnt some good solution out for long times, but the dove policy server does i.e reject mail in smtp session if its allready assured that the mailbox is definite ... percent over quota ( configurable by grace parameter ), at that point it does not mater which size the incomming mail has, it will be i.e rejected anyway also it honors ( some kind overides ) other quota setting in i.e lmtp or lda, cause if it wouldnt, a mailbox would never become overquota by rejecting mail before by lmtp/lda settings Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Re: [Dovecot] Postfix aliases with quota-status service
* /dev/rob0 r...@gmx.co.uk 2013.08.06 20:49: Personally, I'd much rather allow the last overquota mail, even in cases where the user goes far over the quota. Apparently Thomas intends to have a solid, inflexible quota. The point I'm trying to make is mail being queued by Postfix because it has no means to validate the mail would take the user over quota. In the scenarios I tested with SIZE being part of MAIL FROM the mail gets rejected at SMTPD stage, while without SIZE supplied it will get queued and eventually be rejected by the MDA. AFAIK the whole endeavour was undertaken to avoid queue injection of mails knowing they would bounce. This seems to work when the SMTPD receives enough detail. As to Timo's example: This also seems to work given the quota is *over* the limit incl. grace: $ doveadm quota get -u ph...@trashheap.net Quota name TypeValue Limit % user STORAGE 10914 10240 106 user MESSAGE 5 - 0 Aug 6 20:56:31 spectre postfix/smtpd[27201]: connect from mail-oa0-f44.google.com[209.85.219.44] Aug 6 20:56:32 spectre postfix/smtpd[27201]: Anonymous TLS connection established from mail-oa0-f44.google.com[209.85.219.44]: TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits) Aug 6 20:56:32 spectre postfix/smtpd[27201]: NOQUEUE: reject: RCPT from mail-oa0-f44.google.com[209.85.219.44]: 554 5.2.2 ph...@trashheap.net: Recipient address rejected: Quota exceeded (mailbox for user is full); from=u...@googlemail.com to=ph...@trashheap.net proto=ESMTP helo=mail-oa0-f44.google.com Now everything in between seems to create SMTPD rejections in some cases _or_ queue the mail and let it hit the quota in other cases. That's my whole point... signature.asc Description: Digital signature
Re: [Dovecot] Postfix aliases with quota-status service
Thomas Leuxner skrev den 2013-08-06 18:25: * Timo Sirainen t...@iki.fi 2013.08.06 18:15: Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work. quota_grace was meant to solve that. You'll allow the user to become a bit over quota. What I meant is before the mail enters the Postfix queues. If the SIZE extension is not used during MAIL FROM by the remote server, then there's no way to reject an over-quota mail upfront, losing the benefit of the policy service. dovecot dict sql qouta, then in postfix smtpd_end_of_data_restricttions check sql qoutas in policy deamons or just simple sql qoury will not solve it ?, correct if sender need to send data first to get sizes it begins to be impraktical since if there is just 1 byte free last sender can still send more then 1 byte, but next sender cant
Re: [Dovecot] Postfix aliases with quota-status service
On Tue, Jul 30, 2013 at 03:20:47PM +0200, Thomas Leuxner wrote: This is probably intended behaviour, just want to make sure that I'm not missing a point here. For now the only fix that comes to my mind to create quota aware aliases - is creating 'dummy' users in Dovecot which point to the same mailbox rather than performing aliasing on the Postfix end. Open to suggestions... It depends on your user and/or alias database (i.e. how and where they're stored). I use LDAP, and configured Dovecot's userdb lookup to handle both main and alias addresses, like so: user_filter = (|(mail=%u)(mailAlternateAddress=%u)) (As far as Dovecot is concerned, there's actually no difference between main and alias.) If you store your mailbox and alias information in the same data source (LDAP, SQL, ...), you should be able to do the same.
[Dovecot] Postfix aliases with quota-status service
The latest HG commits seem to have fixed some underlying problems with the 'quota-status' service. Now doing some quick tests I wonder if this can be used with aliases on the Postfix side. Appears the 'check_policy_service' used in the example below will query existing users via Dovecot's Auth Backend: http://sys4.de/en/blog/2013/04/05/dovecot-quota-mit-postfix-abfragen/ http://www.postfix.org/SMTPD_POLICY_README.html This works fine for Dovecot user 'philo' where the mail is rejected upfront before it enters the queue: == /var/log/mail.log == Jul 30 13:41:18 spectre postfix/smtpd[31072]: NOQUEUE: reject: RCPT from host.example.com[1.2.3.4]: 554 5.2.2 ph...@trashheap.net: Recipient address rejected: Quota exceeded (mailbox for use r is full); from=u...@example.com to=ph...@trashheap.net proto=ESMTP helo=host.example.com Now user 'gunge' is an alias for 'philo' in Postfix, thus the quota check will not recognize the user, will queue the message and reject it at the MDA stage: == /var/log/mail.log == Jul 30 14:41:02 spectre postfix/lmtp[31460]: 3c4HSD5ZkBzBP: to=ph...@trashheap.net, orig_to=gu...@trashheap.net, relay=spectre.leuxner.net[private/dovecot-lmtp], delay=1.2, delays=1.1/0.01/0/0.13, dsn=4.2.2, status=deferred (host spectre.leuxner.net[private/dovecot-lmtp] said: 452 4.2.2 ph...@trashheap.net Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) This is probably intended behaviour, just want to make sure that I'm not missing a point here. For now the only fix that comes to my mind to create quota aware aliases - is creating 'dummy' users in Dovecot which point to the same mailbox rather than performing aliasing on the Postfix end. Open to suggestions... Regards Thomas signature.asc Description: Digital signature
