Re: postsuper manpage: message expiration

[Emanuel]

Thanks.!! the option is very useful.

Regards,

El 18/1/20 a las 21:45, Wietse Venema escribió:

I'm adding a new field to the Postfix queue file that says if a
message was forcibly expired. The field is set with the postsuper
command, using syntax that is similar to other postsuper commands.

The new option behaves like 'delete' except of course that it sets
a flag in a file instead of deleting it.

-e queue_id
   Expire  one  message  with  the named queue ID in the named mail
   queue(s) (default: hold, incoming, active and deferred).

   This only marks the message as  expired.  The  message  will  be
   returned  to  the  sender  when  the  queue manager opens it for
   delivery (but mail in the  hold  queue  stays  in  that  queue).
   Deferred  messages  are  returned with the actual reason for the
   delay; other messages are returned with the reason  "5.7.0  mes-
   sage is administratively expired".

   To  expire multiple files, specify the -e option multiple times,
   or specify a queue_id of - to read queue IDs from standard input
   (see the -d option for an example).

As implemented, this does not release a file from 'hold'. It only
sets a flag in a queue file. That is the purist approach. However
I suspect that some people would not want to have to say

To delete one file:
postsuper -e queueid -H queueid

To delete a bunch of files:
postqueue -pj | script | postsuper -e -
postqueue -pj | script | postsuper -H -

If the intent of expiration is to make messages 'go away' then it
makes sense that '-e' automagically un-holds a message, so that the
queue manager can return it as undeliverable.

Wietse

--
envialosimple.com   
Emanuel Gonzalez
IT / Departamento Emails
emanuel.gonza...@donweb.com 
www.envialosimple.com 
by donweb 

Nota de confidencialidad: Este mensaje y archivos adjuntos al mismo son 
confidenciales, de uso exclusivo para el destinatario del mismo. La 
divulgación y/o uso del mismo sin autorización por parte de DonWeb.com 
queda prohibida.
DonWeb.com no se hace responsable del mensaje por la falsificación y/o 
alteración del mismo.
De no ser Ud el destinatario del mismo y lo ha recibido por error, por 
favor, notifique al remitente y elimínelo de su sistema.
Confidentiality Note: This message and any attachments (the message) are 
confidential and intended solely for the addressees. Any unauthorised 
use or dissemination is prohibited by DonWeb.com.

DonWeb.com shall not be liable  for the message if altered or falsified.
If you are not the intended addressee of this message, please cancel it 
immediately and inform the sender
Nota de Confidencialidade: Esta mensagem e seus eventuais anexos podem 
conter dados confidenciais ou privilegiados.
Se você os recebeu por engano ou não é um dos destinatários aos quais 
ela foi endereçada, por favor destrua-a e a todos os seus eventuais 
anexos ou copias realizadas, imediatamente.
É proibida a retenção, distribuição, divulgação ou utilização de 
quaisquer informações aqui contidas.
Por favor, informenos sobre o recebimento indevido desta mensagem, 
retornando-a para o autor.

Re: postsuper manpage: message expiration

[Marvin Renich]

* Wietse Venema  [200119 17:54]:
> Marvin Renich:
> > Postsuper isn't overflowing with options; perhaps a more flexible and
> > simpler design would be to have -e expire and -E expire and release from
> > hold (if it is held).
> 
> Yes, the idea of a third option came to mind.
> 
> > The only con I see to this is "feature creep", which is minimal at this
> > point.  It only becomes an issue when postsuper starts getting more and
> > more options.
> 
> There is a limit to what postsuper can do, because by design it has
> no dependencies on a running Postfix system.

I'm a little confused.  Doesn't postsuper already have -H?  If you add
-e, what is hard about having -E set the flag, and then effectively act
like the user also specified -H with the same queue_id?

...Marvin

Re: postsuper manpage: message expiration

[Wietse Venema]

Viktor Dukhovni:
> That said, instead of a queue file record, this could be one
> of the "mode" bits of the file, in particular, we could set
> the queue file group execute bit to indicate expiration,
> this would be cheap for the queue-manager to re-check before
> deciding whether to defer the message, and could alternatively
> be used for expedited processing from the deferred queue.

There is prior art with using Postfix queue file permission bits
for signaling:

- The flush daemon sets the queue file group read permission bit,
  to indicate that the queue manager should unthrottle message
  delivery transports and next-hop destinations that are used to
  deliver the message (the queue manager clears the bit).

- The cleanup daemon sets the queue file owner execute permission
  bit, so that the queue manager can reliably skip junk from a
  crashed cleanup process.

So, the suggestion to use a group execute permission bit seems safe,
modulo interference from unhelpful 'security' software.

But I want to avoid a significant delay before messages start being
removed from the queue after someone runs 'postsuper -e'. Judicious
manipulation of queue file mtimes can take care of that.

I'll explore this alternative further.

Wietse

Re: postsuper manpage: message expiration

[Wietse Venema]

Marvin Renich:
> * Wietse Venema  [200119 08:04]:
> > A radically different approach would be to introduce a new 'expired'
> > queue and move messages there when they need to be expired, and to
> > introduce a new postqueue flag to flush the expired queue (the queue
> > manager should not normally scan the expired queue because that
> > would slow down all deliveries).
> > 
> > Pro: Less complexity. No 'expired' state inside queue files.
> > 
> > Con: More complexity. When expired mail needs to be deferred because
> > the bounce daemon is failing, it should go back to the expired
> > queue, not to the deferred queue.
> 
> Postsuper isn't overflowing with options; perhaps a more flexible and
> simpler design would be to have -e expire and -E expire and release from
> hold (if it is held).

Yes, the idea of a third option came to mind.

> The only con I see to this is "feature creep", which is minimal at this
> point.  It only becomes an issue when postsuper starts getting more and
> more options.

There is a limit to what postsuper can do, because by design it has
no dependencies on a running Postfix system.

That is, until someone turns postsuper into a kind of busybox program
that has all the Postfix code linked into it :-)

Wietse

Re: postsuper manpage: message expiration

[Viktor Dukhovni]

On Sun, Jan 19, 2020 at 08:03:14AM -0500, Wietse Venema wrote:

> A radically different approach would be to introduce a new 'expired'
> queue and move messages there when they need to be expired, and to
> introduce a new postqueue flag to flush the expired queue (the queue
> manager should not normally scan the expired queue because that
> would slow down all deliveries).

My instinct is to not optimize for very large numbers of expired
messages, and add another queue for the administrator to think
about.

Bounces back to the sender for messages one is expiring rather
than simply deleting are presumably for an internal authorized
user, and so their delivery should not incur much latency.

IMHO expiration should leave all messages in whatever queue they were
found (including "hold"), and if the message was deferred I would
try to preserve its (postdated) modification time, so as not to
trigger expedited processing.  It will bounce before the next
delivery attempt.

If the message is active, the queue manager will not have seen
the "expired" flag yet, and will likely defer the message, when
the pending delivery attempts complete, it will bounce before
the next retry.

That said, instead of a queue file record, this could be one
of the "mode" bits of the file, in particular, we could set
the queue file group execute bit to indicate expiration,
this would be cheap for the queue-manager to re-check before
deciding whether to defer the message, and could alternatively
be used for expedited processing from the deferred queue.

-- 
Viktor.

Re: postsuper manpage: message expiration

[Marvin Renich]

* Wietse Venema  [200119 08:04]:
> A radically different approach would be to introduce a new 'expired'
> queue and move messages there when they need to be expired, and to
> introduce a new postqueue flag to flush the expired queue (the queue
> manager should not normally scan the expired queue because that
> would slow down all deliveries).
> 
> Pro: Less complexity. No 'expired' state inside queue files.
> 
> Con: More complexity. When expired mail needs to be deferred because
> the bounce daemon is failing, it should go back to the expired
> queue, not to the deferred queue.

Postsuper isn't overflowing with options; perhaps a more flexible and
simpler design would be to have -e expire and -E expire and release from
hold (if it is held).

The only con I see to this is "feature creep", which is minimal at this
point.  It only becomes an issue when postsuper starts getting more and
more options.

...Marvin

Re: postsuper manpage: message expiration

[azurit]

Citát Wietse Venema :


I'm adding a new field to the Postfix queue file that says if a
message was forcibly expired. The field is set with the postsuper
command, using syntax that is similar to other postsuper commands.

The new option behaves like 'delete' except of course that it sets
a flag in a file instead of deleting it.

   -e queue_id
  Expire  one  message  with  the named queue ID in the named mail
  queue(s) (default: hold, incoming, active and deferred).

  This only marks the message as  expired.  The  message  will  be
  returned  to  the  sender  when  the  queue manager opens it for
  delivery (but mail in the  hold  queue  stays  in  that  queue).
  Deferred  messages  are  returned with the actual reason for the
  delay; other messages are returned with the reason  "5.7.0  mes-
  sage is administratively expired".

  To  expire multiple files, specify the -e option multiple times,
  or specify a queue_id of - to read queue IDs from standard input
  (see the -d option for an example).

As implemented, this does not release a file from 'hold'. It only
sets a flag in a queue file. That is the purist approach. However
I suspect that some people would not want to have to say

To delete one file:
postsuper -e queueid -H queueid

To delete a bunch of files:
postqueue -pj | script | postsuper -e -
postqueue -pj | script | postsuper -H -

If the intent of expiration is to make messages 'go away' then it
makes sense that '-e' automagically un-holds a message, so that the
queue manager can return it as undeliverable.

Wietse




Looks cool! Thank you.

Re: postsuper manpage: message expiration

[Wietse Venema]

A radically different approach would be to introduce a new 'expired'
queue and move messages there when they need to be expired, and to
introduce a new postqueue flag to flush the expired queue (the queue
manager should not normally scan the expired queue because that
would slow down all deliveries).

Pro: Less complexity. No 'expired' state inside queue files.

Con: More complexity. When expired mail needs to be deferred because
the bounce daemon is failing, it should go back to the expired
queue, not to the deferred queue.

Wietse

Re: postsuper manpage: message expiration

[Jose Luis Tallon]

On 19/1/20 1:45, Wietse Venema wrote:

I'm adding a new field to the Postfix queue file that says if a
message was forcibly expired. The field is set with the postsuper
command, using syntax that is similar to other postsuper commands.

The new option behaves like 'delete' except of course that it sets
a flag in a file instead of deleting it.

-e queue_id
   Expire  one  message  with  the named queue ID in the named mail
   queue(s) (default: hold, incoming, active and deferred).

   This only marks the message as  expired.  The  message  will  be
   returned  to  the  sender  when  the  queue manager opens it for
   delivery (but mail in the  hold  queue  stays  in  that  queue).
   Deferred  messages  are  returned with the actual reason for the
   delay; other messages are returned with the reason  "5.7.0  mes-
   sage is administratively expired".

   To  expire multiple files, specify the -e option multiple times,
   or specify a queue_id of - to read queue IDs from standard input
   (see the -d option for an example).

As implemented, this does not release a file from 'hold'. It only
sets a flag in a queue file. That is the purist approach. However
I suspect that some people would not want to have to say

To delete one file:
postsuper -e queueid -H queueid


As long as one can specify both commands in the same invocation, I don't 
believe it'd be a great deal...


Counter-argument: less "unixy" --- "do just one thing and do it well"


To delete a bunch of files:
postqueue -pj | script | postsuper -e -
postqueue -pj | script | postsuper -H -


Some script's filtering criteria could have marked a message in the 
queue as "expired"


...but it'd still be held ---say, for manual inspection/forensic 
whathever--- by some other's selection logic



If the intent of expiration is to make messages 'go away' then it
makes sense that '-e' automagically un-holds a message, so that the
queue manager can return it as undeliverable.


Yes, save for the above reasoning. I know I'd use it as described above 
--- of course others will do it differently.



Just my .02€

Thanks,

    / J.L.

postsuper manpage: message expiration

[Wietse Venema]

I'm adding a new field to the Postfix queue file that says if a
message was forcibly expired. The field is set with the postsuper
command, using syntax that is similar to other postsuper commands.

The new option behaves like 'delete' except of course that it sets
a flag in a file instead of deleting it.

   -e queue_id
  Expire  one  message  with  the named queue ID in the named mail
  queue(s) (default: hold, incoming, active and deferred).

  This only marks the message as  expired.  The  message  will  be
  returned  to  the  sender  when  the  queue manager opens it for
  delivery (but mail in the  hold  queue  stays  in  that  queue).
  Deferred  messages  are  returned with the actual reason for the
  delay; other messages are returned with the reason  "5.7.0  mes-
  sage is administratively expired".

  To  expire multiple files, specify the -e option multiple times,
  or specify a queue_id of - to read queue IDs from standard input
  (see the -d option for an example).

As implemented, this does not release a file from 'hold'. It only
sets a flag in a queue file. That is the purist approach. However
I suspect that some people would not want to have to say

To delete one file:
postsuper -e queueid -H queueid 

To delete a bunch of files:
postqueue -pj | script | postsuper -e -
postqueue -pj | script | postsuper -H -

If the intent of expiration is to make messages 'go away' then it
makes sense that '-e' automagically un-holds a message, so that the
queue manager can return it as undeliverable.

Wietse