Re: per-recipient configuration
On Mon, Jul 26, 2010 at 10:27 PM, Ask Bjørn Hansen a...@develooper.com wrote: On Jul 25, 2010, at 2:43, Jared Johnson wrote: it will soon be hopelessly forked from normal QP on account of per-recipient config directives, etc. Having a common API for per-recipient things have long been on the todo list. We've talked about it a couple times before; but the requirements/needs/wishes never sunk in deep enough in my head to do anything about it. So: How do you use this? My experience porting the Advenge SMTPD to be a qpsmtpd plug-in indicated that the current interface is entirely adequate for per-recipient things, as long as the plug-in manages its own persistence. I don't think any new hooks are required. Per-recipient features can be provided as a layer consisting of coordinating handlers for existing hooks, the most that needs to be done in my opinion is perhaps to establish a registry of name spaces in the existing per-message notes data, to prevent different plugins from clobbering e/o's data. One of the problems is that SMTP doesn't support per-recipient results after DATA. Is QPSMTPD used widely enough that we could introduce new ESMTP extensions with a chance that they could become widely adopted? I believe I have seen a proposed per-recipient results after DATA ESMTP specification, but ... okay I'll search for it ... it was in a thread started by me, in 2006. http://www.ietf.org/mail-archive/web/asrg/current/msg12816.html
Re: per-recipient configuration
My experience porting the Advenge SMTPD to be a qpsmtpd plug-in indicated that the current interface is entirely adequate for per-recipient things, as long as the plug-in manages its own persistence. I don't think any new hooks are required. On principal I'd agree that it's possible to do it all in the plugins, but if we're talking about making per-recip generally supported officially, it seems best to make that as easy as possible. I really like having the user_config and a $rcpt-config() method to go along with the hook_config and corresponding $self-qp-config(). And the more API support we have, the less extra code is added to plugins to support per-recipient stuff, e.g. the less forked they are from the non-per-recip counterparts. IMO Could you show some code examples? What do you do with the queue plugin(s) if recipients ended up having different headers or bodies due to tagging etc.? Per-recipient features can be provided as a layer consisting of coordinating handlers for existing hooks, the most that needs to be done in my opinion is perhaps to establish a registry of name spaces in the existing per-message notes data, to prevent different plugins from clobbering e/o's data. Is this still necessary if Qpsmtpd::Address::notes() exists? One of the problems is that SMTP doesn't support per-recipient results after DATA. Is QPSMTPD used widely enough that we could introduce new ESMTP extensions with a chance that they could become widely adopted? I believe I have seen a proposed per-recipient results after DATA ESMTP specification, but ... okay I'll search for it ... it was in a thread started by me, in 2006. http://www.ietf.org/mail-archive/web/asrg/current/msg12816.html This would be flippin awesome. Although even if QP has more pull than I think it does, it would take a bazillion years for all of the interwebs to adopt any protocol change. Nevertheless the sooner it starts to change, the better :) -Jared
Re: per-recipient configuration
I assume you meant to CC the list originally, I've added it :)
Could you show some code examples?
not immediately, but sure
What do you do with the queue
plugin(s) if recipients ended up having different headers or bodies due
to
tagging etc.?
Advenge only deals with deliver/defer with some fanciness on the defer
side because the bounce message asks for some grease.
That makes the problem simpler but may be an incomplete solution for some.
If we wanted to punt on the problem in mainline QP for a while, I suppose
plugins could just only ever modify the global body and avoid or
generalize any modifications that wanted to be different per-recipient.
But again this is extra burden on the plugins
Per-recipient
features can be provided as a layer consisting of coordinating
handlers for existing hooks, the most that needs to be done in my
opinion is perhaps to establish a registry of name spaces in the
existing per-message notes data, to prevent different plugins from
clobbering e/o's data.
Is this still necessary if Qpsmtpd::Address::notes() exists?
Is that a persistent data structure keyed on address?
Well it's persistent data stored in the Qpsmtpd::Address object itself:
sub notes { # in Qpsmtpd::Address.pm
my ($self,$key) = (shift,shift);
# Check for any additional arguments passed by the caller -- including
undef
return $self-{_notes}-{$key} unless @_;
return $self-{_notes}-{$key} = shift;
}
I submitted that patch a while ago and I think it was accepted.
What's efolder.net?
My new employer (it used to be @nmgi.com). They bought DoubleCheck and
all its IP, infrastructure, and employees from Network Management Group.
Their own product is a cloud backup solution, at some point we'll probably
be working on email archiving integrated with that. It was really a nice
deal, since the switch I get to work at home and I have a new awesome boss
:)
-Jared
Re: Rewritten URIBL plugin
On Sun, 25 Jul 2010, Jared Johnson wrote: The plugin has the following advantages over the original: - Uses MIME::Parser to unpack message text so that we can look for URI's in base64-encoded data, etc., and _not_ look for URI's in noise. I think we should probably consider putting support for parsed messages into core, with the parsing done lazily if requested by the API. Thoughts? Matt.
Re: Rewritten URIBL plugin
I think we should probably consider putting support for parsed messages
into core, with the parsing done lazily if requested by the API.
It's a good idea, it's only that I haven't yet had the time and
justification to write and to test a move from the plugin model (see the
attached plugin in my newer post) to an internal API model. The plugin I
attached represents a trivially-edited version of a plugin we've been
testing in production for quite some time. It probably wouldn't be *that*
difficult or risky, though, to take the very same code and put it into the
lazy API of your choice. And I think it'd be great, even though we
wouldn't actually benefit from the laziness ourselves ;)
If you don't hate storing the resulting parsed mime in
$txn-notes('mime_body'), though, that would rock, since then it wouldn't
break our existing plugin-based forked code ;)
-Jared
Re: Rewritten URIBL plugin
I think we should probably consider putting support for parsed messages into core, with the parsing done lazily if requested by the API. I forgot, we did kinda think of a couple of reasons not to want an API. depending on where you put it, you may find QP in general depending on MIME::Parser even if it never uses it. The benefit of the plugin is that /etc/qpsmtpd/plugins controls whether you wind up having to install, and take up your memory with, MIME::Parser You could conditionally 'require', although that does do different things to memory management that I don't understand Another thought, there are different reasons people might want mime to be parsed that benefit from different implementations. If you only want the text data, maybe you want to optimize for that somehow and throw away the binary attachment data, or vice versa (unfortunately there doesn't seem to be a way to do this with MIME::Parser that i've found though...). If you only want the text data *and* you don't mind having it not perfectly reconstructed (like the URIBL) plugin, then perhaps you could come up with a way to parse without MIME::Parser that is more efficient and more lazy (not in the sense you were suggesting earlier :P) -Jared
Re: Rewritten URIBL plugin
Jared Johnson wrote: I think we should probably consider putting support for parsed messages into core, with the parsing done lazily if requested by the API. I forgot, we did kinda think of a couple of reasons not to want an API. depending on where you put it, you may find QP in general depending on MIME::Parser even if it never uses it. The benefit of the plugin is that /etc/qpsmtpd/plugins controls whether you wind up having to install, and take up your memory with, MIME::Parser One thing we've discussed in the past (at least in my imagination) although not quite figured out how to implement, is making plugins act a little more like normal modules, so that one plugin can use another. So if you're interested in the parsed mime functionality, your plugin can plugin_use util/parsed-mime and the right magic happens. -R
