Re: smtpd - help needed tranlsating to new virtual map syntax [FIXED]
On Tue, Jan 22, 2019 at 01:11:44AM +0100, Eric Elena wrote: > On Mon, 21 Jan 2019 11:08:02 +0100 Gilles Chehade wrote: > > I may sound a bit harsh, but starting a thread with "this is my last try > > or I'll switch" (as if it actually matters) right before telling someone > > who wants to help you that you actually tried _nothing_ then blaming the > > code improvements for a use-case that could have never worked because it > > not only uses the wrong _documented_ mechanism but also because the code > > to make your use-case work has never existed, kinds of irritates me. > > > > I don't get royalties on smtpd install, please install whatever software > > fits your use case, this is how proper engineering works. > > First of all thank you Gilles (and all the others who contributed to > this project) for your amazing work on OpenSMTPD! > > That said, there is a kind of sender rewriting mechanism in OpenSMTP. > Well, it works for me (tm) I'm not saying it's perfect, it might be an > overkill but at least it does what I want it to do. The conf is > included below (only the part for rewriting the sender > address): > > [...] > > When a mail is received (listen on all): > - check if it is rejected > - if not, if the email if for toto@my.domain, forward it to the very > same OpenSMTP daemon on port 10030 using the authenticated user foo and > using masq@my.domain as the MAIL-FROM in the SMTP session (enveloppe) > - when an email is received on port 10030, tag it with the label MASQ. > The authenticated user is allowed to send an email as the user > masq@my.domain. The keyword masquerade modifies the From header (the > message itself) to match the address given in the SMTP session > - at that point, the sender address is rewritten both in the SMTP > session and the headers > - if the email is for toto@my.domain and is tagged with the label MASQ, > the virtual user address is expanded to the real email address > - continue like a normal message > > There is probably room for improvement but I hope this helps. > indeed, a bit overkill and now that we have removed the blockers we must come up with a simpler way to achieve that... but what you did, that's smart :-) -- Gilles Chehade @poolpOrg https://www.poolp.org tip me: https://paypal.me/poolpOrg
Re: smtpd - help needed tranlsating to new virtual map syntax [FIXED]
On Mon, Jan 21, 2019 at 01:04:16PM -0600, Adam Thompson wrote: > > > Also, this is a recipient translation mechanism, similar to aliases, and > > not a sender rewriting mechanism which we do not have at this point. > > [...] > > virtual _now_ only works on recipients, not senders ? > > the virtual code hasn't changed, it works the way it always did. > > > > there is no way it could ever do what you're describing or attempting to > > do given that it doesn't operate at all anywhere near the message. there > > is no way it has ever parsed: > > This is all very surprising to hear. The existing system works (somehow). > So I am apparently misunderstanding what is happening, because with the > configuration as shown, telling the various broken email senders to use that > box as their mailhost _somehow_ fixes the bogus From: headers and envelopes. > the entire virtual expansion happens between the client sending RCPT TO, and the server responding Ok to that RCPT TO. virtual does not know of a sender, never, and it is done before the message is actually received so it doesn't know headers, which is why i'm 100% confident there isn't one chance it could ever do what you describe. > Oh, this just occurred to me as I'm writing: I really hope I didn't switch > to a different MTA on that system years ago, and then just forgot to check > which MTA was actually running. If that's the case, I'm not going to bother > posting an update, because I'll be busy banging my head on the wall and then > hiding in shame. > that is a more likely possibility. > > > I'm not convinced the new smtpd.conf grammar improves anything at > > > all, but I assume it must help someone or it wouldn't have > > > changed... but I believe my use case got thrown out with the > > > bathwater, so to speak. Oh, well. :-( > > This is bullshit. > > The grammar doesn't reduce the functional scope, it can only expand it. > > I'm taking your word for it - you will know far better than I do! > > > > What you are describing has never existed in smtpd, there's never been > > code to translate sender addresses and there's a good reason for that: > > Good reasons aside, I still need to accommodate other vendor's broken mail > implementations, because I can't fix them. I know of multiple reasons > source rewriting is a bad idea, in general, but I get paid to make stuff > work, not just say that it's broken. > oh, don't get me wrong, i'm not saying there's a good reason not to have this rewriting, what i was saying is that there was a good reason why it was not doable before the grammar change. it is a useful feature which is part of my todo and which i will work on as time allows. > > it not considered doable before the grammar change... > > But sure, blame it on the grammar. > > I believed that the grammar change had rendered my use case impossible > because was now limited to local delivery methods. Clearly I was > wrong... and not even in the way I thought I might be wrong. > yes, that's true. using 'virtual' on relay rules didn't transform anything whatsoever, the code had an explicit check to not enter the transformation lookups if we were in a relay rule. the new grammar just made it clear that what you were trying to do could not work rather than accepting the criteria and disregarding it. > > I may sound a bit harsh, but starting a thread with "this is my last try > > or I'll switch" (as if it actually matters) > > My apologies - that was meant to sound more like "I have a plan B so if this > isn't possible, that's OK but I've wasted so much time on this I'm kinda > running out of time, please tell me if I should just stop now and switch". > I know *exactly* how much OpenBSD devs care if I use their code or not! I > do not want to be "that asshole", although it seems I've succeeded again - > sorry. > > Thank you for taking the time to reply. Now I'm going to go check that mail > server a 7,000,000th time, this time to see what MTA is actually *running*, > not just *configured*. I'm not sure whether I want it to be such a blatant > mistake on my part or not... if yes, this all makes sense but I'm an idiot, > whereas if no, then WTF, how is it working at all? > > FWIW: I am much happier with OpenSMTPd than with other MTAs because of its > forward-declarative configuration syntax. Thank you for your work on > bringing a modern, lean, secure(-er) MTA into existence. > np ;-) -- Gilles Chehade @poolpOrg https://www.poolp.org tip me: https://paypal.me/poolpOrg
Re: smtpd - help needed tranlsating to new virtual map syntax [FIXED]
On Mon, 21 Jan 2019 11:08:02 +0100 Gilles Chehade wrote:
> I may sound a bit harsh, but starting a thread with "this is my last try
> or I'll switch" (as if it actually matters) right before telling someone
> who wants to help you that you actually tried _nothing_ then blaming the
> code improvements for a use-case that could have never worked because it
> not only uses the wrong _documented_ mechanism but also because the code
> to make your use-case work has never existed, kinds of irritates me.
>
> I don't get royalties on smtpd install, please install whatever software
> fits your use case, this is how proper engineering works.
First of all thank you Gilles (and all the others who contributed to
this project) for your amazing work on OpenSMTPD!
That said, there is a kind of sender rewriting mechanism in OpenSMTP.
Well, it works for me (tm) I'm not saying it's perfect, it might be an
overkill but at least it does what I want it to do. The conf is
included below (only the part for rewriting the sender
address):
o /etc/mail/smtpd.conf
listen on all tls pki my.domain auth-optional
listen on lo0 port 10030 smtps pki my.domain tag MASQ auth senders { foo =
masq@my.domain } masquerade
table masquser { "toto@my.domain" }
table masq-alias{ "toto@my.domain" = "t...@example.com" }
table secrets file:/etc/mail/secrets
action masq01 mbox virtual
action masq02 relay host smtps://masqlabel@127.0.0.1:10030 auth
mail-from "masq@my.domain"
match tag MASQ rcpt-to action masq01
match from any rcpt-to action masq02
o /etc/mail/secrets
masqlabel foo:asuperpassword
When a mail is received (listen on all):
- check if it is rejected
- if not, if the email if for toto@my.domain, forward it to the very
same OpenSMTP daemon on port 10030 using the authenticated user foo and
using masq@my.domain as the MAIL-FROM in the SMTP session (enveloppe)
- when an email is received on port 10030, tag it with the label MASQ.
The authenticated user is allowed to send an email as the user
masq@my.domain. The keyword masquerade modifies the From header (the
message itself) to match the address given in the SMTP session
- at that point, the sender address is rewritten both in the SMTP
session and the headers
- if the email is for toto@my.domain and is tagged with the label MASQ,
the virtual user address is expanded to the real email address
- continue like a normal message
There is probably room for improvement but I hope this helps.
Re: smtpd - help needed tranlsating to new virtual map syntax [FIXED]
On 2019-01-21 04:08, Gilles Chehade wrote: In this test case, my translations map had: What is a translation map ? There is no such thing in OpenSMTPD (as of today). A virtual map that happened to be called . You're feeding the virtual table with invalid values. Apparently, yes. Also, this is a recipient translation mechanism, similar to aliases, and not a sender rewriting mechanism which we do not have at this point. [...] virtual _now_ only works on recipients, not senders ? the virtual code hasn't changed, it works the way it always did. there is no way it could ever do what you're describing or attempting to do given that it doesn't operate at all anywhere near the message. there is no way it has ever parsed: This is all very surprising to hear. The existing system works (somehow). So I am apparently misunderstanding what is happening, because with the configuration as shown, telling the various broken email senders to use that box as their mailhost _somehow_ fixes the bogus From: headers and envelopes. Oh, this just occurred to me as I'm writing: I really hope I didn't switch to a different MTA on that system years ago, and then just forgot to check which MTA was actually running. If that's the case, I'm not going to bother posting an update, because I'll be busy banging my head on the wall and then hiding in shame. I'm not convinced the new smtpd.conf grammar improves anything at all, but I assume it must help someone or it wouldn't have changed... but I believe my use case got thrown out with the bathwater, so to speak. Oh, well. :-( This is bullshit. The grammar doesn't reduce the functional scope, it can only expand it. I'm taking your word for it - you will know far better than I do! What you are describing has never existed in smtpd, there's never been code to translate sender addresses and there's a good reason for that: Good reasons aside, I still need to accommodate other vendor's broken mail implementations, because I can't fix them. I know of multiple reasons source rewriting is a bad idea, in general, but I get paid to make stuff work, not just say that it's broken. it not considered doable before the grammar change... But sure, blame it on the grammar. I believed that the grammar change had rendered my use case impossible because was now limited to local delivery methods. Clearly I was wrong... and not even in the way I thought I might be wrong. I may sound a bit harsh, but starting a thread with "this is my last try or I'll switch" (as if it actually matters) My apologies - that was meant to sound more like "I have a plan B so if this isn't possible, that's OK but I've wasted so much time on this I'm kinda running out of time, please tell me if I should just stop now and switch". I know *exactly* how much OpenBSD devs care if I use their code or not! I do not want to be "that asshole", although it seems I've succeeded again - sorry. Thank you for taking the time to reply. Now I'm going to go check that mail server a 7,000,000th time, this time to see what MTA is actually *running*, not just *configured*. I'm not sure whether I want it to be such a blatant mistake on my part or not... if yes, this all makes sense but I'm an idiot, whereas if no, then WTF, how is it working at all? FWIW: I am much happier with OpenSMTPd than with other MTAs because of its forward-declarative configuration syntax. Thank you for your work on bringing a modern, lean, secure(-er) MTA into existence. -Adam
Re: smtpd - help needed tranlsating to new virtual map syntax [FIXED]
sorry, I obviously f-up my last mail, this one is fixed ;-) On Sun, Jan 20, 2019 at 04:14:05PM -0600, Adam Thompson wrote: > As it turns out, no, that doesn't work. > Trying to fix up broken sender mail domain-parts only simply gets me a "5.2.4 > Mailing list expansion problem" error, with no debug output to suggest why. > > In this test case, my translations map had: > > @bad.athompso.net @good.athompso.net > What is a translation map ? There is no such thing in OpenSMTPD (as of today). > in it. Obviously, this is a test setup :). > Smtpd.conf itself consisted of: > > listen on all received-auth > smtp max-message-size 100M > table translations file:/etc/mail/translations # ORIG->NEW > mappings > table allowed-hosts file:/etc/mail/allowed-hosts# Who can > connect? (bare IP addresses or CIDR subnets) > action translate lmtp "/var/run/lmtp.sock" virtual > # 1st pass on allowed rewrite mail > action forward forward-only > # and now it's not our problem anymore > match for any from local action forward # 2nd pass for > reinjected mail, this time just forward it > match for any from src action translate # inbound mail > - hand it to LMTP, translating as we go > > from table(5): Aliasing tables Aliasing tables are mappings that associate a recipient to one or many destinations. They can be used in two contexts: primary domain aliases and virtual domain mapping. [...] In a virtual domain context, the key is either a user part, a full email address or a catch all, following selection rules described in smtpd.conf(5), and the value is one or many recipients as described in aliases(5): user1 otheruser us...@example.org otheruser1,otheruser2 @example.orgotheru...@example.com @ catch...@example.com You're feeding the virtual table with invalid values. Also, this is a recipient translation mechanism, similar to aliases, and not a sender rewriting mechanism which we do not have at this point. > A cursory glance at the source code (yikes, it's been a long time since I was > a programmer) suggests that virtual now only works on recipients, not > senders. Which is too bad for me, as that means I'll have to switch at least > one box to use Postfix. > virtual _now_ only works on recipients, not senders ? the virtual code hasn't changed, it works the way it always did. there is no way it could ever do what you're describing or attempting to do given that it doesn't operate at all anywhere near the message. there is no way it has ever parsed: @bad.athompso.net @good.athompso.net and the only thing that changed is that such errors are now visible from the session as: 5.2.4 Mailing list expansion problem instead of an invalid recipient error like it probably did in 6.3 > I'm not convinced the new smtpd.conf grammar improves anything at all, but I > assume it must help someone or it wouldn't have changed... but I believe my > use case got thrown out with the bathwater, so to speak. Oh, well. :-( > This is bullshit. The grammar doesn't reduce the functional scope, it can only expand it. What you are describing has never existed in smtpd, there's never been code to translate sender addresses and there's a good reason for that: it not considered doable before the grammar change... But sure, blame it on the grammar. > (If anyone cares, the bad sender addresses are mostly alerts coming from > older Sun ALOMs and at least one Lexmark printer that also sends email with > broken From addresses.) > I may sound a bit harsh, but starting a thread with "this is my last try or I'll switch" (as if it actually matters) right before telling someone who wants to help you that you actually tried _nothing_ then blaming the code improvements for a use-case that could have never worked because it not only uses the wrong _documented_ mechanism but also because the code to make your use-case work has never existed, kinds of irritates me. I don't get royalties on smtpd install, please install whatever software fits your use case, this is how proper engineering works. -- Gilles Chehade @poolpOrg https://www.poolp.org tip me: https://paypal.me/poolpOrg
