Re: New to Mutt, unable to send messages in *any* attempted way
On Fri, May 20, 2022 at 05:54:55PM +1000, raf wrote: > I'm sending a patch that adds an error check for shell > meta-characters when $sendmail is used. It might have been > better to check right after reading .muttrc, but this seemed > like a more natural place to put the code (i.e., right after > the check that $sendmail exists). I tried to send the patch to mutt-...@mutt.org a few times but never received it, so I'll send it here instead. cheers, raf > Subject: [PATCH] Add error when $sendmail has shell metachars --- sendlib.c | 8 1 file changed, 8 insertions(+) diff --git a/sendlib.c b/sendlib.c index 430b5d73..05039714 100644 --- a/sendlib.c +++ b/sendlib.c @@ -2706,6 +2706,14 @@ mutt_invoke_sendmail (ADDRESS *from, /* the sender */ return -1; } + /* check for shell meta-characters that won't do what the user expects */ +#define SHELL_NON_SPACE_META_CHARACTERS "|&;()<>[]{}$`'~\"\\*?" + if (Sendmail[strcspn(Sendmail, SHELL_NON_SPACE_META_CHARACTERS)] != '\0') + { +mutt_error(_("$sendmail cannot contain shell meta-characters.")); +return -1; + } + ps = s; i = 0; while ((ps = strtok (ps, " "))) -- 2.30.2
Re: New to Mutt, unable to send messages in *any* attempted way
On Fri, May 20, 2022 at 04:20:55PM +1000, raf wrote: > On Thu, May 19, 2022 at 12:05:59PM -0700, "Kevin J. McCarthy" > wrote: > > > On Thu, May 19, 2022 at 01:12:31PM -0500, Derek Martin wrote: > > > On Thu, May 19, 2022 at 11:06:38AM -0700, Kevin J. McCarthy wrote: > > > > Sorry, this isn't currently possible in Mutt. The $sendmail variable is > > > > handled specially: it's tokenized by space and invoked directly via > > > > execvp(). So you won't be able to use arguments with spaces in them > > > > inside > > > > the variable, or any shell quoting since it doesn't pass through the > > > > shell. > > > > > > Couldn't you do something like the following? > > > > > > set my_sendmail=/usr/bin/msmtp [...] --passwordeval=$(gpg --no-tty -q -d > > > ~/.user.gpg)" > > > set sendmail=$my_sendmail > > > > > > Don't know, haven't tried it, but as long as the first thing evaluates > > > to something that sendmail's tokenization can handle, seems like it > > > should work... > > > > $sendmail goes through muttrc evaluation like other variables, but when > > invoked to send an email, the resulting string is tokenized by space, '--' > > and various arguments inserted as needed, and then passed to execvp(). > > > > Yes, this should be documented and I'll add it to the config option shortly. > > > > I don't know why it was written this way, perhaps some security concerns, > > but I agree that it's surprising. > > > > -- > > Kevin J. McCarthy > > GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA > > I think the reason would be that mutt is modifying the > command, not just executing it. It's modifying a > command in a complex external language that it can't > parse by itself unless it imposes very strict > limitations on the allowable syntax of the command. > I don't think it's a security decision. It's just > necessary. How would mutt know how to modify > sendmail="blah1 | blah2 | blah3" ? > > I think this limitation should be documented, but not > fixed. It could be fixed by adding something like % > interpolation placeholders for the things that are > added. But that might break existing commands that > contain %. Maybe # would be a better choice. But it's > still a bad choice. It requires the user to know and > use even more notation. > > An elegant solution already exists. Put complex shell > commands in a file and mutt only needs to see its name. > > But users need to be told when they are required to do > that. So in addition to documenting the limitation, a > new error message when the evaluated $sendmail contains > any (non-space) shell meta-characters would be helpful. > That's a clear indication that the user is expecting > shell parsing that isn't going to happen. Actually, the documentation already did state that $sendmail is a "program and arguments". So it was probably fine. It just didn't explicitly state that it's not an arbitrary shell command. But I see that it's already been improved. I'm sending a patch that adds an error check for shell meta-characters when $sendmail is used. It might have been better to check right after reading .muttrc, but this seemed like a more natural place to put the code (i.e., right after the check that $sendmail exists). cheers, raf
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 19, 2022 at 12:05:59PM -0700, "Kevin J. McCarthy" wrote: > On Thu, May 19, 2022 at 01:12:31PM -0500, Derek Martin wrote: > > On Thu, May 19, 2022 at 11:06:38AM -0700, Kevin J. McCarthy wrote: > > > Sorry, this isn't currently possible in Mutt. The $sendmail variable is > > > handled specially: it's tokenized by space and invoked directly via > > > execvp(). So you won't be able to use arguments with spaces in them > > > inside > > > the variable, or any shell quoting since it doesn't pass through the > > > shell. > > > > Couldn't you do something like the following? > > > > set my_sendmail=/usr/bin/msmtp [...] --passwordeval=$(gpg --no-tty -q -d > > ~/.user.gpg)" > > set sendmail=$my_sendmail > > > > Don't know, haven't tried it, but as long as the first thing evaluates > > to something that sendmail's tokenization can handle, seems like it > > should work... > > $sendmail goes through muttrc evaluation like other variables, but when > invoked to send an email, the resulting string is tokenized by space, '--' > and various arguments inserted as needed, and then passed to execvp(). > > Yes, this should be documented and I'll add it to the config option shortly. > > I don't know why it was written this way, perhaps some security concerns, > but I agree that it's surprising. > > -- > Kevin J. McCarthy > GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA I think the reason would be that mutt is modifying the command, not just executing it. It's modifying a command in a complex external language that it can't parse by itself unless it imposes very strict limitations on the allowable syntax of the command. I don't think it's a security decision. It's just necessary. How would mutt know how to modify sendmail="blah1 | blah2 | blah3" ? I think this limitation should be documented, but not fixed. It could be fixed by adding something like % interpolation placeholders for the things that are added. But that might break existing commands that contain %. Maybe # would be a better choice. But it's still a bad choice. It requires the user to know and use even more notation. An elegant solution already exists. Put complex shell commands in a file and mutt only needs to see its name. But users need to be told when they are required to do that. So in addition to documenting the limitation, a new error message when the evaluated $sendmail contains any (non-space) shell meta-characters would be helpful. That's a clear indication that the user is expecting shell parsing that isn't going to happen. cheers, raf
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 19, 2022 at 05:47:06PM -0500, X Tec wrote: Do you think this could be modified in Mutt's source? As in, kind of a Github-like ticket/issue petition to make it work with arguments with spaces... I try asking because you make it sound as if there were doubts regarding why this is currently coded like this. Most likely I'm wrong, but, do you think a small analysis could be in place? Could it be changed *without* compromising security indeed? I think technically it could be changed (albeit it would require careful analysis of potential issues). And of course you are welcome to open a ticket. However, there are a few things weighing against it likely being done: * Right now, Mutt is in maintenance mode - no new development is taking place. I'm just fixing bugs and security issues in the 2.2.x branch. That may change eventually, or it may not. * The $sendmail behavior has been this way since the first git commit recorded. I'm not saying your attempted usage was incorrect. But given this is the first complaint I've heard, I think it's fair to say almost no one tries to parameterize all the config values to the MTA. So there isn't much demand for the change. * There is some risk of introducing behavior changes, or even a security issue. So again, feel free to open a ticket (noting that tickets should *not* be used as a petition), but don't get your expectations very high. Sorry about that. -- Kevin J. McCarthy GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA signature.asc Description: PGP signature
Re: New to Mutt, unable to send messages in *any* attempted way
On 2022-05-19 12:05:59, Kevin J. McCarthy wrote: > On Thu, May 19, 2022 at 01:12:31PM -0500, Derek Martin wrote: > > On Thu, May 19, 2022 at 11:06:38AM -0700, Kevin J. McCarthy wrote: > > > Sorry, this isn't currently possible in Mutt. The $sendmail variable is > > > handled specially: it's tokenized by space and invoked directly via > > > execvp(). So you won't be able to use arguments with spaces in them > > > inside > > > the variable, or any shell quoting since it doesn't pass through the > > > shell. > > > > Couldn't you do something like the following? > > > > set my_sendmail=/usr/bin/msmtp [...] --passwordeval=$(gpg --no-tty -q -d > > ~/.user.gpg)" > > set sendmail=$my_sendmail > > > > Don't know, haven't tried it, but as long as the first thing evaluates > > to something that sendmail's tokenization can handle, seems like it > > should work... > > $sendmail goes through muttrc evaluation like other variables, but when > invoked to send an email, the resulting string is tokenized by space, '--' > and various arguments inserted as needed, and then passed to execvp(). > > Yes, this should be documented and I'll add it to the config option shortly. > > I don't know why it was written this way, perhaps some security concerns, > but I agree that it's surprising. > Thanks very much sir. I think this finally closes the last question of this -long- email thread... I'm not sure if marking as "SOLVED", like in forum threads, applies here in mailing lists. But if it does, before actually doing it, just a small last question: Do you think this could be modified in Mutt's source? As in, kind of a Github-like ticket/issue petition to make it work with arguments with spaces... I try asking because you make it sound as if there were doubts regarding why this is currently coded like this. Most likely I'm wrong, but, do you think a small analysis could be in place? Could it be changed *without* compromising security indeed? Of course, if it doesn't turn too impractical, neither; and if it's worth Thanks very much for your help.
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 19, 2022 at 01:12:31PM -0500, Derek Martin wrote: On Thu, May 19, 2022 at 11:06:38AM -0700, Kevin J. McCarthy wrote: Sorry, this isn't currently possible in Mutt. The $sendmail variable is handled specially: it's tokenized by space and invoked directly via execvp(). So you won't be able to use arguments with spaces in them inside the variable, or any shell quoting since it doesn't pass through the shell. Couldn't you do something like the following? set my_sendmail=/usr/bin/msmtp [...] --passwordeval=$(gpg --no-tty -q -d ~/.user.gpg)" set sendmail=$my_sendmail Don't know, haven't tried it, but as long as the first thing evaluates to something that sendmail's tokenization can handle, seems like it should work... $sendmail goes through muttrc evaluation like other variables, but when invoked to send an email, the resulting string is tokenized by space, '--' and various arguments inserted as needed, and then passed to execvp(). Yes, this should be documented and I'll add it to the config option shortly. I don't know why it was written this way, perhaps some security concerns, but I agree that it's surprising. -- Kevin J. McCarthy GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA signature.asc Description: PGP signature
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 19, 2022 at 11:06:38AM -0700, Kevin J. McCarthy wrote: > On Thu, May 19, 2022 at 12:36:39PM -0500, X Tec wrote: > > set sendmail="/usr/bin/msmtp [...] --passwordeval=$(gpg --no-tty -q -d > > ~/.user.gpg)" [...] > > Hope someone else could give other advise, or if this is really not > > possible in Mutt, to confirm *why*. > > Sorry, this isn't currently possible in Mutt. The $sendmail variable is > handled specially: it's tokenized by space and invoked directly via > execvp(). So you won't be able to use arguments with spaces in them inside > the variable, or any shell quoting since it doesn't pass through the shell. Couldn't you do something like the following? set my_sendmail=/usr/bin/msmtp [...] --passwordeval=$(gpg --no-tty -q -d ~/.user.gpg)" set sendmail=$my_sendmail Don't know, haven't tried it, but as long as the first thing evaluates to something that sendmail's tokenization can handle, seems like it should work... -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. signature.asc Description: PGP signature
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 19, 2022 at 12:36:39PM -0500, X Tec wrote: But this specific part I'm having trouble with seems to be *not* from Bash or system's shell, but from Mutt *itself*; as in, Mutt's own "local shell" or configuration syntax (not sure how to call it...). Nevertheless, I tried your advise, and got this: msmtp: unrecognized option '--no-tty' Error sending message, child exited 64 (Bad usage.). Could not send the message. Hope someone else could give other advise, or if this is really not possible in Mutt, to confirm *why*. Sorry, this isn't currently possible in Mutt. The $sendmail variable is handled specially: it's tokenized by space and invoked directly via execvp(). So you won't be able to use arguments with spaces in them inside the variable, or any shell quoting since it doesn't pass through the shell. -- Kevin J. McCarthy GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA signature.asc Description: PGP signature
Re: New to Mutt, unable to send messages in *any* attempted way
On 2022-05-19 00:25:32, Jon LaBadie wrote: > So I have NO IDEA IF IT WILL WORK, but you might try: > > set sendmail="/usr/bin/msmtp [...] --passwordeval=$(gpg --no-tty -q -d > ~/.user.gpg)" > Thanks very much for your advise. Actually I had already read about all the Bash stuff (and also other shells) regarding quoting rules, as well as variable/command substitutions. But this specific part I'm having trouble with seems to be *not* from Bash or system's shell, but from Mutt *itself*; as in, Mutt's own "local shell" or configuration syntax (not sure how to call it...). Nevertheless, I tried your advise, and got this: msmtp: unrecognized option '--no-tty' Error sending message, child exited 64 (Bad usage.). Could not send the message. You can see all what else I have tried already in my previous messages in this email "thread"/topic. Hope someone else could give other advise, or if this is really not possible in Mutt, to confirm *why*. Thanks again for your attention.
Re: New to Mutt, unable to send messages in *any* attempted way
Parting from this: > printf "%b\n" "$msg" | mutt -s "Test message" -e 'set > my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set record=""; set > from="Send User<$my_user>"; set envelope_from=yes; set > sendmail="/path/to/msmtp --port=587 --tls=on --host=$my_url --auth=on > --user=$my_user"' recei...@domain.tld > > Trying to add "--" did not work at all, no matter where I put it: > [...] sendmail="[...] --"' recei...@domain.tld > [...] sendmail="[...]"' -- recei...@domain.tld > By searching in the old Msmtp mailing lists, it seems that Msmtp is intended primarily to send email from standard input (console); when called by an external program, such as Mutt in this case, direct access to tty is normally lost and thus no prompt possible. So this seems actually expected behavior. > > - Why you say '"--password" is not a valid command line option'. > > > Yes, according to manual "password" is a valid command for .msmtprc; but > there's no "--password" option for command line. I tried this last one and > got something like "invalid or unknown command". > This seems to be by design as well. However, I still haven't been able to solve this: > The passwordeval with gpg method also works, albeit I wasn't able to figure > out how to use it directly in command line... Trying to use > > set sendmail="/usr/bin/msmtp [...] --passwordeval="gpg --no-tty -q -d > ~/.user.gpg"" > > got this: > > Error in command line: --no-tty: unknown variable > gpg: WARNING: no command supplied. Trying to guess what you mean ... > gpg: processing message failed: Unknown system error > msmtp: cannot read output of 'gpg' > Error sending message, child exited 78 (). > Could not send the message. > > I did figure it's an issue with wrongly used quotes (and the gpg command > needs them), but still haven't been able to solve it... > I tried asking in a Bash forum, but they said this is Mutt explicit question rather than shell quoting stuff. Indeed, actually trying it in .muttrc and using 'printf "%s\n" "some text" | mutt -s "Some subject" recei...@domain.tld' in command line for sending email yielded the very same error. I tried everything I could imagine: single/double quotes mixing, escaping space chars with '\', etc. None of them worked. Could someone help with this last issue by chance, please? Thanks again beforehand.
Re: New to Mutt, unable to send messages in *any* attempted way
On Wed, May 11, 2022 at 09:51:28PM +0100, Sam Kuper wrote: > Once you know your shell's quoting syntax, you will see that you can > probably achieve your goal in any of several different ways. Which to > use is a matter of taste. > > E.g. > > '/home/.user.gpg' > > vs > > "~"'/.user.gpg' Sorry, my first example above should have been: '/home//.user.gpg' -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
Re: New to Mutt, unable to send messages in *any* attempted way
On Wed, May 11, 2022 at 10:54:19AM -0500, X Tec wrote: > Managed to briefly test in a more updated rig with Mutt 2.2.4 and > Msmtp 1.8.11. Good job. I'm in haste, so am skipping over the msmtp password oddness. Maybe another list subscriber will take that up. > The passwordeval with gpg method also works, albeit I wasn't able to > figure out how to use it directly in command line... Trying to use > > set sendmail="/usr/bin/msmtp [...] --passwordeval="gpg --no-tty -q -d > ~/.user.gpg"" > > got this: > > Error in command line: --no-tty: unknown variable > gpg: WARNING: no command supplied. Trying to guess what you mean ... > gpg: processing message failed: Unknown system error > msmtp: cannot read output of 'gpg' > Error sending message, child exited 78 (). > Could not send the message. > > I did figure it's an issue with wrongly used quotes (and the gpg > command needs them), but still haven't been able to solve it... https://tldp.org/LDP/abs/html/quotingvar.html https://mywiki.wooledge.org/Quotes Once you know your shell's quoting syntax, you will see that you can probably achieve your goal in any of several different ways. Which to use is a matter of taste. E.g. '/home/.user.gpg' vs "~"'/.user.gpg' If you still find yourself stuck (or even if you don't), you might want to try using Pass as a wrapper around GPG: https://en.wikipedia.org/wiki/Pass_%28software%29 > On 2022-05-11 09:03:56, Cameron Simpson wrote: >> On 10May2022 07:25, Sam Kuper wrote: >>> On Mon, May 09, 2022 at 11:01:20PM -0500, x...@trimaso.com.mx wrote: ---To respond a received email in Mutt pager I hit 'r', and all the rest. I only change the destination email address, and eventually send. But even after successfully sent, the "responded" email in Mutt pager is not marked with 'r'. Why? >>> >>> Maybe because in your examples above, you set the record variable to >>> ""? >>> >>> I may be wrong, but: I think that in order for Mutt to know whether >>> a message has been replied to, it checks the mailbox specified by >>> the record variable - so if that variable is empty, Mutt has no way >>> of checking. >> >> I thought it just set a flag on the message. >> >> I forget, is XTec using a local or IMAP mail folder? > > Current IMAP settings (using Starttls): > set folder = "imap://u...@domain.tld@smtp.domain.tld" > set spoolfile = +INBOX > mailboxes = +INBOX > > Also, > set record = "" > set date_format = "%F %T" > set index_format = "%4C %Z %D %-15.15L (%?l?%4l&%4c?) %s" > set sort = reverse-date > > By using '$' I just get "Mailbox is unchanged", even when I know there > are new messages. The only thing that *seemed* to work was "bind > index G imap-fetch-mail" in .muttrc. With this, I could use 'G' to > refresh for newly received emails Still, I kind of expected a > "default" key for that... > > > > ---Just like Msmtp, if the better idea is to not store passwords in > plain text, doesn't Mutt have some type of password "masking" or > something? With GPG, hashed into another file somewhere else, etc... Yes, Mutt does have a way to achieve that. I hate so say "Read The Friendly Manual", but within https://www.mutt.org/doc/manual/#muttrc-syntax you will find this example: set imap_pass="`gpg --batch -q --decrypt ~/.mutt/account.gpg`" https://www.mutt.org/doc/manual/#ex-backtick-dblquotes Best, Sam -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
Re: New to Mutt, unable to send messages in *any* attempted way
Managed to briefly test in a more updated rig with Mutt 2.2.4 and Msmtp 1.8.11. On Tue, May 10, 2022 at 07:25:02AM +, Sam Kuper wrote: > Consider adding ` --` (without backticks) to the end of the > sendmail string in your example above. > printf "%b\n" "$msg" | mutt -s "Test message" -e 'set my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set record=""; set from="Send User<$my_user>"; set envelope_from=yes; set sendmail="/path/to/msmtp --port=587 --tls=on --host=$my_url --auth=on --user=$my_user"' recei...@domain.tld Trying to add "--" did not work at all, no matter where I put it: [...] sendmail="[...] --"' recei...@domain.tld [...] sendmail="[...]"' -- recei...@domain.tld If using the more standard method, without all the msmtp parameters above and using .msmtprc instead, it must have *all* the complete needed commands; if it only has the "password" line it fails with the same error as well. So for this to work, I had to make a ~/.netrc file with content: machine smtp.domain.tld login u...@domain.tld password p4ss (Yes, Msmtp manual says this method is deprecated...) > - Why you say '"--password" is not a valid command line option'. > Yes, according to manual "password" is a valid command for .msmtprc; but there's no "--password" option for command line. I tried this last one and got something like "invalid or unknown command". The passwordeval with gpg method also works, albeit I wasn't able to figure out how to use it directly in command line... Trying to use set sendmail="/usr/bin/msmtp [...] --passwordeval="gpg --no-tty -q -d ~/.user.gpg"" got this: Error in command line: --no-tty: unknown variable gpg: WARNING: no command supplied. Trying to guess what you mean ... gpg: processing message failed: Unknown system error msmtp: cannot read output of 'gpg' Error sending message, child exited 78 (). Could not send the message. I did figure it's an issue with wrongly used quotes (and the gpg command needs them), but still haven't been able to solve it... On 2022-05-11 09:03:56, Cameron Simpson wrote: > On 10May2022 07:25, Sam Kuper wrote: > >On Mon, May 09, 2022 at 11:01:20PM -0500, x...@trimaso.com.mx wrote: > >> ---To respond a received email in Mutt pager I hit 'r', and all the > >> rest. I only change the destination email address, and eventually > >> send. But even after successfully sent, the "responded" email in Mutt > >> pager is not marked with 'r'. Why? > > > >Maybe because in your examples above, you set the record variable to ""? > > > >I may be wrong, but: I think that in order for Mutt to know whether a > >message has been replied to, it checks the mailbox specified by the > >record variable - so if that variable is empty, Mutt has no way of > >checking. > > I thought it just set a flag on the message. > > I forget, is XTec using a local or IMAP mail folder? > Current IMAP settings (using Starttls): set folder = "imap://u...@domain.tld@smtp.domain.tld" set spoolfile = +INBOX mailboxes = +INBOX Also, set record = "" set date_format = "%F %T" set index_format = "%4C %Z %D %-15.15L (%?l?%4l&%4c?) %s" set sort = reverse-date By using '$' I just get "Mailbox is unchanged", even when I know there are new messages. The only thing that *seemed* to work was "bind index G imap-fetch-mail" in .muttrc. With this, I could use 'G' to refresh for newly received emails Still, I kind of expected a "default" key for that... ---Just like Msmtp, if the better idea is to not store passwords in plain text, doesn't Mutt have some type of password "masking" or something? With GPG, hashed into another file somewhere else, etc... Thanks again for your attention.
Re: New to Mutt, unable to send messages in *any* attempted way
On 10May2022 07:25, Sam Kuper wrote: >On Mon, May 09, 2022 at 11:01:20PM -0500, x...@trimaso.com.mx wrote: >> ---To respond a received email in Mutt pager I hit 'r', and all the >> rest. I only change the destination email address, and eventually >> send. But even after successfully sent, the "responded" email in Mutt >> pager is not marked with 'r'. Why? > >Maybe because in your examples above, you set the record variable to ""? > >I may be wrong, but: I think that in order for Mutt to know whether a >message has been replied to, it checks the mailbox specified by the >record variable - so if that variable is empty, Mutt has no way of >checking. I thought it just set a flag on the message. I forget, is XTec using a local or IMAP mail folder? Cheers, Cameron Simpson
Re: New to Mutt, unable to send messages in *any* attempted way
On Mon, May 09, 2022 at 11:01:20PM -0500, x...@trimaso.com.mx wrote: > [...] In the end, I chose to begin with Msmtp instead of Postfix... > Why? Because after just installation, I realized that Postfix is kind > of "overkill" if just wanting to send: it's designed as a full-blown > server for sending and receiving. Msmtp, on the other hand, is just a > SMTP client; no need to setup a whole server. [...] Good call :) If you're using `msmtp`, you may already have read about `msmtpq`. If not, have a quick look at this thread in case it is helpful to you :) http://lists.mutt.org/pipermail/mutt-users/Week-of-Mon-20210208/002485.html > So first read the Msmtp documentation, and then for sending I first > tried: > > printf "%b\n" "$msg" | mutt -s "Test message" -e 'set > my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set > record=""; set sendmail="/path/to/msmtp --port=587 --tls=on > --host=$my_url --from="SendUser<$my_user>" --auth=on --user=$my_user"' > recei...@domain.tld > > Incientally "--password" is not a valid command line option; so I was > expecting to be prompted for the password, but instead I got this: > > msmtp: authentication method PLAIN needs a password > msmtp: could not send mail > Error sending message, child exited 69 (Service unavailable.). > Could not send the message. I'm not sure: -Why the command line you gave above did not prompt you for a password. As the msmtp docs ( https://marlam.de/msmtp/msmtp.html ) say, "Password method 5: Do not specify a password. Msmtp will then prompt you for it. This means you need to be able to type into a terminal when msmtp runs." Recent versions of Mutt automatically insert a `--` delimiter between the `sendmail` string and the recipient addresses before invoking the chosen MTA. Perhaps your version of Mutt does not, and perhaps this is somehow causing a problem? Consider adding ` --` (without backticks) to the end of the sendmail string in your example above. - Why you say '"--password" is not a valid command line option'. According to msmtp docs, "all settings can also be configured on the command line", and there is indeed a "--password" option: "‘password [secret]’ Set the password for authentication. An empty argument unsets the password. Consider using the ‘passwordeval’ command or a key ring instead of this command, to avoid storing cleartext passwords in the configuration file." > Even after making a very minimalist .msmtprc with *only* the > "password" line (trying plain password for now just for testing), I > kept getting the above error. I did chmod .msmtprc to 600, BTW. > > So had to make a full .msmtprc according to documentation, and tried > this command: > > printf "%b\n" "$msg" | mutt -s "Test message" -e 'set > my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set > record=""; set from="Send User<$my_user>"; set envelope_from=yes; set > sendmail="/path/to/msmtp"' recei...@domain.tld > > This time it succeeded. > > > > ---So, do I really need to set password apart in some of Msmtp's other > 4 ways in order to successfully use it? In principle, no. msmtp's method 5 (entering the password manually at the terminal) should work. If you cannot get it to work with your version of Mutt, then try it with your version of heirloom mailx. (Perhaps your version of Mutt, which is very old, has a bug that is somehow causing a problem here?) If for some reason it still doesn't work, then perhaps someone else here will spot the reason why. If not, then *maybe* msmtp has a bug. That said, do you really want to manually enter the password each time you send an email? In the long run, you might be better off using method 1 or 2 instead. > ---To respond a received email in Mutt pager I hit 'r', and all the > rest. I only change the destination email address, and eventually > send. But even after successfully sent, the "responded" email in Mutt > pager is not marked with 'r'. Why? Maybe because in your examples above, you set the record variable to ""? I may be wrong, but: I think that in order for Mutt to know whether a message has been replied to, it checks the mailbox specified by the record variable - so if that variable is empty, Mutt has no way of checking. Sam -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
Re: New to Mutt, unable to send messages in *any* attempted way
Sorry for late answering... On Sun, May 08, 2022 at 04:25:05PM +1000, Cameron Simpson wrote: You're failing in this because you have $smtp_url defined. Comment out it and the other $smtp* variables. When you send with $sendmail (the local system's mail system, postfix for you) the smtp settings come from the main.cf file, not from mutt. OUCH So all this whole time I have been using Mutt's builtin SMTP, the less orthodox way according to Mutt wiki... And seemingly Heirloom Mailx also works this exact way when specifying SMTP options as well >By the way, in Mutt default pager for reading emails, your words between underscores '_' (_not_, _may_, etc...) are not being displayed... Why? I think someone suggested turning off any colouring settings you have active. Possibly some colours match the terminal background colour. Are the words invisible (gaps in the text) or missing (no gaps)? You all were right: setting "uncolor body *" solved it, at least in the Mutt version I'm currently using. I'd really need to try latest release and compare. Hope I'm able to do soon... So finally tried using Mutt the orthodox way: with external sendmail SMTP/MTA. In the end, I chose to begin with Msmtp instead of Postfix... Why? Because after just installation, I realized that Postfix is kind of "overkill" if just wanting to send: it's designed as a full-blown server for sending and receiving. Msmtp, on the other hand, is just a SMTP client; no need to setup a whole server. So first read the Msmtp documentation, and then for sending I first tried: printf "%b\n" "$msg" | mutt -s "Test message" -e 'set my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set record=""; set sendmail="/path/to/msmtp --port=587 --tls=on --host=$my_url --from="SendUser<$my_user>" --auth=on --user=$my_user"' recei...@domain.tld Incientally "--password" is not a valid command line option; so I was expecting to be prompted for the password, but instead I got this: msmtp: authentication method PLAIN needs a password msmtp: could not send mail Error sending message, child exited 69 (Service unavailable.). Could not send the message. Even after making a very minimalist .msmtprc with *only* the "password" line (trying plain password for now just for testing), I kept getting the above error. I did chmod .msmtprc to 600, BTW. So had to make a full .msmtprc according to documentation, and tried this command: printf "%b\n" "$msg" | mutt -s "Test message" -e 'set my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set record=""; set from="Send User<$my_user>"; set envelope_from=yes; set sendmail="/path/to/msmtp"' recei...@domain.tld This time it succeeded. ---So, do I really need to set password apart in some of Msmtp's other 4 ways in order to successfully use it? ---To respond a received email in Mutt pager I hit 'r', and all the rest. I only change the destination email address, and eventually send. But even after successfully sent, the "responded" email in Mutt pager is not marked with 'r'. Why? Thanks beforehand yet again.
Re: New to Mutt, unable to send messages in *any* attempted way
On 07May2022 11:11, X Tec wrote: >On 2022-05-07 08:45:35, Cameron Simpson wrote: >> Probably not? If you've been debugging the $smtp* variables (in >> particular, having to debug the password) then it sounds like mutt is >> actually sending directly with SMTP and _not_ using the local postfix. >> >> When you send with the local postfix, mutt just hands the message off to >> the sendmail command and doesn't say anything about delivery - that is >> sendmail's job. >> >I double-checked Mutt's current $sendmail variable, and it's certainly the >default "/usr/sbin/sendmail" command mentioned in the manual. >The route exists, and it's by Postfix package in the system I use. The manual says that setting $smtp_url overrides $sendmail. So your $sendmail setting is not in play. >Also, I tried sending email to myself, and checked the full headers. It does >have the TLS thing you mention BTW, but on the other hand it has "User-Agent: >Mutt 1.10.1"... These are not in conflict. Anything you send with mutt normally has a mutt user-agent header. Have a look at the received headers. They should trace the path from where you sent the email from through to how it was delivered. >Also, Mutt does not say additional stuff or command line output (just exit >code '0') when sending email... >I'm trying to use Mutt with external MTA/SMTP (Postfix in this case), as the >correct way intended, instead of Mutt's builtin SMTP. >Am I failing? If so, why? You're failing in this because you have $smtp_url defined. Comment out it and the other $smtp* variables. When you send with $sendmail (the local system's mail system, postfix for you) the smtp settings come from the main.cf file, not from mutt. >By the way, in Mutt default pager for reading emails, your words between >underscores '_' (_not_, _may_, etc...) are not being displayed... Why? I think someone suggested turning off any colouring settings you have active. Possibly some colours match the terminal background colour. Are the words invisible (gaps in the text) or missing (no gaps)? >If sending email from the webmail, I'm sure it gets send from the email >address account/SMTP. This is because the ISP itself is running the delivery, either by internal SMTP or its own sendmail equivalent. >Same if sending from the popular "official" email clients (Outlook, >Thunderbird...) That's because they're using SMTP, probably the the ISP SMTP service with a username/password (or better) authentication. >But *not* sure when sending from Mutt... According to the manual, if you set $smtp_url mutt will send with SMTP (or try) and send with $sendmail otherwise. >And finally, what key to manually force to check for new mail, instead >of waiting or quitting and starting Mutt again? The "$" key is usually bound to the command, which updates the state of the current folder. >Evidently just doing "any" activity in Mutt does not refresh/fetch new >email... See http://www.mutt.org/doc/manual/#new-mail (particularly new mail Detection) and http://www.mutt.org/doc/manual/#mailboxes Cheers, Cameron Simpson
Re: New to Mutt, unable to send messages in *any* attempted way
On Sat, May 07, 2022 at 11:11:35AM -0500, X Tec wrote: By the way, in Mutt default pager for reading emails, your words between underscores '_' (_not_, _may_, etc...) are not being displayed... Why? Possibly your color settings, or the color settings shipped with your packaged version of mutt. Try adding 'uncolor body *' to the bottom of your muttrc to remove all body coloring add see if that makes a difference. "man mutt" says that: -n Do not read the system-wide Muttrc configuration file. If you're getting an exit 1, I'd expect some kind of error message as well. No command line output whatsoever; only an exit '1' code when checking with "echo $?" I can't duplicate this, but it's possible a bug got fixed related to this. If you could specify the exact command you are running I'll try to look at it again. When browsing emails in Mutt, why do sizes inside () -for example, (1.1K)- change to something different and smaller -( 6)- when hitting enter to display the message? The IMAP code updates the message body size when it actually downloads the message. When opening the mailbox, it only displays an estimate based on the subset of headers it downloads. -- Kevin J. McCarthy GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA signature.asc Description: PGP signature
Re: New to Mutt, unable to send messages in *any* attempted way
Hi X Tec, Not sure if you have read this web page? https://gitlab.com/muttmua/mutt/-/wikis/MailConcept You may find life simpler if, following the advice there, you apply the concept of "separation of concerns". https://en.wikipedia.org/wiki/Separation_of_concerns In other words: - use an MRA such as Getmail, Fetchmail, or Retchmail to retrieve your mail from the server; and - either: - (If you don't need complex filtering) use the MRA's built-in MDA to write the retrieved mail to a local inbox. Ideally, a Maildir directory in some sensible location like ~/inbox ; or - (If you do need complex filtering) use a separate MDA like Procmail or Maildrop to perform that processing and to deliver the mails into relevant Maildirs, e.g. ~/mail/personal and ~/mail/work ; and - use Mutt solely as an MUA; and - use either Postfix as your MTA (if you're already good at Postfix), or perhaps better still, use a lightweight MTA like msmtp, which is much easier to configure and manage, and is more than powerful enough for a typical single user. This way, you are playing to each tool's strengths. You also will be able to troubleshoot more easily under this approach, because MTA-related config should live *solely* within the MTA's config file(s); MUA-related config *solely* within the MUA's config file(s); etc. On Sat, May 07, 2022 at 11:11:35AM -0500, X Tec wrote: > Also, Mutt does not say additional stuff or command line output (just > exit code '0') when sending email... 0 means "success", so that's reasonable. > I'm trying to use Mutt with external MTA/SMTP (Postfix in this case), > as the correct way intended, instead of Mutt's builtin SMTP. > > Am I failing? If so, why? Unsure. Consider a simpler MTA such as msmtp. > By the way, in Mutt default pager for reading emails, your words > between underscores '_' (_not_, _may_, etc...) are not being > displayed... Why? Have you searched the bug tracker? https://gitlab.com/muttmua/mutt/-/issues/ >>> Then how do I know the email is really being sent from >>> u...@domain.tld 's account/SMTP? >> >> Hahahaha! You don't! >> [...] >> > Maybe I'm completely confused and lost... > > If sending email from the webmail, I'm sure it gets send from the > email address account/SMTP. Many webmail interfaces map directly to an underlying email account. I guess that's the situation you've encountered. Not all of them do, though! Some webmail instances let users send email from multiple different accounts, or even from one account via another account's SMTP server! (I'm not making a value judgement here: not saying that one approach is right and the other is wrong. Just stating facts.) > Same if sending from the popular "official" email clients (Outlook, > Thunderbird...) Yes, those clients typically map one email address to one SMTP server. > But *not* sure when sending from Mutt... Correct. Mutt is a MUA. It is a very powerful MUA. It does have some MTA functionality, but it's up to you to configure that functionality (or, perhaps better, configure a standalone MTA) to suit you. > And finally, what key to manually force to check for new mail, instead > of waiting or quitting and starting Mutt again? > > Evidently just doing "any" activity in Mutt does not refresh/fetch new > email... In recent(ish) versions of Mutt, activity (e.g. using a cursor key to move up or down the index) *does* cause Mutt to check for new mail in the currently-displayed mailbox (i.e. mbox or Maildir). That's not the same as fetching mail, though. If you want to *fetch* mail, then you need to invoke your MRA (which could be Mutt, but could also be Getmail, Fetchmail, or Retchmail, etc). Sam
Re: New to Mutt, unable to send messages in *any* attempted way
On 2022-05-07 08:45:35, Cameron Simpson wrote: > Probably not? If you've been debugging the $smtp* variables (in > particular, having to debug the password) then it sounds like mutt is > actually sending directly with SMTP and _not_ using the local postfix. > > When you send with the local postfix, mutt just hands the message off to > the sendmail command and doesn't say anything about delivery - that is > sendmail's job. > I double-checked Mutt's current $sendmail variable, and it's certainly the default "/usr/sbin/sendmail" command mentioned in the manual. The route exists, and it's by Postfix package in the system I use. Also, I tried sending email to myself, and checked the full headers. It does have the TLS thing you mention BTW, but on the other hand it has "User-Agent: Mutt 1.10.1"... Also, Mutt does not say additional stuff or command line output (just exit code '0') when sending email... I'm trying to use Mutt with external MTA/SMTP (Postfix in this case), as the correct way intended, instead of Mutt's builtin SMTP. Am I failing? If so, why? By the way, in Mutt default pager for reading emails, your words between underscores '_' (_not_, _may_, etc...) are not being displayed... Why? > >Then how do I know the email is really being sent from > >u...@domain.tld 's account/SMTP? > > Hahahaha! You don't! > [...] > Maybe I'm completely confused and lost... If sending email from the webmail, I'm sure it gets send from the email address account/SMTP. Same if sending from the popular "official" email clients (Outlook, Thunderbird...) But *not* sure when sending from Mutt... > "man mutt" says that: > > -n Do not read the system-wide Muttrc configuration file. > > If you're getting an exit 1, I'd expect some kind of error message as > well. > No command line output whatsoever; only an exit '1' code when checking with "echo $?" > >---When browsing emails in Mutt, why do sizes inside () -for example, > >(1.1K)- change to something different and smaller -( 6)- when hitting > >enter to display the message? > > They don't for me. Are you sure you're not confusing the index lines > (one per message, specified by $index_format) with the header shown > above a specific message when you're viewing it? Got an example? > 1 2022-05-06 17:45:35 Cameron Simpson ( 11K) Re: New to Mutt, unable to send messages in *any* attempted way And when hitting enter to display message: 1 2022-05-06 17:45:35 Cameron Simpson ( 158) Re: New to Mutt, unable to send messages in *any* attempted way (I changed $date_format and $index_format to custom values) And finally, what key to manually force to check for new mail, instead of waiting or quitting and starting Mutt again? Evidently just doing "any" activity in Mutt does not refresh/fetch new email...
Re: New to Mutt, unable to send messages in *any* attempted way
On 06May2022 12:28, x...@trimaso.com.mx wrote: >Thanks again. > >As mentioned in first message, system has Postfix by default, with its >corresponding /usr/sbin/sendmail and /etc/postfix/main.cf . >Since I don't have anything for $sendmail variable, it's defaulting >indeed for Postix sendmail. Probably not? If you've been debugging the $smtp* variables (in particular, having to debug the password) then it sounds like mutt is actually sending directly with SMTP and _not_ using the local postfix. When you send with the local postfix, mutt just hands the message off to the sendmail command and doesn't say anything about delivery - that is sendmail's job. >On Thu, May 05, 2022 at 19:19:45 -0500, Cameron Simpson wrote: >>Might be. It won't be a "real" Message-ID line as it looks like the log >>tries to include the username of the user who sent the email. >> >>However, you're using SMTP to smtp.domain.tld, which means you're not >>using the local mail system, which means that it should not be logging >>locally for email you send using mutt. >> >Sorry... By "log" here I meant the default ~/sent one, default set for >$record variable Ah, do this is a real message-id field then. Message ids are specified here: https://tools.ietf.org/rfcmarkup/5322#section-3.6.4 I'm not sure you can put a second "@" in a message id - my reading of the grammar says it basicly looks like "". >>No, it is just what gets written into the heder line so that people >>know >>what address to use for replies and citations. >> >Then how do I know the email is really being sent from >u...@domain.tld 's account/SMTP? Hahahaha! You don't! There are various headers which _may_ be added to messages and which a receiver _may_ consult to verify identity, but they're not appplied by default. (They inherently require extra setup, as you need to provide the extra identity in your config.) There are headers which confirm that an ISP sent the message - that would tell the receiver that it can from _where_ it says it comes from, but not from _who_. There are DNS records (SPF) which specify where messages for a domain may come _from_. These can be used by receiving systems to reject messages from other sources claiming to be from that domain. These are often loosely defined though, if present at all. Of course, you're then trusting the receiving system on this point when you collect your email. >>Mutt has some debug flags. Try using the "-d 5" option. >> >Got: "DEBUG was not defined during compilation. Ignored." Oh, that is unfortunate. You might need to built it from source unless your vendor provides some kind of "debugging enabled" version of the mutt package. >Any other ways to tell whether email is really being sent with STARTTLS? The Received: headers of the message (when you get it back) record how the message was received (possibly there will be several of these). For example, this message of yours as received by me here has this header for the final hop: Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.cskk.id.au (Postfix) with ESMTPS id 773F343999 for ; Fri, 6 May 2022 17:35:24 + (UTC) The second line doesn't say that STARTTLS was used, just that TLS was used. That implies either a cleartext connect with a STARTTLS, or a TLS/SSL connection directly. Encrypted either way. Of course, you need to trust the content of those headers. >On Thu, May 05, 2022 at 23:06:15 -0500, Nathan Stratton Treadway wrote: >>Some ISPs specifically block outgoing traffic on TCP port 25 (which is >>the default port for "smtp:" traffic) >> >Which means, if not specifying port then Mutt (or Postfix?) takes port >25 as SMTP default... >Wasn't/shouldn't port 587 the new default for SMTP? >So Heirloom Mailx also takes port 25 as default? Maybe? You could check its docs or help messages. You could get really funky and watch it send your email. here's me testing the "nc" command on a local Ubuntu system: [~]borg*> strace -e trace=network nc 127.0.0.1 25 Also, some more doubts aroused: > >---If adding the "-n" option to whole email sending command, it does >*nothing*, just immediately getting an exit status of '1'. What's >happening with this? "man mutt" says that: -n Do not read the system-wide Muttrc configuration file. If you're getting an exit 1, I'd expect some kind of error message as well. >---Where in the manual can I find the meaning of each of the %Z >letters here http://www.mutt.org/doc/manual/#index-format ? I mean, >the letters norON, DdSPsK... It's right there in the table: %Za three character set of message status flags. the first character is new/read/replied flags (“n”/“o”/“r”/“O”/“N”). the second is deleted or encryption flags (“D”/“d”/“S”/“P”/“s”/“K”).
Re: New to Mutt, unable to send messages in *any* attempted way
Thanks again. As mentioned in first message, system has Postfix by default, with its corresponding /usr/sbin/sendmail and /etc/postfix/main.cf . Since I don't have anything for $sendmail variable, it's defaulting indeed for Postix sendmail. On Thu, May 05, 2022 at 19:19:45 -0500, Cameron Simpson wrote: Might be. It won't be a "real" Message-ID line as it looks like the log tries to include the username of the user who sent the email. However, you're using SMTP to smtp.domain.tld, which means you're not using the local mail system, which means that it should not be logging locally for email you send using mutt. Sorry... By "log" here I meant the default ~/sent one, default set for $record variable No, it is just what gets written into the heder line so that people know what address to use for replies and citations. Then how do I know the email is really being sent from u...@domain.tld 's account/SMTP? Mutt has some debug flags. Try using the "-d 5" option. Got: "DEBUG was not defined during compilation. Ignored." Any other ways to tell whether email is really being sent with STARTTLS? On Thu, May 05, 2022 at 23:06:15 -0500, Nathan Stratton Treadway wrote: Some ISPs specifically block outgoing traffic on TCP port 25 (which is the default port for "smtp:" traffic) Which means, if not specifying port then Mutt (or Postfix?) takes port 25 as SMTP default... Wasn't/shouldn't port 587 the new default for SMTP? So Heirloom Mailx also takes port 25 as default? Also, some more doubts aroused: ---If adding the "-n" option to whole email sending command, it does *nothing*, just immediately getting an exit status of '1'. What's happening with this? ---Where in the manual can I find the meaning of each of the %Z letters here http://www.mutt.org/doc/manual/#index-format ? I mean, the letters norON, DdSPsK... ---When browsing emails in Mutt, why do sizes inside () -for example, (1.1K)- change to something different and smaller -( 6)- when hitting enter to display the message? Thanks again.
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 05, 2022 at 15:11:48 -0500, x...@trimaso.com.mx wrote: > I didn't mean email provider, but ISP internet service I'm connected to. > And did test again: I connected to an internet network, did not specify > port in smtp_url, tried send email, and got: > Could not connect to smtp.domain.tld (No route to host) > Tried connecting to a different internet network, tried to send email > using *exact* command, without port, and *it succceeded*. Some ISPs specifically block outgoing traffic on TCP port 25 (which is the default port for "smtp:" traffic) -- one such example is documented at: https://www.xfinity.com/support/articles/list-of-blocked-ports My experience with Comcast residental-internet service is that the port 25 packets are simply dropped, so SMTP connection attempts hang and eventually time out... but it seems that the ISP you used in your first attempt instead blocks the traffic with a "no route to host" reply. This seems a bit misleading, but if port 587 traffic to that same host does get through successfully while port 25 traffic does not, it seems port-specific blocking of your traffic by that partiticlar ISP is the most likely explanation Nathan
Re: New to Mutt, unable to send messages in *any* attempted way
On 05May2022 15:11, x...@trimaso.com.mx wrote: >On Thu, May 05, 2022 at 12:07:44PM -0500, Kevin J. McCarthy wrote: >>1) On the command line, the shell will expand shell variables inside >>double quoted strings, before Mutt even sees it. [...] >First: had to delete/rename ~/.muttrc, because some previous settings >were perhaps conflicting... Always good to get the config as simple as possible. >This seemed to *finally* work (had to play with both single and double >quotes...): > >printf "%b\n" "$msg" | mutt -s "Test message" -e 'set >my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set >from="Send User "; set >smtp_url="smtp://$my_user:my_p4ss@$my_url:587"; set >ssl_starttls="yes"; set ssl_force_tls="yes"' recei...@domain.tld This is why using a config file is often better. If you had: set my_user="u...@domain.tld" set my_url="smtp.domain.tld" set from="Send User " set smtp_url="smtp://$my_user:my_p4ss@$my_url:587" set ssl_starttls="yes" set ssl_force_tls="yes" in some file "muttrc-test" you can then test with: printf "%b\n" "$msg" | mutt -F muttrc-test -s "Test message" recei...@domain.tld much more easily. >Finally worked -seemingly-, but popped just many doubts: >---There's a "sent email" log in local system, where sent emails are >logged. There's always this line: >Message-ID: < @ >mylocalpcu...@localhost.myisp.com > >Is this correct? Might be. It won't be a "real" Message-ID line as it looks like the log tries to include the username of the user who sent the email. However, you're using SMTP to smtp.domain.tld, which means you're not using the local mail system, which means that it should not be logging locally for email you send using mutt. Personally, I configure my local postfix install's main.cf (+etc) to send email from my laptop, and just have mutt deliver to the local sendmail. That gets me: local logging, local queue (good if offline), etc and a single place for the config (postfix). And anything else on my laptop can also send email. Such as cron etc. >---Without "set from=" field, sender becomes >"mylocalpcu...@localhost.myisp.com"... What is this? Probably defaults. In the good old days of shared standalone machines, localusername@machine_hostname would sometimes be a valid working address. And it is still all that can be guessed without any other config. However, these days machine_hostname is almost never a valid email domain, and you want to configure the domain to be your usual ISP based domain. For example, my postfix main.cf has: mydomain = cskk.id.au myorigin = cskk.id.au even though that domain is actually hosted on another machine, so that email from my laptop has a meaningful From: line. You're doing the same thing for mutt with "set from". >Does the "from" field kind of guarantee that email is being *really* >sent from the u...@domain.tld address, and not from *local rig*? No, it is just what gets written into the heder line so that people know what address to use for replies and citations. >---In "set smtp_url" field, if using "$my_user:my_p4ss" notation, this >seems to override the "set smtp_pass" field completely? > >---Seemingly not needed smtp_authenticators... ?? Some ISPs allow unauthenticated email from "trusted" networks. For example, my home server accepts unauthenticated email from machines on my local LAN. An ISP _might_ accept it from the clients, but that seems more dubious. >---Without smtp_url and smtp_pass fields, where does email go? >recei...@domain.tld doesn't receive it... Probably delivered to the local mail system. If that can already deliver email, it will be doing the outbound SMTP for you. >---Is email really being sent with STARTTLS, as wanted? How can I tell? Mutt has some debug flags. Try using the "-d 5" option. >---In "set from=" field, spaces between "Send User" and actual email >address... don't seem to matter? The $from is for the "From:" header line, which accepts a full RFC2822 email address which has a full name and username@hostname part, which can be written like: Cameron Simpson which I prefer or: c...@cskk.id.au (Cameron Simpson) which is very common, and I find less pleasant to look at. Only the c...@cskk.id.au part matters for email delivery. (And the "From:" doesn't affect delivery for you, but it gets used by people replying, so it affects them.) >>Are you able to send email via that account using other applications? > >Yes, I used to use Heirloom Mailx. That should give you something to copy. >I didn't mean email provider, but ISP internet service I'm connected >to. >And did test again: I connected to an internet network, did not >specify port in smtp_url, tried send email, and got: >Could not connect to smtp.domain.tld (No route to host) >Tried connecting to a different internet network, tried to send email >using *exact* command, without port, and *it succceeded*. >WTH? Maybe that only works from a specifi
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 05, 2022 at 03:11:48PM -0500, x...@trimaso.com.mx wrote: > On Thu, May 05, 2022 at 12:07:44PM -0500, Kevin J. McCarthy wrote: >> 1) On the command line, the shell will expand shell variables inside >> double quoted strings, before Mutt even sees it. >> >> So in the part "set smtp_url=smtp://$my_user@$my_url" Mutt is >> probably only seeing "set smtp_url=smtp://@", because $my_user and >> $my_url are unset in your shell. You may want to use single-quotes >> around the whole "-e" expression, but then be careful with nested >> single quotes, e.g. the set from='User' part. > > First: had to delete/rename ~/.muttrc, because some previous settings > were perhaps conflicting... Good call. > This seemed to *finally* work (had to play with both single and double > quotes...): > > printf "%b\n" "$msg" | mutt -s "Test message" -e 'set > my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set > from="Send User "; set > smtp_url="smtp://$my_user:my_p4ss@$my_url:587"; set > ssl_starttls="yes"; set ssl_force_tls="yes"' recei...@domain.tld > > Finally worked -seemingly- Great! > but popped just many doubts: > > ---There's a "sent email" log in local system, where sent emails are > logged. There's always this line: > Message-ID: < @ > mylocalpcu...@localhost.myisp.com > > Is this correct? Almost certainly, yes. Understandably, you may prefer not to divulge that information to the recipients of your emails. I forget how to effect that for the username part, but for the hostname I think it's as simple as setting Mutt's `$hostname` variable. If you have multiple accounts, you may wish to do that via hooks, e.g.: send2-hook '~f @account1\.net$' 'set hostname = "account1.net"' send2-hook '~f @account2\.com$' 'set hostname = "account2.com"' send2-hook '~f @account3\.org$' 'set hostname = "account3.org"' etc. > ---Without "set from=" field, sender becomes > "mylocalpcu...@localhost.myisp.com"... What is this? That is a default value, derived from your system information. > Does the "from" field kind of guarantee that email is being *really* > sent from the u...@domain.tld address, and not from *local rig*? What do you mean? > ---In "set smtp_url" field, if using "$my_user:my_p4ss" notation, this > seems to override the "set smtp_pass" field completely? Mutt has to make one of those override the other. Otherwise, what would Mutt do if a user specified one password in $smtp_url and a different one in $smtp_pass? > ---Seemingly not needed smtp_authenticators... ?? What do you mean? > ---Without smtp_url and smtp_pass fields, where does email go? > recei...@domain.tld doesn't receive it... Depends on your system configuration. Have you looked in /var/mail/ ? > ---Is email really being sent with STARTTLS, as wanted? How can I > tell? Wireshark? :) Probably there are easier ways - someone else might have suggestions. Perhaps setting debug level to `-d 5` would give you the information you need? > ---In "set from=" field, spaces between "Send User" and actual email > address... don't seem to matter? > > > >> Are you able to send email via that account using other applications? > > Yes, I used to use Heirloom Mailx. Thanks for confirming. >> Yes. Whether or not you have to explicitly specify the port may >> depend on the mail server's configuration. This configuration can >> vary from ISP to ISP. > > I didn't mean email provider, but ISP internet service I'm connected > to. > > And did test again: I connected to an internet network, did not > specify port in smtp_url, tried send email, and got: Could not connect > to smtp.domain.tld (No route to host) Tried connecting to a different > internet network, tried to send email using *exact* command, without > port, and *it succceeded*. > > WTH? Does that behaviour happen consistently? I.e. with one ISP connection *never* succeeds without specifying the port, and with the other ISP it *always* succeeds without specifying the port? If so, then that is puzzling. Hopefully someone else on this list can offer a hypothesis. Anyway, congratulations on your progress! Sam -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 05, 2022 at 12:07:44PM -0500, Kevin J. McCarthy wrote: 1) On the command line, the shell will expand shell variables inside double quoted strings, before Mutt even sees it. So in the part "set smtp_url=smtp://$my_user@$my_url" Mutt is probably only seeing "set smtp_url=smtp://@", because $my_user and $my_url are unset in your shell. You may want to use single-quotes around the whole "-e" expression, but then be careful with nested single quotes, e.g. the set from='User' part. First: had to delete/rename ~/.muttrc, because some previous settings were perhaps conflicting... This seemed to *finally* work (had to play with both single and double quotes...): printf "%b\n" "$msg" | mutt -s "Test message" -e 'set my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set from="Send User "; set smtp_url="smtp://$my_user:my_p4ss@$my_url:587"; set ssl_starttls="yes"; set ssl_force_tls="yes"' recei...@domain.tld Finally worked -seemingly-, but popped just many doubts: ---There's a "sent email" log in local system, where sent emails are logged. There's always this line: Message-ID: < @ mylocalpcu...@localhost.myisp.com > Is this correct? ---Without "set from=" field, sender becomes "mylocalpcu...@localhost.myisp.com"... What is this? Does the "from" field kind of guarantee that email is being *really* sent from the u...@domain.tld address, and not from *local rig*? ---In "set smtp_url" field, if using "$my_user:my_p4ss" notation, this seems to override the "set smtp_pass" field completely? ---Seemingly not needed smtp_authenticators... ?? ---Without smtp_url and smtp_pass fields, where does email go? recei...@domain.tld doesn't receive it... ---Is email really being sent with STARTTLS, as wanted? How can I tell? ---In "set from=" field, spaces between "Send User" and actual email address... don't seem to matter? Are you able to send email via that account using other applications? Yes, I used to use Heirloom Mailx. Yes. Whether or not you have to explicitly specify the port may depend on the mail server's configuration. This configuration can vary from ISP to ISP. I didn't mean email provider, but ISP internet service I'm connected to. And did test again: I connected to an internet network, did not specify port in smtp_url, tried send email, and got: Could not connect to smtp.domain.tld (No route to host) Tried connecting to a different internet network, tried to send email using *exact* command, without port, and *it succceeded*. WTH?
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 05, 2022 at 11:24:36AM -0500, x...@trimaso.com.mx wrote: printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_url=smtp://$my_user@$my_url; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld Two general suggestions to try: 1) On the command line, the shell will expand shell variables inside double quoted strings, before Mutt even sees it. So in the part "set smtp_url=smtp://$my_user@$my_url" Mutt is probably only seeing "set smtp_url=smtp://@", because $my_user and $my_url are unset in your shell. You may want to use single-quotes around the whole "-e" expression, but then be careful with nested single quotes, e.g. the set from='User' part. Alternatively, consider using the 'source' command and putting the configs into a file. 2) You most likely *do* want to set a port number in $smtp_url, as you've done below, e.g. smtp://xxx:587 or smtps://xxx:465. [Since you're new to Mutt, I'll also gently remind to please reply to the mailing list, not to me directly.] -- Kevin J. McCarthy GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA signature.asc Description: PGP signature
Re: New to Mutt, unable to send messages in *any* attempted way
On Thu, May 05, 2022 at 11:24:36AM -0500, x...@trimaso.com.mx wrote: > *Beforehand*, if you'll answer just with questions without advise, or > mock, just ban me better. I do have questions, but these are to help understand what might be going wrong. > printf "%b\n" "$msg" | mutt -s "Test message" -e "set > my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; > set use_from=yes; set smtp_url=smtp://$my_user@$my_url; set > smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" > recei...@domain.tld > > Could not find the host "" > Could not send the message. Are you able to send email via that account using other applications? If so, which other applications? For instance, have you succeeded with the `mail` application? If you're not sure whether you have the `mail` application installed, try entering at your terminal: which mail > Have to explicitly specify the port; though it varies from ISP to > ISP!? In some I need, in some other I don't! Yes. Whether or not you have to explicitly specify the port may depend on the mail server's configuration. This configuration can vary from ISP to ISP. Thankfully, this shouldn't be a problem once you have debugged your issue, because you can save the relevant details (ports, URLs) in a configuration file so that Mutt will know which details to use for which ISP. > What's the difference between Mutt and Neomutt? Which one preferable? Neomutt is a fork of Mutt. Neomutt is maintained by a different person to Mutt. Neomutt includes various third-party extensions to Mutt, and some other changes. If you want a smaller TCB and simpler configuration, Mutt is probably the best option. If you want fancier features, Neomutt is probably the best option. Given that you are experiencing configuration difficulties, I would suggest trying to overcome those first, perhaps with the help of people on this mailing list. If no success, then maybe try a newer version of Mutt next. (I know you said you can't do it, but where there's a will there's a way.) If, once you've got Mutt working and you've lived with it for a while, you decide you want some features that are more readily available in Neomutt then you could switch to Neomutt at that point - but many people find they are delighted with pure Mutt. Good luck! Sam -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
New to Mutt, unable to send messages in *any* attempted way
Hello. I already tried every way I could imagine to debug my problem, and simply no dice; thus trying to post everything here as last resort. Using Mutt 1.10.1 (cannot update for now; please don't ask why) Using Postfix /usr/sbin/sendmail ($sendmail variable is empty) Remote mail server just accepts "login" mode IMAP does work correctly *Beforehand*, if you'll answer just with questions without advise, or mock, just ban me better. printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_url=smtp://$my_user@$my_url; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld Could not find the host "" Could not send the message. Tried with and without simple/double quotes for $smtp_url variable, as well as using "%40" instead of '@' printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_url=smtp://$my_u...@smtp.domain.tld; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld Could not connect to smtp.domain.tld (No route to host). Could not send the message. Have to explicitly specify the port; though it varies from ISP to ISP!? In some I need, in some other I don't! printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_url=smtp://$my_u...@smtp.domain.tld:587; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld No authenticators available Could not send the message. I have to explicitly specify $smtp_authenticators variable... Shouldn't Mutt check all possible ones one by one if empty variable, according to manual? printf "%b\n" "$msg" | mutt -n -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_url=smtp://$my_u...@smtp.domain.tld:587; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld; echo $? 1 Just what?? printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_authenticators=login:gssapi; set smtp_url=smtp://$my_u...@smtp.domain.tld:587; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld login authentication failed, trying next method No authenticators available Could not send the message. printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_authenticators=gssapi; set smtp_url=smtp://$my_u...@smtp.domain.tld:587; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld No authenticators available Could not send the message. printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_authenticators=whatNonsense; set smtp_url=smtp://$my_u...@smtp.domain.tld:587; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld No authenticators available Could not send the message. printf "%b\n" "$msg" | mutt -s "Test message" -e "set my_user=u...@domain.tld; set my_url=smtp.domain.tld; set from='User'; set use_from=yes; set smtp_authenticators=gssapi:login; set smtp_url=smtp://$my_u...@smtp.domain.tld:587; set smtp_pass=p4ss; set ssl_starttls=yes; set ssl_force_tls=yes" recei...@domain.tld SASL authentication failed Could not send the message. And finally: What's the difference between Mutt and Neomutt? Which one preferable?