Re: Compiling a newer version than the latest .deb package

[Cameron Simpson]

On 07Jun2019 23:22, Kurt Hackenberg  wrote:

On 2019-06-07 07:22, Cameron Simpson wrote:
If fetchmail's delivering to a programme, nothing prevents that being 
an arbitrary script to premangle a leading From_ line. Hmm.  I've got 
a script in my bin directory called "unfrom_" for exactly this 
purpose, which reads:


 #!/usr/bin/env sed -f
 #
 # Transmute leading "From " line on email message, if any.
 #    - Cameron Simpson  13jul2002
 #

 1{s/^From /From_: /}

You can just include that script in a pipeline ahead of maildrop.


Yep.

I've also seen From_ lines in maildir message files, and on the stdin 
of delivery agents. Both are wrong, in my opinion. I think mbox From_ 
lines should not exist anywhere outside mbox files.


Well, the From_ isn't just a delimiter for mbox lines, it also 
historically contains the envelope address from the mail system - the 
address used for this delivery (versus whatever may be in the headers).  
So it has some useful forensic information.


I've written a script much like yours, to drop the damn From_ line 
from an incoming message passed to a delivery agent. I think your 
script does something that's not quite right: it transforms the mbox 
From_ line to something that's not RFC 822. You could do, say,


   s/^From /X-Mbox-From: /


I'm digging through RFC 8522:

 https://tools.ietf.org/rfcmarkup/5322

which is the modern incarnation of RFC822, but it doesn't seem very 
prohibitive on header field names. Section 2.2 says:


 Header fields are lines beginning with a field name, followed by a 
 colon (":"), followed by a field body, and terminated by CRLF.  A 
 field name MUST be composed of printable US-ASCII characters (i.e., 
 characters that have values between 33 and 126, inclusive), except 
 colon.


This is confirmed in section 3.6.8:

 optional-field  =   field-name ":" unstructured CRLF
 field-name  =   1*ftext
 ftext   =   %d33-57 /  ; Printable US-ASCII
 %d59-126   ;  characters not including
;  ":".

So I think "_" is permitted :-)

Or -- my recommendation -- just drop it. (Of course, drop it only if 
that first line really is an mbox From_ line.)


I'm a hoarder and dislike discarding anything.

Cheers,
Cameron Simpson 

forwarding in neomutt

[Russell L. Harris]

The "forward-message" command appears not to work in Debian 9
(Stretch, neomutt); when I type ": forward-message" on the Neo-Mutt
command line, the response is "forward-message:  unknown command".

Is neo-Mutt broken, or am I doing something wrong?

Re: Compiling a newer version than the latest .deb package

[Kurt Hackenberg]

On 2019-06-07 07:22, Cameron Simpson wrote:

Are you looking in mbox files or in other places. [...] Hmm, my maildir 
message files also have From_ lines.


If fetchmail's delivering to a programme, nothing prevents that being an 
arbitrary script to premangle a leading From_ line. Hmm. I've got a 
script in my bin directory called "unfrom_" for exactly this purpose, 
which reads:


  #!/usr/bin/env sed -f
  #
  # Transmute leading "From " line on email message, if any.
  #    - Cameron Simpson  13jul2002
  #

  1{s/^From /From_: /}

You can just include that script in a pipeline ahead of maildrop.


Yep.

I've also seen From_ lines in maildir message files, and on the stdin of 
delivery agents. Both are wrong, in my opinion. I think mbox From_ lines 
should not exist anywhere outside mbox files.


I've written a script much like yours, to drop the damn From_ line from 
an incoming message passed to a delivery agent. I think your script does 
something that's not quite right: it transforms the mbox From_ line to 
something that's not RFC 822. You could do, say,


s/^From /X-Mbox-From: /

Or -- my recommendation -- just drop it. (Of course, drop it only if 
that first line really is an mbox From_ line.)

Re: Compiling a newer version than the latest .deb package

[Kurt Hackenberg]

On 2019-06-07 05:08, Frank Watt wrote:


When I look at the headers of most mail, I see an mbox-style From
line. Where do we make use of the 'reformat -f0' and "Return-Path"
advice?


In the stream of data that ends up on the standard input of the delivery 
agent (procmail, maildrop, whatever).


If fetchmail -m works the same as an MTA directed by .forward to hand 
over a message to a delivery agent, then fetchmail sets up a pipe from 
itself to the delivery agent, so the delivery agent reads a single 
message on its standard input. Some MTAs precede the message with an 
mbox-style From_ line; some don't. Fetchmail may or may not add a From_ 
line.


You can find out by having fetchmail hand the message to a script that 
you write, which both saves a copy of its stdin and passes it on to a 
delivery agent. The script would be something like this:


#!/bin/sh

tee -a $HOME/fetchmail.test | maildrop

Re: Compiling a newer version than the latest .deb package

[Patrick Shanahan]

* Cameron Simpson  [06-07-19 19:43]:
> On 07Jun2019 07:37, Patrick Shanahan  wrote:
> > * Cameron Simpson  [06-07-19 07:24]:
> > > It does look that way. I left procmail because I disliked its rule
> > > syntax,
> > > its totally regexp based matching system (ok for subject lines, ghastly 
> > > for
> > > email addresses) and the performance cost incurred by it rereading the
> > > filter files from scratch on every message processed. I've got about 2000
> > > filing rules myself - procmail starts to scale badly for that.
> > 
> > 
> > adjust the order of your rules so that the most likely matchs appear at
> > the top of the rc file
> > 
> 
> I'm aware of that technique. I expect to multifile my email; almost all my
> rules are not match-and-stop, they're match-and-continue. I do have some
> critical match-and-stop rules at the front, but they are few.
> 
> As an example, an email to me personally and also CCed to a list needs to
> land in my inbox (because it is for my attention) but also in the mailing
> list folder (for archiving and general well filedness). So, match and
> continue.

and I abhor receiving duplicate mail.  I read a list and communicate there
and notice email there.  personal mail goes to my inbox, list mail is not
personal and list communication should normally be conducted there.  be we
all have different expectations and utilize linux to achieve them.  :)

-- 
(paka)Patrick Shanahan   Plainfield, Indiana, USA  @ptilopteri
http://en.opensuse.orgopenSUSE Community Memberfacebook/ptilopteri
Registered Linux User #207535@ http://linuxcounter.net
Photos: http://wahoo.no-ip.org/piwigo   paka @ IRCnet freenode

Re: Compiling a newer version than the latest .deb package

[Cameron Simpson]

On 07Jun2019 07:37, Patrick Shanahan  wrote:

* Cameron Simpson  [06-07-19 07:24]:
It does look that way. I left procmail because I disliked its rule 
syntax,

its totally regexp based matching system (ok for subject lines, ghastly for
email addresses) and the performance cost incurred by it rereading the
filter files from scratch on every message processed. I've got about 2000
filing rules myself - procmail starts to scale badly for that.



adjust the order of your rules so that the most likely matchs appear at
the top of the rc file



I'm aware of that technique. I expect to multifile my email; almost all 
my rules are not match-and-stop, they're match-and-continue. I do have 
some critical match-and-stop rules at the front, but they are few.


As an example, an email to me personally and also CCed to a list needs 
to land in my inbox (because it is for my attention) but also in the 
mailing list folder (for archiving and general well filedness). So, 
match and continue.


Cheers,
Cameron Simpson 

Re: mailboxes - example

[Will Yardley]

On Fri, Jun 07, 2019 at 07:15:49AM -0700, Kevin J. McCarthy wrote:
> On Wed, May 15, 2019 at 01:07:28PM +, Dan Ciprus (dciprus) wrote:
> > Obviously all strings defined there are considered mailboxes even though
> > there is no maildir structure nor mailbox file.
> 
> Yes, that's exactly the problem.  The sidebar is a list of mailboxes; there
> is no concept of dividers.  I think the closest you could get would be to
> turn off the display of 0 for all of them using a conditional format string,
> like %?N%.

I haven't used the sidebar feature, but would using directories render
them sort of like the OP wanted?

foo/
  bar
  baz
bat/
  qyz

Re: mailboxes - example

[Kevin J. McCarthy]

On Fri, Jun 07, 2019 at 02:48:16PM +, Dan Ciprus (dciprus) wrote:

Hm, at least there is some creative workaround :-)

Thanks Kevin !


You're welcome.  By the way it looks like I blew the example.  The 
documentation is at 
 and it 
instead would look something like

 %?N?%N?

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature

Re: mailboxes - example

[Dan Ciprus (dciprus)]

Hm, at least there is some creative workaround :-)

Thanks Kevin !

On Fri, Jun 07, 2019 at 07:15:49AM -0700, Kevin J. McCarthy wrote:

On Wed, May 15, 2019 at 01:07:28PM +, Dan Ciprus (dciprus) wrote:
Obviously all strings defined there are considered mailboxes even 
though there is no maildir structure nor mailbox file.


Yes, that's exactly the problem.  The sidebar is a list of mailboxes; 
there is no concept of dividers.  I think the closest you could get 
would be to turn off the display of 0 for all of them using a 
conditional format string, like %?N%.


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA




--




signature.asc
Description: PGP signature

Re: mailboxes - example

[Kevin J. McCarthy]

On Wed, May 15, 2019 at 01:07:28PM +, Dan Ciprus (dciprus) wrote:
Obviously all strings defined there are considered mailboxes even 
though there is no maildir structure nor mailbox file.


Yes, that's exactly the problem.  The sidebar is a list of mailboxes; 
there is no concept of dividers.  I think the closest you could get 
would be to turn off the display of 0 for all of them using a 
conditional format string, like %?N%.


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature

Re: Compiling a newer version than the latest .deb package

[Patrick Shanahan]

* Patrick Shanahan  [06-07-19 07:38]:
> * Cameron Simpson  [06-07-19 07:24]:
>  [...]
> > How's fetchmail run by your system? Cron? Something else?
> 
> not that fetchmail has a daemon, "fetchmail -d 150" runs every 150
> seconds.

s/not/note
 

-- 
(paka)Patrick Shanahan   Plainfield, Indiana, USA  @ptilopteri
http://en.opensuse.orgopenSUSE Community Memberfacebook/ptilopteri
Registered Linux User #207535@ http://linuxcounter.net
Photos: http://wahoo.no-ip.org/piwigo   paka @ IRCnet freenode

Re: Compiling a newer version than the latest .deb package

[Patrick Shanahan]

* Cameron Simpson  [06-07-19 07:24]:
 [...]
> How's fetchmail run by your system? Cron? Something else?

not that fetchmail has a daemon, "fetchmail -d 150" runs every 150
seconds.

 [...] 
> It does look that way. I left procmail because I disliked its rule syntax,
> its totally regexp based matching system (ok for subject lines, ghastly for
> email addresses) and the performance cost incurred by it rereading the
> filter files from scratch on every message processed. I've got about 2000
> filing rules myself - procmail starts to scale badly for that.

 
adjust the order of your rules so that the most likely matchs appear at
the top of the rc file
 

 {...] 
> That said, many people use procmail with little apparent pain.

for many years.  :)
 

-- 
(paka)Patrick Shanahan   Plainfield, Indiana, USA  @ptilopteri
http://en.opensuse.orgopenSUSE Community Memberfacebook/ptilopteri
Registered Linux User #207535@ http://linuxcounter.net
Photos: http://wahoo.no-ip.org/piwigo   paka @ IRCnet freenode

Re: Compiling a newer version than the latest .deb package

[Cameron Simpson]

On 07Jun2019 21:08, Frank Watt  wrote:

First of all, apologies for munging the thread: Gmail didn't deliver
Cameron's response.  I had to get the text from the archives.


That seems to happen to me quite a bit. I harbour some suspicions to do 
with years of maintaining the adzapper project (which is filled with 
spamlike domain strings). Please see if I'm in your spam folder; marking 
my messages as not spam might improve future delivery.



Cameron Simpson wrote:
| Procmail generally relies on being installed in the user's ~/.forward
| file to cause sendmail (the mail system) to deliver email to the
| procmail programme instead of the default.

I don't have a ~/.forward: I have a ~/.fetchmailrc and a ~/.procmail
file, the former having this statement in its first line:

# Configuration created Fri Dec 13 22:30:42 2002 by fetchmailconf

I don't recall doing that, but I edited that file a number of times
since 2002.


Ok, so fetchmail collects your email and hands it to procmail? I'd have 
thought that should continue to work regardless of what you've done with 
the system MTA (sendmail/nullmailer/whatever).


How's fetchmail run by your system? Cron? Something else?


| fetchmail collects email and delivered it to the local email system.
| Which means it delivers to sendmail. And sendmail hands it to
| procmail...
|
| However, fetchmail has a -m option, which can probably deliver directly
| to procmail, bypassing the local mail system entirely.

Looks like that's not as simple as I'd hoped.  From the fetchmail man
page:
  -m  | --mda 
 This option lets fetchmail use a Message or Local
 Delivery Agent (MDA or LDA) directly, rather than
 forward via SMTP or LMTP.

 To avoid losing mail, use this option only with MDAs
 like maildrop or MTAs like sendmail that exit with a
 nonzero status on disk-full and other delivery errors;

[...]

 The well-known procmail(1) package is very hard to
 configure properly, it has a very nasty "fall through to
 the next rule" behavior on delivery errors (even
 temporary ones, such as out of disk space if another
 user's mail daemon copies the mailbox around to purge
 old messages), so your mail will end up in the wrong
 mailbox sooner or later. The proper procmail
 configuration is outside the scope of this
 document. Using maildrop(1) is usually much easier, and
 many users find the filter syntax used by mail‐ drop
 easier to understand.

That seems to be a big incentive not to use procmail.


It does look that way. I left procmail because I disliked its rule 
syntax, its totally regexp based matching system (ok for subject lines, 
ghastly for email addresses) and the performance cost incurred by it 
rereading the filter files from scratch on every message processed. I've 
got about 2000 filing rules myself - procmail starts to scale badly for 
that.


Is your procmailrc extensive? Would it be hard to switch filers?

That said, many people use procmail with little apparent pain.


Meanwhile, in
the man pages for maildrop:

DESCRIPTION

[...]

  maildrop does not accept an mbox-style From_ line before the
  first header line.  maildrop does not accept leading empty
  lines before the first non-blank header line. If the message
  can possibly start with empty lines, and a From_ line, use
  reformail -f0 to remove any initial empty lines, and replace a
  From_ line with a proper “Return-Path:” header; then pipe it to
  maildrop.

When I look at the headers of most mail, I see an mbox-style From
line. Where do we make use of the 'reformat -f0' and "Return-Path"
advice?


Are you looking in mbox files or in other places. [...] Hmm, my maildir 
message files also have From_ lines.


If fetchmail's delivering to a programme, nothing prevents that being an 
arbitrary script to premangle a leading From_ line. Hmm. I've got a 
script in my bin directory called "unfrom_" for exactly this purpose, 
which reads:


 #!/usr/bin/env sed -f
 #
 # Transmute leading "From " line on email message, if any.
 #- Cameron Simpson  13jul2002
 #

 1{s/^From /From_: /}

You can just include that script in a pipeline ahead of maildrop.

Cheers,
Cameron Simpson 

Re: Compiling a newer version than the latest .deb package

[Frank Watt]

First of all, apologies for munging the thread: Gmail didn't deliver
Cameron's response.  I had to get the text from the archives.

Cameron Simpson wrote:

[...]


| Procmail generally relies on being installed in the user's ~/.forward
| file to cause sendmail (the mail system) to deliver email to the
| procmail programme instead of the default.

I don't have a ~/.forward: I have a ~/.fetchmailrc and a ~/.procmail
file, the former having this statement in its first line:

# Configuration created Fri Dec 13 22:30:42 2002 by fetchmailconf

I don't recall doing that, but I edited that file a number of times
since 2002.


|
| So...
|
| fetchmail collects email and delivered it to the local email system.
| Which means it delivers to sendmail. And sendmail hands it to
| procmail...
|
| However, fetchmail has a -m option, which can probably deliver directly
| to procmail, bypassing the local mail system entirely.

Looks like that's not as simple as I'd hoped.  From the fetchmail man
page:
   -m  | --mda 
  (Keyword: mda)

  This option lets fetchmail use a Message or Local
  Delivery Agent (MDA or LDA) directly, rather than
  forward via SMTP or LMTP.

  To avoid losing mail, use this option only with MDAs
  like maildrop or MTAs like sendmail that exit with a
  nonzero status on disk-full and other delivery errors;
  the nonzero status tells fetchmail that deliv‐ ery
  failed and prevents the message from being deleted on
  the server.

[...]

  The well-known procmail(1) package is very hard to
  configure properly, it has a very nasty "fall through to
  the next rule" behavior on delivery errors (even
  temporary ones, such as out of disk space if another
  user's mail daemon copies the mailbox around to purge
  old messages), so your mail will end up in the wrong
  mailbox sooner or later. The proper procmail
  configuration is outside the scope of this
  document. Using maildrop(1) is usually much easier, and
  many users find the filter syntax used by mail‐ drop
  easier to understand.

That seems to be a big incentive not to use procmail. Meanwhile, in
the man pages for maildrop:

DESCRIPTION

[...]

   maildrop does not accept an mbox-style From_ line before the
   first header line.  maildrop does not accept leading empty
   lines before the first non-blank header line. If the message
   can possibly start with empty lines, and a From_ line, use
   reformail -f0 to remove any initial empty lines, and replace a
   From_ line with a proper “Return-Path:” header; then pipe it to
   maildrop.

When I look at the headers of most mail, I see an mbox-style From
line. Where do we make use of the 'reformat -f0' and "Return-Path"
advice?

TIA

Frank