Re: [nmh-workers] INCing of email archives
Hi Bakul,
> Once in a while I download email archives of some mailing list
> and unpack them using "inc -file ". But more
> than once I have seen that inc gets confused and doesn't
> unpack the whole thing. The cause seems to be a line starting
> with From in some message body.
Then it isn't any of the four mbox formats described at
https://en.wikipedia.org/wiki/Mbox#Family ?
> Ideally inc should look that a "From ..." line is immediately followed
> by header lines. And if this is not the case, assume it is in the
> message body.
I agree that would be one heuristic to help, but it would also have
problems:
From the outset, was clear we failed 42
times: the first on attempting to read faulty input...
> fix() {
> grep -n '^From .*[^0-9]$' $1 | sed 's/:.*/s|^|>|/' > ,$1
> if [ -s ,$1 ]; then echo wq >> ,$1; cat ,$1 | ed $1; fi
> rm ,$1
> }
>
> This prepends a > to any line beginning with "From "and not
> ending with a digit.
sed -i '/^From .*[^0-9]$/s/^/> /' "${1?}"
--
Cheers, Ralph.
--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
[nmh-workers] Novel Definition of Deprecation.
Hi Ken, I thought a deprecating a part of nmh meant add the warning it was for the axe in release N and axing it in release M where M>N. But commit 43d9833b, http://git.savannah.nongnu.org/cgit/nmh.git/commit/?id=43d9833bf1dcf38c7892a23951bf1d968028a15e adds this to docs/pending-release-notes: - The generation and verification of Content-MD5 headers is no longer performed. The -check and -nocheck switches to various nmh programs that would control this functionality still exist, but are non-functional and will be removed in the next release. That's axing the feature without deprecation and trying to hoodwink the user that it hasn't happened. :-( Either it should be deprecated, or axed in a release without forewarning. And if the latter then is that agreement we can do it for other things too? After all, many users don't read the release notes and thus miss the deprecation warnings, or they upgrade several of our releases in one bound due to their distro, etc., striding differently, so a feature is effectively axed for them without warning anyway. -- Cheers, Ralph. -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] To/cc decode or not to/cc decode
Hi Ken, > > I may have suggested this before, but shipping a folder of test > > emails that a user can ‘scan +/usr/share/nmh/...’, etc., as a test > > of their configuration, along with pre-formatted ‘golden’ output > > would be an easy way for them to check they've updated to the latest > > bell and whistle. > > That might be useful, although the classic problem of letting people > know that such a thing exists is a always an issue. There's David's context-version check, the FAQ, and this mailing-list's page and appended signature. I've been thinking a bit more about this. A folder of test emails that can be tested by a installed command available to the user. That can either use what it thinks is a good modern configuration so the user can see what's possible in theory, or the user's configuration so they can test their set up on a variety of emails. The former could be built on the fly based on what commands are installed, e.g. lynx(1) v. links(1). -- Cheers, Ralph. -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] Novel Definition of Deprecation.
>Either it should be deprecated, or axed in a release without >forewarning. And if the latter then is that agreement we can do it for >other things too? After all, many users don't read the release notes >and thus miss the deprecation warnings, or they upgrade several of our >releases in one bound due to their distro, etc., striding differently, >so a feature is effectively axed for them without warning anyway. Sigh. Ok, you caught me on this one. But ... in my defense, Content-MD5 is a giant steaming pile of poo and I would argue strongly that it's good that it's gone. First, this has been discussed being removed since 2013. People have argued that they have a use, and at first I was like, "Ok, sure, keep it if people are using it". But ... those arguments upon closer look don't (IMHO) make a lot of sense. Here are my arguments for getting rid of it, from the last time I brought it up: https://lists.gnu.org/archive/html/nmh-workers/2018-07/msg3.html I mean ... can you muster a good argument to keep it? I haven't seen one yet that wasn't kind of vague, "Oh, I think it's good to be able to check integrity", but since we're the only ones who generate it or verify it, that doesn't seem enough justification to me. Now, some people may say, "Oh, leaving it in doesn't harm anything". Well, having been inside of nmh a lot I feel strongly that this is wrong. The INTERNALS of nmh are a mess. The Content-MD5 support was embedded at a bunch of weird layers (like in the base64 encoding and decoding routines), and I understood WHY it was there, but it makes maintenance a nightmare, because when someone looks at that code they have to understand what is going on, and that takes time where they COULD be doing something more useful. Also, in the future where I complete the Great Mime Rewrite there's no way I would implement Content-MD5 (see above for the reasons why). Okay, fine, I understand your complaint isn't really about the removal of Content-MD5; your point is that I didn't really follow the normal rules about deprecating the support and instead I just axed it. I have to plead guily there. But at my sentencing hearing I would argue that there were mitigating circumstances; namely that Content-MD5 support is essentially useless and we are better off without it. Does that justify getting rid of it without going through the normal deprecation schedule? Well, I'll let others decide that. My thinking was that for the few users that still have -check in their .mh_profiles they will hopefully see the change in the release notes and remove those flags. I didn't view it as harming anything because the only people who actually generate or check Content-MD5 headers are nmh users who have -check in their profiles and having those switches be nonfunctional won't really change nmh behavior for them. I have to believe that only a tiny number of people are actually doing that now. If you are suggesting that we just go whole hog and remove the -check and -nocheck flags that now do nothing ... well, I can live with that. As for other features ... well, if they are in the same boat as Content-MD5 then I am fine with treating them the same way (but really, that'a a special case and I can't think of anything close to that). Otherwise we should follow a normal depreciation schedule. I guess in summary: "Yes, Your Honor, I killed Content-MD5, but he had it coming and I'd do it again!". --Ken -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] Novel Definition of Deprecation.
Hi Ken, > > Either it should be deprecated, or axed in a release without > > forewarning. And if the latter then is that agreement we can do it > > for other things too? After all, many users don't read the release > > notes and thus miss the deprecation warnings, or they upgrade > > several of our releases in one bound due to their distro, etc., > > striding differently, so a feature is effectively axed for them > > without warning anyway. ... > As for other features ... well, if they are in the same boat as > Content-MD5 then I am fine with treating them the same way (but > really, that'a a special case and I can't think of anything close to > that). Otherwise we should follow a normal depreciation schedule. But what about our releases not matching many users' upgrade schedules so our one-release deprecation notice is of no use to many? That would suggest more rapid change could be possible without the wait-a-release delay. The alternative is to fudge it by releasing more often so we can hoodwink them in another way that they were given fair notice. :-) -- Cheers, Ralph. -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] 'dist' in "1 line noninteractive script"
Hi Ken: On Tue 7/23/19 21:44 -0400 Ken Hornstein wrote: >> mhannotate=Resent mhaltmsg=~/Mail/sent/bar mhdist=1 send ~/Mail/drafts/foo > >It wouldn't surprise me that the problem is that ~ in mhaltmsg. Is that >actually being expanded? If you run send with -debug, do you maybe >see a message saying "$mhaltmsg mismatch"? $ ( set -x ; x=~ true ) + x=/a/rodmant + true So bash expands ~ before send sees it. >How annotations work is a bit magical. send sees that the 'mhannotate' >variable is set, opens a file descriptor and passes that descriptor down >to post(8) using the -idanno flag. Post then writes the address information >to the anno descriptor (which is a tmp file which has been unlinked) >and then send(1) reads that file and uses it to construct the annotation. >But there's a check in sendsbr.c:anno() to make sure the stat() information >of the filename in mhaltmsg matches what is being passed in as the >original filename; that will fail silently without -debug if there >is a mismatch. Interesting/thanks. I can live without the -annotate working -- it works fine, but not with David's undocumented -file MSG_PATH. -- Tom -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] 'dist' in "1 line noninteractive script"
On Thu, 25 Jul 2019 11:09:38 -0500, n...@trodman.com said: > On Tue 7/23/19 21:44 -0400 Ken Hornstein wrote: > >> mhannotate=Resent mhaltmsg=~/Mail/sent/bar mhdist=1 send > >> ~/Mail/drafts/foo > > > >It wouldn't surprise me that the problem is that ~ in mhaltmsg. Is that > >actually being expanded? If you run send with -debug, do you maybe > >see a message saying "$mhaltmsg mismatch"? > > $ ( set -x ; x=~ true ) > + x=/a/rodmant > + true > > So bash expands ~ before send sees it. But that may not be true for something that invokes execve() rather than system(), or a cronjob (which is what started this discussion). pgpGnHJNlgbPh.pgp Description: PGP signature -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] Novel Definition of Deprecation.
>But what about our releases not matching many users' upgrade schedules >so our one-release deprecation notice is of no use to many? That would >suggest more rapid change could be possible without the wait-a-release >delay. The alternative is to fudge it by releasing more often so we can >hoodwink them in another way that they were given fair notice. :-) Sigh. I think we're overthinking this ... there are no good answers. I'm not so fond of releasing more often when there hasn't been significant changes (we do have some reasonable bug fixes that have occured since 1.7.1). We're going to have users bitten by unexpected changes, and that's just a fact of life. I'm fine with the current "keep for one release" schedule. We have simply killed without deprecating some other features along the way that were unused or didn't make any sense anymore. --Ken -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] 'dist' in "1 line noninteractive script"
Hi David:
On Tue 7/23/19 22:32 -0400 David Levine wrote:
>> Although above 'send' worked, it
>> would be nice to specify ~/Mail/sent/2022 as say ~/Mail/sent/bar,
>
>send(1) doesn't care what the message file is named. After successfully
>sending the message, it will rename the file to, in the case of your example
>above, ~/Mail/sent/,bar (with the default comma backup prefix).
ok/thx
>dist(1) supports a -file switch, but it isn't documented. We should consider
>fixing that.
Great / I'm using "-file" now.
My 1 liner is back to to something like:
dist -whatnowproc ~/Mail/auto_send -from 'nym moob
' -to '"nym moob" ' -fcc resent -cc
myaliasfoo -file ~/Mail/sent/bar
# Adding -annotate does nothing, but I think that is by design / I am
ok with that.
Where:
$ cat ~/Mail/auto_send
#!/bin/bash
if [ $mhdraft ] ; then
# echo mhdraft: $mhdraft mhaltmsg: $mhaltmsg
# cp "$mhdraft" /tmp/foo
# env
set -x
send -verbose "$mhdraft"
else
echo $0: OOPS: mhdraft not set >&2
exit 1
fi
$ head -8 /a/rodmant/Mail/distcomps
%; http://git.savannah.nongnu.org/cgit/nmh.git/tree/etc/distcomps
%;
%<{nmh-from}%|%(void(localmbox))%>%(void(width))%(putaddr Resent-From: )
%<{nmh-to}%(void(width))%(putaddr Resent-To: )%|Resent-To:%>
%<{nmh-cc}%(void(width))%(putaddr Resent-cc: )%|Resent-cc:%>
%<{fcc}%(void(width))%(putaddr Resent-fcc: )%|Resent-fcc:%>
Resent-Comments:
Resent-Comments:
--
Tom
--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] 'dist' in "1 line noninteractive script"
>Interesting/thanks.
>
>I can live without the -annotate working -- it works fine, but not with
>David's undocumented -file MSG_PATH.
Oh, huh, that's explicit:
if (file) {
/*
* Dist a file
*/
anot = false; /* don't want to annotate a file */
I am neutral if that is a good idea or not.
In theory, though, you should be able to annotate anything when you
call send directly as long as the variables are set correctly.
Okay, I double-checked, and it turns out that's a lie. For that all
to work you need to set mhfolder and mhmessages.
Huh, I wonder why? It looks like mhmessages can actually be a LIST of
messages. I guess you can use "forw" to generate more than one message
to be annotated. So yeah, the annotations setup currently really is set
up to only work when they are normal messages in MH folders. Sigh.
That's my "new nmh thing I just learned" for today, I guess.
--Ken
--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] INCing of email archives
>Once in a while I download email archives of some mailing list >and unpack them using "inc -file ". But more >than once I have seen that inc gets confused and doesn't >unpack the whole thing. The cause seems to be a line starting >with From in some message body. Ideally inc should look that >a "From ..." line is immediately followed by header lines. >And if this is not the case, assume it is in the message body. Ralph answered this, but let me expand a bit. The job of inc(1) is to incorporate messages from a 'mail drop' into your MH mailbox. Traditionally it handles mbox-style files and POP (it also does MMDF, but let us not speak of that). As you can see from the Wikipedia entry Ralph linked to, all of the various mbox formats use the same scheme: a line beginning with "From " is the mailbox delimiter (mboxcl and mboxcl2 uses a Content-Length header; I believe they are officially dead at this point). The big differences are in quoting rules. Unfortunately since we're kind of locked in to the mbox format in inc(1) at least, changing that would have some nasty consequences (Ralph gave you an example of a message that it would break on but I am sure there are others). I think your best bet is to preprocess these mailing list archives so they are valid mbox files. --Ken -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [nmh-workers] To/cc decode or not to/cc decode
>I've been thinking a bit more about this. A folder of test emails that >can be tested by a installed command available to the user. That can >either use what it thinks is a good modern configuration so the user can >see what's possible in theory, or the user's configuration so they can >test their set up on a variety of emails. The former could be built on >the fly based on what commands are installed, e.g. lynx(1) v. links(1). Hm, well ... I think notifying the user of that might be tough. But ... we DO have the problem where users say, "Hey, this email doesn't look right" and we first have to figure out what the email contains to try to determine what is going wrong. A set of emails that we know the content of sure would help in debugging that. I think that would be worthwhile to put together if you wanted to do that. It might even help a few long-time MH/nmh users on this mailing list who somehow manage to keep munging 8-bit characters in their replies (you know who you are!). --Ken -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
