Re: [mmh] showing attachments

[markus schnalke]

Hoi,

great! :-)


meillo


[2021-05-27 22:38] Juergen Lorenz 
>
> Hi Meillo,
> thanks for your example. I've copied it.
> Jürgen
> 
> > Hoi.
> >
> > [2021-05-27 19:41] Philipp Takacs 
> > > [2021-05-27 18:54] Juergen Lorenz 
> > > >
> > > > in a previous mail, Philipp explains, why X attachments are problematic
> > > > and recommended using mhstore.
> > > > But in my old nmh-profile, I used programs which transformed the X
> > > > attachments to mere text, and they don't work either.
> > > 
> > > Ok thats strange, most of them should work.
> > > 
> > > > Here are some of these profile-entries
> > > >
> > > > mhshow-show-text/enriched: %p/usr/bin/richtext '%f'
> > > > mhshow-show-text/plain: %p/usr/bin/less '%f'
> > > 
> > > Here is one error. text/plain is handled with less. The
> > > default "%liconv -f  should work.
> > > 
> > > > mhshow-show-application/pdf: %p/usr/bin/pdftotext '%f' -
> > > > mhshow-show-application/odt: %p/usr/bin/odt2txt '%f'
> > > > mhshow-show-application/docx: %p/usr/bin/docx2txt '%f' - 
> > > > mhshow-show-application/msword: %p/usr/bin/antiword '%f' 
> > > > mhshow-show-application/mspowerpoint: %p/usr/bin/catppt '%f'
> > > > mhshow-show-application/excel: %p/usr/bin/xls2csv '%f'
> > > > mhshow-show-application/rtf: %p/usr/bin/catdoc '%f'
> > > > mhshow-show-application/octet-stream: %p/usr/bin/less '%f'
> > > > mhshow-show-application: %p/usr/bin/less '%f'
> > > 
> > > These two also don't work. Also I'm not sure if this is a good idea.
> > > 
> > > > mhshow-show-text/html: %p/usr/bin/w3m '%f'
> > > > mhshow-show-text/html: %p/usr/bin/lynx '%f'
> > > >
> > > > Why don't they work? And what should I do to make it work?
> > > 
> > > I belive I have seen the problem. mmh don't support %p (for pause). nmh
> > > also removed the support, but handle %p like %l. So replace %p with %l
> > > and everithing should work.
> >
> >
> > For example, I have these lines in my profile:
> >
> > mhshow-show-text/html: %l w3m -dump -T text/html %f
> > mhshow-show-application/pdf: %l pdftotext -layout %f -
> > mhshow-show-application/msword: %l antiword %f
> >
> >
> > meillo
> >
> 

Re: [mmh] showing attachments

[markus schnalke]

Hoi.

[2021-05-27 19:41] Philipp Takacs 
> [2021-05-27 18:54] Juergen Lorenz 
> >
> > in a previous mail, Philipp explains, why X attachments are problematic
> > and recommended using mhstore.
> > But in my old nmh-profile, I used programs which transformed the X
> > attachments to mere text, and they don't work either.
> 
> Ok thats strange, most of them should work.
> 
> > Here are some of these profile-entries
> >
> > mhshow-show-text/enriched: %p/usr/bin/richtext '%f'
> > mhshow-show-text/plain: %p/usr/bin/less '%f'
> 
> Here is one error. text/plain is handled with less. The
> default "%liconv -f  should work.
> 
> > mhshow-show-application/pdf: %p/usr/bin/pdftotext '%f' -
> > mhshow-show-application/odt: %p/usr/bin/odt2txt '%f'
> > mhshow-show-application/docx: %p/usr/bin/docx2txt '%f' - 
> > mhshow-show-application/msword: %p/usr/bin/antiword '%f' 
> > mhshow-show-application/mspowerpoint: %p/usr/bin/catppt '%f'
> > mhshow-show-application/excel: %p/usr/bin/xls2csv '%f'
> > mhshow-show-application/rtf: %p/usr/bin/catdoc '%f'
> > mhshow-show-application/octet-stream: %p/usr/bin/less '%f'
> > mhshow-show-application: %p/usr/bin/less '%f'
> 
> These two also don't work. Also I'm not sure if this is a good idea.
> 
> > mhshow-show-text/html: %p/usr/bin/w3m '%f'
> > mhshow-show-text/html: %p/usr/bin/lynx '%f'
> >
> > Why don't they work? And what should I do to make it work?
> 
> I belive I have seen the problem. mmh don't support %p (for pause). nmh
> also removed the support, but handle %p like %l. So replace %p with %l
> and everithing should work.


For example, I have these lines in my profile:

mhshow-show-text/html: %l w3m -dump -T text/html %f
mhshow-show-application/pdf: %l pdftotext -layout %f -
mhshow-show-application/msword: %l antiword %f


meillo

Re: [mmh] Problems with mmh

[markus schnalke]

Hoi.

[2021-05-24 20:31] Philipp Takacs 
> [2021-05-24 17:14] Juergen Lorenz 
> >
> > I'm new to mmh, but not to nmh and mutt and the like.
> > This Mail is written in mutt, because mmh causes trouble.
> 
> Nice to see a new user.

Indeed. :-)


> > For example, the scan command produces the following output:
> >
> > ...
> > 2021-05-08 13:49 2845~ 
> > 2021-05-10 16:42 286 4489~ 
> > 2021-05-11 11:59 2878~ 
> > 2021-05-15 15:48 289   70~ 
> > 2021-05-22 22:23 292   m...@marmaro.de  3  [mmh] Welcome 
> > j...@jugilo.de!
> > 2021-05-23 15:17 297   Jürgen Lorenz5  enc
> > 2021-05-23 15:45 298   -> Juergen Lorenz6  [BCC] "Keine 
> > Jobsteuerung in dieser Shell"
> 
> This looks like some of the files are mboxes and not mails. If there are
> any lines starting with "From " in the header, then mmh can't parse the
> mail. nmh and old versions of mmh could hadle this. I can look up our
> converting scripts if you need them.

Maybe we should document this in some manpage ... I cannot recall if
we had done so. I can't even recall if we have a ``Transition from
nmh'' document. (Maybe you, Juergen, want to start with one that we
can include in the code, maybe as a manpage?)


> > and the command show 297 the following error-message:
> >
> > /bin/sh: Zeile 1: fg: Keine Jobsteuerung in dieser Shell.
> >
> > I have no idea, what happened.
> 
> This looks like one of your mhshow-show- entries is triggert. One of
> them with a ``|less''. You can check this with ``show -debug 297''.
> 
> Why do you have these entries? show will run a pager by default, when
> you call it from a shell and without a pipe. In the other cases the mail
> is just printed to stdout.

Seems as if this could come from nmh. In mmh the profile can be very
small, as the modern features are available out of the box. In nmh
you needed a lot of profile entries to have a resonable user
experience. As Juergen transitions from nmh, he probably copied his
nmh profile and thus has all those lines in there.


> > As you can see from my profile below, I tested a lot of settings, but
> > the problems still go on. In particular, I tried utf-8 as well as
> > latin1. Here is the profile, mostly copied from my nmh-profile:
> 
> Your profile looks intresting. Have you tried with nearly empty profile.
> For most stuff you only need to set the Path. As far as I see your config
> for encoding is correct, because the ü in the scan output is correct.

In the message he wrote to me personally, yesterday, the text actually
was UTF-8 encoded but the header read:
Content-Type: text/plain; charset="iso-8859-1"

Not sure what caused that problem. (It's not the case for the
message to the list.)


meillo

Re: [mmh] Previous-Sequence

[markus schnalke]

Hoi.

[2020-03-23 21:46] Philipp Takacs 
> [2020-03-22 07:53] markus schnalke 
> > [2020-03-22 00:21] Philipp Takacs 
> > > [2020-03-21 09:54] markus schnalke 
> > > > Before we remove it, I just would like to know why it was
> > > > introduced in MH the first time and what possible other usecases
> > > > there could be. This means asking on the nmh-workers mailing list.
> > > > Both, out of curiosity and as a double-check in case we've not
> > > > thought on something. Do you wanna write this message or should
> > > > I?
> > > 
> > > This is a good idea. I want to write this mail.
> > > 
> > > So conclusion ask the nmh guys for use cases we missed and if there
> > > aren't any, remove it.
> >
> > Yes! :-)
> >
> > Please as well ask them when and why the previous sequence was
> > introduced. I'd like to have confirmed that it was because of lack
> > of a shell history, or told why else.
> 
> I have asked[0] them.
>
> [0] https://lists.nongnu.org/archive/html/nmh-workers/2020-03/msg0.html

Thanks for asking. i've read through the mails there.

> Some conclusion:
> 
> We have missed that a command may have side effects. Here the example
> from Ralph:
> 
>   show next:3
>   scan !$
> 
> I see, this isn't possible to solve without a sequence. This is a
> problem, when you notice you want to use these messages after you
> started show. I would say this corner case isn't sufficient to keep the
> Previous-Sequence.

If you did
show n:3
you can use
scan c:-3
to address the same messages. For further commands on the same
messages, you keep repeating `c:-3', as the commands place `c'
again on the last message of the range.

(You have to be careful, however, if the first command was
mark(1), for instance, because mark(1) does not change the
current message, thus the second command would still take
`n:3'.)

Yes, that requires some brain action, compared to the pseq, but
it's feasible if you know the commands.


Btw: I agree with what Valdis wrote about muscle memory that was
set in times before command line editing was available. A lot of
the nmh guys are that old. I have such hard-wired typing strokes
too ... only that they were burned in some decades later. ;-)


But still not enough reason to keep it.

> I would say remove it.

Yes.

> A updated version of my patches is attached, including the updated
> man pages.

Please add a HISTORY section to the mh-sequence(7) manpage with
some explanation about the removal of the Previous-Sequence, so
that people, who search for it, are informed that it was removed
and why.

Then it's ready to commit, I'd say.


meillo

Re: [mmh] folder_read with scandir()

[markus schnalke]

[2020-03-23 20:57] Philipp Takacs 
> [2020-03-22 07:20] markus schnalke 
> > [2020-03-22 01:08] Philipp Takacs 
> > > [2020-03-21 23:01] markus schnalke 
> > > > [2020-03-21 17:48] Philipp Takacs 
> > > > > [2020-03-21 10:28] markus schnalke 
> > > > > > [2020-03-19 12:45] Philipp Takacs 
> > > > > > >
> > > > > > > +static int others;
> > > > > > > +
> > > > > > > +static int
> > > > > > > +msgcmp(const struct dirent **d1, const struct dirent **d2)
> > > > > > > +{
> > > > > > > + size_t l1, l2;
> > > > > > > +
> > > > > > > + l1 = strlen((*d1)->d_name);
> > > > > > > + l2 = strlen((*d2)->d_name);
> > > > > > > + if (l1 < l2) {
> > > > > > > + return -1;
> > > > > > > + }
> > > > > > > + if (l1 > l2) {
> > > > > > > + return 1;
> > > > > > > + }
> > > > > > > + return strcmp((*d1)->d_name, (*d2)->d_name);
> > > > > > > +}
> > > > > >
> > > > > > Isn't that a strange comparison algorithm, for first compare
> > > > > > string lengths and only if eaqual their content? Here does mmh
> > > > > > sort by length of folder name?
> > > > > 
> > > > > This is only for msg-files and sorts by numeric order. So "9" is
> > > > > smaller the "10". But actually it's complete irrelevant, because the
> > > > > m_atoi() and the storage in the array will automatic get the right
> > > > > order. Therefor we could completly remove this function.
> > > >
> > > > Even better. ;-)
> > > 
> > > OK then I'll remove the msgcmp function, check if I have overseen some
> > > errors and commit it in the next days.
> >
> > Great.
> 
> Sorry I was wrong. The sort is necessary, because I need to know
> `mp->lowmsg' and `mp->hghmsg' to allocate `mp->msgstats'. Without
> the right order I can't do this. I could do this with two loops
> like the old code, but I one of the reasons I don't like the old
> version.
> 
> So I would suggest to rename msgcmp to msgnumcmp.

Sounds good.


meillo

Re: [mmh] folder_read with scandir()

[markus schnalke]

Hoi.

[2020-03-22 13:17] Philipp Takacs 
> [2020-03-22 07:20] markus schnalke 
> > [2020-03-22 01:08] Philipp Takacs 
> > > [2020-03-21 23:01] markus schnalke 
> > > > [2020-03-21 17:48] Philipp Takacs 
> > > > > [2020-03-21 10:28] markus schnalke 
> > > > > >
> > > > > > - Files starting with a comma. These were backups of deleted 
> > > > > > messages
> > > > > > (see the section on the trash folder in my master's thesis). Thus
> > > > > > they are considered MH files here but not further regarded. I'm
> > > > > > not sure if the code had been this way all the time, because there
> > > > > > used to the a `--with-hash-prefix' option (or similar) that
> > > > > > replaced the comma with a hash symbol as the prefix for deleted
> > > > > > messages backups. Anyways, we don't have such comma files anymore.
> > > > > > Mmh uses the trash folder for that case, i.e. we can get rid of
> > > > > > the comma prefix file handling ... possibly. Does anyone know the
> > > > > > situation in nmh: have they switched to a trash folder as well?
> > > > > > Because we're still (basically) mail storage compatible to nmh (I
> > > > > > think), we should not be fully ignorant of that situation. But we
> > > > > > could handle those backup files as `other' files. 
> > > > > 
> > > > > The files starting with a comma/dot aren't the problem they are 
> > > > > totally
> > > > > ignored. The problem are files witch aren't totally ignored. I have a 
> > > > > ugly
> > > > > workaround in the filter and I want to get rid of it.
> > > >
> > > > How I understand the situation (without looking at the code): A
> > > > folder can contain:
> > > >
> > > > - messages
> > > > - `.' and `..'
> > > > - `.mh_sequences'
> > > > - subfolders
> > > > - other files
> > > >
> > > > Other files are everything else, no matter if they are named
> > > > `.foo' or `,foo' or just `foo'.
> > > >
> > > > When we read a folder, only messages are of relevance. All the
> > > > rest is ignored.
> > > 
> > > expect for the OTHERS bit set for files which aren't messages and don't
> > > start with a '.' or a ','.
> >
> > Why for files starting with a comma? Is there a sense to do that
> > in mmh? Or is it only nmh legacy/compatiblity? And still: What
> > sense does it make to ignore backup files starting with a comma,
> > if the nmh installation might be configured to use hash prefixes
> > instead of comma prefixes?
> 
> I believe this is for legacy/compatiblity. The idea is, you don't have
> other files if you only have backup files in this directory.
> 
> > Either we should add '#' as well or remove ','. Can you follow my
> > reasoning?
> 
> For compatiblity reasons, I would say add the '#' while we have the
> other bit.

Agreed.

Please add a comment to the ',' and '#' case, as their reasons for
existance can not be found within mmh. Like:

For compatibility with nmh, ignore rmm backup files.


meillo

Re: [mmh] Previous-Sequence

[markus schnalke]

Hoi.

[2020-03-22 00:21] Philipp Takacs 
> [2020-03-21 09:54] markus schnalke 
> > [2019-12-15 17:55] Philipp Takacs 
> > >
> > > During that I found the Previous-Sequence again. I don't see a reason
> > > why we need this feature. The usecases, I see, for the feature are better
> > > handled with mark or the backlog of your shell. But I don't use this
> > > feature.
> 
> During reading some old mails on the nmh mailing list I found a use case.
> Put mails in a sequence after a refile[0].

> [0] https://lists.nongnu.org/archive/html/nmh-workers/2005-12/msg00086.html

That's a valid usecase. But I don't think that we need to have the
previous sequence to cover that. The refiled messages will always
be at the end of the destination folder. Thus, you can solve it
this way as well:

b=`mhpath b +dest`
refile -src +src ...msg... +dest
mark -seq pseq +dest "$b"-l

(If you've forgotten to remember the beyond message beforehand,
a human can still look at the destination folder and know which
the first refiled message was.)

As it is a rare usecase and there is a workaround, I'd rather
document the ``workaround'' (actually not really a workaround but
rather a different approach to solve the problem) in the man
page.

(I think it's good to have hints and examples in the man pages to
help newer users to figure out how to effectively work with mmh.
See the manpage of pick(1) for example.)

> Not sure if this works, but
> for this I would suggest to add a -preserve switch to refile instead of
> using previous-sequence for this, like nmh has done.

The `-preserve' switch changes how the refiling works in nmh ...
using it makes the refiled message spread over the destination
folder (trying to preserve the previous message numbers from the
source folder), thus the above solution won't work. In this case
you need to set something like a previous sequence to find the
refiled messages again. But in mmh I've dropped the `-preserve'
switch a long time ago.

Here from my master's thesis (PDF page 33):

­[no]preserve of refile was removed [ 8edc5aa] because what use was
it anyway? Quoting nmh's man page refile(1):
   Normally when a message is refiled, for each destination folder it
   is assigned the number which is one above the current highest
   message number in that folder. Use of the ­preserv [sic!] switch
   will override this message renaming, and try to preserve the
   number of the message. If a conflict for a particular folder occurs
   when using the ­preserve switch, then refile will use the next
   available message number which is above the message number
   you wish to preserve.

That means: We have a valid usecase but still no real need for the
previous sequence. The `refile -preserve' case does not exist in
mmh.



> > Besides: It does introduce a lot of write accesses to the sequence
> > files.
> 
> Would it only be the write access itself, it wouldn't be that bad. You
> still need to enable it.

Do you mean that it is disabled by default? This kinda contradicts
the idea of mmh, where all (new/good) features should be enabled
by default. I kept it disabled by default because of the many
write accesses, i.e. a lot of stuff it does though it is almost
never used. Actually it's only still there because I wasn't sure
enough that we can safely drop it.


> > Before we remove it, I just would like to know why it was
> > introduced in MH the first time and what possible other usecases
> > there could be. This means asking on the nmh-workers mailing list.
> > Both, out of curiosity and as a double-check in case we've not
> > thought on something. Do you wanna write this message or should
> > I?
> 
> This is a good idea. I want to write this mail.
> 
> So conclusion ask the nmh guys for use cases we missed and if there
> aren't any, remove it.

Yes! :-)

Please as well ask them when and why the previous sequence was
introduced. I'd like to have confirmed that it was because of lack
of a shell history, or told why else.


meillo

Re: [mmh] show sequences racecondition

[markus schnalke]

Hoi.

[2020-03-21 16:15] Philipp Takacs 
> [2020-03-21 09:32] markus schnalke 
> > [2020-03-19 12:14] Philipp Takacs 
> > > [2019-12-15 17:55] Philipp Takacs 
> > > >
> > > > I have looked at the problem with open pager for show and change the
> > > > sequences during the pager is open. This happen often, if you show a
> > > > message and recieve a new message.
> >
> > This is an annoying inconvenience (actually a bug but not too
> > severe) that we probably all suffer from. Improving the situation
> > would be great.
> >
> > We can't get fully rid of the race condition, or? We have multiple
> > processes writing the same file, thus we will inevitably have a
> > conflict window. Correct? Could you explain quickly how we react
> > in this case of conflict?
> 
> So currently and after my path last writer winns. Currenly a programm
> reads the sequences and does it work. If the work includes changing
> the sequences, it writes the sequences to disk.
> 
> My patches reads the sequences after show is finisched with the main
> work. The changes to the sequences are then aplied to the new set of
> sequences. This makes the conflict window smaler, but the race
> condition is still there. My patch does this only for show, but I want
> to add this to seq_save.

Thanks for the explanation. Show(1) is the only case where I've
noticed the problem. Thus it's the most important.


> I found then, that seq_read and seq_list ignores mails witch it thinks
> don't exist. This I have disabled this, to avoid a secound read of the 
> of the whole directory.
> 
> Now the conflict window is small, but still there. The best way to
> handle this is locking.

This whole topic is a special one ... One case that one should
have in mind as well: Stuff shouldn't mess up, if you have show(1)
running while deleting/refiling/incing messages, not only to the
end but possibly on previously used message numbers. For instance,
I use `folder -pack' on the inbox quite often ... there might be a
`repl' job in background at the same time ...


I really don't want to burden you with this stuff. I think that
improving the most annoying problem is already a great step
forward. The rest is more of a separate topic to dig into and
solve consistently and generally. I only wanted to put the
information onto the list, to have it documented.


meillo


P.S.
Not related to this topic, more in general: I really appreciate
that you supported your changes with test cases. This is good
work. It will make mmh increasingly stronger. Thank you!

Re: [mmh] folder_add and msgstats

[markus schnalke]

Hoi.

[2020-03-21 18:43] Philipp Takacs 
> [2020-03-21 10:05] markus schnalke 
> > [2020-03-19 12:33] Philipp Takacs 
> > > I have also removed the
> > > clear_msg_flags loop from folder_read and folder_realloc. In most cases
> > > we use calloc, so this isn't necesarry. At one place in folder_realloc
> > > realloc is used. There I have add a memset.
> > > 
> > > comment?
> >
> > Why were (in the section down below) only the flags outside the
> > current message range cleared? Now you clear all flags.
> 
> Not sure whats make you think this. But if I had to guess I would
> say it's the "- lo".

No, it's the comment a bit more down in the code:

> > -   /*
> > -   ** Clear all the flags for entries outside
> > -   ** the current message range for this folder.
> > -   */

Maybe I was irritated that you removed this comment but still
performed a similar action. The memset() line is pretty complex
to understand. You could add a similar two-line comment to it,
explaining what you explained to me here (only a bit more
destilled):

> So a shourt exircourse to the msgstats. msgstats
> is an array of the lenght ``mp->hghoff - mp->lowoff''. lowoff is the
> starting offset, so if you want the stats of message 5 you access
> ``mp->msgstats[5 - mp->mp->lowoff]. hghoff is the end offset so you
> can change flags for all messagestats in the range lowoff <= i <= hghoff.
> folder_realloc in this code path increases the hghoff. Now I have to
> clear all data after the old hghoff. by the way there was a off by one
> in my path. I fixed it now it looks like this:
> 
> > > memset(mp->msgstats + mp->hghoff - lo + 1, 0, hi - mp->hghoff);
> 
> This actually clears less then the code before. The code before
> started at ``mp->hghmsg + 1''. Now it starts hghoff which is
> greater or equal to hghmsg.
> 
> > Does that change the behavior?
> 
> Yes, like the pathes for seq_read it don't clears flags for msgs
> which it belive don't exists. But this is actually the reason
> I created this patch.

Alright. That makes sense.

> > Or, if your memset() clears only specific
> > parts, you probably should put a comment there to explain it, as
> > this might not be what the programmer usually expects from a
> > memset() after a (re)alloc().
> 
> To me this looks completly like this what I would expect.

Of course; you've written the code. ;-)

> But I'll think about a good comment.

Is what you actually clear only the newly allocated memory? If so
then you could write something like:

/* zero the newly allocated memory, i.e. no flags set */

Or:

/* clear the newly allocated msg flag space */


(I really need to look at the code, not only at the diffs and the
mails ...)


meillo

Re: [mmh] folder_read with scandir()

[markus schnalke]

Hoi.

[2020-03-21 17:48] Philipp Takacs 
> [2020-03-21 10:28] markus schnalke 
> >
> > Some comments:
> >
> > [2020-03-19 12:45] Philipp Takacs 
> > >
> > > Also noticed the other flag, not shoure if we need this realy. It's
> > > only used in folder and I don't see the point of knowing that there
> > > are other files in a folder.
> >
> > We should be a bit careful at that point to understand the
> > situation. Here are some aspects to think about:
> >
> > - Files starting with a comma. These were backups of deleted messages
> > (see the section on the trash folder in my master's thesis). Thus
> > they are considered MH files here but not further regarded. I'm
> > not sure if the code had been this way all the time, because there
> > used to the a `--with-hash-prefix' option (or similar) that
> > replaced the comma with a hash symbol as the prefix for deleted
> > messages backups. Anyways, we don't have such comma files anymore.
> > Mmh uses the trash folder for that case, i.e. we can get rid of
> > the comma prefix file handling ... possibly. Does anyone know the
> > situation in nmh: have they switched to a trash folder as well?
> > Because we're still (basically) mail storage compatible to nmh (I
> > think), we should not be fully ignorant of that situation. But we
> > could handle those backup files as `other' files. 
> 
> The files starting with a comma/dot aren't the problem they are totally
> ignored. The problem are files witch aren't totally ignored. I have a ugly
> workaround in the filter and I want to get rid of it.

How I understand the situation (without looking at the code): A
folder can contain:

- messages
- `.' and `..'
- `.mh_sequences'
- subfolders
- other files

Other files are everything else, no matter if they are named
`.foo' or `,foo' or just `foo'.

When we read a folder, only messages are of relevance. All the
rest is ignored.

Do we need to know about subfolders and other files? I'd say that
it would be nice, because there are mmh tools that use such
information.

Does it have to be part of this function? Not necessarily.

(Sorry for not looking at the code right now. I hope my thoughts
help nonetheless.)


> > - `Other' files seem to be files in the mail storage that have
> > nothing to do with MH. E.g. any random file someone created in the
> > mail storage without any connection to mmh. Normally mmh would
> > simply ignore them, i.e. act as if they wouldn't be there at all.
> 
> Not completely other files are also sub directories. folder uses this to
> check if it's required to search for sub directories with the recursive
> flag. This is an other ugly code path I found.

Like I layed out above, these are different classes of files that
could be in a folder. Merging subdirectories and other files (or
ignoring everything that starts with a dot) does not match the
problem structure. It might work and it might have been good
enough -- it might still be good enough -- but the goal should be
that the code reflects the problem domain.


> > There's one place, however, where `other' files are relevant:
> > rmf(1). We should not delete a mail folder that includes `other'
> > files. It seems this check is missing there. Hence, I would keep
> > the `other' files marker. It adds few complexity and can be
> > useful. Plus, we should use it in rmf(1).
> 
> rmf(1) just don't use folder_read(), therefor it's completely useless
> for it. It does check for other files on it's own.

Sounds a bit like code duplication. Surely both cases -- reading
message numbers from a folder and deleting a folder -- are quite
different, but the classification of files within a folder is the
same and maybe should be outfactored and reused (maybe with an
enum fileclass or such).


> So conclusion, currently I would suggest to let in in. But if I rewrite
> the recursive code path of folder(1), I'll bring this up again.

Would be okay for me.


> > > +static int others;
> > > +
> > > +static int
> > > +msgcmp(const struct dirent **d1, const struct dirent **d2)
> > > +{
> > > + size_t l1, l2;
> > > +
> > > + l1 = strlen((*d1)->d_name);
> > > + l2 = strlen((*d2)->d_name);
> > > + if (l1 < l2) {
> > > + return -1;
> > > + }
> > > + if (l1 > l2) {
> > > + return 1;
> > > + }
> > > + return strcmp((*d1)->d_name, (*d2)->d_name);
> > > +}
> >
> > Isn't that a strange comparison algorithm, for first compare
> > string lengths and only if eaqual their content? Here does mmh
> > sort by length of folder name?
> 
> This is only for msg-files and sorts

Re: [mmh] folder_read with scandir()

[markus schnalke]

Hoi.

[2020-03-19 12:45] Philipp Takacs 
>
> I have also looked at folder_read and found it't a bit ugly ;-)

A good indicator for starting to rework. :-)

> I have rewritten it with scandir. This saves about 40 loc and improves
> the reading flow for this funktion.

Sounds good!


Some comments:

> Also noticed the other flag, not shoure if we need this realy. It's
> only used in folder and I don't see the point of knowing that there
> are other files in a folder.

We should be a bit careful at that point to understand the
situation. Here are some aspects to think about:

- Files starting with a comma. These were backups of deleted messages
(see the section on the trash folder in my master's thesis). Thus
they are considered MH files here but not further regarded. I'm
not sure if the code had been this way all the time, because there
used to the a `--with-hash-prefix' option (or similar) that
replaced the comma with a hash symbol as the prefix for deleted
messages backups. Anyways, we don't have such comma files anymore.
Mmh uses the trash folder for that case, i.e. we can get rid of
the comma prefix file handling ... possibly. Does anyone know the
situation in nmh: have they switched to a trash folder as well?
Because we're still (basically) mail storage compatible to nmh (I
think), we should not be fully ignorant of that situation. But we
could handle those backup files as `other' files. 

- `Other' files seem to be files in the mail storage that have
nothing to do with MH. E.g. any random file someone created in the
mail storage without any connection to mmh. Normally mmh would
simply ignore them, i.e. act as if they wouldn't be there at all.
There's one place, however, where `other' files are relevant:
rmf(1). We should not delete a mail folder that includes `other'
files. It seems this check is missing there. Hence, I would keep
the `other' files marker. It adds few complexity and can be
useful. Plus, we should use it in rmf(1).


> +static int others;
> +
> +static int
> +msgcmp(const struct dirent **d1, const struct dirent **d2)
> +{
> + size_t l1, l2;
> +
> + l1 = strlen((*d1)->d_name);
> + l2 = strlen((*d2)->d_name);
> + if (l1 < l2) {
> + return -1;
> + }
> + if (l1 > l2) {
> + return 1;
> + }
> + return strcmp((*d1)->d_name, (*d2)->d_name);
> +}

Isn't that a strange comparison algorithm, for first compare
string lengths and only if eaqual their content? Here does mmh
sort by length of folder name?


> +static int
> +msgfilter(const struct dirent *e)
> +{
> + int i;
> + if (e->d_name[0] == '.' || e->d_name[0] == ',') {

About this comma: see above!

> + return 0;
> + }
> + for (i = 0; e->d_name[i]; i++) {
> + //if ((i > 1 && e->d_name[i] < '0') || e->d_name[i] < '1' || 
> e->d_name[i] > '9') {

C99-style comment ... looks to be not finished at this point.

> + if (e->d_name[i] < '0' || e->d_name[i] > '9') {
> + others = 1;
> + return 0;
> + }
> + }
> + return 1;
> +}


Wow, it feels so good to have some touch with the mmh development
again! I'm really looking forward to dig into the code as well.
Hoping I get that managed soon.


meillo

Re: [mmh] folder_add and msgstats

[markus schnalke]

Hoi.

[2020-03-19 12:33] Philipp Takacs 
>
> I have also removed the
> clear_msg_flags loop from folder_read and folder_realloc. In most cases
> we use calloc, so this isn't necesarry. At one place in folder_realloc
> realloc is used. There I have add a memset.
> 
> comment?

Why were (in the section down below) only the flags outside the
current message range cleared? Now you clear all flags. Does that
change the behavior? Or, if your memset() clears only specific
parts, you probably should put a comment there to explain it, as
this might not be what the programmer usually expects from a
memset() after a (re)alloc().

Apart from that, your changes look and sound sensible.


meillo


> From 937c01f62952147a8ff853d8a103dbe8465cb90e Mon Sep 17 00:00:00 2001
> From: Philipp Takacs 
> Date: Wed, 18 Mar 2020 21:46:30 +0100
> Subject: [PATCH 3/3] use memset to clear the msgstats in folder_realloc
> 
> ---
>  sbr/folder_realloc.c | 16 +---
>  1 file changed, 1 insertion(+), 15 deletions(-)
> 
> diff --git a/sbr/folder_realloc.c b/sbr/folder_realloc.c
> index 64a66409..781e8bdc 100644
> --- a/sbr/folder_realloc.c
> +++ b/sbr/folder_realloc.c
> @@ -46,6 +46,7 @@ folder_realloc(struct msgs *mp, int lo, int hi)
>   ** just realloc the message status array.
>   */
>   mp->msgstats = mh_xrealloc(mp->msgstats, MSGSTATSIZE(mp, lo, 
> hi));
> + memset(mp->msgstats + mp->hghoff - lo, 0, hi - mp->hghoff);
>   } else {
>   /*
>   ** We are changing the offset of the message status
> @@ -69,20 +70,5 @@ folder_realloc(struct msgs *mp, int lo, int hi)
>   mp->lowoff = lo;
>   mp->hghoff = hi;
>  
> - /*
> - ** Clear all the flags for entries outside
> - ** the current message range for this folder.
> - */
> - if (mp->nummsg > 0) {
> - for (msgnum = mp->lowoff; msgnum < mp->lowmsg; msgnum++)
> - clear_msg_flags(mp, msgnum);
> - for (msgnum = mp->hghmsg + 1; msgnum <= mp->hghoff; msgnum++)
> - clear_msg_flags(mp, msgnum);
> - } else {
> - /* no messages, so clear entire range */
> - for (msgnum = mp->lowoff; msgnum <= mp->hghoff; msgnum++)
> - clear_msg_flags(mp, msgnum);
> - }
> -
>   return mp;
>  }
> -- 
> 2.20.1
> 

Re: [mmh] Previous-Sequence

[markus schnalke]

Hoi.

[2019-12-15 17:55] Philipp Takacs 
>
> During that I found the Previous-Sequence again. I don't see a reason
> why we need this feature. The usecases, I see, for the feature are better
> handled with mark or the backlog of your shell. But I don't use this
> feature.

Just for reference, the section from mh-sequence(7):

   The Previous Sequence
   Mmh provides the ability to remember the `msgs' or  `msg'
   argument last given to an mmh command.  The entry `Previ‐
   ous-Sequence' should be defined in the mmh  profile;  its
   value  should  be  a  sequence  name or multiple sequence
   names separated by spaces.  If  this  entry  is  defined,
   when   an  mmh  command  finishes,  it  will  define  the
   sequence(s) named in the value of this entry to be  those
   messages  that  were  specified to the command.  Hence, a
   profile entry of

Previous-Sequence: pseq

   directs any mmh command that accepts a  `msg'  or  `msgs'
   argument  to define the sequence `pseq' as those messages
   when it finishes.

   Note: there can be a performance  penalty  in  using  the
   `Previous-Sequence'  facility.   If  it  is used, all mmh
   programs have to write the sequence  information  to  the
   .mh_sequences file for the folder each time they run.  If
   the `Previous-Sequence' profile entry  is  not  included,
   only pick and mark will write to the .mh_sequences file.

I myself don't use the previous sequence neither, yet I was
reluctant to throw it away as I wondered if I just haven't
understood it fully enough to find use for it. Also, I have the
feeling that I haven't really understood its reason for existance
well enough.

To me, it seems that it might have been introduced in times before
the shell history feature and today, the shell history covers the
usecase up fully.

The usecase I have in mind is something like:

show 16 25-29 33 34 41-l
refile pseq +foo

However, with shell history and line editing facilities, I simply
do:
Escape k cw refile
to get the second command with out the previous sequence. The
argument set doesn't get lost if you have a shell history.

The shell history covers up the case of a wrong command you like
to undo, too. E.g. I have ``mark unread'' and ``mark read''
commands (which wrap around mark(1)).


And regarding the situation apart from interactive shells with
history, in scripts you'll have the msg arguments in variables
anyways, thus you have them remembered.


This brings me to the conclusion that I don't see a relevant need
for the previous sequence in mmh. I find all the usecaes I can
think of covered otherwise (and more convenient) and I haven't
used the previous sequence myself at all.

Besides: It does introduce a lot of write accesses to the sequence
files.


Before we remove it, I just would like to know why it was
introduced in MH the first time and what possible other usecases
there could be. This means asking on the nmh-workers mailing list.
Both, out of curiosity and as a double-check in case we've not
thought on something. Do you wanna write this message or should
I?


meillo

Re: [mmh] show sequences racecondition

[markus schnalke]

Hoi,

good to see you being active again, Philipp. I try to put my
share in as well, at least comment on your messages. So, let's
start here:


[2020-03-19 12:14] Philipp Takacs 
> [2019-12-15 17:55] Philipp Takacs 
> >
> > I have looked at the problem with open pager for show and change the
> > sequences during the pager is open. This happen often, if you show a
> > message and recieve a new message.

This is an annoying inconvenience (actually a bug but not too
severe) that we probably all suffer from. Improving the situation
would be great.

We can't get fully rid of the race condition, or? We have multiple
processes writing the same file, thus we will inevitably have a
conflict window. Correct? Could you explain quickly how we react
in this case of conflict?

(For once I sadly am not able to dig into the code at the moment,
but as well it often is valuable to explain the code to someone
else in order to check the approach taken.)


meillo

Re: [mmh] pick rework the secound

[markus schnalke]

Hoi.

[2019-07-09 16:04] Philipp Takacs 
>
> So if you find a test case is to little bug me to add one ;-).

Some suggestions:

- `pick -subject foo' for lower, upper and mixed case ``foo'' in
  the subject

- `pick -subject übung' for encoded header

- `pick -search übung' for encoded body (does fail currently)

- `pick -search foo' for some messages have ``foo'' only in the
  subject and some have ``foo'' only in the body (It doesn't so
  much matter here where `-search' should search, but just that
  we can detect regression bugs.)


> [2019-07-08 16:15] markus schnalke 
> >
> > One thing I haven't noticed up to now, regarding this now deleted
> > comment in the old version of uip/pick.c:
> >
> > ** Unfortunately,
> > ** pick advertises that lowercase characters matches characters of both
> > ** cases.  Since re_exec() doesn't exhibit this behavior, we are stuck
> > ** with this version.
> >
> > As you couldn't keep this case-sensitivity behavior with regex(3),
> > you switched to case-insensitive matching only. That was probably
> > the best choice. But we need to document this in the HISTORY section
> > of the pick manpage.
> >
> >
> > I think we also need to document there that `-search' does no longer
> > search the header fields. AFAICS there is no longer a possibility to
> > search all header values and the body. You have to list each header
> > field explicitely. You neither can cover it with grep(1) as the
> > header values and body text are likely to be encoded. But, well, we
> > can live without it, I'd say. -- Did I understand the situation
> > corrently?
> 
> Yes, this is the current situation.

Thanks for clarifying.


> Aditional: Encoding wasn't handled
> at all before my rewrite. The search switch still don't handle
> encoding. I have plans to add this.

That sounds great!


> Also I think about adding regex also to the header name search.

What usecases do you have in mind for that?


> Also the search switch could search in all field values.

It would be nice if `-search' would search in about everywhere in
the messages, so that you don't have to think about it ... like
grepping through the files.

If we want specific only body searching, we rather add a `-body'
switch.


> But this isn't
> well-thought-out, so maybe my plans how to implement this change.

See how it goes during implementation. See what makes sense from
the implementation view.


meillo

Re: [mmh] pick rework the secound

[markus schnalke]

Hoi.

Thanks for the work!

Unfortunately, I haven't had a closer look, no testing and probably
won't find the time to do so. What you wrote sounds good. I
appreciate that you added a testcase to catch regression bugs!


One thing I haven't noticed up to now, regarding this now deleted
comment in the old version of uip/pick.c:

** Unfortunately,
** pick advertises that lowercase characters matches characters of both
** cases.  Since re_exec() doesn't exhibit this behavior, we are stuck
** with this version.

As you couldn't keep this case-sensitivity behavior with regex(3),
you switched to case-insensitive matching only. That was probably
the best choice. But we need to document this in the HISTORY section
of the pick manpage.


I think we also need to document there that `-search' does no longer
search the header fields. AFAICS there is no longer a possibility to
search all header values and the body. You have to list each header
field explicitely. You neither can cover it with grep(1) as the
header values and body text are likely to be encoded. But, well, we
can live without it, I'd say. -- Did I understand the situation
corrently?


meillo



> From 67e3a94cdfddfbf0630a2c78f2817d1c8d7bf856 Mon Sep 17 00:00:00 2001
> From: Philipp Takacs 
> Date: Mon, 10 Jun 2019 13:55:02 +0200
> Subject: [PATCH 3/3] remove unused defines in uip/pick.c
> 
> These defines were used for the old pattern matching.
> Since we use regex(3) they aren't used.

>  uip/pick.c | 25 -
>  1 file changed, 25 deletions(-)
> 
> diff --git a/uip/pick.c b/uip/pick.c
> index e9783318..d74c7310 100644
> --- a/uip/pick.c
> +++ b/uip/pick.c
> @@ -515,31 +515,6 @@ static struct swit parswit[] = {
>   { NULL, 0 }
>  };
>  
> -/* DEFINITIONS FOR PATTERN MATCHING */
> -
> -/*
> -** We really should be using re_comp() and re_exec() here.  Unfortunately,
> -** pick advertises that lowercase characters matches characters of both
> -** cases.  Since re_exec() doesn't exhibit this behavior, we are stuck
> -** with this version.  Furthermore, we need to be able to save and restore
> -** the state of the pattern matcher in order to do things "efficiently".
> -**
> -** The matching power of this algorithm isn't as powerful as the re_xxx()
> -** routines (no \(xxx\) and \n constructs).  Such is life.
> -*/

Re: Mail spool locking (was: [mmh] mmh-0.4 is released! (meillo's mail handler))

[markus schnalke]

Hoi.

[2019-01-25 21:38] Dmitry Bogatov 
> [2019-01-24 09:53] markus schnalke 
> > Concerning the setgid of inc(1), the Debian Policy writes:
> >
> > The mail spool is 2775 root:mail, and MUAs should be
> > setgid mail to do the locking mentioned above (and must obviously
> > avoid accessing other users’ mailboxes using this privilege).
> >
> > The Debian Policy even motivates us to make inc(1) setgid ...
> > because it simply has to be for reliable locking.
> >
> > Thus I suggest that you keep the setgid part as it is upstream.
> 
> Seems I failed to clearify my point.
> 
> Of course, inc(1mh) on user system is setgid. But due the nature of
> binary packages, setgid can't be set on build system, it is set on user
> system during installation. I (and any other distribution maintainer)
> have to remove attempts to setgit inc(1mh) on buildtime.

Ah okay. ;-)

So, the change request is to ease the package builds on Debian and
probably other distributions. Is this correct?

Thus no change of the default behavior is requested but only an
additional (configure) option. Correct?


If there is no better solution, I could go with an additional
configure option.

Does anyone know how it is solved in nmh and its Debian package? Do
they have that configure option?

Is such a configure option the preferred way to solve the issue?
I mean, every set[ug]id program on the system has the same problem.
At least from the times when I maintained the MTA masqmail, I
cannot remember anything like that ... but, of course, that's years
ago, Debian packaging has evolved since. I'm just a bit irritated
that you would request all upstreams to add a configure option just
for the build system. I'd rather use a chmod(1) wrapper on the build
systeme, which ignores the set[ug]id request, if I were in Debian's
place. Looks more hazzle-free to me. This is why I ask so many
questions ...

You must realize that I worked hard to cut the configure options
down to where we are now. Mmh is an effort to counter feature-creep.
Adding new configure options need a certain amount of need to have
me like them. ;-)


Does someone on the list know the situation for different GNU/Linux
distributions or BSD derivates?


meillo

Re: [mmh] mmh-0.4 is released! (meillo's mail handler)

[markus schnalke]

Hoi.

[2019-01-21 20:24] Dmitry Bogatov 
> 
> Thank you. Packaged and upload.

Great to hear! Thanks a lot.


> Please, consider following patches I
> maintain in Debian, you may wish to apply them upstream.
> 
> 1. Spelling errors in manpages
> 2. Hyphen usage in manpage

We'll be pleased to apply those two.

> 3. Install manpages as show.1mh, not show.1 to avoid conflicts with
>other programs.

Not sure about that one. It's mainly a distribution issue not so
much an upstream issue, I'd say. But tell me otherwise!
Oppinions?

> 4. Currently, if LOCKTYPE=dot, ./configure script forces Makefile to
>setgid installed `inc' binary. I patch it out, but you may want to
>add option to /not/ use setgid. It is headache for two-stage
>installation, used by most package managers (dpkg, rpm, ...)

AFAICS, if you use dotlocking and the mail spool directory is not
writable for all possible users, then inc(1) has to have setgid
or you would have no locking. Or am I wrong?

I recall that Debian has some kind of special libdotlocking or
such which needs to be used for mail spool locking by Policy,
IIRC. Does that solve the issue in a different way? IIRC you
cannot use other lock methods as the mail spool could be on an
NFS share, which only offers dotlocking. ... not sure about all
that. It's too far past when I dug into in the last time.

Questions: What's the situation exactly on Debian? Is this a
distribution-specific patch or does it fix a general issue?


meillo

Re: [mmh] mmh-0.4 is released! (meillo's mail handler)

[markus schnalke]

Hoi.

[2019-01-08 12:37] Dmitry Bogatov 
> [2019-01-06 20:40] Philipp Takacs 
> >
> > After more then a year I can announce the release of mmh-0.4
> > 
> > http://marmaro.de/prog/mmh/files/mmh-0.4.tar.gz
> 
> Good news. You seems to forgot to upload signature (mmh-0.4.tar.gz.asc).
> Would you be so kind to do it?

Philipp wanted to provide a signature, he just hasn't managed to do
so yet. I'm sure we'll have it anytime soon ...


meillo

Re: [mmh] mmh-0.4 is released! (meillo's mail handler)

[markus schnalke]

Hoi,

thanks for the all the good work and for preparing this release,
Philipp!

It was a pleasure reading the release announcement. Thanks a lot!


meillo

Re: [mmh] mmh 0.4 whatnow2 and pick

[markus schnalke]

Hoi.

[2019-01-05 15:01] Philipp Takacs 
>
> I have changed my mind about the deprecation of whatnow. The best
> option I see is to add a warning to whatnow and the manpage mention
> whatnow2. The warning could look somithing like this.
> 
> > whatnow is deprecated and will removed in mmh-0.5. Consider switching to 
> > whatnow2.

This plan goes okay with me.

Maybe just you could shorten then message to fit 70 chars or so.
Maybe like this:

whatnow is deprecated. Consider switching to whatnow2.

(Besides: We should avoid predicting the future. You never know
if the 0.5 release will actually turn out to be what you now
think it will. If at all, then write ``will be removed soon'' or
``in some of the next releases'', but actually, the word
``deprecated'' says this already.)

Nonetheless, I like the warning message.


> I'll also do some test today, but see no problem with the relase at
> 06.01.2019.

I have problems finding free time to have closer looks. But that
shall not delay anything. Release as planned. If we discover
problems afterwards, the following release only comes sooner. ;-)


Keep going!


meillo

[mmh] whatnow, whatnow2, draft metadata

[markus schnalke]

[I wrote this draft at 2018-11-25, but never came to go over it
a second time and send it.]


Hoi,

Philipp brought the whatnow topic up again (in private
conversation). I realized that I still don't use whatnow2, which
is a bad thing when we would want to drop whatnow in the next
release. Our last discussions on the topic are a year or so ago.

Bear with me that I start afresh and try to summarize the
situation and think aload about ways to go ...


Philipp wrote that in his eyes, the old whatnow would be dead and
we should switch to whatnow2 or something similar.


If we want to go this way, I need to drop whatnow from my workflow
and start using whatnow2 or alternatives, now. ;-)

Thus I had a(nother) look at whatnow2. AFAICS, it currently works
on the current message in the draft folder only. This collides
with my typical work scheme of drafting multiple messages in
different tmux shells in parallel. Hence I would need to switch
the current message in the draft folder explicitely all the time.
(It might be possible to solve that problem with different
contexts and private sequences, but I'm not sure, as I've never
used private messages.)

In the core of the problem lies the question: How does whatnow2
know which draft to work on?

The answer is: It cannot know it, unless we explicitely tell it.
The current message is the only automatic hint we have. This
however collides with parallel working on multiple drafts.


But let us look on the topic from a different angle: Let's think
without having any sort of whatnow/whatnow2. Comp/repl/forw/dist
would invoke the first editor on their own and after it exited
just leave the draft in the draft folder and exit to the shell.

Then we need replacements to these whatnow commands:

- edit already covered by `comp -use', we just need to care
   for last-editor handling

- list already covered by `show' (no separate listproc
   necessary, i'd say)
- send already covered by `send'
- delete   already covered by `rmm +drafts'
- refile   already covered by `refile -src +drafts'

- display  new wrapper necessary for `show', which honors
   mhaltmsg (i.e. soon a header in the draft)
- attach   new wrapper necessary for `anno'
- alistnew wrapper necessary for `anno'
- detach   new wrapper necessary for `anno'


Thus the question: Why do we want whatnow2 at all? Wouldn't it be
much better to go for the above changes right away?


First, we should to move the draft environment variables into the
draft. Currently, Philipp uses a separate meta file for that in
whatnow2. I think we already had consensus for putting it as
mmh-* headers (or similar) in the draft. I'd say that this is
what we should do at first, because this change is conceptionally
worthwhile anyways, independent of any whatnow rework.

@Philipp: Are you already working on this or should I start with
it? [This I wrote on 2018-11-25 -- I was really keen to start
working on it ... but personal life made it impossible just at
that time. :-/ ]


Second, cover the yet uncovered tasks: display, attach, detach,
and alist. Two ways come to mind to do so:

1) Add each of those commands as a tool (shell script or C code),
wrapping show or anno, respectively.

2) Merge the tools:

- Display could be added as a switch to show.

- Attach/detach/alist could all be one tool:
attach
attach -d
attach -l

... I'm unsure what I like more ...


Nonetheless, I'd favor to drop whatnow/whatnow2 altogether, making
a compatibility-breaking release, and go for the conceptionally
better approach right away.

The first step, of moving the environment variables of whatnow or
meta files of whatnow2 into the draft, is worthwhile anyways and
can be done right away, independent of further decisions. This
should be done.


What do you think?


meillo

Re: [mmh] mhl trailing white space

[markus schnalke]

[2018-11-22 10:53] Philipp Takacs 
> [2018-11-22 08:35] markus schnalke 
> > [2018-11-19 03:41] Philipp Takacs 
> > >
> > > After I have looked at the ali bug, I have looked at the other tests
> > > which are skipped. I have noticed the mhl with space feature is not this
> > > hard to port from nmh. So I have done this. A patch for this is attached.
> >
> > Seems to be very useful to have that feature. I've always been
> > dissatisfied with lines containing only "> " in my replies.
> >
> > Is there a need to update some files in etc/ to use the rtrim
> > function by default?
> 
> I don't think so. The rtrim for lines with only "> " is done automatic.
> only the rtrim of contant have to be requested by the component settings.

Okay. Thanks for the clarification.

Oh man! How much I love to see the trailing whitespace of "> "
lines to be removed! :-D  Really happy about that feature!


> > > Ps: the RFC 2231 support is on the todo list but not on top
> >
> > Okay.
> 
> To my todo list: I have some features and patches I'll send the next
> days to the list.

I'm much looking forward to that! :-)


> > > diff --git a/uip/mhl.c b/uip/mhl.c
> > > index a84703b8..b0ba02ed 100644
> > > --- a/uip/mhl.c
> > > +++ b/uip/mhl.c
> >
> > > @@ -884,7 +887,10 @@ putcomp(struct mcomp *c1, struct mcomp *c2, int flag)
> >
> > >   if (c1->c_flags & CLEARTEXT) {
> > > - putstr(c1->c_text);
> > > + putstr(c1->c_flags & RTRIM ? rtrim(c1->c_text) : c1->c_text);
> >
> > For those bit-check-plus-ternary-operator cases, parenthesis
> > around the bit check should be used to clarify the precedence. I'm
> > no fan of using too many parenthesis but in cases like this one,
> > I think they improve the readability. Thus:
> >
> > putstr((c1->c_flags & RTRIM) ? rtrim(c1->c_text) : c1->c_text);
> >
> > There are some more occurences of this pattern in the patch ...
> 
> As I said this patch is only ported from the nmh code, but I can add
> parenthesis.
> 
> > > @@ -970,16 +980,23 @@ putcomp(struct mcomp *c1, struct mcomp *c2, int 
> > > flag)
> > >   }
> > >   count += c1->c_offset;
> > >  
> > > - if ((cp = oneline(c2->c_text, c1->c_flags)))
> > > -putstr(cp);
> > > + if ((cp = oneline(c2->c_text, c1->c_flags))) {
> > > +putstr(c1->c_flags & RTRIM ? rtrim(cp) : cp);
> >
> > Here's an indent with spaces. (That's the mixed tabs and spaces
> > indent style of nmh. In mmh all indents should be tabs only.)
> 
> I have also noticed this, but the indention is already in our code.
> I wanted to write a secound patch to fix the style. But I can include
> this in the patch as well.

I just wanted to point on stuff I noticed. It's fine if you you it
in a separate commit. I don't care how. Do as you think it fits
best.


meillo

Re: [mmh] Alias and blind lists (was: Fixed two tests)

[markus schnalke]

[2018-11-18 22:43] Philipp Takacs 
> [2018-11-16 00:40] Philipp Takacs 
> > > Currently, only the alias expanding test is still failing. Does
> > > anyone know if nmh has solved this already? Can we use their
> > > solution? @Philipp: IIRC, you had a look at that some time ago.
> > 
> > I have looked at this some time ago, I'm just remember that the
> > code is a bit ugly. I'll look at this again.
> 
> I have looked at the bug, nmh already fixed this. I'm not shure
> why we have the test but not the fix.
> 
> I have cherry-picked the commit and pushed it.

Thanks.


meillo

Re: [mmh] [PATCH] Ignore folders with an empty sequence in new

[markus schnalke]

Hoi.

[2018-10-01 16:01] c_14 
>
> This solves the issue where `new' lists folders that have an empty
> sequence (e.g. the unseen sequence), this also prevents `fnext' and
> `fprev' from navigating to such folders.

Thanks for the patch. It is applied now. Plus, I've added a
regression test for it.


meillo

Re: [mmh] Dig into MIME! (was: Accept header lines of exactly 998 characters)

[markus schnalke]

Hoi.

[2018-07-19 10:52] Vasilii Kolobkov 
> [2018-07-19 10:02 +0200] markus schnalke 
> > [2018-07-19 03:24] Philipp Takacs 
> > > [2018-07-18 23:13] Vasilii Kolobkov 
> > > > The other broken part is second to last case in test/send/test-mimeify.
> > > > The sizes reported for multipart/... types differ from expected
> > > > values. I'll be looking further into it, but wonder if it's broken
> > > > on your systems as well?
> > > 
> > > This also a known bug and behaves realy strange, because sometimes
> > > the numbers differ. I have looked at this some time ago, but because
> > > the mime implementation is realy complex I coldn't find it.
> > 
> > These could have been my words as well. ;-)
> > 
> > If one could track that down, that would be great.
> 
> You got me intrigued guys, but not scared. I'll give it a try -
> heard a lot about how mime was notorious for complex code, but
> haven't experienced that myself yet.

It's an experience worthwhile to have ... once in your life! ;-D


But seriously, MH's MIME stuff is really interesting to have a
look at, in regards of MIME's coming to life. AFAIK, MH was the
first implementation of MIME. It goes closely with the concepts
of the RFCs. The idea of MIME as a means to encode all sorts of
contents (like newspaper articles and such stuff) is visible in
MH's MIME implementation, which I guess is different if you
look at other mail software. MH can present MIME messages as a
multimedia performance, invoking all sorts of commands to handle
specific parts, and, the other way round, you can create most
complex MIME messages -- stuff that most mail clients would not
regard as mail messages at all.

True that, MIME implementations have complex code -- MH's has
all the downsides of any complex code -- but also true, that
MH's MIME implementation is worth a closer look. If you have
the time and the brain capacity to lay out all those data
structures and program flows, then do dig into it! (You know,
m_getfld was much worse, and still, it tought great lessons!)


meillo

Re: [mmh] Accept header lines of exactly 998 characters

[markus schnalke]

Our replies interleaved ...


[2018-07-19 09:38] Vasilii Kolobkov 
>
> Thanks for the comments, Philipp!
> 
> [2018-07-19 03:24 +0200] Philipp Takacs 
> > [2018-07-18 23:13] Vasilii Kolobkov 
> > > diff --git a/test/runtest b/test/runtest
> > > index 9f35ade..37a23ac 100755
> > > --- a/test/runtest
> > > +++ b/test/runtest
> > > @@ -1,13 +1,15 @@
> > >  #!/bin/sh
> > >  
> > >  set -e
> > >  
> > >  export MH_TEST_COMMON="$PWD/common.sh"
> > >  
> > > +. ${MH_TEST_COMMON}
> > > +
> > 
> > Would be nice if this patch would also remove the source in
> > the single tests.
> 
> I do like it that way it is for it makes this dependency a bit more
> explicit.  Let me know if you see no merit in it and i'll dump
> sourcing in each test.

As written in the other message: I'd favor to just have these helper
functions available in all tests (like we have all of sbr/ available
in all of uip/). It's a library thing ...


> > > diff --git a/test/tests/mhsign/test-mhsign b/test/tests/mhsign/test-mhsign
> > > index 3c2bb97..4f24b09 100755
> > > --- a/test/tests/mhsign/test-mhsign
> > > +++ b/test/tests/mhsign/test-mhsign
> > > @@ -3,14 +3,21 @@
> > >  #
> > >  # Test mhsign (correct alias expansion with -enc)
> > >  #
> > >  ##
> > >  
> > >  . "$MH_TEST_COMMON"
> > >  
> > > +phostname()
> > > +{
> > > + case `uname -s` in
> > > + OpenBSD) hostname ;;
> > > + *) hostname -f ;;
> > > + esac
> > > +}
> > 
> > oh hostname, markus has summarised[3] this very good. I wouldn't
> > change the test, because it does exact the same as mhsing. Maybe
> > after fixing mhsign.
> 
> I missed the mhsign(1) discussion, but do agree here. Though it'd
> have to be `hostname -f 2>/dev/null | uname -n` so no to fill the
> output with errors about unrecognized flag.

Yes, do so, but put the second pipe symbol in place again! ;-)


meillo

Re: [mmh] Accept header lines of exactly 998 characters

[markus schnalke]

Hoi.

[2018-07-19 03:24] Philipp Takacs 
> [2018-07-18 23:13] Vasilii Kolobkov 
> > I'm having the tests green... well technically they are black, but
> > function much better to their end on OpenBSD :)
> 
> Thanks.
> 
> > Yet two of them do fail, but that'd be another patch. The ali tests
> > for the blind lists fail - it's the first time i came to know of
> > this feature and frankly don't know if i'd want to work on fixing
> > that. Does anybody use them?
> 
> If you talk about the test which fails on markus nightly tests[0],
> it's a known bug, but down on the priority list. I don't know if
> someone use this feature[1], but I would like to have this sane
> implemented.

My view is the same.


> > The other broken part is second to last case in test/send/test-mimeify.
> > The sizes reported for multipart/... types differ from expected
> > values. I'll be looking further into it, but wonder if it's broken
> > on your systems as well?
> 
> This also a known bug and behaves realy strange, because sometimes
> the numbers differ. I have looked at this some time ago, but because
> the mime implementation is realy complex I coldn't find it.

These could have been my words as well. ;-)

If one could track that down, that would be great.


> In general I like your patch, but I have some comments:
> 
> > p.s. i prefixed all the portable functions with p* so to not shadown
> > the system ones just in case.

Now that I thought about that, I wonder if we should name the
compat functions in the shell scripts the same way as we name
them in the C code: by prefixing them with ``mh''. But OTOH, these
functions in the C code usually don't provide 1:1 functionality
but rather slightly adjusted functionality. Still, the ``mh'' or
``mh_'' prefix shows quicker what's going on, IMO.

Your oppinion on this?


> > ---8<---
> > From 204ae88524838063493242f012a1b1fe2fcf2d05 Mon Sep 17 00:00:00 2001
> > From: Vasilii Kolobkov 
> > Date: Sun, 15 Jul 2018 23:05:09 +0200
> > Subject: [PATCH] Fix tests to run on OpenBSD
> > 
> >   For there's no seq(1) in OpenBSD, Use pseq function from common.sh
> >   for generating sequence numbers. Credits for the function goes
> >   to Markus Schnalke.
> > 
> >   He also had a one-liner, which having less features is so freaking
> >   nice to not mention it here:
> > < /dev/zero tr \\000 \\n | sed 10q  | nl -b a
> > 
> >   awk(1) doesn't support interval regular expression syntax ({n,m})
> >   on some BSDs.
> > 
> >   hostname(1) has no -f flag on OpenBSD and, while invocation without
> >   flags is supposed to return FQDN allright. Use the same phostname
> >   function in mhsign(1).
> > 
> >   mktemp(1) is limited to at least 6 placeholder symbols on OpenBSD.
> > ---
> >  test/common.sh| 18 ++
> >  test/runtest  |  4 +++-
> >  test/tests/inc/test-eom-align |  2 +-
> >  test/tests/mhsign/test-mhsign | 11 +--
> >  test/tests/mhstore/test-filenames |  2 +-
> >  test/tests/show/test-longlines|  6 ++
> >  uip/mhsign.sh | 11 ++-
> >  7 files changed, 44 insertions(+), 10 deletions(-)
> > 
> > diff --git a/test/common.sh b/test/common.sh
> > index 805afea..056eac2 100644
> > --- a/test/common.sh
> > +++ b/test/common.sh
> > @@ -105,7 +105,25 @@ runandcheck()
> >  
> > if [ "$diff" ]; then
> > echo "$0: $1 failed"
> > echo "$diff"
> > failed=`expr "${failed:-0}" + 1`
> > fi
> >  }
> > +
> > +pseq() {
> > +   start=1
> > +   inc=1
> > +
> > +   case $# in
> > +   1) end="$1" ;;
> > +   2) start="$1"; end="$2" ;;
> > +   3) start="$1"; inc="$2" ; end="$3" ;;
> > +   *) echo "Usage: pseq end | start end | start step end" >&2; return 1 ;;
> > +   esac
> > +
> > +   awk 'BEGIN{
> > +   for (i='"$start"'; i<='"$end"'; i+='"$inc"') {
> > +   print i
> > +   }
> > +   }'
> > +}
> 
> I'm not sure, if we realy nead this funktion, because it's used on
> two places. In the one place it's easy to remove. On the other place
> I would remove the hole test.
> 
> > diff --git a/test/runtest b/test/runtest
> > index 9f35ade..37a23ac 100755
> > --- a/test/runtest
> > +++ b/test/runtest
> > @@ -1,13 +1,15 @@
> >  #!/bin/sh
> >

Re: [mmh] Accept header lines of exactly 998 characters

[markus schnalke]

Hoi.

[2018-07-16 21:45] markus schnalke 
> [2018-07-16 20:56] Vasilii Kolobkov 
> >
> > I'd rate the given options (in order of descending preference) -
> > 1. tr|sed|nl 2. sh w/ expr(1) and 3. awk. What about you?
> 
> First step, I'd say: Write some enum() or countto() or even still
> name it seq() function in test/common.sh, which only accepts a
> single argument and does what seq(1) does if it has exactly one
> argument.

Well, if we choose the awk implementation, the we can easily support
two and three arguments as well:

seq() {
start=1
inc=1

case $# in
1) end="$1" ;;
2) start="$1"; end="$2" ;;
3) start="$1"; inc="$2" ; end="$3" ;;
*) echo "Usage: FIXME" >&2; return 1 ;;
esac

awk 'BEGIN{
for (i='"$start"'; i<='"$end"'; i+='"$inc"') {
print i
}
}'
}

Not robust, but already working. The tr|sed|nl code doesn't grow
the same way. This is why I would favor the awk approach.


meillo

Re: [mmh] Accept header lines of exactly 998 characters

[markus schnalke]

Hoi.

[2018-07-16 20:56] Vasilii Kolobkov 
> [2018-07-16 12:23 +0200] markus schnalke 
> > [2018-07-15 23:29] Vasilii Kolobkov 
> > > 1. Generate sequences vi shell script as BSDs lack seq(1)
> > 
> > That's unfortunate, as seq(1) is such a great tool. It's not part
> > of POSIX though. (It originated in Research Unix after v7, i.e. on
> > the way to Plan 9 but after the SystemV and BSD have split off, IIRC.)
> 
> Yeah, read something like 'in Linux you do `seq 1 10` and in BSDs
> - `jot 10 1`'. Guess you got to learn to appreciate the fun bits
> like that, even if just to keep your nervous system intact when
> doing cross-platform stuff.

> Enough ground to keep a software anthropologist busy for a long
> while, heh? til, just keep interval expressions out of awk programms.

Yeah, an interesting topic in itself, this Unix history stuff.
I like it a lot. Of course, it makes portability a difficult job.
But, now that we have POSIX ... okay wait: eventually, when all
systems will implement it ... ;-P


> > Although $((...)) ist part of POSIX. I'd like to avoid it, if
> > possible. All uses of $((...)) in our test cases can be easily
> > converted to expr(1). Would that be okay for you or do do you have
> > strong opinions pro $((...))?
> 
> I hold no special feeling about arithmetic expressions in sh(1) -
> they do get clunky fast, but i still prefer shell constructs to
> involving some heavy hitters like awk. Mind sharing why you choose
> against them?

POSIX's goal was to document the common behavior across various
Unix derivates. In situations, when different, incompatible
behavior was present, it sometimes chose one of the alternatives
or none, if one could live without. For the most part, it refused
to standardize new behavior, especially if there was no important
reason to do so. To me, arithmetic expressions in sh are kind of
superfluous. There already exist alternatives in POSIX that are
older and more widespread: expr(1) for the simple stuff, bc(1)
for the complex stuff, test(1) for boolean, and awk(1) for the full
power of $((...)). Their only downside is time performance, which
I hardly like to accept as a good motivation.

Arithmetic expressions are an extension of the shell command
language, which introduces new syntax and a new thinking model,
whereas there already existed solutions, which either fitted
into the existing thinking model (expr(1), test(1)) or already
introduced new thinking models (bc(1), awk(1)).

IMO arithmetic expressions were unnecessary to add (from the Unix
standard shell POV (not from David Korn's POV, as I like to
admit)). (Even more, their syntax is badly chosen, as it is too
close to $(...) and (...).)

These are (conceptual) reasons I don't like arithmetic
expressions.

And, of course, old shells don't support them. Which is a
(increasingly less important) practical reason.


> The tr|sed|nl is such nice little gem i'd definitely go with it,
> readability objections well deserved still.

The coolnes factor is definitely high. ;-)

It's only acceptable if it is wrapped in a function with a name
that clearly describes what's going on.


> I'd rate the given options (in order of descending preference) -
> 1. tr|sed|nl 2. sh w/ expr(1) and 3. awk. What about you?

First step, I'd say: Write some enum() or countto() or even still
name it seq() function in test/common.sh, which only accepts a
single argument and does what seq(1) does if it has exactly one
argument. Then use this helper function in the test cases. The
actual implementation becomes a minor topic thus. We can change
it easily then.


> I believe it'll be best to keep this last patch as a draft and i'll
> keep adding more fixes to it.

Yes, keep going! Definitely worthwhile, your work!


meillo

Re: [mmh] Accept header lines of exactly 998 characters

[markus schnalke]

[2018-07-16 21:03] Vasilii Kolobkov 
> [2018-07-16 14:17 +0200] Philipp Takacs 
> > [2018-07-16 09:20] Vasilii Kolobkov 
> > > [2018-07-16 03:18 +0200] Philipp Takacs 
> > > > What a strange BSD are you using, maybe OpenBSD ;-)? seq is part of Net,
> > > > Free and DragonflyBSD. Ok, the awk bug/missing feature is also in 
> > > > FreeBSD.
> > > 
> > > There's no hiding from you, Philipp!
> > > 
> > > I thought those issues to be rather common in BSD-land and am sorry
> > > for badmouthing all the good ones. Can you please do a s/BSDs/OpenBSD/g
> > > on the patches?
> > 
> > This was more a joke, then a real comment.
> 
> And funny enough - like the first time i choose not to name my OS
> directly, i'm being greeted with 'why you must be running OpenBSD!'
> :) Which i do allright.

:-)

Even OpenBSD guys are welcome here. :-D Just joking. They do some
real good work and have good values in their system. I'm glad they
are around!

Please keep reporting stuff that fails on OpenBSD. It's important
to ensure mmh is working well there. (Otherwise my friend Phil --
an OpenBSD convinced guy and mmh's very first user, btw -- won't
switch back to mmh again.)


meillo

Re: [mmh] Accept header lines of exactly 998 characters

[markus schnalke]

Hoi.

[2018-07-15 23:29] Vasilii Kolobkov 
> [2018-07-15 13:50 +0200] markus schnalke 
> > Fortunately you don't have to slap me. *phew* ;-)
> 
> Seems like i missed out on some fun :)

Actually, the fun didn't happen. (Fortunately for me.) :-)


Some comments:

> ---8<---
> From a8035ca600799e10b1aab2fd9f1b9fe97e2f9aad Mon Sep 17 00:00:00 2001
> From: Vasilii Kolobkov 
> Date: Sun, 15 Jul 2018 23:05:09 +0200
> Subject: [PATCH] Make show/test-longlines test more cross-platform
> 
> 1. Generate sequences vi shell script as BSDs lack seq(1)

That's unfortunate, as seq(1) is such a great tool. It's not part
of POSIX though. (It originated in Research Unix after v7, i.e. on
the way to Plan 9 but after the SystemV and BSD have split off, IIRC.)


> 2. awk(1) doesn't support interval regular expression syntax ({n,m})
> on BSDs

Interesting.

Just FYI, POSIX writes:

Regular expressions in awk have been extended
somewhat from historical implementations to make
them a pure superset of extended regular
expressions, as defined by POSIX.1-2017 (see XBD
Extended Regular Expressions). The main extensions
are internationalization features and interval
expressions.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html

OpenBSD seems to be more historic than POSIX. ;-P



> diff --git a/test/runtest b/test/runtest
> index 9f35ade..2e6866d 100755
> --- a/test/runtest
> +++ b/test/runtest
> @@ -32,15 +32,15 @@ cat >"$MMH/profile" <<-!
>   Inbox: +inbox
>  !
>  folder -create `mhparam inbox` >/dev/null
>  folder -create `mhparam trashfolder` >/dev/null
>  folder -create `mhparam draftfolder` >/dev/null
>  
>  # create 10 basic messages
> -for i in `seq 1 10`;
> +for i in `i=0; while [ $(( i += 1 )) -le 10 ]; do echo $i; done`;

In the actual case, one could just write down the ten numbers:

for i in 1 2 3 4 5 6 7 8 9 10;

For flexibility, we ca use awk (which is even shorter):

for i in `awk 'BEGIN{ for (i=1; i<=10; i++) {print i} }'`;

Here another POSIX-compatible implementation (although a bit too
tricky, I'd say):

 diff --git a/test/tests/show/test-longlines b/test/tests/show/test-longlines
> index 138c724..c15c1cb 100644
> --- a/test/tests/show/test-longlines
> +++ b/test/tests/show/test-longlines
> @@ -9,21 +9,19 @@ set -e
>  
>  expected=$MH_TEST_DIR/$$.expected
>  actual=$MH_TEST_DIR/$$.actual
>  
>  genlongsubject() {
>   len="${1:-998}"
>   awk -v len="$len" 'BEGIN {
> - prefix = "Subject: " len
> + prefix = "Subject: " len " "
>   while (i++   s = s "x"
>   }
> - re = ".{" length(prefix) "}."
> - sub(re, prefix " ", s)
> - print s
> + print prefix substr(s, length(prefix) + 1)

Accepted. That's better.


meillo

Re: [mmh] Accept header lines of exactly 998 characters

[markus schnalke]

[2018-07-11 13:49] Vasilii Kolobkov 
>
> Holla mmh folks, hope you're doing fine!

Thanks a lot for the patch ... and for bringing some action
into this list again. We've been much too quiet (and lazy)
lately!


> Lately I've been getting issues with messages containing header
> lines of 998 characters. This patch fixes it.

I'm gonna apply it (plus add a test case to detect this situation).

If it's not done till Sunday night, slap me and do it yourself. ;-P


meillo

Re: [mmh] Re: Comments on the Debian package

[markus schnalke]

[2018-02-17 17:12] Philipp Takacs <phil...@bureaucracy.de>
> [2018-02-16 23:16] kact...@gnu.org
> > 
> > Hello. Sorry for late reply, was away from keyboard for very long time.
> 
> Nice to hear from you again.

Yeah, it's great to see you back here!


> > [2017-04-17 15:36] markus schnalke <mei...@marmaro.de>
> > > 
> > > 2) This is a bit harsh: ``[...] rather mmh breaks compatibility
> > > to nmh in order to modernize and simplify it.'' To some ears it
> > > could sound like we would not care for compatibility at all.
> > > (Compared to the nmh-situation in 2010 this might look correct.)
> > > But we rather are willing to sacrifice compatibility for (in
> > > our eyes) higher goals like modernity and simplicity, for
> > > instance. Maybe you could reword this a bit.
> > 
> > This description is from README, too. So, probably I am not right person
> > to reword.

Oh, it were my own words from 2012. ;-)

> Thanks for pointing this out. I'll look for a better wording.

I cared for it:
http://git.marmaro.de/?p=mmh;a=commitdiff;h=b3fde4704


> > By the way, you are welcome to release more often. With
> > better description, hm?
> 
> Oh...
> Yes we should release more often. Sadly I don't have much
> time at the moment. I plan to put more time in mmh mid/end of
> May. So maybe end of June[0] a new release is there.

> [0] I realy hope I can manage this

Yes. Let's go with that timeline.


> > > 3) The license information is not exactly correct. The license
> > > included in debian/copyright is not the license of mmh but only
> > > the closest one of the generic licenses.
> > > [...]
> > 
> > Fine. I will change 'debian/changelog' and insert content of COPYRIGHT
> > verbatim.

Thanks.


> > > 4) The mmh package is still unreproducible (same problem as nmh).
> > > This comes from compiling the compile time into the executables.
> > > This way, it cannot be reproducible. I have no strong oppinion
> > > either way. Just want to mention it here, for the record.
> > 
> > Thank you for warning. I will patch out source of non-reproducibility.
> 
> The time bug should be fixed.

Yes.

$ folder -V
folder -- mmh-0.3+403e719

... no more compile-time and -machine information.


> If not, contact me. But there
> is a problem with the hostname[1]. I haven't understand yet,
> so if you find and fix this, please let me know.

> [1] see 
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/mmh.html

I don't fully understand the message neither. It's not clear if
the hostname or a username is the problem. Onthe hostname side it
might be related to LocalName() in sbr/mts.c or to `hostname -f'
in uip/mhsign.sh. Probably it's rather the former function. There
is also code that discovers the current user to generate the From
header line.

I couldn't find more detailled information on what this test
actually fails and what the rationale behind it is. It would be
great if you could provide more information to help us understand
the problem and to fix it.


meillo

Re: [mmh] Dmitry Bogatov

[markus schnalke]

Hoi.

FYI, via Debian's Bits of the DPL:

Dmitry Bogatov ("KAction") will be cleared of charges

http://tass.com/society/986636


meillo


[2017-07-02 17:58] markus schnalke <mei...@marmaro.de>
>
> Hoi,
> 
> just read the monthly ``Bits of the Debian Project Leader'' and
> there my eyes stuck on these words:
> 
>   the situation around Dmitry Bogatov
> 
> In case you haven't heard about it before, have a look at, for
> instance:
> 
>   https://www.debian.org/News/2017/20170417
>   
> https://www.reddit.com/r/debian/comments/64qat1/debian_maintainer_dmitry_bogatov_was_arrested_in/
>   ... or other places on the web
> 
> Well, this is already two months old news, but nonetheless ...
> 
> 
> meillo
> 
> 

Re: [mmh] pick(1) putting zero at the end

[markus schnalke]

Hoi.

[2017-07-08 15:33] Vasily Kolobkov <polezaivs...@openmailbox.org>
> [2017-07-08 14:48 +0200] markus schnalke <mei...@marmaro.de>
> > [2017-07-08 12:08] Vasily Kolobkov <polezaivs...@openmailbox.org>
> > > While at it, do you think man pages could benefit of freshening
> > > them up with mdoc? I'm up for it.
> > 
> > I'm much of a fan of good old nroff, thus I feel no need to
> > change anything there. The nroff code is well readable, everything
> > obvious and crystal clear. What could there be improved? ;-D
> 
> I've mostly missed out on the (n)roff stuff and wouldn't judge on
> how it compares to mdoc. If you say it's good, i'm good with it too
> :)

Sorry, I wasn't fair by not marking the irony well. True is, that
I like roff a lot and I can deal with it well. The ``crystal clear'',
however, was ironically, because roff is most unreadable.

I've never investigated into mdoc, because, despite it's obscure
syntax, I like plain roff still.

AFAICS, mdoc is just another macro package that tries to do some
things better than the man macros. At a first glance, they appear
to be more semantics and more macros. The Wikipedia writes:

The default format of the man pages is troff, with either
the macro package man (appearance oriented) or mdoc
(semantic oriented).
https://en.wikipedia.org/wiki/Man_page

As mdoc is larger than the man macros and still is restricted to
two-character identifiers, I wonder if it will really help to
improve the readability, because unfamiliar readers will have to
learn much more macros in order to understand the man page
sources. (It will help indexing man pages and such tasks, for
sure, but they don't have to do with mmh.)

How about the availability of mdoc? Could we expect to have it
available on all systems mmh runs on?


Currently I don't see the big improvement we would get from
converting to mdoc. I see no readability or maintainability
improvements of relevance.


meillo

Re: [mmh] pick(1) putting zero at the end

[markus schnalke]

[2017-07-07 12:32] Vasily Kolobkov <polezaivs...@openmailbox.org>
> [2017-07-07 11:58 +0200] markus schnalke <mei...@marmaro.de>
> > Aren't we getting rid of this problem altogether when Philipp
> > finished his pick/scan merge? Well, hmm, no not really as there'll
> > probably always remain the problem of supporting both:
> > 
> > anymhcommand `pick`
> > and:
> > pick | wc -l
> > 
> > I see no better solution, at the moment.
> 
> Another option could be a  flag for most other
> programs, but that's quite some code for such a small issue.

Sure, but that would be a dramatic change in the user interface.
It would require huge benefits afterwards in order to break the
compatiblity that much.


> > > Maybe a note in the pick.1 will do?
> > 
> > The note is already there! :-D
> 
> I can't seem to find it! It mentions -[no]zero, which are unrelated
> to this issue though. Any pointers?

End of the page, under BUGS:

   If pick is used in a back-quoted operation, such as

scan `pick -from jones`

   and  pick selects no messages (e.g., no messages are from
   `jones'), then the shell will still run the outer command
   (e.g.,  scan).  Since no messages were matched, pick pro‐
   duced no output, and the argument given to the outer com‐
   mand  as  a  result of backquoting pick is empty.  In the
   case of mmh programs, the outer command now  acts  as  if
   the  default  `msg' or `msgs' should be used (e.g., `all'
   in the case of scan).  To prevent this unexpected  behav‐
   ior,  if  -list  was given, and if its standard output is
   not a tty, then pick outputs the illegal  message  number
   `0'  when  it  fails.   This  lets the outer command fail
   gracefully as well.


meillo

Re: [mmh] pick(1) putting zero at the end

[markus schnalke]

Hoi,

this message only about the mainling list problem.

[2017-07-07 00:18] Vasily Kolobkov 
>
> p.s. seems like a n string in previous message's subject
> caused mail list software(?) to err.

You are right, this was a problem ... and it is fixed now.

The problem was:

echo "Subject: $subject" >>"$hdrs"

I changed it to:

printf "Subject: %s\n" "$subject" >>"$hdrs"

Well, if echo(1) would just echo, then the world would be fine, but
as today, it does some strange things, printf(1) needs to come to
rescue.

Feel free to give it a test.


Btw: If you don't know McIlroy's Tale of the Echo, in The UNIX
Programming Environment on page 79, then have a look at it!

http://cs2.ist.unomaha.edu/~stanw/172/csci4500/UNIXProgrammingEnvironment.pdf
Although not directly related to the here occured problem, it still
is the main text on the design of echo(1). One page earlier is
a note on the echo's interpretation of escape sequences as well.


meillo

[mmh] Dmitry Bogatov

[markus schnalke]

Hoi,

just read the monthly ``Bits of the Debian Project Leader'' and
there my eyes stuck on these words:

the situation around Dmitry Bogatov

In case you haven't heard about it before, have a look at, for
instance:

https://www.debian.org/News/2017/20170417

https://www.reddit.com/r/debian/comments/64qat1/debian_maintainer_dmitry_bogatov_was_arrested_in/
... or other places on the web

Well, this is already two months old news, but nonetheless ...


meillo

Re: [mmh] Add optional "from" flag to sendfiles

[markus schnalke]

Hoi.

[2017-04-18 19:39] mor0i...@gmail.com
> scrīpsit markus schnalke <mei...@marmaro.de>:
> | [2017-03-16 21:24] mor0i...@gmail.com
> | >
> | > The attached patch adds an optional flag to sendfiles for specifying
> | > the address from which to send files.  I'm sending the patch for
> | > anyone else that may find such functionality useful and not because I
> | > wish it be applied to the code base.
> |
> | Could you explain the use case for this feature? I'm curious.
> | (Mainly because I don't use sendfiles(1) at all myself.)
> 
> I don't often use sendfiles and I haven't used it recently.  But I
> believe that, when I added the optional flag to sendfiles, I needed
> to send documents to myself to be opened on another device, such as
> my cell phone; email was the most convenient means of doing so at the
> time.

https://xkcd.com/949/
;-)


Okay, but that doesn't explain the need to set the from address.

If you would want to set the from address generally (in order to
not use the automatically detected one), then you can use the
Default-From profile entry. (Maybe you used it before this
profile entry was added?)

IMO such a -from flag makes only sense if you want to set the
from address differently for different messages. That's were I
currently fail to see a usecase (but don't want to say that there
is none, thus I'm curious).


meillo

Re: [mmh] scan and pick (was: How "modern" are you willing to go?)

[markus schnalke]

Hoi.

[2016-09-18 13:21] Philipp Takacs 
> [2016-09-09 19:45] Philipp Takacs 
> > It wouldn't be that complicate. We would to replace the
> > 
> > > printf("%s\n", m_name(msgnum));
> > 
> > with a call to scan() and add the ``-format'' switch.
> > 
> > In other words pick supports only the format '=%(msg)' at
> > the moment. If we have a format switch in pick we could replace
> > scan by a symlink to pick.

Philipp convinced me with the pick and scan merge. We should go
for it.


> I have implemented this to have a code example we can discouse
> not only an idea. The patch is attached.

Thanks for the patch to get a better picture of the change! Before
you commit it, there's some cleanup to do, e.g.:

> +void printmsg(FILE *f, struct msgs *mp, int msgnum, char *fmtstr, int width);

Prototypes without argument names.

> + if (strcmp(invo_name,"scan")==0) {

Space after comma missing.


Should we prepare it in a branch? (As there are several parts
to change: pick, scan, format files, manpages, ...)


Am I right, that the combination:
scan `pick -sub foo`
can be replaced by this single call to pick/scan:
pick -form scan.meillo -sub foo
Or maybe even by:
scan -sub foo

Do we then still need the -list switch? It could be replaced
by `pick -form scan.msgnum'.

Anyway, ... I do start to like the pick/scan merge. :-)


meillo

Re: [mmh] Add optional "from" flag to sendfiles

[markus schnalke]

[2017-03-16 21:24] mor0i...@gmail.com
>
> Hello,
> 
> The attached patch adds an optional flag to sendfiles for specifying
> the address from which to send files.  I'm sending the patch for
> anyone else that may find such functionality useful and not because I
> wish it be applied to the code base.

Thanks for sharing!

Could you explain the use case for this feature? I'm curious.
(Mainly because I don't use sendfiles(1) at all myself.)


meillo

Re: [mmh] Changes to ali.man1

[markus schnalke]

[2016-12-13 20:14] Larry Hynes 
>
> Following is a start on the man pages, beginning with ali.man1.

Great, thanks!

> As noted on the nmh-devel list (apologies to those who are subscribed
> to both!) grep tells me that 'switch' is used 193 times whereas
> 'option' is used 51 times, so I have tried to standardise on 'switch'.

(I'm not reading nmh-workers ... which you might mean, because I
know of no nmh-devel mailing list.)

AFAIK, some use ``switch'' if it's only boolean and ``option'' if
you can give a value. I don't know if it was done like that in mh
and I don't really care for this detail. It's okay for me to use
``switch''.

> There seems to be a tendency to have blank lines in mmh man pages.
> Are these to make scanning between Sections easier, or are they
> intended to be line breaks? In either case, I'm not a fan: If they're
> just for layout, they may introduce unintended empty lines, depending
> on the processor, and if they're to be line breaks I think there's
> a better way to handle them.

As an active troff user, I know well, that blank lines are not
really roff-like, but, on the other hand, roff sources quickly
become optically unstructured, thus I like to add blank lines
before sections (.SH), where they don't harm but provide much
optical structure. .PP provides enough optical separation because
there's nothing more on the line, but .SH lines (which are the
higher-level separators) just vanish optically in the surrounding
text. Blank lines compensate this problem. Hence I would like to
keep them, but only before every .SH.

Would that be okay for you?


Apart from that, your patches are fine. I'm gonna apply them.


meillo

Re: [mmh] better thread support

[markus schnalke]

Sorry that I'm quite inactive these days ...


[2016-11-18 18:06] Philipp Takacs 
>
> Simon complained about the missing thread support in mmh,

Right so! :-)

> so I have written a small script to get the hole thread
> associated with a message. What do you think about this
> script?

Seems we have to start somewhere ... it doesn't matter so
much how we start as long as we start, because threading
will not drop from heaven ... In this sense: Well done!


What I think about for a long time is the use of shell
scripts as mmh commands. I see the great leverage we get
from shell scripting, but OTOH I see the inconsistency
that creeps into our so consistent C framework. All C tools
behave just the same. The shell scripts behave differently
and if we want to change that, then we need to build up a
new sbr/ library but now in sh. -- I see the problem but I
don't see the answer.

Some things, however, are clear to me: Shell scripting is
powerful, we should use it in cases like this threading
prototype. Our C code base is wonderful, as well, and we
should try to preserve it's good quality and high consistency.
Maybe we should consider rewriting the shell scripts in C
when they are established (like mhsign(1) and mhpgp(1), for
instance).


> part 2 text/x-shellscript 364 pickthread  
> #!/bin/sh
> 
> getthread()
> {

Have you used jwz's threading algorithm here?
https://www.jwz.org/doc/threading.html
If so, please document it. If not, then why not? ;-)
AFAIK it's the best way to do it.

>   id=`anno -list -component References "$1"`
>   if [ -z "$id" ];then
>   id=`anno -list -component Message-ID "$1"`
>   fi
>   id=`/usr/local/mmh/lib/ap "$id"|sed 1q`

You cannot be sure ap(1) is installed there. To do it right,
you have to add a placeholder at that point and generate
pickthread from pickthread.sh during installation. Some of
the other scripts are handled this way, just have a look at
them..

>   if [ -z "$id" ];then
>   pick $1
>   fi
>   pick --References "$id" -or --Message-ID "$id"
> }

You should keep in mind, that pick(1) is an mmh tool for which
most users have default switches in the profile. Thus, you
should explicitly set -[no]list and -sequence.

This of course is part of a larger problem. Actually we should
set all the default switches to ensure correct behavior. Maybe
we should add some -noprofile (or -nodefaults or -usedefaults)
option to all tools ...

> if [ -z "$@" ];then
>   msgs=c
> else
>   msgs=$@
> fi

Shorter is:  msgs="${@:-c}"

> for msg in `pick "$msgs"`;do

You probably meant  `mhpath "$msgs"`  here?

>   getthread $msg
> done


Actually, what's the (intended) output of your command?


Maybe, the algorithm should be implemented as an option to
sortm(1):  sortm -thread

Maybe we could add some fmt function to scan(1) to draw
such little branches before the subject, if we're on the
same thread as the message before.


However, thanks for providing this script. It is a good
start into a larger topic. First steps are important to
gather experience.

... and btw thanks for the optimistic subject:
*better* thread support
not that there would have been any before. :-D


meillo

Re: [mmh] Patches for man page cleanup.

[markus schnalke]

Hoi.

(Larry is not subscribed to the list. Thus the full quote below.)


First of all, thanks for your interest in mmh and for the wish
to help improving it, Larry. That's great!

Grammar and spelling improvements are definitely welcome.

I understand your feeling about the prose style. However, I'm
not really sure what my opinion is about it. For sure, it is a
bit uncommon, compared to some other manpages and it does not
provide such a quick access to the information, but on the other
hand it provides a lot of background information because of the
wordy style. I would not want to lose that in favor for some
``mechanical'' listing of the options. (There's always some
dealing with history involved in every bit of mmh.) I.e. if you
want to rearrange the layout (as it appears in the patch), then
that'll surely improve the readability and I would like that.
Please just be careful when it involves cutting down history
or other additional information, just to get it all into the
new layout.

I assume that there are a bunch of ``simple'' manpages, like
ali(1), where your new layout works out fine. Beyond that, you
should also have a look at some larger manpages like repl(1)
or spost(8) where the switches description is not the main
content. Check if your layout concept works there too. Of
course, these could benefit from a new layouting and regrouping
of information as well ... likely they would even benefit the
most of it.


I suggest, that you submit one patch with grammar and spelling
fixes over all manpages, because there will be no discussion
to apply it.

Then, as Philipp suggested, one patch per manpage that you
reworked. (To me it would be okay as well, if you send only one
patch over a bunch of simple manpages that you changed in the
same style. But for larger manpages with more rework, please
use one patch per manpage.)


And you should consider subscribing to the mailing list. ;-)


meillo



[2016-11-17 18:28] Philipp Takacs 
> [2016-11-15 18:47] Larry Hynes 
> >
> > I started making (grammar and spelling) corrections to the man
> > pages, and then I remembered that I don't like the (n|m)mh man pages
> > because of their 'prose' style... The options are contained in the
> > paragraphs and not listed as individual switches. So I started to
> > wonder what I would do differently... Following is an (example)
> > patch for ali.man1. (There are still things that I would to change
> > in this, like "...will try to track down the official hostname of
> > the address".)
> 
> In general this is good some comments for the specific patch:
> 
> 1. Don't hardcode versions or other dynamic content.
> 2. The ``-file'' switch description has some display issues
>It looks a bit strage. ``-file  aliasfile'' is not in an extra line
>and at ``aliasfile .'' shouldn't be a space.
> 
> If you change this and change the things you want to change,
> I can add your patch.
> 
> > I can:
> > 
> > 1. submit minimal patches, with just spelling and grammar fixes
> > 2. as 1., plus remove old mmh and MH references
> > 3. as 2., plus 'de-prose' and general clean up
> > 4. as 3, plus work on wording
> > 5. whatever you're having yourself
> 
> The remove of old nmh/MH references should be done carefull,
> because at some points it is intended to talk about MH.
> 
> Just send your patches (one per manpage), the way you think
> they are good and we say what we don't like/want to change.
> 
> It would be nice, if you could also check the correctness of
> the manpages.
> 
> Philipp

Re: [mmh] [PATCH 9/9] Factor out draft file creation from replout()

[markus schnalke]

[2016-09-09 18:58] Dmitry Bogatov 
>
> When I started working on patch, which would extract file creation
> with `m_gmprot', I asked myself more general question -- what is the
> point of `m_gmprot'?
> 
>   $ man mh-profile
>  Msg-Protect:0644
>   An octal number which defines the permission bits for new 
> message
>   files.  See chmod(1) for an explanation of the octal number.
>   (profile, default: 0600)
> 
> In code, it is passed to umask(2) anyway. So, as I understand matter,
> Msg-Protect is just a way to specify mmh-local umask.

Yes.

> If my understanding is correct, we can eliminate it fully. If not, please,
> I need clarification.

As I couldn't come up with a clear oppinion about it, back then,
I left it as it was. Haven't made much fuzz about it, but at the
same time I haven't seen much worth in it, equally.

I always thought that it existed because mail could be regarded
as needing more privacy concern, thus the files should be stronger
protected (the default is 0600, that equals a umask of 077) in
contrast to normal umasks, which are often 02 only.

For the mail storage we can solve the privacy problem by setting
the permissions for ~/Mail appropriately, but mhstore(1) seems to
be the relevant point. If you set Nmh-Storage in the profile to
store all files of to a central directory, which all users use,
then Msg-Protect is relevant.

I would like to have Msg-Protect removed, but I am not sure if we
cut off valuable possibilities at the same time. Actually, the
point is that my feeling is that I don't see far enough in this
topic to be conviced of any decision.

But I would say that we can remove it from everywhere except
from mhstore(1), without problem.


More oppinions or ideas on this?


meillo

Re: [mmh] mhstore(1) uses CD filename now

[markus schnalke]

[2016-09-05 12:08] markus schnalke <mei...@marmaro.de>
>
> Eventually, I fixed the long annoying problem that mhstore(1)
> didn't respect the ``filename'' attribute of Content-Disposition.

>   http://git.marmaro.de/?p=mmh;a=commitdiff;h=6306992c

Just pushed a patch for this commit, because it introduced a
segfault for messages that include MIME parts without CD headers:

http://git.marmaro.de/?p=mmh;a=commitdiff;h=9a990c3302


We should apply RFC2047-decoding to these filenames. Is there
anyone who wants to do that. (It should be mainly, finding the
right place and applying the decode_rfc2047() function to the
string.)

And maybe someone who wants to write a testcase for mhstore(1),
so that we discover such segfaults earlier. ;-)

If no one's interested, I'll do it.


meillo

Re: [mmh] [PATCH 9/9] Factor out draft file creation from replout()

[markus schnalke]

[2016-09-08 15:33] Dmitry Bogatov 
> 
> I would move to sbr/ after I locate another call site.

I suggest that this could be done in one go: try to rework all or
most of the places we do the similar thing.


> About lookup --
> you look at function name and understand that file is created and
> FILE* is returned. Since it is assigned to `out' variable, it is
> created for write. No need to lookup actual code.

To explain my view of the situation:

You assume that abstractions would be perfect. I believe that
in reality abstractions are leaky, hence I don't read over the
function call but do the lookup ... because you never know! I
do want to understand code right down to the bottom. Maybe I
look as being arcane in this respect ... I can assure you, I'm
not. :-)

If there is exactly one call of such a small function, then the
lookup overhead is large. If however the function is used in
many places, then the overhead decreases as I have to do the
lookup only once for all of them.


> And, it is much easier to locate code duplication when you have many
> "several lines" functions, than when you have many code in every
> function.

Of course. I would appreciate to have such a function outfactored.
I do only suggest to first examine the other places that have
similar code and try to convert them in one go. Because only if
you know about the other places, you know if the shape of function
you created from this one places fits for all the others as well.

To sum it up again: I think that this patch contains a worthwhile
idea, but I don't think it is finished already. Take the idea
further and then create a patch that solves the issue more
completely.


meillo

Re: [mmh] [PATCH 9/9] Factor out draft file creation from replout()

[markus schnalke]

Should this be a generic function (included into sbr/)?
There probably are other places where we create files
with reduced permissions. This patch here does not fully
convince me -- it pays the encapsulation of a few lines
with an additional lookup --, but it might be a starting
point for a larger improvement. Then the deal could be
much better.


meillo


[2016-09-08 13:04] kact...@gnu.org
>
> From: Dmitry Bogatov 
> 
> ---
>  uip/repl.c | 22 +-
>  1 file changed, 13 insertions(+), 9 deletions(-)
> 
> diff --git a/uip/repl.c b/uip/repl.c
> index 8a40543..9e39b2a 100644
> --- a/uip/repl.c
> +++ b/uip/repl.c
> @@ -370,8 +370,18 @@ docc(char *cp, int ccflag)
>   }
>  }
>  
> +static FILE*
> +fcreate_with_gmprot(char *filepath)
> +{
> + int mask = umask(~m_gmprot());
> + FILE *out = fopen(filepath, "w");
>  
> -
> + if (!out) {
> + adios(EX_CANTCREAT, filepath, "unable to create");
> + }
> + umask(mask);
> + return out;
> +}
>  
>  static void
>  replout(FILE *inb, char *drft, struct msgs *mp,
> @@ -382,16 +392,10 @@ replout(FILE *inb, char *drft, struct msgs *mp,
>   int i;
>   struct comp *cptr;
>   char **ap;
> - int char_read = 0, format_len, mask;
> + int char_read = 0, format_len;
>   char *scanl;
>   unsigned char *cp;
> - FILE *out;
> -
> - mask = umask(~m_gmprot());
> - if ((out = fopen(drft, "w")) == NULL)
> - adios(EX_CANTCREAT, drft, "unable to create");
> -
> - umask(mask);
> + FILE *out = fcreate_with_gmprot(drft);
>  
>   /* get new format string */
>   cp = new_fs(form, NULL);
> -- 
> I may be not subscribed. Please, keep me in carbon copy.
> 
> 
> 

Re: [mmh] [PATCH 8/9] Refactor `ismymbox()' function

[markus schnalke]

The idea of your changes is good. On the first look, your
code looks good as well. Just one thing: Please use shorter
identifiers.


meillo


[2016-09-08 13:03] kact...@gnu.org
>
> From: Dmitry Bogatov 
> 
> Move initialization of alternate mailnames list into separate
> function.
> ---
>  sbr/addrsbr.c | 151 
> +-
>  1 file changed, 65 insertions(+), 86 deletions(-)
> 
> diff --git a/sbr/addrsbr.c b/sbr/addrsbr.c
> index db5b42e..d1c454b 100644
> --- a/sbr/addrsbr.c
> +++ b/sbr/addrsbr.c
> @@ -229,6 +229,68 @@ adrformat(struct mailname *mp)
>  #define W_HOST  (W_HBEG | W_HEND)
>  #define WBITS   "\020\01MBEG\02MEND\03HBEG\04HEND"
>  
> +static void
> +fill_mailname_type(struct mailname *mp)
> +{
> + char *cp = mp->m_mbox + strlen(mp->m_mbox) - 1; /* last symbol */
> +
> + if (*cp == '*') {
> + mp->m_type |= W_MEND;
> + *cp = '\0';
> + }
> + if (*mp->m_mbox == '*') {
> + mp->m_type |= W_MBEG;
> + mp->m_mbox++;
> + }
> + if (mp->m_host) {
> + char *cp2 = mp->m_host + strlen(mp->m_host) - 1; /* last symbol 
> */
> + if (*cp2 == '*') {
> + mp->m_type |= W_HEND;
> + *cp2 = '\0';
> + }
> + if (*mp->m_host == '*') {
> + mp->m_type |= W_HBEG;
> + mp->m_host++;
> + }
> + }
> +}
> +
> +static struct mailname**
> +append_alternate_mailboxes(struct mailname **mp, char *data)
> +{
> + char *cp;
> + while ((cp = getname(data))) {
> + struct mailname *next = getm(cp, NULL, 0, AD_NAME, NULL);
> + if (!next) {
> + advise(NULL, "invalid address in profile: %s", cp);
> + }
> + *mp = next;
> + (*mp)->m_type = W_NIL;
> + fill_mailname_type(*mp);
> +
> + mp = &(*mp)->m_next;
> + }
> + return mp;
> +}
> +
> +static const struct mailname*
> +get_alternate_mailboxes()
> +{
> + static struct mailname *mq = NULL;
> +
> + if (!mq) {
> + struct mailname **mp = 
> + char *alt;
> +
> + alt = context_find("alternate-mailboxes");
> + mp = append_alternate_mailboxes(mp, alt);
> +
> + alt = context_find("Default-From");
> + mp = append_alternate_mailboxes(mp, alt);
> + }
> + return mq;
> +}
> +
>  /*
>  ** Check if this is my address
>  */
> @@ -236,93 +298,11 @@ adrformat(struct mailname *mp)
>  int
>  ismymbox(struct mailname *np)
>  {
> - int oops;
>   int len, i;
>   char *cp;
>   char *pp;
>   char buffer[BUFSIZ];
> - struct mailname *mp;
> - static char *am = NULL;
> - static struct mailname mq;
> -
> - /*
> - ** If this is the first call, initialize
> - ** list of alternate mailboxes.
> - */
> - if (am == NULL) {
> - mq.m_next = NULL;
> - mq.m_mbox = getusername();
> - mp = 
> - if ((am = context_find("alternate-mailboxes")) == NULL) {
> - am = getusername();
> - } else {
> - oops = 0;
> - while ((cp = getname(am))) {
> - if ((mp->m_next = getm(cp, NULL, 0, AD_NAME, 
> NULL)) == NULL) {
> - admonish(NULL, "illegal address: %s", 
> cp);
> - oops++;
> - } else {
> - mp = mp->m_next;
> - mp->m_type = W_NIL;
> - if (*mp->m_mbox == '*') {
> - mp->m_type |= W_MBEG;
> - mp->m_mbox++;
> - }
> - if (*(cp = mp->m_mbox + 
> strlen(mp->m_mbox) - 1) == '*') {
> - mp->m_type |= W_MEND;
> - *cp = '\0';
> - }
> - if (mp->m_host) {
> - if (*mp->m_host == '*') {
> - mp->m_type |= W_HBEG;
> - mp->m_host++;
> - }
> - if (*(cp = mp->m_host + 
> strlen(mp->m_host) - 1) == '*') {
> - mp->m_type |= W_HEND;
> - *cp = '\0';
> - }
> - }
> - if ((cp = getenv("MHWDEBUG")) && *cp) {
> -

Re: [mmh] [PATCH 6/9] Remove unused function `cpydgst'

[markus schnalke]

I know that this code is not used. I kept it available in case
it would be useful again. It wasn't because of the copy code
but because of the dash-stuffing code. I just wasn't sure if
we would need that again. So much for the background.

... but well, not that I read the comment about the sed
equivalent ... seems as if we can apply your patch then. If
we need dash-stuffing again, we'd better use sed for it.


meillo


[2016-09-08 13:03] kact...@gnu.org
>
> From: Dmitry Bogatov 
> 
> ---
>  sbr/Makefile.in |  3 +--
>  sbr/cpydgst.c   | 68 
> -
>  2 files changed, 1 insertion(+), 70 deletions(-)
>  delete mode 100644 sbr/cpydgst.c
> 
> diff --git a/sbr/Makefile.in b/sbr/Makefile.in
> index f60bd9a..e264454 100644
> --- a/sbr/Makefile.in
> +++ b/sbr/Makefile.in
> @@ -49,7 +49,7 @@ SRCS = addrsbr.c ambigsw.c brkstring.c  \
>   charset.c concat.c context_del.c  \
>   context_find.c context_read.c  \
>   context_replace.c context_save.c \
> - cpydata.c cpydgst.c crawl_folders.c  \
> + cpydata.c crawl_folders.c  \
>   dtime.c dtimep.c  \
>   error.c execprog.c ext_hook.c folder_addmsg.c folder_delmsgs.c  \
>   folder_free.c folder_read.c  \
> @@ -128,4 +128,3 @@ subdir = sbr
>  
>  Makefile: Makefile.in ../config.status
>   cd .. && ./config.status $(subdir)/$@
> -
> diff --git a/sbr/cpydgst.c b/sbr/cpydgst.c
> deleted file mode 100644
> index 5e5ef87..000
> --- a/sbr/cpydgst.c
> +++ /dev/null
> @@ -1,68 +0,0 @@
> -/*
> -** cpydgst.c -- copy from one fd to another in encapsulating mode
> -**   -- (do dashstuffing of input data).
> -**
> -** This code is Copyright (c) 2002, by the authors of nmh.  See the
> -** COPYRIGHT file in the root directory of the nmh distribution for
> -** complete copyright information.
> -*/
> -
> -#include 
> -#include 
> -#include 
> -
> -/*
> -** We want to perform the substitution
> -**
> -** \n(-.*)\n-->\n- \1\n
> -**
> -** This is equivalent to the sed substitution
> -**
> -** sed -e 's%^-%- -%' < ifile > ofile
> -**
> -**  but the routine below is faster than the pipe, fork, and exec.
> -*/
> -
> -#define S1 0
> -#define S2 1
> -
> -#define output(c)  if (bp >= dp) {flush(); *bp++ = c;} else *bp++ = c
> -#define flush()  if ((j = bp - outbuf) && write(out, outbuf, j) != j) \
> - adios(EX_IOERR, ofile, "error writing"); \
> - else \
> - bp = outbuf
> -
> -
> -void
> -cpydgst(int in, int out, char *ifile, char *ofile)
> -{
> - int i, j, state;
> - char *cp, *ep;
> - char *bp, *dp;
> - char buffer[BUFSIZ], outbuf[BUFSIZ];
> -
> - dp = (bp = outbuf) + sizeof outbuf;
> - for (state = S1; (i = read(in, buffer, sizeof buffer)) > 0;)
> - for (ep = (cp = buffer) + i; cp < ep; cp++) {
> - if (*cp == '\0')
> - continue;
> - switch (state) {
> - case S1:
> - if (*cp == '-') {
> - output('-');
> - output(' ');
> - }
> - state = S2;  /* fall */
> -
> - case S2:
> - output(*cp);
> - if (*cp == '\n')
> - state = S1;
> - break;
> - }
> - }
> -
> - if (i == -1)
> - adios(EX_IOERR, ifile, "error reading");
> - flush();
> -}
> -- 
> I may be not subscribed. Please, keep me in carbon copy.
> 
> 
> 

Re: [mmh] [PATCH 4/9] Remove unused function `pref_encoding()'

[markus schnalke]

This seems to be part of the code move from nmh. Philipp took the
QP code but not (yet) the base64 code for RFC 2047 encoding of
header fields. That's the reason why we have this outcommented code.

Actually, I think that we don't need base64 encoding of header
fields. The only reason for it is that it usually leads to shorter
headers if you encode mostly non-ASCII values. I don't really care
if the encoded header is 50, 100, or 400 chars long. As long as we
don't drop into problems here, I'd like to save the additional
code and stay with the simple solution of only supporting one RFC
2047 encoding. Thus apply the proposed patch.

Philipp?


btw: Patch #6 should be part of this patch.


meillo


[2016-09-08 13:03] kact...@gnu.org
>
> From: Dmitry Bogatov 
> 
> ---
>  sbr/encode_rfc2047.c | 36 
>  1 file changed, 36 deletions(-)
> 
> diff --git a/sbr/encode_rfc2047.c b/sbr/encode_rfc2047.c
> index 3c4a6b0..00310b3 100644
> --- a/sbr/encode_rfc2047.c
> +++ b/sbr/encode_rfc2047.c
> @@ -58,7 +58,6 @@ static int field_encode_quoted(const char *, char **, const 
> char *, int,
>   int, int);
>  static int scanstring(const char *, int *, int *, int *);
>  static int utf8len(const char *);
> -/*static int pref_encoding(int, int, int);*/
>  
>  /*
>  ** Encode a message header using RFC 2047 encoding.  We make the assumption
> @@ -633,38 +632,3 @@ scanstring(const char *string, int *asciilen, int 
> *eightbitchars,
>  
>   return *eightbitchars > 0;
>  }
> -
> -#if 0
> -
> -/*
> -** This function is to be used to decide which encoding algorithm we should
> -** use if one is not given.  Basically, we pick whichever one is the shorter
> -** of the two.
> -**
> -** Arguments are:
> -**
> -** ascii - Number of ASCII characters in to-be-encoded string.
> -** specials  - Number of ASCII characters in to-be-encoded string that
> -**still require encoding under quoted-printable.  Note that
> -**these are included in the "ascii" total.
> -** eightbit  - Eight-bit characters in the to-be-encoded string.
> -**
> -** Returns one of CE_BASE64 or CE_QUOTED.
> -**/
> -static int
> -pref_encoding(int ascii, int specials, int eightbits)
> -{
> - /*
> - ** The length of the q-p encoding is:
> - **
> - ** ascii - specials + (specials + eightbits) * 3.
> - **
> - ** The length of the base64 encoding is:
> - **
> - ** base64len(ascii + eightbits) (See macro for details)
> - */
> - return base64len(ascii + eightbits) < (ascii - specials +
> - (specials + eightbits) * 3) ? CE_BASE64 : CE_QUOTED;
> -}
> -
> -#endif
> -- 
> I may be not subscribed. Please, keep me in carbon copy.
> 
> 
> 

Re: [mmh] [PATCH 2/9] Remove unused function unset_unseen

[markus schnalke]

This function is not used, correct. AFAIR, Philipp added it for
symmetry to set_unseen(). I have no strong oppinion whether to
keep it or to remove it. Philipp, what do you think?


meillo


[2016-09-08 13:03] kact...@gnu.org
>
> From: Dmitry Bogatov 
> 
> ---
>  sbr/seq_msgstats.c | 7 ---
>  1 file changed, 7 deletions(-)
> 
> diff --git a/sbr/seq_msgstats.c b/sbr/seq_msgstats.c
> index 50ae7c6..7ffbde1 100644
> --- a/sbr/seq_msgstats.c
> +++ b/sbr/seq_msgstats.c
> @@ -142,13 +142,6 @@ unset_selected(struct msgs *mp, int msgnum)
>   }
>  }
>  
> -void
> -unset_unseen(struct msgs *mp, int msgnum)
> -{
> - assert_msg_range(mp, msgnum);
> - mp->msgstats[msgnum - mp->lowoff] &= ~SELECT_UNSEEN;
> -}
> -
>  
>  /*
>  **  private/public sequences
> -- 
> I may be not subscribed. Please, keep me in carbon copy.
> 
> 
> 

Re: [mmh] git clone 403

[markus schnalke]

[2016-09-08 13:02] Dmitry Bogatov 
>
> It works on machine without Tor. Do you use cloudflare or like-this
> annoying stuff?

No, nothing of that kind.

I have some (few) IP addresses blocked because of annoying attacks.
But with Tor you change your outgoing IP address frequenctly, IIRC.
Hence this should not be a problem.

Hmm ...


meillo

Re: [mmh] git clone 403

[markus schnalke]

[2016-09-08 11:57] Dmitry Bogatov <kact...@gnu.org>
> [2016-09-08 10:38] markus schnalke <mei...@marmaro.de>
> > [2016-09-08 11:13] Dmitry Bogatov <kact...@gnu.org>
> > >
> > >   Fetching origin
> > >   error: The requested URL returned error: 403 Forbidden (curl_result = 
> > > 22, http_code = 403, sha1 = adb0e89adb8f2a34e692f9
> > >   8b05760b03d94e833d)
> > >   error: Unable to find adb0e89adb8f2a34e692f98b05760b03d94e833d under 
> > > http://git.marmaro.de/mmh
> > >   Cannot obtain needed blob adb0e89adb8f2a34e692f98b05760b03d94e833d
> > >   while processing commit 94ac6bbeccca6dcf41f4e84d3a71b437929f98df.
> > >   error: fetch failed.
> > >   error: Could not fetch origin
> > >
> > > Hello! Today I received this error on `git fetch --all'. Any ideas?
> >
> > The offending commit is not part of the history:
> 
> No, problem is not in git. Problem is that git.marmaro.de returns me
> 403 (Access denied). What is your web server 403 policy? I do use Tor,
> just in case it matter.

I don't have anything special setup. I can't believe that Tor matters
(at least not within my domain of control).

Web access to the specific blob works:

http://git.marmaro.de/?p=mmh;a=blob_plain;f=man/Makefile.in;h=adb0e89adb8f2a34e692f98b05760b03d94e833d;hb=94ac6bbeccca6dcf41f4e84d3a71b437929f98df


Was it the first time you got this problem? Did it work earlier?
Have you tried from another machine?

Can you tell me when exactly it did happen?

The accesses to adb0e89adb8f2a34e692f98b05760b03d94e833d in the webserver
logs of the last two days look good.

The only 403 status code within the last five days is this one:

180.76.15.150 git.marmaro.de - [07/Sep/2016:08:27:03 +0200]
"GET 
/?p=mmh;a=search;h=900d7ecb383b331d5fc17bc12c3463a2157cdcd9;s=;st=author 
HTTP/1.1"
403 2360 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; 
+http://www.baidu.com/search/spider.html)"

... which is a different hash.

I don't see a problem on my side, up to now.

Can you reproduce the failure?


meillo

Re: [mmh] git clone 403

[markus schnalke]

[2016-09-08 11:13] Dmitry Bogatov 
>
>   Fetching origin
>   error: The requested URL returned error: 403 Forbidden (curl_result = 
> 22, http_code = 403, sha1 = adb0e89adb8f2a34e692f9
>   8b05760b03d94e833d)
>   error: Unable to find adb0e89adb8f2a34e692f98b05760b03d94e833d under 
> http://git.marmaro.de/mmh
>   Cannot obtain needed blob adb0e89adb8f2a34e692f98b05760b03d94e833d
>   while processing commit 94ac6bbeccca6dcf41f4e84d3a71b437929f98df.
>   error: fetch failed.
>   error: Could not fetch origin
> 
> Hello! Today I received this error on `git fetch --all'. Any ideas?

The offending commit is not part of the history:

git log | grep adb0e89adb8f2a34e692f98b05760b03d94e833d | wc -l
0

But in my local repo I get something displayed when I invoke:

git show adb0e89adb8f2a34e692f98b05760b03d94e833d

It's not a changeset that I see but some version of man/Makefile.in.


Unfortunately my git knowledge is quite limited. Maybe someone
else can debug it.


meillo

Re: [mmh] [PATCH] Fix unreproducible build

[markus schnalke]

[2016-09-07 10:17] Philipp Takacs 
>
> Thanks for your feedback a updated patch is attached, this contains most
> of your suggestion.

I've tested it now. Some further comments below ...


> > > +VERSION= `head -n 1 $(srcdir)/VERSION`
> > 
> > Better portable and even less to type:
> > 
> > VERSION= `sed q $(srcdir)/VERSION`

Btw: For the background: The reason why there was tail(1) since
7th Edition but not head(1), is that head(1) could easily be
replaced by  sed 10q  (or  sed 11q  for faster typing). Hence
there just was no need to have head(1) ... until people found
sed(1) strange (apart from s///) and wanted something easier.

See also: http://harmful.cat-v.org/software/


Now about your new patch:

> diff --git a/version.sh b/version.sh
> new file mode 100755
> index 000..3d7764b
> --- /dev/null
> +++ b/version.sh
> @@ -0,0 +1,23 @@
> +#!/bin/sh
> +#
> +# version.sh -- script to create version string(s) for mmh.
> +#
> +# You need to pass the script the top source directory.
> +#
> +
> +if [ -z "$1" ] || [ ! -d "$1" ]; then
> +echo "usage: version.sh topdir" 1>&2
> +exit 1

The indent here are spaces instead of tabs.

> +fi
> +
> +cd $1

Instead of the above code, I suggest you use:

if [ -d "$1" ]; then
cd "$1" # <-- quotes!
fi

Then we can invoke version.sh with or without argument. (For
testing I usually call `./version.sh' ... it would be tedious
to always supply a dot.) Then you should update ./Makefiles.in
and remove the dot there.

> +
> +VERSION="`sed q VERSION`"

Use lowercase for the shell variable.

> +
> +git_info=""
> +
> +if [ -d "$1/.git" ]; then

There is a bug! Use:
if [ -d .git ]; then
instead, as we already are in the right directory.

> + git_info="+`git log -n 1 --pretty'=format:%h' HEAD`"

This always adds git hashes if we have .git, even if we are
at a release version. My suggestion is:

current=`git log -n 1 --pretty=format:%h HEAD`
release=`git log -n 1 --pretty=format:%h "mmh-$version"`
if [ "$current" != "$release" ]; then
git_info="+$current"
fi

> +fi
> +
> +echo mmh-"$VERSION""$git_info"


One further thing we have to update as well is configure.ac.
It appears to be enough to replace:
AC_INIT(mmh, m4_normalize(m4_include([VERSION])))
with:
AC_INIT(mmh, m4_normalize(`./version.sh`))
(Don't know if this is autoconf style but it works.)


meillo


P.S.
About the `+' in the version number suffix: I used the `+'
instead of a `-' because I wanted to express that e.g. mmh-0.3+dev
is mmh-0.3 plus further development and not be confused with
mmh-0.3-rc1 (which is ``not yet 0.3''). Now as we have commit
hashes, I don't see this ambiguity anymore, we could use `-'
again to append the git hash if you would prefer that. I don't
care.

[mmh] Date parsing (can be fun)

[markus schnalke]

Hoi,

as my attention was pointed to dtimep.lex I had a look at it,
read through it, and worked on it.

First of all: It is better readable than I thought. I haven't
done much with lex yet, but the code was easy to adjust.

Adjusting was what I did. First I replaced optimized code that
had bad readability with simpler code of better readability.
dd2e898a13855acd7034a2ee73584995d7452f4f
dc08e69006fdf9edb458dc23885dd669f3b4a176

Then I added support for basic ISO 8601/RFC 3339 dates (at the
same time dropping support for XX-XX-YY dates).
9c9972821e53c2e5ac94431e3c9be3aeb63ef26c

Seems as if these days I tackle a lot of those annoying
problems I suffer from since many months. :-)

It was quite some fun working in the lex code ... but still
I am curious for the re2c version Dmitry wants to provide.


meillo

Re: [mmh] build in diffrent directory don't work

[markus schnalke]

[2016-09-07 09:51] Philipp Takacs 
> 
> Since the commit 94ac6bb build in a diffrent directory don't work.
> This means the test suite can't be used. I have a patch attached
> and will push this today.

Good catch! Your patch looks good. Thanks.


> @Markus
> For what is the gettitle() funktion in man/gettitles.sh?

Oops, remove it.


meillo

[mmh] mhstore(1) uses CD filename now

[markus schnalke]

Hoi.

Eventually, I fixed the long annoying problem that mhstore(1)
didn't respect the ``filename'' attribute of Content-Disposition.


Many attachments have such MIME headers:

Content-Type: application/pdf; name="foo.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="foo.pdf"

They are no problem, because mhstore(1) used the ``name''
attribute of Content-Type.

But there are also attachments with MIME headers like that one:

Content-Type: application/pdf
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="foo.pdf"

In this case, mhstore(1) used to store the file as ``20.2.pdf''
or the like, as it didn't make use of the ``filename''
attribute.

I change this today:
http://git.marmaro.de/?p=mmh;a=commitdiff;h=6306992c

(I admit that it is not the most elegant solution I can think
of. It seems as if Content-Disposition wasn't part of the
original MIME implementation but was added later on, thus the
clumsiness.)


Left to do is making mhlist(1) display this information as
well.


meillo

Re: [mmh] m_convert refactor

[markus schnalke]

[2016-09-04 09:53] Dmitry Bogatov 
>
> > The big vision is to merge the MIME part specification with the
> > message specification. Currently we can either deal with messages
> > or with MIME parts. Those worlds are strictly separated. The
> > vision is to be able to invoke ``show 15.3'' for example.
> > (Currently we have to use ``show -part 3 15''.)
> 
> What would mean `15.3-20.1'?  As for single message, 15.3 looks fine,
> but I am afraid it will compilcate things.

Message parts make only sense for single messages, I think, but
that would be part of the grammer of the parser. Your example
would be rejected in the same way ``l-f'' gets rejected. Currently
we have two different concepts for naming messages and for naming
parts. I would like to have one single concept.


> > Currently, mmh has hardly any dependencies. Lex is no
> > build-dependency for users of releases, it is only a dependency
> > for development. Autoconf is available everywhere (and anyway
> > there's the plan to move away from autoconf). Hence one can build
> > mmh on almost every system without resolving build-dependencies.
> > This is good, I think. But re2c could be used in the same way as
> > lex, I think, just that it is not as widespread and known.
> 
> My personal rule of thumb -- if it is in Debian stable, it is find to
> rely on.

I think that mmh now and in the future is mainly used by people
who care for the Unix philosophy and who like simple software and
simple systems. Some of them will surely use Debian, but many will
rather use smaller Distros or non-GNU/Linux Unix systems.
Furthermore, I think that many users will compile mmh manually.

Crux (which I use on one machine) has no port of re2c:
https://crux.nu/portdb/?a=search=re2c
Debian is not necessarily the reference for the userbase of mmh.


> > Btw: As I understood, re2c generates code that can parse some
> > regexps. The generated parser is then static, i.e. it recognizes
> > exactly the compiled in regexps but no others. What about using
> > a dynamic regexp library? (regex(3) is part of POSIX.) Would it
> > be possible to do the same that way? I don't know ... am just
> > thinking aload to understand the situation.
> 
> Why? Sure, we can use rexec(3) or libpcre (not much of dependency,
> even grep depends on it). But if we can make it static, it would be
> faster and more reliable.

The reason I suggested this is because regex(3) is part of POSIX,
and thus just available. I seldom care about something being faster
and if we use dynamic matching of static regexps it would be
equally reliable. But I am not at all sure that the problem is
solvable this way.

Most likely only GNU grep depends on libpcre, because a POSIX
implementation of grep does not support PCRE. I would love to only
depend on a POSIX system and ANSI C. (lex is specified by POSIX.
Autoconf is not.)

(btw: rexec(3) appears to be something else. ;-) )


meillo

Re: [mmh] maildir

[markus schnalke]

[2016-09-03 13:30] Dmitry Bogatov 
>
> > For import and export, search for a good mbox to Maildir
> > conversion tool. (btw: Does anyone know one?)
> 
>   #!/bin/sh
>   for i in new/* cur/* ; do
>   formail < ”$i” >> /path/to/resulting/mbox
>   done
> 
> formail(1) is part of procmail suite, which is part of many (all?)
> distributions.

Thanks.

If the task is only to import the maildir into mmh, we can omit
the mbox format and import directly:

for i in new/* cur/* ; do
rcvstore +folder <"$i"
done


For the other direction (mbox to maildir), the program mb2md
appears to be a good choice.

To move messages directly from mmh to maildir, something like
this could be used:

for i in `mhpath +folder a` ; do
(cp "$i" "maildir/new/`date +%s`.$$.`hostname`")
done

It is not really how it should be done, but it might work this
way. (For details see: http://www.qmail.org/man/man5/maildir.html)


meillo

Re: [mmh] maildir

[markus schnalke]

[2016-09-03 03:48] Link Satonaka 
>
> I must confess I've only just discovered mh/nmh/mmh, but I'm very interested 
> in
> the concept and very interested in this project that intends to modernize it.

Welcome to mmh!


> I briefly played with nmh before discovering mmh, and I noticed that nmh
> supports maildir. Unfortunately, if mmh supports maildir, I cannot  figure out
> how to get inc to recognize it, I can only get it to read mbox. As it turns
> out, mbox is surprisingly hard to get working with other modern tools.
> 
> If maildir is supported, how do I pull from it? If not, will it be?

In contrast to nmh, mmh does not support maildir. This is for
simplicity reasons. If you want to import a maildir, convert it
into mbox and import it with inc(1), or import it file by file
with rcvstore(1).

The storage of mmh is always in MH format (AFAIK this is the same
in nmh). Maildir support is only an importing and exporting
question.

For mmh, there are no plans to support maildir in any way (because
of the complexity it would add), but you can have a maildir
storage indirectly if you sync your MH storage via IMAP
synchronization with an IMAP server that stores into maildir.

For import and export, search for a good mbox to Maildir
conversion tool. (btw: Does anyone know one?)


meillo


P.S.
I believe that maildir support is not relevant for mmh, but I
more and more think that good support for some IMAP sync is of
relevance, because it enables interfacing the outside mail
processing world. (It is hardly relevant for myself, but it is
for the project.) Ongoing efforts in this direction are
appreciated.

Re: [mmh] [PATCH] Fix unreproducible build

[markus schnalke]

[2016-09-02 14:16] Dmitry Bogatov 
> > > +git_info="$(git show --pretty'=format: [%h -- %cD]' HEAD | head -n1)"
> >
> > I don't agree with this line, because it requires git to be
> > available for compilation. If we want to have the commit hash
> > available, then we should store it into a file on release, as
> > it is done for VERSION and DATE.
> 
> Well, since script is not -e, it results empty git_info, if git is not
> availiable. It is perfectly fine for me, as potential Debian
> maintainer.

Actually, it results in:

sh: git: not found

But that would be solveable.

My thought was more general: Do we need this information? We
have to deal with several problems (more code; out-of-sync
information; with or without a commit hash dependent on git
being available at compilation time) for having this
information. Hence, I wonder what the information is worth.
Maybe it is relevant, then what exactly is the relevance?
Maybe we can extract that piece of information better. Or if
there hardly is relevance, then we better drop the information
and its problems.

For packaged versions of mmh all this is not relevant, AFAIS.
It is only relevant for manual compilation and installation.

The time of installation can be found out with:
ls -l /usr/local/mmh/bin/show

The time of compilation (usually the same) can be found out
with:
ls -l .../src/mmh/uip/show.o

The hostname is hardly relevant anymore today, I think. In
most cases it is obvious where it was built.

The commit hash could be useful. To be exact, I'd rather want
it to be part of the normal version number, such as: 0.3-be582c4,
instead of 0.3+dev. (Releases should be without such a suffix.)
Then there should be no difference between version_num and
version_str. If we have such a more detailed version number,
then we must ensure that the correct one is used in each build
result (i.e. each program) not an outdated one in some build
source (i.e. the library).


However, for me this build information has little relevance.
Hence it would be fine for me to drop it until we have a good
solution for including commit hashes in the normal version
numbers (for all versions that are no releases). But btw: What
about locally modified and not committed files? ... Maybe it's
not worth all the complexity ...


Further opinions or experiences from other projects?


meillo

Re: [mmh] msg-hook config option

[markus schnalke]

[2016-08-31 17:46] Philipp Takacs 
> 
> I have removed the ``msg-hook'' config option, because it's unnessarry and
> it's not a good idea to pass a user provided string in a format string. If
> someone realy needs this option, we may reimplement this, but in a better way.

I agree. The default message is fully sufficient.


meillo

Re: [mmh] [PATCH] Fix unreproducible build

[markus schnalke]

[2016-08-30 02:35] kact...@gnu.org
> From: Dmitry Bogatov 
> 
> Hostname and date of compilation are no longer embeded in version
> string. See https://reproducible-builds.org

This problem also appeared in the nmh package in Debian. See:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=835985


> Signed-off-by: Dmitry Bogatov 
> ---
>  config/version.sh | 30 ++
>  1 file changed, 2 insertions(+), 28 deletions(-)
> 
> diff --git a/config/version.sh b/config/version.sh
> index 5a9628c..6396306 100755
> --- a/config/version.sh
> +++ b/config/version.sh
> @@ -11,34 +11,8 @@ if [ -z "$1" ]; then
>  fi
>  
>  VERSION=$1
> -OFS="$IFS"
> -IFS=:
> -HOSTNAME=unknown
>  
> -# Find out the name of the host we are compiling on
> -for prog in uname hostname
> -do
> - for dir in $PATH
> - do
> - if [ ! -f $dir/$prog ]; then
> - continue
> - fi
> - case $prog in
> - uname)
> - HOSTNAME=`$prog -n`
> - ;;
> - hostname)
> - HOSTNAME=`$prog`
> - ;;
> - esac
> - break
> - done
> - if [ X"$HOSTNAME" != X  -a  X"$HOSTNAME" != Xunknown ]; then
> - break
> - fi
> -done
> +git_info="$(git show --pretty'=format: [%h -- %cD]' HEAD | head -n1)"

I don't agree with this line, because it requires git to be
available for compilation. If we want to have the commit hash
available, then we should store it into a file on release, as
it is done for VERSION and DATE.

But all in all, I tend to move away from storing such information
in the binaries. We also have the problem of this information
being compiled into libmh.a but not into the tools. Hence when
changing uip/$foo.c and recompiling, the tool still contains the
old information, because libmh.a was not recompiled.

Maybe we should just drop all that. Or let's tackle it from the
other side: Why do we need this information?


> -IFS=" "
> -
> -echo "char *version_str = \"mmh-$VERSION [compiled on $HOSTNAME at 
> `date`]\";"
> +echo "char *version_str = \"mmh-${VERSION}${git_info}\";"
>  echo "char *version_num = \"mmh-$VERSION\";"


meillo

Re: [mmh] mmh-0.3 issues with 'message error format'

[markus schnalke]

[2016-08-24 10:22] Dmitry Bogatov <kact...@ruggedinbox.com>
> 
> Hello!
> 
> All this time I used mmh-0.2, which worked flawless (module repl
> quotation). I decided to upgrade to 0.3 (and upload into Debian),
> but encountered issues with displaying any of my email.
> 
> Typescript and example message attached (in typescript note 'message
> error format in component #1', which is *before* actual message
> content)

The problem is the From_ line:

> From kaction  Wed Aug 24 09:33:27 2016
> Return-path: <debb...@buxtehude.debian.org>
> Envelope-to: kact...@gnu.org
...


See the attached message and the discussion around and before it for
explanations.

You have to remove these lines. They are not RFC 822 conforming. Up
to now they were only tolerated.


Looks as if we forgot to add the conversion script to the distribution
and I also forgot to mention it in the release notes/announcement.
Sorry.


meillo
--- Begin Message ---
[2016-05-10 16:34] markus schnalke <mei...@marmaro.de>
> 
> Yet, I have a simpler solution (sorry for being so late with it!):
> 
>   #!/bin/sh
>   #
>   # remove From_ lines from MH messages in-place
> 
>   for i do
>   ed - "$i" <<\!
>   1,/^$/g/^From /d
>   w
>   !
>   done

This code isn't good enough. It cannot handle header-only messages,
nor does it check for read-only files. This is a better version:

for i do
if [ ! -w "$i" ] ; then
echo "$i: not writeable"
continue
fi
if grep -q '^$' "$i" ; then
range='1,/^$/'
else
range='1,$'
fi
ed - "$i" <--- End Message ---

[mmh] mmh-0.3 is released! (meillo's mail handler)

[markus schnalke]

Hello everyone,

it is my pleasure to announce the release of mmh-0.3:

http://marmaro.de/prog/mmh/files/mmh-0.3.tar.gz
(md5sum: 880017112da72d7d67856c13a252d436)
http://marmaro.de/prog/mmh/files/mmh-0.3.tar.gz.asc
http://marmaro.de/prog/mmh/files/mmh-0.3.tar.gz.asc2

The most important improvements:

- m_getfld() is replaced by m_getfld2() (i.e. We killed the beast!)
- mmh became portable over stdio implementations, e.g. musl libc
- repl now decodes MIME messages in the quotated text
- send invokes mhbuild automatically on every message
- send became able to send from folders other than +drafts
- inc learned to read from stdin
- mhsign encryption works with aliases
- whatnow2 is included as a probable future replacement for whatnow

See the NEWS file in the tarball for an extended summary of the changes.
Alternatively online: http://git.marmaro.de/?p=mmh;a=blob;f=NEWS;t=mmh-0.3
For the full changelog have a look at the ChangeLog file in the
distribution, use `git log', or view: http://git.marmaro.de/?p=mmh;a=log

You can clone the code with: git clone http://git.marmaro.de/mmh

Mmh's website is: http://marmaro.de/prog/mmh/


An overview on mmh's code base in numbers:

In total, mmh-0.3 has 32,645 SLOC, compared to 32,382 SLOC of mmh-0.2
and 28,757 SLOC in mmh-0.1. Regarding C code only, it's 25,930 (-430)
compared to 26,360 and 25,342. The subdirectories in detail:

SLOCChangeDirectory SLOC-by-Language (Sorted)
19059   (+100)uip ansic=18223,sh=836
6959(-250)sbr ansic=6885,awk=74
3017(+400)testsh=3017
2732  top_dir sh=2732
762   h   ansic=762
92config  ansic=60,sh=32
24man sh=24
0 docs(none)
0 etc (none)

The largest growth again is in the testsuite. The main code in C
shrunk (which is a good sign). The growth in uip comes from
including both, whatnow in C (420 SLOC) and whatnow2 in sh (270 SLOC).
(All numbers calculated by sloccount.)


Thanks to everyone who contributed to this release. Again, it was
Philipp Takacs, who did most of the work. The commits:

 41 Author: Philipp Takacs <phil...@bureaucracy.de>
 39 Author: markus schnalke <mei...@marmaro.de>
 12 Author: m...@arascio.xyz <m...@arascio.xyz>
  2 Author: Vasily Kolobkov <polezaivs...@openmailbox.org>
  2 Author: Dmitry Bogatov <kact...@gnu.org>
  1 Author: Jan Düpmeier <j.duepme...@gmail.com>

Further contributions on the mailing list by: Marcin Cieslak,
Joshua Haase, j...@ssh.bio, Alba Pompeo, and Sören Tempel.


We welcome comments, bug reports, and suggestions. Please send them
to the mailing list at <mmh@marmaro.de>.


meillo


pgpP4ftOI3D0h.pgp
Description: PGP signature

[mmh] Release takes a few more days

[markus schnalke]

Hoi.

I want to inform you that (because of other life aspects) couldn't
manage to release mmh-0.3 yet. The release will happen soon.


meillo

Re: [mmh] Building with musl libc

[markus schnalke]

[2016-08-11 14:41] Sören Tempel <soe...@soeren-tempel.net>
> On 01.08.16, markus schnalke wrote:
> > [2016-07-29 00:26] markus schnalke <mei...@marmaro.de>
> > >
> > > @all: Has anyone verified that the latest version runs fine with
> > > musl libc?
> 
> Yes, it builds fine on Alpine Linux with gcc 6.1 and musl 1.1.15. Head
> was fa3da68dfa43341a8b6722d2e8a787c14cef02b9 when I tested it. The
> configure and make output is attached.

Great! Thanks a lot for confirming that.


Could you please run the test suite as well:

cd test
./runalltests

and post the output to the list.


meillo

Re: [mmh] Testcases

[markus schnalke]

[2016-08-10 09:39] markus schnalke <mei...@marmaro.de>
>
> If I run the whole testsuite with LC_ALL=C (as is the case for
> the nightly test run (but should be changed to UTF-8 I think))

I now changed the nightly test runs to use LC_ALL=en_US.UTF-8.

Then I started one run manually. The current results are already
created with the UTF-8 locale.


meillo

Re: [mmh] Testcases

[markus schnalke]

[2016-08-10 09:39] markus schnalke <mei...@marmaro.de>
>
> Great, you did a start with tests/send/test-mimeify! I'll add some
> more cases.

Done.

http://git.marmaro.de/?p=mmh;a=commitdiff;h=fa3da68dfa43341a8b6722d2e8a787c14cef02b9


meillo

[mmh] Testcases

[markus schnalke]

Hoi.

Philipp, thanks for your work in the test suite. The result is
much better now.

Great, you did a start with tests/send/test-mimeify! I'll add some
more cases.


Besides the ali test, this is the one failing (as Philipp already
mentioned):

tests/repl/test-decode-addr: cat `mhpath +drafts l` failed
--- -   2016-08-10 09:12:35.858863138 +0200
+++ /tmp/tmp.RxsUilrI1m 2016-08-10 09:12:35.856917739 +0200
@@ -1,13 +1,13 @@
-To: J?rgen 
-Cc: b...@example.com, K?the 
+To: J?rgen
+Cc: b...@example.com, K?the
 Fcc: +sent
 Subject: Re: repl addr decode
 In-reply-to: <1as2n8-1q9...@deseo.home.schnalke.org>
 References: <1as2n8-1q9...@deseo.home.schnalke.org>
-Comments: In-reply-to J?rgen 
+Comments: In-reply-to J?rgen
message dated "Mon, 18 Apr 2016 08:36:14 +0200."
 
-[2016-04-18 08:36] J?rgen 
+[2016-04-18 08:36] J?rgen
 >
 > part   text/plain   4
 > foo
Test tests/repl/test-decode-addr
FAIL

Why are the email addresses not output? It might be related to
using
%(void(width))%(void(decode))%(putaddr To: )
in the replcomps (and similar in replgroupcomps), introduced by
09f632a7c9e4eea0

What fails is the last test in the testcase, i.e. the one that
is run with LC_ALL=C.

I agree with Philipp's commit message:
This should create the same output as with utf-8.
But the Umlaut should be displayed as '?'. This isn't
the case at the moment.

We have to dig deeper to understand why the email address
misses.



If I run the whole testsuite with LC_ALL=C (as is the case for
the nightly test run (but should be changed to UTF-8 I think))
this other failure appears:

tests/mhbuild/test-linebreak: sed "/^Content-ID:/s/:.*/: /" 
"$draft" failed
--- -   2016-08-10 09:12:29.060968585 +0200
+++ /tmp/tmp.s3rYrn73OO 2016-08-10 09:12:29.058794252 +0200
@@ -3,7 +3,7 @@
 Date: Thursday, 28 Sep 2006 00:02:00
 Subject: mhbuild line breaking with quoted-printable
 MIME-Version: 1.0
-Content-Type: text/plain; charset="UTF-8"
+Content-Type: text/plain; charset="US-ASCII"
 Content-ID: 
 Content-Transfer-Encoding: quoted-printable
 
Test tests/mhbuild/test-linebreak   
FAIL

The draft contains non-ASCII characters, thus US-ASCII is not
correct, but in the C locale one is just not able to handle the
characters in the draft correctly, AFAIK.

What do you think that the right solution is?

Is the test only valid for UTF-8 locales? And thus add

require_locale en_US.utf-8 en_US.utf8
LC_ALL=en_US.UTF-8
export LC_ALL

to those testcases?


meillo

Re: [mmh] Release plan for mmh-0.3

[markus schnalke]

Hoi,

we're in the freeze now. We had a lot of good work recently! :-)

[2016-08-06 17:38] Philipp Takacs 
>
> as far as I see everything for the release is done.

This of course applies to the features only. There's still a lot
to do. Freeze does not mean lazytime. ;-) Freeze means, testing
(writing testcases!), fixing bugs, checking and updating the
manpages, etc. Some tasks in this area SHOULD be done; others
(like writing additional testcases) are not urgent and MAY be
done ... but only if you add a few more during each freeze,
you'll end up in a good test coverage. All of this less pleasant
work is important!

Thus let's start with the second part of our release sprint!


The release is to be expected on Sunday or Monday.


meillo

Re: [mmh] Always MIMEify?

[markus schnalke]

Hoi,

the commit is pushed.


@Philipp: I've used your code nearly as you proposed it. Just the
field matching condition had to be improved:
- strlen() instead of sizeof() -- surely an accidental mistake
- check that the character after the field name is a colon,
  otherwise the check would have matched for
MIME-Version-Test: xxx
  as well.


I did update the man page, but I haven't added testcases. We
should have such test cases, but I'm currently short of free time.
I'll write them during the freeze (if not someone else provides
them earlier).


meillo

Re: [mmh] Release plan for mmh-0.3

[markus schnalke]

[2016-08-01 07:13] markus schnalke <mei...@marmaro.de>
> 
> Okay, so let's freeze next Sunday.

$ date
Freeze minus 3 days
$

;-)


Work already done:

- Allow inc(1) to read from stdin
- Allow send(1) to send from other than draftfolder
- Whatnow2 as a prototype/preview


Work still to do:

- Always MIMEify patch: I'm working on it. (The If-has-MIME-Version
  handling is not yet implemented.)

- broken mhsign(1): Philipp is working on it

- musl libc: Can someone help with testing the compatibility?
  Maybe there's a LiveCD that bases on musl, which we could use
  to compile and run the testsuite.

- There are some tests that fail. See:
http://marmaro.de/prog/mmh/nightly/tests-log/runalltests.log
  * tests/ali/test-ali  that's okay, we need to port code from nmh
  * tests/bad-input/test-header adjust the testcase or m_getfld2()?
  * tests/mhparam/test-mhparam  ?
  * tests/mhsign/test-mhsign(see above)
  In the release, only the ali(1) test should still fail.


Of course, bugfixing can still be done in the freeze ... but why
wait?


meillo

Re: [mmh] Building with musl libc (was: Release plan for mmh-0.3)

[markus schnalke]

[2016-07-29 00:26] markus schnalke <mei...@marmaro.de>
>
> @all: Has anyone verified that the latest version runs fine with
> musl libc?

I tried it myself today. On Debian oldstable, I downloaded musl-1.1.15,
extracted it, then configure, make, make install.

Then in the mmh sources directory, I tried to configure mmh:

$ CC=/usr/local/musl/bin/musl-gcc ./configure -q
configuring for mmh-0.2+dev
configuring for mmh dated 2015-11-02
configure: error: in `/home/meillo/src/mmh':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

$ CC=/usr/local/musl/bin/musl-gcc LDFLAGS=-static ./configure -q
configuring for mmh-0.2+dev
configuring for mmh dated 2015-11-02
Could not find tgetent() in any library.
Is there a ncurses-devel package that you can install?

libncurses5-dev is installed. (configure runs find with gcc.)

Is our configure script broken? Or what am I missing? (I have no
experience with musl libc up to now.)


meillo

Re: [mmh] Release plan for mmh-0.3

[markus schnalke]

[2016-07-31 23:53] Philipp Takacs <phil...@bureaucracy.de>
> [2016-07-29 00:26] markus schnalke <mei...@marmaro.de>
> >
> > More concrete: Start to freeze this Sunday or start to freeze in
> > a week? What do you think?
> 
> Because it's already Sunday I would say in a week.

Okay, so let's freeze next Sunday.

> > Stuff currently in the pipe (in decreasing priority for me):
> > 
> > - Allow inc(1) to read from stdin (recent topic)
> > - Allow send(1) to send from other than draftfolder (not yet announced)
> > - Always MIMEify patch (message of today)
> > - Whatnow2 as a prototype/preview (message of today)
> 
> I would say, we get all this features in.

Agreed.


And defer the rest for after the release:

> > - Abort if mkstemp() fails (message of 2015-09-20 14:24:11)
> 
> Oh, this again. Has anyone looked at it and tested?

Seems to be not ... :-(

> > - Booleans (message of 2016-04-21)
> 
> I send a mail on this tomorrow (monday)
> 
> > - Solving the part line question (message of 2016-03-06 00:04)
> 
> I think this is a big issue, but I don't know how to fix this without
> big changes. So I would say we discous this a bit more.
> 
> > - Error conditions in m_getfld2 (messages of 2016-05)
> 
> AFAIK this is already fixed.
> 
> > - From_ quoting and un-quoting (message of 2015-11-12 23:25:55)
> 
> This is also an bigger change, I'll write a mail to this tomorrow.
> 
> > @Philipp: Is your scan problem of the message you sent at
> > 2015-09-19 14:43:56 solved in the meantime?
> 
> This was fixed by not using the m_getfld-patches from nmh


meillo

Re: [mmh] new whatnowproc

[markus schnalke]

Hoi Philipp,

thanks for:

commit 501f523802eeaed49dc095c9577a9e5f20447a6d
Author: Philipp Takacs 
Date:   Sat Feb 27 19:58:42 2016 +0100

add my whatnow replacement whatnow2

I just discovered this typo:

> draftfolder=`mhparam Draft-Folder`
> if [ -z "$dratffolder" ]
> then
> draftfolder="+drafts"
> fi

Should be ``draftfolder'' instead of ``dratffolder'' in the
is-empty test.


Further more, if you want the compile-time defaults of e.g. the
draft folder, you can use:

mhparam draftfolder

(See ``mhparam -debug'' for the full list.)

Someone might change these compile-time defaults ... it's not
that it would be common and at least the test suite would fail,
because one would have to use `mhparam seq-first` instead of
`f' in the tests ... but anyway.

Unfortunately, the profile entry is ``Draft-Folder'' and the
compile-time value is ``draftfolder''. I'm not sure if they
should match. All those values in the debugging output were
the result of ``Let's print the full list of those compile-time
values from h/mh.h i.e. config/config.c and use the variable
names''. I'm not really satisfied with the result ist was just
better than the situation before.

Okay, debugging output is not really to be used in scripts
... but maybe still better than hardcoding the value.

It's more that I wanted to tell you this, in case you didn't
know already. ;-)


meillo

[mmh] Allow send(1) to send draft from folder other than +drafts

[markus schnalke]

Hoi,

attached is a patch to be able to use send(1) to send a draft
that is located in a folder other than +drafts. Currently this
could only be achieved by:

send `mhpath +folder 17`

... using the uncodumented behavior, that send treats it as a
file if an argument start with a slash.


Objections to applying this patch?


meillo
From c872da5fb2337a573ccba720b33c7269b4160c48 Mon Sep 17 00:00:00 2001
From: markus schnalke <mei...@marmaro.de>
Date: Fri, 29 Jul 2016 11:18:36 +0200
Subject: [PATCH] Allow send(1) to send from folder other than +drafts

This allows having multiple draft folders or queueing drafts
for automatic sending.
---
 man/send.man1 |   15 +--
 uip/send.c|   21 ++---
 2 files changed, 23 insertions(+), 13 deletions(-)

diff --git a/man/send.man1 b/man/send.man1
index b4a9fcc..02e64fe 100644
--- a/man/send.man1
+++ b/man/send.man1
@@ -8,8 +8,9 @@ send \- send a message
 .HP 5
 .na
 .B send
-.RB [ \-verbose " | " \-noverbose ]
+.RI [ +folder ]
 .RI [ msgs ]
+.RB [ \-verbose " | " \-noverbose ]
 .RB [ \-Version ]
 .RB [ \-help ]
 .ad
@@ -70,13 +71,14 @@ will request verbose information of the transport system.
 .PP
 .B Send
 with no
+.I +folder
+and
 .I msgs
-argument will send the current message in the draft folder.
+arguments will send the current message in the draft folder.
 .B Send
-always takes messages from the draft folder.
-(But, a
+sends messages from the draft folder, unless
 .I +folder
-argument might be added in the future.)
+is given.
 Consult the
 .BR mh-draft (7)
 man page for more information.
@@ -167,7 +169,8 @@ comp(1), dist(1), forw(1), repl(1), mh\-alias(5), spost(8)
 
 .SH DEFAULTS
 .nf
-.RB ` msgs "' defaults to the current message in the draft folder"
+.RB ` msgs "' defaults to the current message"
+.RB ` +folder "' defaults to the draft folder"
 .RB ` \-noverbose '
 .fi
 
diff --git a/uip/send.c b/uip/send.c
index 619ff62..c1665b2 100644
--- a/uip/send.c
+++ b/uip/send.c
@@ -81,7 +81,7 @@ main(int argc, char **argv)
 	int msgnum, status;
 	int in, out;
 	int n;
-	char *cp, *maildir = NULL;
+	char *cp, *maildir = NULL, *folder = NULL;
 	char buf[BUFSIZ], **argp, **arguments;
 	char *msgs[MAXARGS], *vec[MAXARGS];
 	char *files[MAXARGS];
@@ -125,6 +125,12 @@ main(int argc, char **argv)
 vec[vecp++] = --cp;
 continue;
 			}
+		} else if (*cp == '+' || *cp == '@') {
+			if (folder) {
+adios(EX_USAGE, NULL, "only one folder at a time!");
+			} else {
+folder = mh_xstrdup(expandfol(cp));
+			}
 		} else {
 			if (*cp == '/') {
 files[nfiles++] = cp;
@@ -139,17 +145,18 @@ main(int argc, char **argv)
 	}
 
 	if (nmsgs) {
-		maildir = toabsdir(draftfolder);
+		folder = folder ? folder : draftfolder;
+		maildir = toabsdir(folder);
 		if (chdir(maildir) == NOTOK) {
 			adios(EX_OSERR, maildir, "unable to change directory to");
 		}
-		if (!(mp = folder_read(draftfolder))) {
-			adios(EX_IOERR, NULL, "unable to read draft folder %s",
-	draftfolder);
+		if (!(mp = folder_read(folder))) {
+			adios(EX_IOERR, NULL, "unable to read folder %s",
+	folder);
 		}
 		if (mp->nummsg == 0) {
-			adios(EX_DATAERR, NULL, "no messages in draft folder %s",
-	draftfolder);
+			adios(EX_DATAERR, NULL, "no messages in folder %s",
+	folder);
 		}
 		/* parse all the msgranges/sequences and set SELECTED */
 		for (msgnum = 0; msgnum < nmsgs; msgnum++) {
-- 
1.7.10

Re: [mmh] new whatnowproc

[markus schnalke]

[2016-02-27 16:40] Philipp Takacs 
>
> I have written a small whatnowshell replacement. It's a little shell
> wrapper for all the features of the whatnowshell. The improvement is,
> that you use it directly from the shell, so you can use all your shell
> features.

As the topic popped up again, I now took a closer look at your
script (although not tested it practically, yet).


> Some meta-info (i.e. last-editor) are stored in a metafile so this is
> available over different sessions.

Just some thoughts about that ... mainly to structure my own
mind:

- Mmh stores the configuration on a mmh-profile level. A user
  can have one or more profiles.

- Mmh stores public sequences (typically the current message)
  on a mail folder level. This is shared among all users of
  this mail folder.

- Mmh stores private sequences and the current folder on the
  mmh-context level. A user can have one or more contexts.

- Whatnow (the old one) stores its state (mail draft, last
  editor, ...) on a Unix process level. Each draft editing
  session is separate. Only whatnow behaves this way.

How should your new whatnow behave?

Your new whatnow stores its state in a metafile next to the
draft. No other mmh tool uses metafiles ... but storing this
information connected to the draft appears to be the right
thing in my eyes. It allows editing multiple drafts at the
same time with multiple processes, one after each other, per
draft.

The next question is: Which one of the drafts is the one we
are currently working on? You use the current message in the
draft folder. This is connected to the folder, which is not
what I would want, because I couldn't work on multiple drafts
in parallel. I think the ``current'' draft should be
connected to the context or to the shell session (env var).
This would allow editing multiple drafts in parallel. Using
the current message does not allow concurrent editing of
drafts but only alternated editing of drafts, by switching
the current message each time. Seems as if I would want to
have concurrent editing ...


> Some features that I want to implement in the next time:

> - attach msgs: this is actually a new feature. I think it's useful

It is no new feature, we already have it. Just use forw(1) and
have a look at the draft!

You do it this way:

Attach: +folder 123 124

... or do you only mean your attach() function below? Maybe
you could use mhpath(1) to check if the parameters form
valid messages in mmh notation. Either they do or they all
have to be valid files in the filesystem. Otherwise it's an
error.



> part 1.2   text/x-shellscript4185 mywhatnow

I think you should rename it to whatnow2 and add it to the
VCS ... maybe installing it into lib/ (like mhtest) until
we settled on your version and it has a man page.


> #!/bin/sh
> 
> printhelp()
> {
>   echo "usage"

:-)

> }
> 
> usage()
> {
>   if [ $1 -eq 0 ]

It might be worthwhile to quote some more of your variable
expansions ...

>   then
>   printhelp
>   exit 0
>   fi
>   printhelp 1>&2
>   exit $1
> }
> 
> config()
> {
>   [ -n "$MMHP" ] || return

Means: ``nonempty $MMHP? otherwise return''. For better
understandability (i.e. two negations less), invert it:
[ -z "$MMHP" ] && return
``empty $MMHP? then return''.

>   MMHP=$HOME/.mmh/profile
> }
> 
> get_editor()
> {
>   if [ -f "$mhmetafile" ]
>   then
>   lasteditor=`anno -list -component 'last-editor' $mhmetafile`
>   if [ -n "$lasteditor" ]
>   then
>   mheditor=`anno -list -component "$lasteditor-next" 
> $MMHP`

Much easier:
mhparam "$lasteditor-next"
:-)

>   [ -n "$mheditor" ] && return
>   fi
>   fi
>   if [ -n "$MMHEDITOR" ]
>   then
>   mheditor=$MMHEDITOR
>   return
>   fi
>   mheditor=`anno -list -component 'Editor' $MMHP`
>   if [ -n "$mheditor" ]
>   then
>   return
>   fi
>   if [ -n "$VISUAL" ]
>   then
>   mheditor=$VISUAL
>   return
>   fi
>   if [ -n "$EDITOR" ]
>   then
>   mheditor=$EDITOR
>   return
>   fi
>   mheditor=vi
> }
> 
> get_showproc()
> {
>   mhshowproc=`anno -list -component 'listproc' $MMHP`
>   if [ -n "mhshowproc" ]
>   then
>   return
>   fi
>   mhshowproc=show

In MH terms the showproc is called ``listproc'' (because you
list a message to display it).

You can replace this whole function with:
mhparam listproc
It will use the value in the profile, or, if there is none,
then the default value.

> }
> 
> get_realpath()
> {
>   reldir=`dirname $1`
>   filename=`basename $1`
>   cd $reldir
>   echo "$PWD/$filename"
>   cd -
> }
> 
> get_attachmentheader()
> {
>   header=`anno -list -component 'Attachment-Header' $MMHP`
>   if [ -z 

Re: [mmh] Release plan for mmh-0.3

[markus schnalke]

[2016-04-21 10:33] markus schnalke <mei...@marmaro.de>
> 
> we are coming closer to the next release.

*hehe*


> The largest work task left to do is merging the m_getfld2-meillo
> branch into master.

This is done.

> The other one missing is the quote decoding in repl, everyone
> is wishing so much. We're right before pushing that. :-)

This is done.

> There might be some more smaller improvements that we put in as
> we go, but shortly after the m_getfld() replacement is done,
> we'll start the freeze.

This actually is where we're currently at ... unfortunately
since months ... although the merge was made the very next day.

> P.S.
> It took 3 years after mmh-0.1 to get 0.2 done. From there to the
> upcoming 0.3 it'll be about 6 months. If we extrapolate this,
> 0.4 can be expected to be finished by the end of May. ;-)

Oh, well, you know ... :-/

But there's new activity now! We already did some more stuff
and there's even more in the pipe.

Should be just put a label on what we have currently or should
we include what we are working on currently?

More concrete: Start to freeze this Sunday or start to freeze in
a week? What do you think?

Then we'll have a one-week freeze (only updating documentation,
writing tests, fixing bugs) before the release. After that, we
can go for new interesting stuff (like the whatnow question,
for instance).


Stuff currently in the pipe (in decreasing priority for me):

- Allow inc(1) to read from stdin (recent topic)
- Allow send(1) to send from other than draftfolder (not yet announced)
- Always MIMEify patch (message of today)
- Whatnow2 as a prototype/preview (message of today)
- Abort if mkstemp() fails (message of 2015-09-20 14:24:11)

- Booleans (message of 2016-04-21)
- Solving the part line question (message of 2016-03-06 00:04)
- Error conditions in m_getfld2 (messages of 2016-05)
- From_ quoting and un-quoting (message of 2015-11-12 23:25:55)
- ... some more long-term ideas

@Philipp: Is your scan problem of the message you sent at
2015-09-19 14:43:56 solved in the meantime?

@all: Has anyone verified that the latest version runs fine with
musl libc?

@all: Has anyone explanations for the symlink patch in the 
message of 2016-05-23 20:13?


meillo

[mmh] Always MIMEify?

[markus schnalke]

Hoi,

attached is a patch that simplifies the code by not checking if
we need to MIMEify the draft, but just do it always.

In the case of a message, which would not have been MIMEified
with the old code (because it is compatible to the old email
format), the difference is that these additional headers are
inserted:

MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <18983.146972256...@deseo.home.schnalke.org>

I see no reason why this should be problematic. (At least the
nmh guys didn't wanted to create MIME messages if possible to
avoid ... but if the MIMEification differs just by these three
header lines, I can hardly imagine why one would want to prevent
that.)

The patch is more of a first step for discussion, because I
think we should have an additional check for a MIME-Version
header. Currently, if such a header is present, mhbuild(1)
(which does the MIMEification) would refuse to work. Now that
we invoke mhbuild(1) on each draft, we are no longer able to
pass a draft around it. Hence I think that if a MIME-Version
header is present, we assume to have an already MIMEified
draft, thus the mhbuild(1) invocation should be omited.

Does this sound sensible to you?

If so, I would implement it.


meillo
diff --git a/uip/send.c b/uip/send.c
index 619ff62..4eb8ddf 100644
--- a/uip/send.c
+++ b/uip/send.c
@@ -320,27 +320,11 @@ sendsbr(char **vec, int vecp, char *drft, struct stat *st)
 }
 
 static int
-contains_non_ascii(char *str)
-{
-	unsigned char *cp;
-
-	for (cp = str; *cp; cp++) {
-		if (*cp > 127) {
-			return 1;
-		}
-	}
-	return 0;
-}
-
-static int
 attach(char *draft_file_name)
 {
 	char buf[MAXPATHLEN + 6];
 	int c;
-	int has_attachment;
-	int has_body = 0;
-	int non_ascii = 0; /* msg body or hdr contains non-ASCII chars */
-	int length;  /* length of attachment header field name */
+	int length = strlen(attach_hdr);
 	char *p;
 
 	if (!(draft_file = fopen(draft_file_name, "r"))) {
@@ -351,47 +335,7 @@ attach(char *draft_file_name)
 	field = mh_xcalloc(field_size = 256, sizeof(char));
 
 	/*
-	** Scan the draft file for an attachment header field name.
-	*/
-	length = strlen(attach_hdr);
-	has_attachment = 0;
-	while (get_line() != EOF && *field != '\0' && *field != '-') {
-		if (strncasecmp(field, attach_hdr, length)==0 &&
-field[length] == ':') {
-			has_attachment = 1;
-		}
-		if (contains_non_ascii(field)) {
-			non_ascii = 1;
-		}
-	}
-
-	/*
-	** Look for at least one non-blank line in the body of the
-	** message which indicates content in the body.
-	** Check if body contains at least one non-blank (= not empty)
-	** and if it contains any non-ASCII chars (= need MIME).
-	*/
-	while (get_line() != EOF) {
-		if (contains_non_ascii(field)) {
-			non_ascii = 1;
-		}
-		for (p = field; *p; p++) {
-			if (isgraph(*p)) {
-has_body = 1;
-			}
-		}
-		if (has_body && non_ascii) {
-			break;  /* that's been already enough information */
-		}
-	}
-
-	if (!has_attachment && non_ascii==0) {
-		/* We don't need to convert it to MIME. */
-		return DONE;
-	}
-
-	/*
-	** Else: mimify
+	** MIMEify
 	*/
 
 	/* Make names for the temporary files.  */
@@ -402,12 +346,10 @@ attach(char *draft_file_name)
 			m_mktemp(toabsdir(invo_name), NULL, NULL),
 			sizeof (composition_file_name));
 
-	if (has_body) {
-		body_file = fopen(body_file_name, "w");
-	}
+	body_file = fopen(body_file_name, "w");
 	composition_file = fopen(composition_file_name, "w");
 
-	if ((has_body && !body_file) || !composition_file) {
+	if (!body_file || !composition_file) {
 		clean_up_temporary_files();
 		adios(EX_IOERR, NULL, "unable to open all of the temporary files.");
 	}
@@ -422,17 +364,15 @@ attach(char *draft_file_name)
 	}
 	fputs("\n", composition_file);
 
-	if (has_body) {
-		/* Copy the message body to the temporary file. */
-		while ((c = getc(draft_file)) != EOF) {
-			putc(c, body_file);
-		}
-		fclose(body_file);
-
-		/* Add a mhbuild MIME composition file line for the body */
-		/* charset will be discovered/guessed by mhbuild */
-		fprintf(composition_file, "#text/plain %s\n", body_file_name);
+	/* Copy the message body to the temporary file. */
+	while ((c = getc(draft_file)) != EOF) {
+		putc(c, body_file);
 	}
+	fclose(body_file);
+
+	/* Add a mhbuild MIME composition file line for the body */
+	/* charset will be discovered/guessed by mhbuild */
+	fprintf(composition_file, "#text/plain %s\n", body_file_name);
 
 	/*
 	** Now, go back to the beginning of the draft file and look for
@@ -674,7 +614,6 @@ sendaux(char **vec, int vecp, char *drft, struct stat *st)
 		}
 	}
 
-
 	return status ? NOTOK : status;
 }
 

Re: [mmh] Allow inc(1) to read from stdin

[markus schnalke]

[2016-07-27 21:40] m...@arascio.xyz
> scrīpsit markus schnalke <mei...@marmaro.de>:
> | [2016-07-27 11:25] m...@arascio.xyz
> | > 
> | > + if (fstat(0, ) == NOTOK) {
> | > + adios(EX_IOERR, "fstat", "unable to read from stdin");
> | > + } else if (!(S_ISREG(s1.st_mode) || S_ISFIFO(s1.st_mode))) {
> | > + adios(EX_USAGE, NULL, "missing input from stdin");
> |
> | Why do you do these two checks? To control the error message?
> 
> Exactly.  But your subsequent comment (below) has led me to consider
> the second check unnecessary.  Do you consider the first check also
> unnecessary?

What conditions could lead to a failure of the first check? The
manpage of fstat(2) says:

   EACCES Search  permission  is  denied for one of the directories in the
  path prefix of path.  (See also path_resolution(7).)

   EBADF  fd is bad.

   EFAULT Bad address.

   ELOOP  Too many symbolic links encountered while traversing the path.

   ENAMETOOLONG
  path is too long.

   ENOENT A component of path does not exist, or path is an empty string.

   ENOMEM Out of memory (i.e., kernel memory).

   ENOTDIR
  A component of the path prefix of path is not a directory.

   EOVERFLOW
  path or fd refers to a file whose size, inode number, or  number
  of  blocks  cannot  be  represented  in, respectively, the types
  off_t, ino_t, or blkcnt_t.  This error can occur when, for exam‐
  ple,  an  application  compiled  on  a  32-bit platform without
  -D_FILE_OFFSET_BITS=64 calls stat() on a file whose size exceeds
  (1<<31)-1 bytes.

Is one of them relevant and could not be detected by actually
reading from the fd? I'd say no, hence, drop the pre-checks and
just try to read.

Anyway, it is uncommon, that one would not be able to read
from stdin, hence a pre-check like in the maildrop-empty
case (which is common) is unnecessary.

But, you know, I haven't borrowed wisdom. So, if someone has a
different opinion or just a thought to consider, please speak
up!


> I've remove the second of the two checks from the patch.

Good, because it would have prevented us from reading a character
device. With your first patch:

inc  -file -  | > @@ -480,7 +489,7 @@ giveup:;
> | > else
> | > admonish(newmail, "error zero'ing");
> | > }
> | > -   } else if (noisy) {
> | > +   } else if (noisy && newmail) {
> | > printf("%s not zero'd\n", newmail);
> | > }
> |
> | ... although it might confuse users I would have liked the humor
> | in: ``- not zero'd'' or ``stdin not zero'd''. ;-)
> 
> :)

I'm still undecided if my comment was just for fun, or if would
like to have this message printed. To be fair, the version with
the dash is confusing and setting `newmail' to "stdin" might
have other disadvantages ... Well, maybe it really was just for
the humorous aspect ... :-)


> I've attached to this message also a second patch that updates inc(1)
> to mention that inc can read stdin.

Thanks for that, too. When we've decided about the fstat(2)
check and you provide a third patch, include the testcase in
the patch as well.


meillo

Re: [mmh] Confused while getting here

[markus schnalke]

[2016-07-27 22:07] Vasily Kolobkov 
> 
> My first try of usual subject but with empty body, sent on 21st got
> bounced on 26th with following report:
> 
> 8<
> Reporting-MTA: dns; mail.openmailbox.org
> X-Postfix-Queue-ID: E05ED2066F6
> X-Postfix-Sender: rfc822; polezaivs...@openmailbox.org
> Arrival-Date: Thu, 21 Jul 2016 17:51:43 +0200 (CEST)
> 
> Final-Recipient: rfc822; mmh-requ...@marmaro.de
> Action: failed
> Status: 4.0.0
> Remote-MTA: dns; mx1.marmaro.de
> Diagnostic-Code: smtp; 451 Queuing declined or disabled; try again later
> >8
> 
> I'm not versed in smtp parlor and am not sure as to whos mta choked
> up on the message.

Seems as if it was mine. From the qpsmtpd logs:

Thu Jul 21 17:51:45 2016 quagga[18965]: 250 , 
sender OK - how exciting to get mail from you!
Thu Jul 21 17:51:45 2016 quagga[18965]: dispatching RCPT 
TO:
Thu Jul 21 17:51:45 2016 quagga[18965]: 250 , recipient 
ok
Thu Jul 21 17:51:45 2016 quagga[18965]: dispatching DATA
Thu Jul 21 17:51:45 2016 quagga[18965]: 354 go ahead
Thu Jul 21 17:51:45 2016 quagga[18965]: FATAL PLUGIN ERROR [spamassassin]:  
Can't call method "as_string" on an undefined value at 
/usr/share/qpsmtpd/plugins/spamassassin line 163.

Thu Jul 21 17:51:45 2016 quagga[18965]: FATAL PLUGIN ERROR [spamassassin]:  
Can't call method "get" on an undefined value at 
/usr/share/qpsmtpd/plugins/spamassassin line 257.

Thu Jul 21 17:51:45 2016 quagga[18965]: FATAL PLUGIN ERROR [spamassassin]:  
Can't call method "get" on an undefined value at 
/usr/share/qpsmtpd/plugins/spamassassin line 257.

Thu Jul 21 17:51:45 2016 quagga[18965]: forwarding to localhost:25
Thu Jul 21 17:51:45 2016 quagga[18965]: FATAL PLUGIN ERROR 
[queue::smtp_2dforward]:  Can't call method "as_string" on an undefined value 
at /usr/share/qpsmtpd/plugins/queue/smtp-forward line 60.

Thu Jul 21 17:51:45 2016 quagga[18965]: 451 Queuing declined or disabled; try 
again later
Thu Jul 21 17:51:45 2016 quagga[18965]: dispatching QUIT
Thu Jul 21 17:51:45 2016 quagga[18965]: 221 quagga closing connection. Have a 
wonderful day.


Seems to be an qpsmtpd problem. I wrote to the their mailing list for help.


meillo

Re: [mmh] Allow inc(1) to read from stdin

[markus schnalke]

[2016-07-27 11:25] m...@arascio.xyz
> scrīpsit markus schnalke <mei...@marmaro.de>:
> 
> | it should be possible to do:  packf | inc
>  
> | Patches (including a test case ;-) ) are welcome.
> 
> The attached patch makes inc able to read stdin and named pipes; so
> both of the following become possible.
> 
>   f=/tmp/mbox; packf >"$f" && inc -file - <"$f"
>   packf |inc -file -
> 
> I've also attached a very basic test script.

Great!

It works.


> One problem that I've encountered with my patch's changes occurs when
> stdin is empty; for example, in the following.
> 
>   printf '' |inc -file -
> 
> Currently, inc calls adios if the file read for new mail is empty.
> 
>   if (stat(newmail, ) == NOTOK || s1.st_size == 0)
>   adios(EX_DATAERR, NULL, "no mail to incorporate");

IMO this is a special-case for the system maildrop, where this
condition is common. It shouldn't be used or relevant in all
other cases.

The usual way to go is: Read the file until EOF and incorporate
each message you find while doing this. If you read some messages
-- fine. If you got EOF before you read a message (i.e. empty
file) -- fine as well. No special handling needed. Checking for
the empty file is just a performance hack for a really common
condition in the maildrop case ... likely to make up to 90% of
all inc(1) calls in some setups.


Some comments about the patch:

> diff --git a/uip/inc.c b/uip/inc.c

> + if (from && !strcmp(from, "-")) {/* We'll get mail from stdin. */

Please use
strcmp(s, t)==0
to check for equality, not the bang. The reason is, that the
return value of strcmp(3) is not logically boolean.


> + if (fstat(0, ) == NOTOK) {
> + adios(EX_IOERR, "fstat", "unable to read from stdin");
> + } else if (!(S_ISREG(s1.st_mode) || S_ISFIFO(s1.st_mode))) {
> + adios(EX_USAGE, NULL, "missing input from stdin");

Why do you do these two checks? To control the error message?
Why not just try to open the file (not caring what kind of file
it is) and if it succeeds, then everything is fine, and if it
fails, we know the reason then. Likewise, as written above: Just
try to read from it, not caring if it is empty or not.


> @@ -480,7 +489,7 @@ giveup:;
> else
> admonish(newmail, "error zero'ing");
> }
> -   } else if (noisy) {
> +   } else if (noisy && newmail) {
> printf("%s not zero'd\n", newmail);
> }

... although it might confuse users I would have liked the humor
in: ``- not zero'd'' or ``stdin not zero'd''. ;-)


meillo

Re: [mmh] Test with dot on a line on its own

[markus schnalke]

[2016-07-27 21:10] markus schnalke <mei...@marmaro.de>
> 
> I'm testing the mailing list with this file that includes a dot
> on a line on its own.

Well, the solution was easy: Invoke sendmail(8) with -i. ;-)

Sorry for the inconvenience and noise.


meillo

[mmh] Test with dot on a line on its own

[markus schnalke]

Hey,

I'm testing the mailing list with this file that includes a dot
on a line on its own.


meillo
#!/bin/sh
##
# Test incorporating mail from stdin with `inc -file -`. #
##
. "$MH_TEST_COMMON"

# Test empty stdin.
runandcheck "printf '' |inc -file -" <<.
Incorporating new mail into inbox...

inc: no messages incorporated, continuing...
.

# Test FIFO.
runandcheck "packf +inbox |inc -file -" <<.
Incorporating new mail into inbox...

  11+ 2006-09-29 00:00  Test1  Testing message 1
  12  2006-09-29 00:00  Test2  Testing message 2
  13  2006-09-29 00:00  Test3  Testing message 3
  14  2006-09-29 00:00  Test4  Testing message 4
  15  2006-09-29 00:00  Test5  Testing message 5
  16  2006-09-29 00:00  Test6  Testing message 6
  17  2006-09-29 00:00  Test7  Testing message 7
  18  2006-09-29 00:00  Test8  Testing message 8
  19  2006-09-29 00:00  Test9  Testing message 9
  20  2006-09-29 00:00  Test10 Testing message 10
.

# Test redirected stdin.
cd "$MMH_TEST_DIR"
f=mbox
runandcheck "packf +inbox 1-10 >$f && inc -file - <$f" <<.
Incorporating new mail into inbox...

  21+ 2006-09-29 00:00  Test1  Testing message 1
  22  2006-09-29 00:00  Test2  Testing message 2
  23  2006-09-29 00:00  Test3  Testing message 3
  24  2006-09-29 00:00  Test4  Testing message 4
  25  2006-09-29 00:00  Test5  Testing message 5
  26  2006-09-29 00:00  Test6  Testing message 6
  27  2006-09-29 00:00  Test7  Testing message 7
  28  2006-09-29 00:00  Test8  Testing message 8
  29  2006-09-29 00:00  Test9  Testing message 9
  30  2006-09-29 00:00  Test10 Testing message 10
.
rm "$f"
unset f

Re: [mmh] Allow inc(1) to read from stdin

[markus schnalke]

[2016-07-27 11:50] m...@arascio.xyz
> scrīpsit m...@arascio.xyz:
> | scrīpsit m...@arascio.xyz:
> | | scrīpsit markus schnalke <mei...@marmaro.de>:
> | | 
> | | | it should be possible to do:  packf | inc
> | |  
> | | | Patches (including a test case ;-) ) are welcome.
> | | 
> | | I've also attached a very basic test script.
> | 
> | The previously-sent test script was incomplete for some reason.  I'm
> | attempting to send the complete version with this message.
> 
> I see that it failed again for some reason.  I'll try once more before
> giving up for now.

Hey, nice catch!

It's a bug in the mailing list software or some other part of my
mail setup. The message I received directly was fine.


> part 2 text/x-shellscript1841 mmh-0.2-inc-test-read-stdin 
> #!/bin/sh
> ##
> # Test incorporating mail from stdin with `inc -file -`. #
> ##
> . "$MH_TEST_COMMON"
> 
> # Test empty stdin.
> runandcheck "printf '' |inc -file -" <<.
> Incorporating new mail into inbox...
> 
> inc: no messages incorporated, continuing...
> .

The problem is this dot on a line on it's own. Well, of course,
it should work though, but reality shows that it is less reliable
in the email world. Maybe you better switch to some other EOF
marker.

Did you send the message with mmh? Which version? Because
mmh-0.2 includes this encode-dot-on-a-line-of-its-own code,
to prevent such cases to come appear.
commit: 92e279e1255b12b02a2faede6e1d66b46f731807


meillo


P.S.
The rest of the test file:

> # Test FIFO.
> runandcheck "packf +inbox |inc -file -" <<.
> Incorporating new mail into inbox...
> 
>   11+ 2006-09-29 00:00  Test1  Testing message 1
>   12  2006-09-29 00:00  Test2  Testing message 2
>   13  2006-09-29 00:00  Test3  Testing message 3
>   14  2006-09-29 00:00  Test4  Testing message 4
>   15  2006-09-29 00:00  Test5  Testing message 5
>   16  2006-09-29 00:00  Test6  Testing message 6
>   17  2006-09-29 00:00  Test7  Testing message 7
>   18  2006-09-29 00:00  Test8  Testing message 8
>   19  2006-09-29 00:00  Test9  Testing message 9
>   20  2006-09-29 00:00  Test10 Testing message 10
> .
> 
> # Test redirected stdin.
> cd "$MMH_TEST_DIR"
> f=mbox
> runandcheck "packf +inbox 1-10 >$f && inc -file - <$f" <<.
> Incorporating new mail into inbox...
> 
>   21+ 2006-09-29 00:00  Test1  Testing message 1
>   22  2006-09-29 00:00  Test2  Testing message 2
>   23  2006-09-29 00:00  Test3  Testing message 3
>   24  2006-09-29 00:00  Test4  Testing message 4
>   25  2006-09-29 00:00  Test5  Testing message 5
>   26  2006-09-29 00:00  Test6  Testing message 6
>   27  2006-09-29 00:00  Test7  Testing message 7
>   28  2006-09-29 00:00  Test8  Testing message 8
>   29  2006-09-29 00:00  Test9  Testing message 9
>   30  2006-09-29 00:00  Test10 Testing message 10
> .
> rm "$f"
> unset f

Re: [mmh] Confused while getting here

[markus schnalke]

[2016-07-26 22:42] Vasily Kolobkov 
> Hey folks,
> 
> can someone comment on subscribing to this list:
> 
>  - http://marmaro.de/prog/mmh says you need to _send_ ``subscibe''
>  - while the README in the root of the repository mentions sending
>a message with such a _subject_
> 
> I tried initially the second path just to get an empty message with
> this subject bounced back after a while. The first one seem to work
> though. Though i might have sent the second request with the codeword
> in both - subject and body. If my assumption is correct, attached
> is a remedy.

Here's my version of your activities:

I received this message from you:

To: mmh@marmaro.de
Subject: A pair of smalltime mmh fixes
Date: Wed, 13 Jul 2016 21:04:15 +0200
From: Vasily Kolobkov 

As you were not already subscribed, your message was filtered out
for moderation. I pushed it to the list twenty minutes later. Then
I added your address to the whitelist.

You successfully subscribed to the list at 2016-07-26 18:56 +0200.

And the same in the mailing list logfile:

2016-07-13 21:04:23 received message, wrote to /tmp/listig.osvU5z
2016-07-13 21:04:23 from: 
2016-07-13 21:04:23 reply-to: <>
2016-07-13 21:04:23 mailto: 
2016-07-13 21:04:23 subject: 
2016-07-13 21:04:23 date: <2016-07-13>
2016-07-13 21:04:23 to owner

2016-07-13 21:24:00 received message, wrote to /tmp/listig.bA8cD3
2016-07-13 21:24:00 from: 
2016-07-13 21:24:00 reply-to: <>
2016-07-13 21:24:00 mailto: 
2016-07-13 21:24:00 subject: 
2016-07-13 21:24:00 date: <2016-07-13>
2016-07-13 21:24:00  whitelisted
2016-07-13 21:24:00 to list

2016-07-26 18:56:49 received message, wrote to /tmp/listig.xZKOuL
2016-07-26 18:56:49 from: 
2016-07-26 18:56:49 reply-to: <>
2016-07-26 18:56:49 mailto: 
2016-07-26 18:56:49 subject: 
2016-07-26 18:56:49 date: <2016-07-26>
2016-07-26 18:56:49 subscribe 


Correct is to send a message with the subject ``subscribe'' (the
body is ignored) to .

In mail header words:
List-Subscribe: 


I fixed the description on the website.


[copied from above]
> I tried initially the second path just to get an empty message with
> this subject bounced back after a while.

I don't know why this happened. Have others had the same
experience?


meillo

[mmh] Allow inc(1) to read from stdin

[markus schnalke]

Hoi,

if someone is searching for work in mmh, here's an idea:

Currently inc(1) can only read from an mbox file, either
the system maildrop (e.g. /var/mail/$user) or some file
specified with -file. It is, however, not possible to
inc from stdin. This prevents us from doing clever stuff.

For instance, it should be possible to do:  packf | inc

Even if we try to take the way over /dev/stdin, we fail:
inc -file /dev/stdin < /tmp/test.mbox  +test1
inc: unable to lock and fopen /dev/stdin

I think, it would be great if mmh would support:

... | inc -file - +folder


If one of you others (not Philipp or me) wants to work
on something, this could be a good choice.

Patches (including a test case ;-) ) are welcome.


meillo

Re: [mmh] A pair of smalltime mmh fixes

[markus schnalke]

[2016-07-19 15:52] Philipp Takacs 
> [2016-07-13 21:04] Vasily Kolobkov 
> >
> > I found some minor issues during setting up mmh and thought you
> > might want to have a look.
> 
> Thanks for your patches. The changes you make look good.

I agree.

The slocal bug was introduced by me in e711cf1c50a.


meillo

Re: [mmh] [PATCH] simplify whatnow.c/main() function

[markus schnalke]

[2016-07-19 16:02] Philipp Takacs 
> > From: Dmitry Bogatov 
> > 
> > GNU Complexity: 60 -> 59.
> 
> Thanks for the Patch, it's commited.

Thank you both.


meillo

[mmh] Re: [Patch for mmh] symlinks and musl fixes

[markus schnalke]

Hoi.

[2016-05-23 13:53] Alba Pompeo 
>
> Alpine Linux has some local patches that fix a few things about mmh.

It's a pleasure to see that mmh is a so much cared for part of
Alpine Linux! :-)

> It would be great if these were committed upstream.
> http://git.alpinelinux.org/cgit/aports/tree/testing/mmh?h=master

You did a bunch of work to fix the stdio internals accesses!
Well, we plan to release mmh-0.3 soon, which contains a completely
new implementation of m_getfld(), which no longer reaches behind
stdio's back (and is much shorter). It will work fine with any
standard compliant libc. Thus there will be more need for that
patch. :-)

It would be great if you could checkout the latest development
version to verify it runs fine with musl libc.

git clone http://git.marmaro.de/mmh


Concerning the symlink patch: I don't understand the reason for
it. Why shouldn't we hard-link? These links are all in the same
directory, hence we won't cause EXDEV. It would be helpful if you
could explain the reason for that patch.


meillo


P.S.
Please include the list in the further discussion. Even better:
Subscribe! ;-)

P.P.S.
Mmh's license is not exactly 3-clause BSD, but a variant of it.
Not that I (as a CC0 friend) would care for that, but the nmh guys
once pointed me on that, hence they might care.

Re: [mmh] better error handling

[markus schnalke]

[2016-05-08 18:53] Philipp Takacs 
>
> I have added this output, because I like it and not valid mails
> indicates spammer.

Better write a new program that checks for validity.


Btw: Why is it necessary to set `state' here?

case LENERR2:
state = FLD2;
/* FALL */
case FLD2:
...

It would be nicer to be able to write:

case LENERR2:
case FLD2:
...

But that change can be done later.


Please push your changes. Refinements can be done later.


meillo

Re: [mmh] From-line in mh directory

[markus schnalke]

[2016-05-10 16:34] markus schnalke <mei...@marmaro.de>
> 
> Yet, I have a simpler solution (sorry for being so late with it!):
> 
>   #!/bin/sh
>   #
>   # remove From_ lines from MH messages in-place
> 
>   for i do
>   ed - "$i" <<\!
>   1,/^$/g/^From /d
>   w
>   !
>   done

This code isn't good enough. It cannot handle header-only messages,
nor does it check for read-only files. This is a better version:

for i do
if [ ! -w "$i" ] ; then
echo "$i: not writeable"
continue
fi
if grep -q '^$' "$i" ; then
range='1,/^$/'
else
range='1,$'
fi
ed - "$i" <

Re: [mmh] From-line in mh directory

[markus schnalke]

[2016-05-06 00:21] Philipp Takacs 
>
> I have writen a smal script to convert mail contain a ``From ''
> line to mails without a ``From '' line.

Thanks Philipp!

Yet, I have a simpler solution (sorry for being so late with it!):

#!/bin/sh
#
# remove From_ lines from MH messages in-place

for i do
ed - "$i" <<\!
1,/^$/g/^From /d
w
!
done

You can use this way:

remove-from-lines `mhpath +folder a`

Or that way:

find ~/Mail -type f -name '[1-9]*' | xargs -d \\n remove-from-lines 


Running it on a 9000 messages (80 MB) folder took about 10 seconds on
my system. A 2500 messages (400 MB) folder needed about the same time.


meillo

Re: [mmh] Issues following the m_getfld merge

[markus schnalke]

[2016-04-23 20:38] Philipp Takacs <phil...@bureaucracy.de>
> [2016-04-23 19:02] markus schnalke <mei...@marmaro.de>
> >
> > Given these words, I think that long lines should not generally
> > be considered errors (LENERR2). In message reading they should just
> > be considered to be valid input! [...]

> > In conclusion, I think that m_getfld2 should not care about too
> > long lines, because they are perfectly valid in almost all cases.
> > The only case where they are not valid is on message generation
> > .. or maybe more exact: Messages that leave the MH system. I see
> > two places where messages leave MH: send/spost and packf (maybe
> > there are more). Only these should care for folding too long
> > lines.

> Yes for the mmh-parts to long lines aren't a problem, but you miss some
> points.
> 
> 1) Inside the private mh-dir doesn't mean it is only available for mmh.
> I use mailsync to synchronize my mail over imap, and there are other
> possibility's other software access this mail, like using mutt/emacs
> as second MUA.

That's a good point, which I have not thought about, yet.

Okay, what does that mean? All messages created by mmh have to stick 
to the length limit, not just those that leave though send/spost?

What messages does mmh create? That are the drafts and nothing else
I think.

But drafts are not under control of mmh because the user may enter
whatever content he likes with his editor and he writes them
directly to disk. We cannot modify (e.g. QP-encode) them behind the
user's back, as the user must have the freedom to edit them in any
way he wishes. Only when the draft is passed to send(+mhbuild+spost),
mmh gains the right to modify the message as it likes, before sending
it out.

All other mail in the system is mail that is generated somewhere
else and only received and managed by mmh.

And then there are mh_sequences, the profile and such stuff, which
is MH-specific, so the RFC 822 length limits are not binding for us.

This brings me back to my view that send/spost (and packf) might
be the right points to solve the line length issue.


We should not hurry this topic too much. I feel I want some more
time to think about it. 


> 3) m_getfld2 doesn't really care[0] about too long lines, it just notify
> the caller this field contains a to long line or this line in the
> body is to long. The problem is most mmh-parts consider a LENERR2 as
> a cause to fail hard.

Yes. And why do they do so? Because it is named ``length error''.
;-) Coming back to my initial point: LENERR2 should not be named
error, because it usually is no error. A better naming would be
FLDLONG ... but I still like to first evaluate if dealing with
long lines should be done in send/spost and packf only.


meillo

[mmh] Refile: never change the current folder

[markus schnalke]

Hoi,

following a private discussion with Philipp, I now had a closer
look at the topic and implemented the change that refile(1) does
no longer change the current folder. The commit message contains
a detailed explanation.
http://git.marmaro.de/?p=mmh;a=commitdiff;h=a8984c0e490

I could not see any implications to the Previous-Sequence in this
change. I did some manual tests regarding the Previous-Sequence
on my machine, but could not detect any different behavior after
this change.

The test I wrote does only cover the folder changing but not
the previous sequence. If someone wants to add tests for that,
feel free to do so. However, we are considering to drop the
Previous-Sequence completely. (Is there anyone using it ...
now, that we all have shell historys and command line editing?)


Another finished work task, in mmh's words:
refile -src +todo c +done
:-)


meillo

[mmh] Release plan for mmh-0.3

[markus schnalke]

Hoi,

we are coming closer to the next release.

The largest work task left to do is merging the m_getfld2-meillo
branch into master.

The other one missing is the quote decoding in repl, everyone
is wishing so much. We're right before pushing that. :-)

There might be some more smaller improvements that we put in as
we go, but shortly after the m_getfld() replacement is done,
we'll start the freeze.

The last time, we had a one-week freeze, during which only
documentation, tests and bugfixes were done. IMO this was a good
approach.


@Philipp: Do you agree?


meillo


P.S.
It took 3 years after mmh-0.1 to get 0.2 done. From there to the
upcoming 0.3 it'll be about 6 months. If we extrapolate this,
0.4 can be expected to be finished by the end of May. ;-)