[mmh] long header fields

[Philipp Takacs]

Hi all

I have recently had a problem with to long header fields. Therfor I have
implemented some header folding. I have also submitted my patch to nmh.
David Levine and the other nmh developers for helping testing the patch.

Also in the discussion came a problem up that a field may not contain
a white space. Therefor also MIME (RFC 2047) should be used. This is
true, but not a big problem. So if I find some time, I might implement
this. It's not a big priority for me.

Philipp

Re: [mmh] showing attachments

[Philipp Takacs]

Hi

[2021-06-06 15:28] Juergen Lorenz 
> thanks for the info. But I still have a question: What's the format of
> the Metafile-extension profile entry? The man page of whatnow2 only
> speaks of an optional extion, what's that? A filename or a
> file-extension?

It's just a string. whatnow2 will use it as an extention to the filename.
So lets use my config as an example. I have following in my profile:

Metafile-Extension: .meta

Lets asume the draft has the message number 1. So the filename is ``1''
and the filename for the state is ``1.meta''. So you can use all[0]
charrecter your FS supports. I recommend stick to printable ascii.

Philipp

[0] expection: leading and trailing white space will be stript by mmh and
all white space charrecters will be translated to space.

> Cheers
> Jürgen
>
> [2021-06-05 20:23] Philipp Takacs 
> >
> > part   text/plain2659
> > Hi Jürgen
> >
> > > thanks, that seems to work.
> > >
> > > But now I have the next problem: Trying to use whatnow2 via
> > >   comp -whatnowproc whatnow2
> > > I got an input-template starting with
> > >   mmh-last-editor: /usr/bin/vim
> > >   mmh-mhdist: 0
> > > before
> > >   To:
> > >   ...
> > >
> > > The same (almost) with repl.
> >
> > Oh sorry should have warned you, whatnow2 saves it state in the draft.
> > whatnow2 will remove these fields before send. You can set the option
> > Metafile-Extension in your profile to save the sate in an extra file.
> >
> > Currently the metafile-extention is only used by whatnow2, but I plan
> > to have this concept general in mmh.
> >
> > > By the way, whatnow2 is a very bad name. Why not simply use now, because
> > > it always needs an argument like edit, attach ...?
> >
> > I'm realy bad at names, the first version was called mywhatnow. now
> > sounds good, I'll check if there are other programms with such an
> > executable.
> >
> > Philipp
> >
> > > Cheers
> > >
> > > Jürgen
> > >
> > > PS: Sorry, that I have so many questions.
> > >
> > > > part   text/plain1503
> > > > [2021-05-27 18:54] Juergen Lorenz 
> > > > > in a previous mail, Philipp explains, why X attachments are 
> > > > > problematic
> > > > > and recommended using mhstore.
> > > > > But in my old nmh-profile, I used programs which transformed the X
> > > > > attachments to mere text, and they don't work either.
> > > >
> > > > Ok thats strange, most of them should work.
> > > >
> > > > > Here are some of these profile-entries
> > > > >
> > > > > mhshow-show-text/enriched: %p/usr/bin/richtext '%f'
> > > > > mhshow-show-text/plain: %p/usr/bin/less '%f'
> > > >
> > > > Here is one error. text/plain is handled with less. The
> > > > default "%liconv -f  should work.
> > > >
> > > > > mhshow-show-application/pdf: %p/usr/bin/pdftotext '%f' -
> > > > > mhshow-show-application/odt: %p/usr/bin/odt2txt '%f'
> > > > > mhshow-show-application/docx: %p/usr/bin/docx2txt '%f' - 
> > > > > mhshow-show-application/msword: %p/usr/bin/antiword '%f' 
> > > > > mhshow-show-application/mspowerpoint: %p/usr/bin/catppt '%f'
> > > > > mhshow-show-application/excel: %p/usr/bin/xls2csv '%f'
> > > > > mhshow-show-application/rtf: %p/usr/bin/catdoc '%f'
> > > > > mhshow-show-application/octet-stream: %p/usr/bin/less '%f'
> > > > > mhshow-show-application: %p/usr/bin/less '%f'
> > > >
> > > > These two also don't work. Also I'm not sure if this is a good idea.
> > > >
> > > > > mhshow-show-text/html: %p/usr/bin/w3m '%f'
> > > > > mhshow-show-text/html: %p/usr/bin/lynx '%f'
> > > > >
> > > > > Why don't they work? And what should I do to make it work?
> > > >
> > > > I belive I have seen the problem. mmh don't support %p (for pause). nmh
> > > > also removed the support, but handle %p like %l. So replace %p with %l
> > > > and everithing should work.
> > > >
> > > > Philipp
> > > >
> >
>

Re: [mmh] showing attachments

[Philipp Takacs]

Hi Jürgen

> thanks, that seems to work.
>
> But now I have the next problem: Trying to use whatnow2 via
>   comp -whatnowproc whatnow2
> I got an input-template starting with
>   mmh-last-editor: /usr/bin/vim
>   mmh-mhdist: 0
> before
>   To:
>   ...
>
> The same (almost) with repl.

Oh sorry should have warned you, whatnow2 saves it state in the draft.
whatnow2 will remove these fields before send. You can set the option
Metafile-Extension in your profile to save the sate in an extra file.

Currently the metafile-extention is only used by whatnow2, but I plan
to have this concept general in mmh.

> By the way, whatnow2 is a very bad name. Why not simply use now, because
> it always needs an argument like edit, attach ...?

I'm realy bad at names, the first version was called mywhatnow. now
sounds good, I'll check if there are other programms with such an
executable.

Philipp

> Cheers
>
> Jürgen
>
> PS: Sorry, that I have so many questions.
>
> > part   text/plain1503
> > [2021-05-27 18:54] Juergen Lorenz 
> > > in a previous mail, Philipp explains, why X attachments are problematic
> > > and recommended using mhstore.
> > > But in my old nmh-profile, I used programs which transformed the X
> > > attachments to mere text, and they don't work either.
> >
> > Ok thats strange, most of them should work.
> >
> > > Here are some of these profile-entries
> > >
> > > mhshow-show-text/enriched: %p/usr/bin/richtext '%f'
> > > mhshow-show-text/plain: %p/usr/bin/less '%f'
> >
> > Here is one error. text/plain is handled with less. The
> > default "%liconv -f  should work.
> >
> > > mhshow-show-application/pdf: %p/usr/bin/pdftotext '%f' -
> > > mhshow-show-application/odt: %p/usr/bin/odt2txt '%f'
> > > mhshow-show-application/docx: %p/usr/bin/docx2txt '%f' - 
> > > mhshow-show-application/msword: %p/usr/bin/antiword '%f' 
> > > mhshow-show-application/mspowerpoint: %p/usr/bin/catppt '%f'
> > > mhshow-show-application/excel: %p/usr/bin/xls2csv '%f'
> > > mhshow-show-application/rtf: %p/usr/bin/catdoc '%f'
> > > mhshow-show-application/octet-stream: %p/usr/bin/less '%f'
> > > mhshow-show-application: %p/usr/bin/less '%f'
> >
> > These two also don't work. Also I'm not sure if this is a good idea.
> >
> > > mhshow-show-text/html: %p/usr/bin/w3m '%f'
> > > mhshow-show-text/html: %p/usr/bin/lynx '%f'
> > >
> > > Why don't they work? And what should I do to make it work?
> >
> > I belive I have seen the problem. mmh don't support %p (for pause). nmh
> > also removed the support, but handle %p like %l. So replace %p with %l
> > and everithing should work.
> >
> > Philipp
> >

Re: [mmh] showing attachments

[Philipp Takacs]

[2021-05-27 18:54] Juergen Lorenz 
> in a previous mail, Philipp explains, why X attachments are problematic
> and recommended using mhstore.
> But in my old nmh-profile, I used programs which transformed the X
> attachments to mere text, and they don't work either.

Ok thats strange, most of them should work.

> Here are some of these profile-entries
>
> mhshow-show-text/enriched: %p/usr/bin/richtext '%f'
> mhshow-show-text/plain: %p/usr/bin/less '%f'

Here is one error. text/plain is handled with less. The
default "%liconv -f  should work.

> mhshow-show-application/pdf: %p/usr/bin/pdftotext '%f' -
> mhshow-show-application/odt: %p/usr/bin/odt2txt '%f'
> mhshow-show-application/docx: %p/usr/bin/docx2txt '%f' - 
> mhshow-show-application/msword: %p/usr/bin/antiword '%f' 
> mhshow-show-application/mspowerpoint: %p/usr/bin/catppt '%f'
> mhshow-show-application/excel: %p/usr/bin/xls2csv '%f'
> mhshow-show-application/rtf: %p/usr/bin/catdoc '%f'
> mhshow-show-application/octet-stream: %p/usr/bin/less '%f'
> mhshow-show-application: %p/usr/bin/less '%f'

These two also don't work. Also I'm not sure if this is a good idea.

> mhshow-show-text/html: %p/usr/bin/w3m '%f'
> mhshow-show-text/html: %p/usr/bin/lynx '%f'
>
> Why don't they work? And what should I do to make it work?

I belive I have seen the problem. mmh don't support %p (for pause). nmh
also removed the support, but handle %p like %l. So replace %p with %l
and everithing should work.

Philipp

Re: [mmh] Problems with mmh

[Philipp Takacs]

Hi

[2021-05-25 16:12] Jürgen Lorenz 
> thanks for your prompt reply.

This is more the exeption then the rule. ;-)
Not on purpose and I hope I can change this.

So looking at your mail I would say your encoding is wrong. Probaly from
some tries to fix your probles. So I would guess if you just change your
encoding back to utf-8 everything works again. If not to help you I
would need the output of "locale" and "echo $MM_CHARSET".  

> Since all my mailers are based on the MH-Format, I have now mbox mails
> in my system.

Not sure what you want to say. If you want to say you have mbox because
all your mailers use the MH-Format, this is a missunderstanding.
MH-Format works with mails not mboxes, but most implementations have a
lax checking for mbox files. So it kind of work without notice, till
start strict checking. The files comes out of a mix between the lax
checking and some not strict defined interfaces between mta and mda.

> There remain some questions:
>
> If I remove the From: line in components and friends, the from line in
> the message sent is computed to juergen@manjaro, what's definitely not
> what I want. How do I get the correct From: 

We have implemented an option called Default-From. You can set it in the
mh-profile(5) and it will used as from/sender, if you don't send from
one of your Alternate-Mailboxes. See mh-profile(5) and spost(8) for
details or ask on the ml.

> In my profile, I've now uncommented everything whith mhshow. Then the
> errormessage in show disappears and I can see the content. But what if I
> want to see html contents or pdf contents? It was hard to find programs
> which convert them to mere text, and I would like not to be forced to a
> fallback like mutt or sylpheed.

Yes the idea is to have a programm converting a the file to text. Most
of your config already does this. The problem was the pipe to less.
This don't work with your /bin/sh and don't work in the concept of mmh
version of show. The problem is that show calls this in the background
and don't pass throug the terminal[0]. So it will start it and show wait
till it exit. But because less don't have user input it won't exit.

There is a diffrent on your X programs. If they just open create a
window and then exit, this may[1] work but you can't read the body and
the header at the same time. Because the Body is shown before show
display any output. There may also some side efekts for programms which
behave diffrent (i.e. browser).

An other problem is that mmh uses show in repl (to decode mime) this
can have also unknown/supprising side efekts. I only have to text
converters set up for show. For files I want to display with an
Graphical programm I use mhstore and open them manual. 

Yes this isn't the perfect solution, but I don't see any change to
have a good interface for display some mime types with a GUI out
of show. Better solution would be some extentions to mhstore so
that it store it, call a gui and after the gui closes deletes it.

Philipp

Ps: I have CC this to the list, hope this was ok. I would prefere to
have such questions on the list rather then in a private conversation.

[0] Not shure if true, would have to check. But I would define the
Interface in this way so don't expect this will work just because
is does.

[1] Just because it may work it doesn't meen it's a good idea.

>
> Cheers
>
> Juergen
>
> > part   text/plain3667
> > Hi
> >
> > [2021-05-24 17:14] Juergen Lorenz 
> > > I'm new to mmh, but not to nmh and mutt and the like.
> > > This Mail is written in mutt, because mmh causes trouble.
> >
> > Nice to see a new user.
> >
> > > For example, the scan command produces the following output:
> > >
> > > ...
> > > 2021-05-08 13:49 2845~ 
> > > 2021-05-10 16:42 286 4489~ 
> > > 2021-05-11 11:59 2878~ 
> > > 2021-05-15 15:48 289   70~ 
> > > 2021-05-22 22:23 292   m...@marmaro.de  3  [mmh] Welcome 
> > > ju@jugil
> > o.de!
> > > 2021-05-23 15:17 297   Jürgen Lorenz5  enc
> > > 2021-05-23 15:45 298   -> Juergen Lorenz6  [BCC] "Keine 
> > > Jobsteuer
> > ung in dieser Shell"
> >
> > This looks like some of the files are mboxes and not mails. If there are
> > any lines starting with "From " in the header, then mmh can't parse the
> > mail. nmh and old versions of mmh could hadle this. I can look up our
> > converting scripts if you need them.
> >
> > > and the command show 297 the following error-message:
> > >
> > > /bin/sh: Zeile 1: fg: Keine Jobsteuerung in dieser Shell.
> > >
> > > I have no idea, what happened.
> >
> > This looks like one of your mhshow-show- entries is triggert. One of
> > them with a ``|less''. You can check this with ``show -debug 297''.
> >
> > Why do you have these entries? show will run a pager by default, when
> > you call it from a shell and without a pipe. In the other cases the mail
> > 

Re: [mmh] Problems with mmh

[Philipp Takacs]

Hi

[2021-05-24 17:14] Juergen Lorenz 
> I'm new to mmh, but not to nmh and mutt and the like.
> This Mail is written in mutt, because mmh causes trouble.

Nice to see a new user.

> For example, the scan command produces the following output:
>
> ...
> 2021-05-08 13:49 2845~ 
> 2021-05-10 16:42 286 4489~ 
> 2021-05-11 11:59 2878~ 
> 2021-05-15 15:48 289   70~ 
> 2021-05-22 22:23 292   m...@marmaro.de  3  [mmh] Welcome 
> j...@jugilo.de!
> 2021-05-23 15:17 297   Jürgen Lorenz5  enc
> 2021-05-23 15:45 298   -> Juergen Lorenz6  [BCC] "Keine 
> Jobsteuerung in dieser Shell"

This looks like some of the files are mboxes and not mails. If there are
any lines starting with "From " in the header, then mmh can't parse the
mail. nmh and old versions of mmh could hadle this. I can look up our
converting scripts if you need them.

> and the command show 297 the following error-message:
>
> /bin/sh: Zeile 1: fg: Keine Jobsteuerung in dieser Shell.
>
> I have no idea, what happened.

This looks like one of your mhshow-show- entries is triggert. One of
them with a ``|less''. You can check this with ``show -debug 297''.

Why do you have these entries? show will run a pager by default, when
you call it from a shell and without a pipe. In the other cases the mail
is just printed to stdout.

> As you can see from my profile below, I tested a lot of settings, but
> the problems still go on. In particular, I tried utf-8 as well as
> latin1. Here is the profile, mostly copied from my nmh-profile:

Your profile looks intresting. Have you tried with nearly empty profile.
For most stuff you only need to set the Path. As far as I see your config
for encoding is correct, because the ü in the scan output is correct.

If you want I can comment your whole profile, but not today.

> Moreover, I have some minor important questions:
>
> Why do you advertise whatnow2, marking whatnow deprecated?
> I'm happy with whatnow and do not know, why I should take whatnow2 and
> how. So I would prefer the deprecated message to be removed. (As far as I
> remember, I tried the -whatnowppoc switch, but it didn't work.)

whatnow is a dead end. The problem is whatnow must provide a shell which
behave like the shell of the user. This has lead to interesting code.

whatnow2 works a bit diffrent instand of trapping you in a subshell it's
exit normaly and you can just call whatnow2 if you need to interact with
your current draft. All your shell features are provided by your shell.

So an sample reply would look like this:
$ repl -whatnowproc whatnow2
[editor]
$ whatnow2 attach /file/which/was/requested
$ other tool
$ whatnow2 edit
[editor-next]
$ whatnow2 send

Hope this explain why we want to drop whatnow and how to use whatnow2.
Feel free to ask if you have further questions.

> The same applies to switches -sign, -enc and -attach.

There no such switches in mmh.

To attach files you simple create an attachment header in your draft,
this is done by whatnow[2] attach. The magic is done by send and mhbuild.
For encrypt and sign of a mail you add a encrypt or signing header to
your draft. Then send calls also mhsign [-encrypt] after mhbuild. For
more info look at send(1).

> By the way, the handling of pgp and attachments is one of the reasons, I
> prefer mmh over nmh, more or less.

This is more or less the reason for mmh, so thanks for the compliment.

> I hope, you can help me. For example, providing a non-trivial and
> working profile file.

I'll look at it, but not today.

Philipp

[mmh] changes to m_getfld2()

[Philipp Takacs]

Hi

Because of the work on getfields() and readmbox() I have notized some
problems in m_getfield and the API. I have already mention the EOF2
status and the LENERR2. But this thread should collect all changes I
currently plan to m_getfld2().

I mentiont in the first discossion of m_getfld2, that I don't want to
change the API. This is kindof still true. I would like to keep the old
API, but there are problems I would like to fix. These problems could
also seen as bugs in the API. Not critical bugs, but I would like to
fix them.

So where are the problems. First the enum status contains miss the end
of header status. Also the LENERR2 status isn't realy match for this
enumeration. So I would like to add the EOH2 status like in the patches
for getfields and remove the LENERR2.

structs fields contains a boolean (unsigned char) crlf. Actually there
should be more then this. By a quick look I identified three boolean we
should have: crlf, lenerr and obsolet header (rfc 5322 4.5. Obsolete
Header Fields). Maybe there are more I haven't found yet. So I would
suggest to replace the crlf member a general flags member. Also an enum
fields_flags with contains { crlf, lenerr, obs } at the moment. This
way we can easy add more flags.

The biggest problem is the name member. Currently it's a 999 byte big
char array[0]. Conssidering that a line in a mail is most of the time
not longer then 78 bytes and most fields fit in one line, this is a huge
waste of memory. Also in the body part this array isn't even used. In
the old style handle the header field by field, this may be not that
bad, but with the new getfields() and readmbox() function this is a big
waste. So I would change this to a pointer and add an member for the
allocation length.

A small inprovent without changing the API would be to reuse the
allocation of value in the body (and with this changes also the fields).
Currently m_getfld2 frees f->value and reassige a tmp char* which is
allocated by getline(). I don't conssider this as an optimization, more
as removing an degradation.

If we apply these changes I considder the m_getfld2() API finished.
But I'm only a human and also considerd that the the API was finished
last time this was discossed. So now it's the best time to comment on
this API. Did I missed some cases? Have I overseen a big loophole in
the API? Are there other problems with m_getfld2(), the structs/enums?

Philipp

[0] This char array is a relic of the old m_getfld(). The m_getfld()
had a parameter name with the impliziet length NAMESZ. So all tools
just had a char name[NAMESZ] for m_getfld(). I have just took this
parameter and put it in the fields struct without think about the
consequences.

[mmh] mmh on freenode.net

[Philipp Takacs]

Hi all

I have created the channel #mmh on freenode. The main propose is to get
quick support. Because there aren't this much users, implementation and
features can also discussed. Feel free to join and chat about mmh.

Philipp

[mmh] scansbr and mbox support

[Philipp Takacs]

Hi

During the work on getfields I fall a bit over the scan function. This
function reads an mbox or a mail, generates the scan output and writes
the the mail in the mailstore for inc.

I would suggest to create a getfields like funktion wich reads a mail
out of a mailbox and a scan funktion wich can generate a scan listing
out of a fields pointer. The signatures for this functions I would say
are something like:

ssize_t readmbox(FILE *f, const char fn, struct field **fields, size_t *alloc);
void scan(int msgnum, const struct field *fields, size_t len, struct format 
fmt, int width, int curflg, int unseen);

For the readmbox im not sure if return a ssize_t for the length or maybe
a state struct and the length in an extra argument. scan() could also
get the msgflags entry for the mail and the msgattrs for the folder.
This would allow a in_sequence function in the format.

This way the problem that a scan on an mbox don't support filter could
also be solved. Maybe the inc could get a message argument to inc only
some mails out of an mbox. But this is future music, first clean up the
scan function.

What do you thing about this idea?

Philipp

Re: [mmh] getfields() prototype

[Philipp Takacs]

[2020-05-10 18:50] Philipp Takacs 
> Currently version of the patches is attached. The commit messages aren't
> finished yet and sometimes wrong. I'll fix this before I commit.
> Comments?

Of course I have forgett them.

Philipp
From 42901c2c423cffa12ef870233bf1996efd66768f Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 28 Aug 2019 16:50:37 +0200
Subject: [PATCH 01/19] add EOH2 to m_getfld2 return

if you only intrested in the header you don't need to read the first
body line. This is also required for getfields.
---
 h/mh.h   | 1 +
 sbr/m_getfld2.c  | 5 -
 sbr/readconfig.c | 1 +
 sbr/seq_read.c   | 1 +
 uip/distsbr.c| 4 
 uip/mhbuild.c| 2 ++
 uip/mhl.c| 2 ++
 uip/mhparse.c| 2 ++
 uip/new.c| 1 +
 uip/pick.c   | 4 +++-
 uip/prompter.c   | 2 ++
 uip/rcvdist.c| 1 +
 uip/repl.c   | 1 +
 uip/scansbr.c| 3 ++-
 uip/slocal.c | 1 +
 uip/spost.c  | 2 ++
 uip/whom.c   | 1 +
 17 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/h/mh.h b/h/mh.h
index 649582d3..03c9e11c 100644
--- a/h/mh.h
+++ b/h/mh.h
@@ -219,6 +219,7 @@ enum state {
 	FMTERR2 = -3,  /* Format error in message */
 	IOERR2 = -1,   /* Read error */
 	FLD2 = 0,  /* Header field returned */
+	EOH2,  /* End of Header */
 	BODY2, /* Body line returned */
 	FILEEOF2   /* Reached end of input file */
 };
diff --git a/sbr/m_getfld2.c b/sbr/m_getfld2.c
index b9a618d1..7cc8689c 100644
--- a/sbr/m_getfld2.c
+++ b/sbr/m_getfld2.c
@@ -74,7 +74,7 @@ m_getfld2(enum state s, struct field *f, FILE *msg)
 		if (ret == FLD2 && is_separator(tmpline)) {
 			/* header/body separator found */
 			free(tmpline);
-			return m_getfld2(BODY2, f, msg);
+			return EOH2;
 		}
 
 		f->namelen = copyname(f->name, tmpline);
@@ -129,6 +129,9 @@ m_getfld2(enum state s, struct field *f, FILE *msg)
 		free(tmpline);
 		return ret;
 
+	case EOH2:
+		ret = BODY2;
+		/* FALL */
 	case BODY2:
 		*f->name = '\0';
 		f->namelen = 0;
diff --git a/sbr/readconfig.c b/sbr/readconfig.c
index 91d2b4ed..431530ef 100644
--- a/sbr/readconfig.c
+++ b/sbr/readconfig.c
@@ -75,6 +75,7 @@ readconfig(struct node **npp, FILE *ib, char *file, int ctx)
 			advise(NULL, "%s is poorly formatted", file);
 			state = FLD2;
 			continue;
+		case EOH2:
 		case BODY2:
 			adios(EX_CONFIG, NULL, "no blank lines are permitted in %s",
 	file);
diff --git a/sbr/seq_read.c b/sbr/seq_read.c
index 4cc42233..28145fdf 100644
--- a/sbr/seq_read.c
+++ b/sbr/seq_read.c
@@ -83,6 +83,7 @@ seq_public(struct msgs *mp)
 			seq_init(mp, mh_xstrdup(f.name), trimcpy(f.value));
 			continue;
 
+		case EOH2:
 		case BODY2:
 			adios(EX_CONFIG, NULL, "no blank lines are permitted in %s",
 	seqfile);
diff --git a/uip/distsbr.c b/uip/distsbr.c
index 2662c02d..913ade01 100644
--- a/uip/distsbr.c
+++ b/uip/distsbr.c
@@ -67,6 +67,8 @@ distout(char *drft, char *msgnam, char *backup)
 			fprintf(ofp, "%s: %s", f.name, f.value);
 			break;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			for (dp = f.value; *dp; dp++) {
 if (!isspace(*dp)) {
@@ -171,6 +173,8 @@ ready_msg(char *msgnam)
 			fprintf(ofp, "%s: %s", f.name, f.value);
 			break;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			fclose(ofp);
 
diff --git a/uip/mhbuild.c b/uip/mhbuild.c
index a233c685..13dc8510 100644
--- a/uip/mhbuild.c
+++ b/uip/mhbuild.c
@@ -376,6 +376,8 @@ build_mime(char *infile)
 
 			continue;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			fseek(in, (long) (-strlen(f.value)), SEEK_CUR);
 			break;
diff --git a/uip/mhl.c b/uip/mhl.c
index fa83602d..a09cd28b 100644
--- a/uip/mhl.c
+++ b/uip/mhl.c
@@ -683,6 +683,8 @@ mhlfile(FILE *fp, char *mname, int ofilen, int ofilec)
 			}
 			continue;
 
+		case EOH2:
+			continue;
 		case BODY2:
 		case FILEEOF2:
 			column = 0;
diff --git a/uip/mhparse.c b/uip/mhparse.c
index 2b72e7bc..2d8e8ead 100644
--- a/uip/mhparse.c
+++ b/uip/mhparse.c
@@ -275,6 +275,8 @@ get_content(FILE *in, char *file, int toplevel)
 			ct->c_begin = ftell(in) + 1;
 			continue;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			ct->c_begin = ftell(in) - strlen(f.value);
 			break;
diff --git a/uip/new.c b/uip/new.c
index 21435b34..cd206e18 100644
--- a/uip/new.c
+++ b/uip/new.c
@@ -137,6 +137,7 @@ get_msgnums(char *folder, char *sequences[])
 			}
 			continue;
 
+		case EOH2:
 		case BODY2:
 			adios(EX_DATAERR, NULL, "no blank lines are permitted in %s", seqfile);
 			/* FALL */
diff --git a/uip/pick.c b/uip/pick.c
index 0b9205bb..6201b214 100644
--- a/uip/pick.c
+++ b/uip/pick.c
@@ -864,7 +864,7 @@ pmatches(FILE *fp, int msgnum)
 		nexus_debug(head, 0);
 	}
 
-	while (s == FLD2 || s == BODY2) {
+	while (s == FLD2 || s == BODY2 || s == EOH2) {
 		switch (s = m_getfld2(s, , fp)) {
 		case LENERR2:
 			s = FLD2;
@@ -872,6 +872,8 @@ pmatches(FILE *fp, int msgnum)
 		case FLD2:
 			nexus_match(, msgnum, head);

Re: [mmh] getfields() prototype

[Philipp Takacs]

[2019-09-08 11:53] Philipp Takacs 
> [2019-08-31 12:16] Philipp Takacs 
> > As Markus mentioned in the other mail we plan to implement a wrapper
> > around m_getfld2(). I have started with one called getfields() and
> > following interface:
> >
> > [...]
> >
> > So next step is clean up the error messages a bit. I have only copied
> > them so they maybe not that good. Test the filter system. then implement
> > getflieds on all tools.
>
> I have now tested my filter implementation with whom. Today I will
> work on replacing m_getfld() calls with getfields(). New version is
> attached. Still based on my pick patch.

I have now used getfields in most of uip. But there are some places
I don't want to use getfields now and a short explanation why

* uip/distsbr.c I don't get what this code does, also looks like code
  duplication. Maybe rewrite first.
* uip/rcvdist.c complex and ugly code, maybe rewrite/cleanup first.
* uip/mhl.c highly complex code, maybe rewrite/cleanup first.
* uip/scansbr.c see my next mail

Also I have two cases where there are problems with my patch:
* uip/mhparse.c This is currently extreme fail tolerant. I'm not sure
  if we should change this. I have a patch, but with this patch the
  bad-input test fails.
* uip/slocal.c There are no tests, currently porting the tests from nmh

Also noticed that the LENERR2 could only be a flag in the fields struct.
Like we already have for crlf.

Currently version of the patches is attached. The commit messages aren't
finished yet and sometimes wrong. I'll fix this before I commit.
Comments?

Philipp

[mmh] unknow mime types handling

[Philipp Takacs]

Hi

I have recent a mail with a attachment with the mime-type "appliction/pdf".
show then exited with "unknown content type 0" error message.

The problem is that mmh defaults to adios, if the mime-type is unknown.
I also don't get why there is some definitions for types we know but
without a special handling.

I have written a patch and some tests. Comments?

Philipp
From 37f44c80d16257b8bcd436ad74e4279650940856 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Thu, 9 Apr 2020 17:17:21 +0200
Subject: [PATCH] don't fail on unknown mime types

---
 test/tests/mhlist/test-unknow-mime | 60 ++
 test/tests/mhstore/test-bogo-mime  | 79 +
 test/tests/show/test-mime-types| 81 ++
 uip/mhbuild.c  |  2 +
 uip/mhlistsbr.c|  6 +--
 uip/mhshowsbr.c|  5 +-
 uip/mhstore.c  |  5 +-
 7 files changed, 225 insertions(+), 13 deletions(-)
 create mode 100755 test/tests/mhlist/test-unknow-mime
 create mode 100755 test/tests/mhstore/test-bogo-mime
 create mode 100755 test/tests/show/test-mime-types

diff --git a/test/tests/mhlist/test-unknow-mime b/test/tests/mhlist/test-unknow-mime
new file mode 100755
index ..91a7749d
--- /dev/null
+++ b/test/tests/mhlist/test-unknow-mime
@@ -0,0 +1,60 @@
+#!/bin/sh
+##
+#
+# Test mhlist
+#
+##
+
+. "$MH_TEST_COMMON"
+
+
+# check with no options and no current message
+
+
+# Write message with a text/plain subpart.
+
+msgfile=`mhpath b`
+cat > $msgfile < $msgfile < $msgfile <> `mhparam defpath`
+
+# check -part
+runandcheck 'show l' <c_type);
+		return list_content(ct, toplevel, verbose, debug);
 		break;
 	}
 
diff --git a/uip/mhshowsbr.c b/uip/mhshowsbr.c
index 7cf276ad..1d668cfc 100644
--- a/uip/mhshowsbr.c
+++ b/uip/mhshowsbr.c
@@ -212,11 +212,8 @@ show_switch(CT ct, int alternate)
 	case CT_IMAGE:
 	case CT_VIDEO:
 	case CT_APPLICATION:
-		return show_content(ct, alternate);
-		break;
-
 	default:
-		adios(EX_DATAERR, NULL, "unknown content type %d", ct->c_type);
+		return show_content(ct, alternate);
 		break;
 	}
 
diff --git a/uip/mhstore.c b/uip/mhstore.c
index 971ee7ef..e187ffa0 100644
--- a/uip/mhstore.c
+++ b/uip/mhstore.c
@@ -449,11 +449,8 @@ store_switch(CT ct)
 	case CT_AUDIO:
 	case CT_IMAGE:
 	case CT_VIDEO:
-		return store_generic(ct);
-		break;
-
 	default:
-		adios(EX_DATAERR, NULL, "unknown content type %d", ct->c_type);
+		return store_generic(ct);
 		break;
 	}
 
-- 
2.20.1

[mmh] use INSTALL_SCRIPT for shell scripts

[Philipp Takacs]

Hi

During working on a freebsd-port I have noticed, that we install
shell script with INSTALL_PROGRAM. This can lead to problems, when
a user (or the ports) uses the -s flag. I have changed the install
target that for scripts INSTALL_SCRIPT is used. I'll commit this
in tomorrow.

patch is attached, comments?

Philipp
From 628f25d1b714537d38548f42b9f86bce7f709a8e Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Sat, 4 Apr 2020 05:07:23 +0200
Subject: [PATCH] use INSTALL_SCRIPT for shell scripts in uip/Makefile.in

$(INSTALL_PROGRAM) may get the --strip flag. This don't
work on shell scripts.
---
 uip/Makefile.in | 34 +++---
 1 file changed, 27 insertions(+), 7 deletions(-)

diff --git a/uip/Makefile.in b/uip/Makefile.in
index fcab14d7..f10f46f1 100644
--- a/uip/Makefile.in
+++ b/uip/Makefile.in
@@ -39,6 +39,7 @@ LN = ln
 
 INSTALL = @INSTALL@
 INSTALL_PROGRAM = @INSTALL_PROGRAM@
+INSTALL_SCRIPT  = @INSTALL_SCRIPT@
 
 MAIL_SPOOL_GRP = @MAIL_SPOOL_GRP@
 SETGID_MAIL= @SETGID_MAIL@
@@ -50,17 +51,23 @@ SETGID_MAIL= @SETGID_MAIL@
 	$(COMPILE) $<
 
 # commands to build
-CMDS = ali anno burst comp dist flist folder forw mmh mark \
-   mhbuild mhl mhsign mhpgp \
+CMDS = ali anno burst comp dist flist folder forw mark \
+   mhbuild mhl \
mhlist mhmail mhparam mhpath mhstore new packf pick \
-   print-mimetype prompter rcvdist rcvpack rcvstore refile repl rmf \
-   rmm send sendfiles show slocal sortm spost whatnow whatnow2 whom
+   prompter rcvdist rcvpack rcvstore refile repl rmf \
+   rmm send show slocal sortm spost whatnow whom
+
+# shell commands to install
+# this is need to avoid call strip on this scripts
+SCRIPTS = mmh mhsign mhpgp sendfiles print-mimetype whatnow2
 
 # commands that are links to other commands
 LCMDS = flists folders next prev fnext fprev unseen scan
 
 # misc support binaries
-MISC = ap dp fmtdump mhtest mmhwrap
+MISC = ap dp fmtdump mhtest
+
+MISCSCRIPTS = mmhwrap
 
 # commands with 'S'pecial installation needs
 SCMDS = inc
@@ -80,7 +87,7 @@ SRCS = ali.c aliasbr.c anno.c ap.c burst.c comp.c \
 
 # == DEFAULT TARGET ==
 
-all: $(CMDS) $(MISC) $(SCMDS)
+all: $(CMDS) $(MISC) $(SCMDS) $(SCRIPTS) $(MISCSCRIPTS)
 
 # = DEPENDENCIES FOR BUILDING ==
 
@@ -231,7 +238,7 @@ whom: whom.o $(LOCALLIBS)
 # == DEPENDENCIES FOR INSTALLING ==
 
 # install everything
-install: install-cmds install-misc install-lcmds install-scmds
+install: install-cmds install-misc install-lcmds install-scmds install-scripts install-misc-scripts
 
 # install commands
 install-cmds:
@@ -240,6 +247,13 @@ install-cmds:
 	  $(INSTALL_PROGRAM) $$cmd $(DESTDIR)$(bindir)/$$cmd; \
 	done
 
+# install shell commands
+install-scripts:
+	mkdir -p $(DESTDIR)$(bindir)
+	for cmd in $(SCRIPTS); do \
+	  $(INSTALL_SCRIPT) $$cmd $(DESTDIR)$(bindir)/$$cmd; \
+	done
+
 # install links
 install-lcmds: install-cmds
 	rm -f $(DESTDIR)$(bindir)/flists
@@ -266,6 +280,12 @@ install-misc:
 	  $(INSTALL_PROGRAM) $$misc $(DESTDIR)$(libdir)/$$misc; \
 	done
 
+install-misc-scripts:
+	mkdir -p $(DESTDIR)$(libdir)
+	for misc in $(MISCSCRIPTS); do \
+	  $(INSTALL_SCRIPT) $$misc $(DESTDIR)$(libdir)/$$misc; \
+	done
+
 # install commands with special installation needs (thus no $(SCMDS) use here)
 install-scmds:
 	if test x$(SETGID_MAIL) != x; then \
-- 
2.20.1

Re: [mmh] Previous-Sequence

[Philipp Takacs]

[2020-03-24 08:56] markus schnalke 
> [2020-03-23 21:46] Philipp Takacs 
> > A updated version of my patches is attached, including the updated
> > man pages.
>
> Please add a HISTORY section to the mh-sequence(7) manpage with
> some explanation about the removal of the Previous-Sequence, so
> that people, who search for it, are informed that it was removed
> and why.
>
> Then it's ready to commit, I'd say.

OK I have added following:

--- a/man/mh-sequence.man7
+++ b/man/mh-sequence.man7
@@ -263,5 +263,24 @@ in your profile with an empty value.
 .SH "SEE ALSO"
 flist(1), mark(1), pick(1), mh-profile(5)
 
+.SH "HISTORY"
+.SS Previous\-Sequence
+Earlier versions of
+.B mmh
+supported the `Previous\-Sequence'.
+In this sequence every
+.B mmh
+command save the selected messages.
+This feature was disabled by default,
+because it introduces a lot write access to the
+.IR \&.mh_sequence
+file.
+If you used multiple
+.B mmh
+commands parallel or other tools writing the
+.IR \&.mh_sequence
+file, this could lead to race conditions.
+Because of this this feature was removed.
+
 .SH DEFAULTS
 None

>
>
> meillo
>

Re: [mmh] folder_read with scandir()

[Philipp Takacs]

[2020-03-23 21:08] markus schnalke 
> [2020-03-23 20:57] Philipp Takacs 
> > [2020-03-22 07:20] markus schnalke 
> > > [2020-03-22 01:08] Philipp Takacs 
> > > > [2020-03-21 23:01] markus schnalke 
> > > > > [2020-03-21 17:48] Philipp Takacs 
> > > > > > [2020-03-21 10:28] markus schnalke 
> > > > > > > [2020-03-19 12:45] Philipp Takacs 
> > > > > > > >
> > > > > > > > +static int others;
> > > > > > > > +
> > > > > > > > +static int
> > > > > > > > +msgcmp(const struct dirent **d1, const struct dirent **d2)
> > > > > > > > +{
> > > > > > > > +   size_t l1, l2;
> > > > > > > > +
> > > > > > > > +   l1 = strlen((*d1)->d_name);
> > > > > > > > +   l2 = strlen((*d2)->d_name);
> > > > > > > > +   if (l1 < l2) {
> > > > > > > > +   return -1;
> > > > > > > > +   }
> > > > > > > > +   if (l1 > l2) {
> > > > > > > > +   return 1;
> > > > > > > > +   }
> > > > > > > > +   return strcmp((*d1)->d_name, (*d2)->d_name);
> > > > > > > > +}
> > > > > > >
> > > > > > > Isn't that a strange comparison algorithm, for first compare
> > > > > > > string lengths and only if eaqual their content? Here does mmh
> > > > > > > sort by length of folder name?
> > > > > > 
> > > > > > This is only for msg-files and sorts by numeric order. So "9" is
> > > > > > smaller the "10". But actually it's complete irrelevant, because the
> > > > > > m_atoi() and the storage in the array will automatic get the right
> > > > > > order. Therefor we could completly remove this function.
> > > > >
> > > > > Even better. ;-)
> > > > 
> > > > OK then I'll remove the msgcmp function, check if I have overseen some
> > > > errors and commit it in the next days.
> > >
> > > Great.
> > 
> > Sorry I was wrong. The sort is necessary, because I need to know
> > `mp->lowmsg' and `mp->hghmsg' to allocate `mp->msgstats'. Without
> > the right order I can't do this. I could do this with two loops
> > like the old code, but I one of the reasons I don't like the old
> > version.
> > 
> > So I would suggest to rename msgcmp to msgnumcmp.
>
> Sounds good.

A updated version of the patch is attached, I'll commit
this in the next days.

Philipp
From 3fc8b92d56b6913dc8cffd95c26991e5b664edfa Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Thu, 19 Mar 2020 02:50:51 +0100
Subject: [PATCH] use scandir in folder_read

---
 sbr/folder_read.c | 149 +-
 1 file changed, 56 insertions(+), 93 deletions(-)

diff --git a/sbr/folder_read.c b/sbr/folder_read.c
index 011958f8..6a8139a6 100644
--- a/sbr/folder_read.c
+++ b/sbr/folder_read.c
@@ -12,9 +12,6 @@
 #include 
 #include 
 
-/* We allocate the `mi' array 1024 elements at a time */
-#define NUMMSGS  1024
-
 /*
 ** 1) Create the folder/message structure
 ** 2) Read the directory (folder) and temporarily
@@ -24,121 +21,87 @@
 ** 4) Read and initialize the sequence information.
 */
 
+static int others;
+
+static int
+msgnumcmp(const struct dirent **d1, const struct dirent **d2)
+{
+	size_t l1, l2;
+
+	l1 = strlen((*d1)->d_name);
+	l2 = strlen((*d2)->d_name);
+	if (l1 < l2) {
+		return -1;
+	}
+	if (l1 > l2) {
+		return 1;
+	}
+	return strcmp((*d1)->d_name, (*d2)->d_name);
+}
+
+static int
+msgfilter(const struct dirent *e)
+{
+	int i;
+	/* For compatibility with nmh, ignore rmm backup files. */
+	if (e->d_name[0] == '.' || e->d_name[0] == ',' || e->d_name[0] == '#') {
+		return 0;
+	}
+	for (i = 0; e->d_name[i]; i++) {
+		if ((i == 0 && e->d_name[i] == '0') || e->d_name[i] < '0' || e->d_name[i] > '9') {
+			others = 1;
+			return 0;
+		}
+	}
+	return 1;
+}
+
 struct msgs *
 folder_read(char *name)
 {
-	int msgnum, len, *mi;
+	int i;
 	struct msgs *mp;
 	struct stat st;
-	struct dirent *dp;
-	DIR *dd;
+	struct dirent **dp;
 
+	others = 0;
 	name = mh_xstrdup(toabsdir(name));
-	if (!(dd = opendir(name))) {
-		mh_free0();
-		return NULL;
-	}
 
 	if (stat(name, ) == -1) {
 		mh_free0();
-		closedir(dd);
 		return NULL;
 	}
 
 	/* Allocate the main structure for folder information */
 	mp = mh_xcalloc(1, sizeof(*mp))

Re: [mmh] folder_read with scandir()

[Philipp Takacs]

[2020-03-22 07:20] markus schnalke 
> [2020-03-22 01:08] Philipp Takacs 
> > [2020-03-21 23:01] markus schnalke 
> > > [2020-03-21 17:48] Philipp Takacs 
> > > > [2020-03-21 10:28] markus schnalke 
> > > > > [2020-03-19 12:45] Philipp Takacs 
> > > > > >
> > > > > > +static int others;
> > > > > > +
> > > > > > +static int
> > > > > > +msgcmp(const struct dirent **d1, const struct dirent **d2)
> > > > > > +{
> > > > > > +   size_t l1, l2;
> > > > > > +
> > > > > > +   l1 = strlen((*d1)->d_name);
> > > > > > +   l2 = strlen((*d2)->d_name);
> > > > > > +   if (l1 < l2) {
> > > > > > +   return -1;
> > > > > > +   }
> > > > > > +   if (l1 > l2) {
> > > > > > +   return 1;
> > > > > > +   }
> > > > > > +   return strcmp((*d1)->d_name, (*d2)->d_name);
> > > > > > +}
> > > > >
> > > > > Isn't that a strange comparison algorithm, for first compare
> > > > > string lengths and only if eaqual their content? Here does mmh
> > > > > sort by length of folder name?
> > > > 
> > > > This is only for msg-files and sorts by numeric order. So "9" is
> > > > smaller the "10". But actually it's complete irrelevant, because the
> > > > m_atoi() and the storage in the array will automatic get the right
> > > > order. Therefor we could completly remove this function.
> > >
> > > Even better. ;-)
> > 
> > OK then I'll remove the msgcmp function, check if I have overseen some
> > errors and commit it in the next days.
>
> Great.

Sorry I was wrong. The sort is necessary, because I need to know
`mp->lowmsg' and `mp->hghmsg' to allocate `mp->msgstats'. Without
the right order I can't do this. I could do this with two loops
like the old code, but I one of the reasons I don't like the old
version.

So I would suggest to rename msgcmp to msgnumcmp.

Philipp

Re: [mmh] folder_read with scandir()

[Philipp Takacs]

[2020-03-22 07:20] markus schnalke 
> [2020-03-22 01:08] Philipp Takacs 
> > [2020-03-21 23:01] markus schnalke 
> > > [2020-03-21 17:48] Philipp Takacs 
> > > > [2020-03-21 10:28] markus schnalke 
> > > > >
> > > > > - Files starting with a comma. These were backups of deleted messages
> > > > > (see the section on the trash folder in my master's thesis). Thus
> > > > > they are considered MH files here but not further regarded. I'm
> > > > > not sure if the code had been this way all the time, because there
> > > > > used to the a `--with-hash-prefix' option (or similar) that
> > > > > replaced the comma with a hash symbol as the prefix for deleted
> > > > > messages backups. Anyways, we don't have such comma files anymore.
> > > > > Mmh uses the trash folder for that case, i.e. we can get rid of
> > > > > the comma prefix file handling ... possibly. Does anyone know the
> > > > > situation in nmh: have they switched to a trash folder as well?
> > > > > Because we're still (basically) mail storage compatible to nmh (I
> > > > > think), we should not be fully ignorant of that situation. But we
> > > > > could handle those backup files as `other' files. 
> > > > 
> > > > The files starting with a comma/dot aren't the problem they are totally
> > > > ignored. The problem are files witch aren't totally ignored. I have a 
> > > > ugly
> > > > workaround in the filter and I want to get rid of it.
> > >
> > > How I understand the situation (without looking at the code): A
> > > folder can contain:
> > >
> > > - messages
> > > - `.' and `..'
> > > - `.mh_sequences'
> > > - subfolders
> > > - other files
> > >
> > > Other files are everything else, no matter if they are named
> > > `.foo' or `,foo' or just `foo'.
> > >
> > > When we read a folder, only messages are of relevance. All the
> > > rest is ignored.
> > 
> > expect for the OTHERS bit set for files which aren't messages and don't
> > start with a '.' or a ','.
>
> Why for files starting with a comma? Is there a sense to do that
> in mmh? Or is it only nmh legacy/compatiblity? And still: What
> sense does it make to ignore backup files starting with a comma,
> if the nmh installation might be configured to use hash prefixes
> instead of comma prefixes?

I believe this is for legacy/compatiblity. The idea is, you don't have
other files if you only have backup files in this directory.

> Either we should add '#' as well or remove ','. Can you follow my
> reasoning?

For compatiblity reasons, I would say add the '#' while we have the
other bit.

Philipp

Re: [mmh] show sequences racecondition

[Philipp Takacs]

[2020-03-21 23:29] markus schnalke 
> [2020-03-21 16:15] Philipp Takacs 
> Thanks for the explanation. Show(1) is the only case where I've
> noticed the problem. Thus it's the most important.

That's the reason I have started there. I would say I look at the
patches again, to see if I have overseen something and push them in the
next days.

> > I found then, that seq_read and seq_list ignores mails witch it thinks
> > don't exist. This I have disabled this, to avoid a secound read of the 
> > of the whole directory.
> > 
> > Now the conflict window is small, but still there. The best way to
> > handle this is locking.
>
> This whole topic is a special one ... One case that one should
> have in mind as well: Stuff shouldn't mess up, if you have show(1)
> running while deleting/refiling/incing messages, not only to the
> end but possibly on previously used message numbers. For instance,
> I use `folder -pack' on the inbox quite often ... there might be a
> `repl' job in background at the same time ...

I have the problems with sortm and folder -pack in mind. I have some
idea how to prevent most of such problems. But the problem is: We can
help the user to prevent such problems, but there is no way we can stop
the user from shooting him in the food. ;-)

> I really don't want to burden you with this stuff. I think that
> improving the most annoying problem is already a great step
> forward. The rest is more of a separate topic to dig into and
> solve consistently and generally. I only wanted to put the
> information onto the list, to have it documented.

It don't burden me. It's nice to read an other view on mmh. Most
of the time it gives me good idea for future work or how I have
to change my code to get better results.

> P.S.
> Not related to this topic, more in general: I really appreciate
> that you supported your changes with test cases. This is good
> work. It will make mmh increasingly stronger. Thank you!

A test helps a lot, first it's the best argument why my patch is needed.
Second I use the tests to see if I really fixed the problems. Third I
see later, if another patch kills some functionality. So if anybody has
a problem with mmh, a test case help me (and others) to fix this issue.

Philipp

Re: [mmh] folder_read with scandir()

[Philipp Takacs]

[2020-03-21 23:01] markus schnalke 
> [2020-03-21 17:48] Philipp Takacs 
> > [2020-03-21 10:28] markus schnalke 
> > >
> > > Some comments:
> > >
> > > [2020-03-19 12:45] Philipp Takacs 
> > > >
> > > > Also noticed the other flag, not shoure if we need this realy. It's
> > > > only used in folder and I don't see the point of knowing that there
> > > > are other files in a folder.
> > >
> > > We should be a bit careful at that point to understand the
> > > situation. Here are some aspects to think about:
> > >
> > > - Files starting with a comma. These were backups of deleted messages
> > > (see the section on the trash folder in my master's thesis). Thus
> > > they are considered MH files here but not further regarded. I'm
> > > not sure if the code had been this way all the time, because there
> > > used to the a `--with-hash-prefix' option (or similar) that
> > > replaced the comma with a hash symbol as the prefix for deleted
> > > messages backups. Anyways, we don't have such comma files anymore.
> > > Mmh uses the trash folder for that case, i.e. we can get rid of
> > > the comma prefix file handling ... possibly. Does anyone know the
> > > situation in nmh: have they switched to a trash folder as well?
> > > Because we're still (basically) mail storage compatible to nmh (I
> > > think), we should not be fully ignorant of that situation. But we
> > > could handle those backup files as `other' files. 
> > 
> > The files starting with a comma/dot aren't the problem they are totally
> > ignored. The problem are files witch aren't totally ignored. I have a ugly
> > workaround in the filter and I want to get rid of it.
>
> How I understand the situation (without looking at the code): A
> folder can contain:
>
> - messages
> - `.' and `..'
> - `.mh_sequences'
> - subfolders
> - other files
>
> Other files are everything else, no matter if they are named
> `.foo' or `,foo' or just `foo'.
>
> When we read a folder, only messages are of relevance. All the
> rest is ignored.

expect for the OTHERS bit set for files which aren't messages and don't
start with a '.' or a ','.

> Do we need to know about subfolders and other files? I'd say that
> it would be nice, because there are mmh tools that use such
> information.
>
> Does it have to be part of this function? Not necessarily.

That's why I have mentioned it.

> (Sorry for not looking at the code right now. I hope my thoughts
> help nonetheless.)

No problem, it's helps to have a different view on the problems

> > > - `Other' files seem to be files in the mail storage that have
> > > nothing to do with MH. E.g. any random file someone created in the
> > > mail storage without any connection to mmh. Normally mmh would
> > > simply ignore them, i.e. act as if they wouldn't be there at all.
> > 
> > Not completely other files are also sub directories. folder uses this to
> > check if it's required to search for sub directories with the recursive
> > flag. This is an other ugly code path I found.
>
> Like I layed out above, these are different classes of files that
> could be in a folder. Merging subdirectories and other files (or
> ignoring everything that starts with a dot) does not match the
> problem structure. It might work and it might have been good
> enough -- it might still be good enough -- but the goal should be
> that the code reflects the problem domain.
>
>
> > > There's one place, however, where `other' files are relevant:
> > > rmf(1). We should not delete a mail folder that includes `other'
> > > files. It seems this check is missing there. Hence, I would keep
> > > the `other' files marker. It adds few complexity and can be
> > > useful. Plus, we should use it in rmf(1).
> > 
> > rmf(1) just don't use folder_read(), therefor it's completely useless
> > for it. It does check for other files on it's own.
>
> Sounds a bit like code duplication. Surely both cases -- reading
> message numbers from a folder and deleting a folder -- are quite
> different, but the classification of files within a folder is the
> same and maybe should be outfactored and reused (maybe with an
> enum fileclass or such).

Yes it's code duplication. folder_read() covers most cases, but in some
cases there need to be done something a little different. Some refactoring
to have less code duplication would be nice. The best function for this
is the msgfilter().

> > > > +static int others;
> > > > +
> > > > +static int
> > > >

Re: [mmh] Previous-Sequence

[Philipp Takacs]

Hi

[2020-03-21 09:54] markus schnalke 
> [2019-12-15 17:55] Philipp Takacs 
> >
> > During that I found the Previous-Sequence again. I don't see a reason
> > why we need this feature. The usecases, I see, for the feature are better
> > handled with mark or the backlog of your shell. But I don't use this
> > feature.

During reading some old mails on the nmh mailing list I found a use case.
Put mails in a sequence after a refile[0]. Not sure if this works, but
for this I would suggest to add a -preserve switch to refile instead of
using previous-sequence for this, like nmh has done.

> Just for reference, the section from mh-sequence(7):
>
>The Previous Sequence
>Mmh provides the ability to remember the `msgs' or  `msg'
>argument last given to an mmh command.  The entry `Previ‐
>ous-Sequence' should be defined in the mmh  profile;  its
>value  should  be  a  sequence  name or multiple sequence
>names separated by spaces.  If  this  entry  is  defined,
>when   an  mmh  command  finishes,  it  will  define  the
>sequence(s) named in the value of this entry to be  those
>messages  that  were  specified to the command.  Hence, a
>profile entry of
>
> Previous-Sequence: pseq
>
>directs any mmh command that accepts a  `msg'  or  `msgs'
>argument  to define the sequence `pseq' as those messages
>when it finishes.
>
>Note: there can be a performance  penalty  in  using  the
>`Previous-Sequence'  facility.   If  it  is used, all mmh
>programs have to write the sequence  information  to  the
>.mh_sequences file for the folder each time they run.  If
>the `Previous-Sequence' profile entry  is  not  included,
>only pick and mark will write to the .mh_sequences file.

Actually the last sentence is wrong. A lot more mmh programs write
the .mh_sequences file, like show, inc, ...

> Besides: It does introduce a lot of write accesses to the sequence
> files.

Would it only be the write access itself, it wouldn't be that bad. You
still need to enable it.

> Before we remove it, I just would like to know why it was
> introduced in MH the first time and what possible other usecases
> there could be. This means asking on the nmh-workers mailing list.
> Both, out of curiosity and as a double-check in case we've not
> thought on something. Do you wanna write this message or should
> I?

This is a good idea. I want to write this mail.

So conclusion ask the nmh guys for use cases we missed and if there
aren't any, remove it.

Philipp

[0] https://lists.nongnu.org/archive/html/nmh-workers/2005-12/msg00086.html

Re: [mmh] folder_add and msgstats

[Philipp Takacs]

Hi

[2020-03-21 10:05] markus schnalke 
> [2020-03-19 12:33] Philipp Takacs 
> > I have also removed the
> > clear_msg_flags loop from folder_read and folder_realloc. In most cases
> > we use calloc, so this isn't necesarry. At one place in folder_realloc
> > realloc is used. There I have add a memset.
> > 
> > comment?
>
> Why were (in the section down below) only the flags outside the
> current message range cleared? Now you clear all flags.

Not sure whats make you think this. But if I had to guess I would
say it's the "- lo". So a shourt exircourse to the msgstats. msgstats
is an array of the lenght ``mp->hghoff - mp->lowoff''. lowoff is the
starting offset, so if you want the stats of message 5 you access
``mp->msgstats[5 - mp->mp->lowoff]. hghoff is the end offset so you
can change flags for all messagestats in the range lowoff <= i <= hghoff.
folder_realloc in this code path increases the hghoff. Now I have to
clear all data after the old hghoff. by the way there was a off by one
in my path. I fixed it now it looks like this:

> > memset(mp->msgstats + mp->hghoff - lo + 1, 0, hi - mp->hghoff);

This actually clears less then the code before. The code before
started at ``mp->hghmsg + 1''. Now it starts hghoff which is
greater or equal to hghmsg.

> Does that change the behavior?

Yes, like the pathes for seq_read it don't clears flags for msgs
which it belive don't exists. But this is actually the reason
I created this patch.

> Or, if your memset() clears only specific
> parts, you probably should put a comment there to explain it, as
> this might not be what the programmer usually expects from a
> memset() after a (re)alloc().

To me this looks completly like this what I would expect. But I'll
think about a good comment.

Philipp

> Apart from that, your changes look and sound sensible.
>
>
> meillo
>
>
> > From 937c01f62952147a8ff853d8a103dbe8465cb90e Mon Sep 17 00:00:00 2001
> > From: Philipp Takacs 
> > Date: Wed, 18 Mar 2020 21:46:30 +0100
> > Subject: [PATCH 3/3] use memset to clear the msgstats in folder_realloc
> > 
> > ---
> >  sbr/folder_realloc.c | 16 +---
> >  1 file changed, 1 insertion(+), 15 deletions(-)
> > 
> > diff --git a/sbr/folder_realloc.c b/sbr/folder_realloc.c
> > index 64a66409..781e8bdc 100644
> > --- a/sbr/folder_realloc.c
> > +++ b/sbr/folder_realloc.c
> > @@ -46,6 +46,7 @@ folder_realloc(struct msgs *mp, int lo, int hi)
> > ** just realloc the message status array.
> > */
> > mp->msgstats = mh_xrealloc(mp->msgstats, MSGSTATSIZE(mp, lo, 
> > hi));
> > +   memset(mp->msgstats + mp->hghoff - lo, 0, hi - mp->hghoff);
> > } else {
> > /*
> > ** We are changing the offset of the message status
> > @@ -69,20 +70,5 @@ folder_realloc(struct msgs *mp, int lo, int hi)
> > mp->lowoff = lo;
> > mp->hghoff = hi;
> >  
> > -   /*
> > -   ** Clear all the flags for entries outside
> > -   ** the current message range for this folder.
> > -   */
> > -   if (mp->nummsg > 0) {
> > -   for (msgnum = mp->lowoff; msgnum < mp->lowmsg; msgnum++)
> > -   clear_msg_flags(mp, msgnum);
> > -   for (msgnum = mp->hghmsg + 1; msgnum <= mp->hghoff; msgnum++)
> > -   clear_msg_flags(mp, msgnum);
> > -   } else {
> > -   /* no messages, so clear entire range */
> > -   for (msgnum = mp->lowoff; msgnum <= mp->hghoff; msgnum++)
> > -   clear_msg_flags(mp, msgnum);
> > -   }
> > -
> > return mp;
> >  }
> > -- 
> > 2.20.1
> > 
>

Re: [mmh] folder_read with scandir()

[Philipp Takacs]

[2020-03-21 10:28] markus schnalke 
> [2020-03-19 12:45] Philipp Takacs 
> >
> > I have also looked at folder_read and found it't a bit ugly ;-)
>
> A good indicator for starting to rework. :-)
>
> > I have rewritten it with scandir. This saves about 40 loc and improves
> > the reading flow for this funktion.
>
> Sounds good!
>
>
> Some comments:
>
> > Also noticed the other flag, not shoure if we need this realy. It's
> > only used in folder and I don't see the point of knowing that there
> > are other files in a folder.
>
> We should be a bit careful at that point to understand the
> situation. Here are some aspects to think about:
>
> - Files starting with a comma. These were backups of deleted messages
> (see the section on the trash folder in my master's thesis). Thus
> they are considered MH files here but not further regarded. I'm
> not sure if the code had been this way all the time, because there
> used to the a `--with-hash-prefix' option (or similar) that
> replaced the comma with a hash symbol as the prefix for deleted
> messages backups. Anyways, we don't have such comma files anymore.
> Mmh uses the trash folder for that case, i.e. we can get rid of
> the comma prefix file handling ... possibly. Does anyone know the
> situation in nmh: have they switched to a trash folder as well?
> Because we're still (basically) mail storage compatible to nmh (I
> think), we should not be fully ignorant of that situation. But we
> could handle those backup files as `other' files. 

The files starting with a comma/dot aren't the problem they are totally
ignored. The problem are files witch aren't totally ignored. I have a ugly
workaround in the filter and I want to get rid of it.

> - `Other' files seem to be files in the mail storage that have
> nothing to do with MH. E.g. any random file someone created in the
> mail storage without any connection to mmh. Normally mmh would
> simply ignore them, i.e. act as if they wouldn't be there at all.

Not completely other files are also sub directories. folder uses this to
check if it's required to search for sub directories with the recursive
flag. This is an other ugly code path I found.

> There's one place, however, where `other' files are relevant:
> rmf(1). We should not delete a mail folder that includes `other'
> files. It seems this check is missing there. Hence, I would keep
> the `other' files marker. It adds few complexity and can be
> useful. Plus, we should use it in rmf(1).

rmf(1) just don't use folder_read(), therefor it's completely useless
for it. It does check for other files on it's own.

So conclusion, currently I would suggest to let in in. But if I rewrite
the recursive code path of folder(1), I'll bring this up again.

> > +static int others;
> > +
> > +static int
> > +msgcmp(const struct dirent **d1, const struct dirent **d2)
> > +{
> > +   size_t l1, l2;
> > +
> > +   l1 = strlen((*d1)->d_name);
> > +   l2 = strlen((*d2)->d_name);
> > +   if (l1 < l2) {
> > +   return -1;
> > +   }
> > +   if (l1 > l2) {
> > +   return 1;
> > +   }
> > +   return strcmp((*d1)->d_name, (*d2)->d_name);
> > +}
>
> Isn't that a strange comparison algorithm, for first compare
> string lengths and only if eaqual their content? Here does mmh
> sort by length of folder name?

This is only for msg-files and sorts by numeric order. So "9" is
smaller the "10". But actually it's complete irrelevant, because the
m_atoi() and the storage in the array will automatic get the right
order. Therefor we could completly remove this function.

> > +static int
> > +msgfilter(const struct dirent *e)
> > +{
> > +   int i;
> > +   if (e->d_name[0] == '.' || e->d_name[0] == ',') {
>
> About this comma: see above!
>
> > +   return 0;
> > +   }
> > +   for (i = 0; e->d_name[i]; i++) {
> > +   //if ((i > 1 && e->d_name[i] < '0') || e->d_name[i] < '1' || 
> > e->d_name[i] > '9') {
>
> C99-style comment ... looks to be not finished at this point.

Oh thanks, I have forgotten about this.

> > +   if (e->d_name[i] < '0' || e->d_name[i] > '9') {

I have changed this to:

+   if ((i == 0 && e->d_name[i] == '0') || e->d_name[i] < '0' || 
e->d_name[i] > '9') {

> > +   others = 1;
> > +   return 0;
> > +   }
> > +   }
> > +   return 1;
> > +}
>
>
> Wow, it feels so good to have some touch with the mmh development
> again! I'm really looking forward to dig into the code as well.
> Hoping I get that managed soon.

Thanks for your comments. I'm looking forward to hear from you.

Philipp

Re: [mmh] show sequences racecondition

[Philipp Takacs]

Hi

[2020-03-21 09:32] markus schnalke 
> [2020-03-19 12:14] Philipp Takacs 
> > [2019-12-15 17:55] Philipp Takacs 
> > >
> > > I have looked at the problem with open pager for show and change the
> > > sequences during the pager is open. This happen often, if you show a
> > > message and recieve a new message.
>
> This is an annoying inconvenience (actually a bug but not too
> severe) that we probably all suffer from. Improving the situation
> would be great.
>
> We can't get fully rid of the race condition, or? We have multiple
> processes writing the same file, thus we will inevitably have a
> conflict window. Correct? Could you explain quickly how we react
> in this case of conflict?

So currently and after my path last writer winns. Currenly a programm
reads the sequences and does it work. If the work includes changing
the sequences, it writes the sequences to disk.

My patches reads the sequences after show is finisched with the main
work. The changes to the sequences are then aplied to the new set of
sequences. This makes the conflict window smaler, but the race
condition is still there. My patch does this only for show, but I want
to add this to seq_save.

I found then, that seq_read and seq_list ignores mails witch it thinks
don't exist. This I have disabled this, to avoid a secound read of the 
of the whole directory.

Now the conflict window is small, but still there. The best way to
handle this is locking.

Philipp

[mmh] folder_read with scandir()

[Philipp Takacs]

Hi

I have also looked at folder_read and found it't a bit ugly ;-)

I have rewritten it with scandir. This saves about 40 loc and improves
the reading flow for this funktion.

Also noticed the other flag, not shoure if we need this realy. It's
only used in folder and I don't see the point of knowing that there
are other files in a folder.

Philipp
From cd91711daacb8a7383b7c0815b84d909c13e429a Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Thu, 19 Mar 2020 02:50:51 +0100
Subject: [PATCH] use scandir in folder_read

---
 sbr/folder_read.c | 149 +-
 1 file changed, 56 insertions(+), 93 deletions(-)

diff --git a/sbr/folder_read.c b/sbr/folder_read.c
index 011958f8..9eb13727 100644
--- a/sbr/folder_read.c
+++ b/sbr/folder_read.c
@@ -12,9 +12,6 @@
 #include 
 #include 
 
-/* We allocate the `mi' array 1024 elements at a time */
-#define NUMMSGS  1024
-
 /*
 ** 1) Create the folder/message structure
 ** 2) Read the directory (folder) and temporarily
@@ -24,121 +21,87 @@
 ** 4) Read and initialize the sequence information.
 */
 
+static int others;
+
+static int
+msgcmp(const struct dirent **d1, const struct dirent **d2)
+{
+	size_t l1, l2;
+
+	l1 = strlen((*d1)->d_name);
+	l2 = strlen((*d2)->d_name);
+	if (l1 < l2) {
+		return -1;
+	}
+	if (l1 > l2) {
+		return 1;
+	}
+	return strcmp((*d1)->d_name, (*d2)->d_name);
+}
+
+static int
+msgfilter(const struct dirent *e)
+{
+	int i;
+	if (e->d_name[0] == '.' || e->d_name[0] == ',') {
+		return 0;
+	}
+	for (i = 0; e->d_name[i]; i++) {
+		//if ((i > 1 && e->d_name[i] < '0') || e->d_name[i] < '1' || e->d_name[i] > '9') {
+		if (e->d_name[i] < '0' || e->d_name[i] > '9') {
+			others = 1;
+			return 0;
+		}
+	}
+	return 1;
+}
+
 struct msgs *
 folder_read(char *name)
 {
-	int msgnum, len, *mi;
+	int i;
 	struct msgs *mp;
 	struct stat st;
-	struct dirent *dp;
-	DIR *dd;
+	struct dirent **dp;
 
+	others = 0;
 	name = mh_xstrdup(toabsdir(name));
-	if (!(dd = opendir(name))) {
-		mh_free0();
-		return NULL;
-	}
 
 	if (stat(name, ) == -1) {
 		mh_free0();
-		closedir(dd);
 		return NULL;
 	}
 
 	/* Allocate the main structure for folder information */
 	mp = mh_xcalloc(1, sizeof(*mp));
 
-	clear_folder_flags(mp);
 	mp->foldpath = name;
-	mp->lowmsg = 0;
-	mp->hghmsg = 0;
-	mp->curmsg = 0;
-	mp->lowsel = 0;
-	mp->hghsel = 0;
-	mp->numsel = 0;
-	mp->nummsg = 0;
 
 	if (access(name, W_OK) == -1)
 		set_readonly(mp);
 
-	/*
-	** Allocate a temporary place to record the
-	** name of the messages in this folder.
-	*/
-	len = NUMMSGS;
-	mi = mh_xcalloc(len, sizeof(*mi));
-
-	while ((dp = readdir(dd))) {
-		if ((msgnum = m_atoi(dp->d_name)) && msgnum > 0) {
-			/*
-			** Check if we need to allocate more
-			** temporary elements for message names.
-			*/
-			if (mp->nummsg >= len) {
-len += NUMMSGS;
-mi = mh_xrealloc(mi, len * sizeof(*mi));
-			}
-
-			/* Check if this is the first message we've seen */
-			if (mp->nummsg == 0) {
-mp->lowmsg = msgnum;
-mp->hghmsg = msgnum;
-			} else {
-/*
-** Check if this is it the highest or
-** lowest we've seen?
-*/
-if (msgnum < mp->lowmsg)
-	mp->lowmsg = msgnum;
-if (msgnum > mp->hghmsg)
-	mp->hghmsg = msgnum;
-			}
-
-			/*
-			** Now increment count, and record message
-			** number in a temporary place for now.
-			*/
-			mi[mp->nummsg++] = msgnum;
-
-		} else {
-			switch (dp->d_name[0]) {
-			case '.':
-			case ',':
-continue;
-
-			default:
-/*
-** indicate that there are other
-** files in folder
-*/
-set_other_files(mp);
-continue;
-			}
-		}
-	}
+	mp->nummsg = scandir(name, , msgfilter, msgcmp);
 
-	closedir(dd);
-	mp->lowoff = max(mp->lowmsg, 1);
+	if (others)
+		set_other_files(mp);
 
-	/* Go ahead and allocate space for 100 additional messages. */
+	if (mp->nummsg < 0) {
+		mh_free0();
+		mh_free0();
+		return NULL;
+	}
+	if (mp->nummsg > 0) {
+		mp->lowmsg = m_atoi(dp[0]->d_name);
+		mp->hghmsg = m_atoi(dp[mp->nummsg-1]->d_name);
+	}
+	mp->lowoff = max(mp->lowmsg, 1);
 	mp->hghoff = mp->hghmsg + 100;
-
-	/* for testing, allocate minimal necessary space */
-	/* mp->hghoff = max(mp->hghmsg, 1); */
-
-	/* Allocate space for status of each message. */
-
 	mp->msgstats = mh_xcalloc(MSGSTATSIZE(mp, mp->lowoff, mp->hghoff), 1);
-
-	/*
-	** Scan through the array of messages we've seen and
-	** setup the initial flags for those messages in the
-	** newly allocated mp->msgstats area.
-	*/
-	for (msgnum = 0; msgnum < mp->nummsg; msgnum++)
-		set_exists(mp, mi[msgnum]);
-
-	mh_free0();  /* We don't need this anymore */
+	for (i = 0; i < mp->nummsg; i++) {
+		set_exists(mp, m_atoi(dp[i]->d_name));
+		mh_free0(dp + i);
+	}
+	mh_free0();
 
 	/*
 	** Read and initialize the sequence information.
-- 
2.20.1

[mmh] folder_add and msgstats

[Philipp Takacs]

Hi

Because I have looked at the sequence handling a bit more and found a
bug in folder_add.

If the new message number is already in a sequence
folder_add will clear the flags in memmory but don't set the SEQMOD
flag. I have added this to clear_msg_flags. I have also removed the
clear_msg_flags loop from folder_read and folder_realloc. In most cases
we use calloc, so this isn't necesarry. At one place in folder_realloc
realloc is used. There I have add a memset.

comment?

Philipp
From cc0518925f0c38967307600ca864f352de95aa9e Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 18 Mar 2020 19:53:44 +0100
Subject: [PATCH 1/3] set SEQMOD if clear_msg_flags change the flags

This fixes a bug in folder_addmsg. If the sequences file containes
the new msgnum, seq_save won't override this.
---
 sbr/seq_msgstats.c|  2 ++
 test/tests/rcv/test-rcvstore-nounseen | 18 ++
 2 files changed, 20 insertions(+)
 create mode 100755 test/tests/rcv/test-rcvstore-nounseen

diff --git a/sbr/seq_msgstats.c b/sbr/seq_msgstats.c
index 7ffbde16..bfd7fb27 100644
--- a/sbr/seq_msgstats.c
+++ b/sbr/seq_msgstats.c
@@ -27,6 +27,8 @@ void
 clear_msg_flags(struct msgs *mp, int msgnum)
 {
 	assert_msg_range(mp, msgnum);
+	if (mp->msgstats[msgnum - mp->lowoff])
+		mp->msgflags |= SEQMOD;
 	mp->msgstats[msgnum - mp->lowoff] = 0;
 }
 
diff --git a/test/tests/rcv/test-rcvstore-nounseen b/test/tests/rcv/test-rcvstore-nounseen
new file mode 100755
index ..b97f8edc
--- /dev/null
+++ b/test/tests/rcv/test-rcvstore-nounseen
@@ -0,0 +1,18 @@
+#!/bin/sh
+##
+#
+# Test rcvstore
+#
+##
+
+. "$MH_TEST_COMMON"
+
+# check -nounseen
+printf "u: %s\n" $(basename $(mhpath b)) >> $MH_TEST_DIR/Mail/inbox/.mh_sequences
+runandcheck "rcvstore -nounseen <$MH_TEST_DIR/Mail/inbox/1" <From 764e9f71e179406d4eca3af12088dd33279a32df Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 18 Mar 2020 20:02:19 +0100
Subject: [PATCH 2/3] don't manuall clear the message flags

because we use calloc the message flags are cleared
after the allocation.
---
 sbr/folder_read.c | 8 
 1 file changed, 8 deletions(-)

diff --git a/sbr/folder_read.c b/sbr/folder_read.c
index 722ee2fd..011958f8 100644
--- a/sbr/folder_read.c
+++ b/sbr/folder_read.c
@@ -130,14 +130,6 @@ folder_read(char *name)
 
 	mp->msgstats = mh_xcalloc(MSGSTATSIZE(mp, mp->lowoff, mp->hghoff), 1);
 
-	/*
-	** Clear all the flag bits for all the message
-	** status entries we just allocated.
-	** TODO: use memset() ?
-	*/
-	for (msgnum = mp->lowoff; msgnum <= mp->hghoff; msgnum++)
-		clear_msg_flags(mp, msgnum);
-
 	/*
 	** Scan through the array of messages we've seen and
 	** setup the initial flags for those messages in the
-- 
2.20.1

From 937c01f62952147a8ff853d8a103dbe8465cb90e Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 18 Mar 2020 21:46:30 +0100
Subject: [PATCH 3/3] use memset to clear the msgstats in folder_realloc

---
 sbr/folder_realloc.c | 16 +---
 1 file changed, 1 insertion(+), 15 deletions(-)

diff --git a/sbr/folder_realloc.c b/sbr/folder_realloc.c
index 64a66409..781e8bdc 100644
--- a/sbr/folder_realloc.c
+++ b/sbr/folder_realloc.c
@@ -46,6 +46,7 @@ folder_realloc(struct msgs *mp, int lo, int hi)
 		** just realloc the message status array.
 		*/
 		mp->msgstats = mh_xrealloc(mp->msgstats, MSGSTATSIZE(mp, lo, hi));
+		memset(mp->msgstats + mp->hghoff - lo, 0, hi - mp->hghoff);
 	} else {
 		/*
 		** We are changing the offset of the message status
@@ -69,20 +70,5 @@ folder_realloc(struct msgs *mp, int lo, int hi)
 	mp->lowoff = lo;
 	mp->hghoff = hi;
 
-	/*
-	** Clear all the flags for entries outside
-	** the current message range for this folder.
-	*/
-	if (mp->nummsg > 0) {
-		for (msgnum = mp->lowoff; msgnum < mp->lowmsg; msgnum++)
-			clear_msg_flags(mp, msgnum);
-		for (msgnum = mp->hghmsg + 1; msgnum <= mp->hghoff; msgnum++)
-			clear_msg_flags(mp, msgnum);
-	} else {
-		/* no messages, so clear entire range */
-		for (msgnum = mp->lowoff; msgnum <= mp->hghoff; msgnum++)
-			clear_msg_flags(mp, msgnum);
-	}
-
 	return mp;
 }
-- 
2.20.1

Re: [mmh] show sequences racecondition and Previous-Sequence

[Philipp Takacs]

Hi

[2019-12-15 17:55] Philipp Takacs 
> I have looked at the problem with open pager for show and change the
> sequences during the pager is open. This happen often, if you show a
> message and recieve a new message.

after a hint from simon, that in my patch is still a racecondition I have
looked at it again and found the bug. A patch is attached.

Explenation: seq_read and seq_list totaly ignore mails without the
EXISTS flag. So when a new mail arrives during show has it's pager
open, the seq_read will see the new unseen flag and ignores it. Because
it doesn't know this mail exists. To avoid a total reread for some coner
cases I belive it's better just to ignore, if a mail exists or not.

comments?

Philipp
From 9bf8d356af4e3a6ad57d7010b5088c85f91935f8 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 18 Mar 2020 19:53:25 +0100
Subject: [PATCH 3/3] seq_read and seq_list also recognise msg which don't
 exist

This allows better handling of raceconditions, like a new mail gets
deliverd while a unreed mail is shown.
---
 sbr/seq_list.c |  7 +++
 sbr/seq_read.c |  9 +++--
 test/tests/show/test-unseen-update | 13 ++---
 3 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/sbr/seq_list.c b/sbr/seq_list.c
index 3606f6f3..65cc7aa0 100644
--- a/sbr/seq_list.c
+++ b/sbr/seq_list.c
@@ -54,12 +54,12 @@ seq_list(struct msgs *mp, char *seqname)
 
 	bp = buffer;
 
-	for (i = mp->lowmsg; i <= mp->hghmsg; ++i) {
+	for (i = mp->lowmsg; i <= mp->hghoff; ++i) {
 		/*
 		** If message doesn't exist, or isn't in
 		** the sequence, then continue.
 		*/
-		if (!does_exist(mp, i) || !in_sequence(mp, seqnum, i))
+		if (!in_sequence(mp, seqnum, i))
 			continue;
 
 		/*
@@ -91,8 +91,7 @@ seq_list(struct msgs *mp, char *seqname)
 		/*
 		** Scan to the end of this message range
 		*/
-		for (++i; i <= mp->hghmsg && does_exist(mp, i) &&
-in_sequence(mp, seqnum, i); ++i)
+		for (++i; i <= mp->hghoff && in_sequence(mp, seqnum, i); ++i)
 			;
 
 		if (i - j > 1) {
diff --git a/sbr/seq_read.c b/sbr/seq_read.c
index 041f6327..4cc42233 100644
--- a/sbr/seq_read.c
+++ b/sbr/seq_read.c
@@ -210,9 +210,14 @@ seq_init(struct msgs *mp, char *name, char *field)
 			** We iterate through messages in this range
 			** and flip on bit for this sequence.
 			*/
+			if (k > mp->hghoff) {
+if (!(mp = folder_realloc(mp, mp->lowoff, k))) {
+	adios(EX_OSERR, NULL, "unable to allocate folder storage");
+}
+mp->hghoff = k;
+			}
 			for (; j <= k; j++) {
-if (j >= mp->lowmsg && j <= mp->hghmsg &&
-		does_exist(mp, j))
+if (j >= mp->lowmsg && j <= mp->hghoff)
 	add_sequence(mp, i, j);
 			}
 		}
diff --git a/test/tests/show/test-unseen-update b/test/tests/show/test-unseen-update
index a4875c43..61e317ed 100644
--- a/test/tests/show/test-unseen-update
+++ b/test/tests/show/test-unseen-update
@@ -24,7 +24,14 @@ runandcheck "pick u" <

[mmh] show sequences racecondition and Previous-Sequence

[Philipp Takacs]

Hi

I have looked at the problem with open pager for show and change the
sequences during the pager is open. This happen often, if you show a
message and recieve a new message.

During that I found the Previous-Sequence again. I don't see a reason
why we need this feature. The usecases, I see, for the feature are better
handled with mark or the backlog of your shell. But I don't use this
feature.

The problem I see is, that sequences get racy, if you want to handle
your mail in parallel. I.e. have a mail open with show and get a new
mail. Does anyone see a usecase witch can't be handled with mark or the
backlog of your shell? Or an other reason to keep the Previous-Sequence?

So back to show I have two commits attached. One to remove the
Previous-Sequence and one to handle the unseen sequence better. If we
decide to keep the Previous-Sequence, I have to look at the race again.

The unseen sequence has it's own flag beside the sequences in the
msgs struct. The seq_setunseen() function updates the unseen sequence
in this struct. So I have just moved the seq_setunseen() call to the
end and added a seq_read() call before. This way it's still a bit racy
but not this hard. To fix the rest of the race we could add locking.

Comments?

Philipp
From b4b4f102566d1d3d0e4686ebf157d513a6a68f30 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Sun, 8 Dec 2019 17:54:29 +0100
Subject: [PATCH 1/2] remove previus sequence

unfinished, docs need update
---
 config/config.c   |  1 -
 h/mh.h|  1 -
 sbr/Makefile.in   |  2 +-
 sbr/m_draft.c |  1 -
 sbr/seq_setprev.c | 44 
 uip/burst.c   |  1 -
 uip/comp.c|  1 -
 uip/dist.c|  1 -
 uip/folder.c  |  1 -
 uip/forw.c|  1 -
 uip/mhlist.c  |  1 -
 uip/mhparam.c |  2 --
 uip/mhpath.c  |  2 --
 uip/mhshow.c  |  1 -
 uip/mhstore.c |  1 -
 uip/mhtest.c  |  1 -
 uip/packf.c   |  1 -
 uip/pick.c|  1 -
 uip/refile.c  |  2 --
 uip/repl.c|  1 -
 uip/rmm.c |  1 -
 uip/send.c|  1 -
 uip/sortm.c   |  1 -
 23 files changed, 1 insertion(+), 69 deletions(-)
 delete mode 100644 sbr/seq_setprev.c

diff --git a/config/config.c b/config/config.c
index 5e250414..0a63912d 100644
--- a/config/config.c
+++ b/config/config.c
@@ -89,7 +89,6 @@ char *seq_unseen = "u";
 char *seq_neg= "!";
 
 char *usequence = "Unseen-Sequence";
-char *psequence = "Previous-Sequence";
 char *nsequence = "Sequence-Negation";
 
 /* profile entries for storage locations */
diff --git a/h/mh.h b/h/mh.h
index 6337a0ca..649582d3 100644
--- a/h/mh.h
+++ b/h/mh.h
@@ -288,7 +288,6 @@ extern char *msgprot;
 extern char *nmhstorage;
 extern char *nsequence;
 extern char *profile;
-extern char *psequence;
 extern char *rcvdistcomps;
 extern char *replcomps;
 extern char *replgroupcomps;
diff --git a/sbr/Makefile.in b/sbr/Makefile.in
index f2310fb8..142a7f20 100644
--- a/sbr/Makefile.in
+++ b/sbr/Makefile.in
@@ -68,7 +68,7 @@ SRCS = addrsbr.c ambigsw.c brkstring.c  \
 	readconfig.c seq_add.c seq_bits.c  \
 	seq_del.c seq_getnum.c seq_list.c seq_nameok.c  \
 	seq_print.c seq_read.c seq_save.c seq_setcur.c  \
-	seq_setprev.c seq_setunseen.c signals.c  \
+	seq_setunseen.c signals.c  \
 	smatch.c snprintb.c strcasecmp.c  \
 	strindex.c trim.c trimcpy.c uprf.c vfgets.c fmt_def.c  \
 	mf.c utils.c m_mktemp.c seq_msgstats.c \
diff --git a/sbr/m_draft.c b/sbr/m_draft.c
index ee857900..050c7e51 100644
--- a/sbr/m_draft.c
+++ b/sbr/m_draft.c
@@ -53,7 +53,6 @@ m_draft(char *which)
 	*/
 	if (!m_convert(mp, which))
 		exit(EX_SOFTWARE);
-	seq_setprev(mp);
 
 	snprintf(buffer, sizeof(buffer), "%s/%s", mp->foldpath,
 			m_name(mp->lowsel));
diff --git a/sbr/seq_setprev.c b/sbr/seq_setprev.c
deleted file mode 100644
index f4304cc5..
--- a/sbr/seq_setprev.c
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
-** seq_setprev.c -- set the Previous-Sequence
-**
-** This code is Copyright (c) 2002, by the authors of nmh.  See the
-** COPYRIGHT file in the root directory of the nmh distribution for
-** complete copyright information.
-*/
-
-#include 
-#include 
-
-/*
-** Add all the messages currently SELECTED to
-** the Previous-Sequence.  This way, when the next
-** command is given, there is a convenient way to
-** selected all the messages used in the previous
-** command.
-*/
-
-void
-seq_setprev(struct msgs *mp)
-{
-	char **ap, *cp, *dp;
-
-	/*
-	** Get the list of sequences for Previous-Sequence
-	** and split them.
-	*/
-	if ((cp = context_find(psequence))) {
-		dp = mh_xstrdup(cp);
-		if (!(ap = brkstring(dp, " ", "\n")) || !*ap) {
-			mh_free0();
-			return;
-		}
-	} else {
-		return;
-	}
-
-	/* Now add all SELECTED messages to each sequence */
-	for (; *ap; ap++)
-		seq_addsel(mp, *ap, -1, 1);
-
-	mh_free0();
-}
diff --git a/uip/burst.c b/uip/burst.c
index 12d1f9d4..09a60d9f 100644
--- a/uip/burst.c
+++ b

Re: [mmh] pick rework the secound

[Philipp Takacs]

Hi

I have pushed this now.

@markus
you wanted to update the documentation for this.

Philipp

[2019-07-14 14:01] Philipp Takacs 
> [2019-07-09 19:47] markus schnalke 
> > [2019-07-09 16:04] Philipp Takacs 
> > >
> > > So if you find a test case is to little bug me to add one ;-).
> >
> > Some suggestions:
> >
> > - `pick -subject foo' for lower, upper and mixed case ``foo'' in
> >   the subject
> >
> > - `pick -subject übung' for encoded header
> >
> > - `pick -search übung' for encoded body (does fail currently)
> >
> > - `pick -search foo' for some messages have ``foo'' only in the
> >   subject and some have ``foo'' only in the body (It doesn't so
> >   much matter here where `-search' should search, but just that
> >   we can detect regression bugs.)
>
> Thanks for the testcases. With them I found some bugs in my
> implementation. A new version is attach, witch the testcases.
>
> > > [2019-07-08 16:15] markus schnalke 
> > > Also I think about adding regex also to the header name search.
> >
> > What usecases do you have in mind for that?
>
> I'm not shoure, this was just an idea I have, because it's easy to
> implement. So if I find it realy worth to implement, I'll write an
> other mail with the reasons.
>
> > > Also the search switch could search in all field values.
> >
> > It would be nice if `-search' would search in about everywhere in
> > the messages, so that you don't have to think about it ... like
> > grepping through the files.
> >
> > If we want specific only body searching, we rather add a `-body'
> > switch.
>
> Sounds good. I'll look at it.
>
> > > But this isn't
> > > well-thought-out, so maybe my plans how to implement this change.
> >
> > See how it goes during implementation. See what makes sense from
> > the implementation view.
>
> This was more the view from the feature set. Do we realy need this
> features and in this exact form or a bit diffrent. Implementing looks
> like it is possible just straight forword.
>
> Philipp

Re: [mmh] getfields() prototype

[Philipp Takacs]

Hi

[2019-08-31 12:16] Philipp Takacs 
> As Markus mentioned in the other mail we plan to implement a wrapper
> around m_getfld2(). I have started with one called getfields() and
> following interface:
>
> [...]
>
> So next step is clean up the error messages a bit. I have only copied
> them so they maybe not that good. Test the filter system. then implement
> getflieds on all tools.

I have now tested my filter implementation with whom. Today I will
work on replacing m_getfld() calls with getfields(). New version is
attached. Still based on my pick patch.

Philipp
From 710511ce7310e2b850f64243cd8c2c9c0cc41d59 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 28 Aug 2019 16:50:37 +0200
Subject: [PATCH 1/4] add EOH2 to m_getfld2 return

if you only intrested in the header you don't need to read the first
body line. This is also required for getfields.
---
 h/mh.h   | 1 +
 sbr/m_getfld2.c  | 5 -
 sbr/readconfig.c | 1 +
 sbr/seq_read.c   | 1 +
 uip/distsbr.c| 4 
 uip/mhbuild.c| 2 ++
 uip/mhl.c| 2 ++
 uip/mhparse.c| 2 ++
 uip/new.c| 1 +
 uip/pick.c   | 4 +++-
 uip/prompter.c   | 2 ++
 uip/rcvdist.c| 1 +
 uip/repl.c   | 1 +
 uip/scansbr.c| 3 ++-
 uip/slocal.c | 1 +
 uip/spost.c  | 2 ++
 uip/whom.c   | 1 +
 17 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/h/mh.h b/h/mh.h
index 6337a0ca..1aa58655 100644
--- a/h/mh.h
+++ b/h/mh.h
@@ -219,6 +219,7 @@ enum state {
 	FMTERR2 = -3,  /* Format error in message */
 	IOERR2 = -1,   /* Read error */
 	FLD2 = 0,  /* Header field returned */
+	EOH2,  /* End of Header */
 	BODY2, /* Body line returned */
 	FILEEOF2   /* Reached end of input file */
 };
diff --git a/sbr/m_getfld2.c b/sbr/m_getfld2.c
index b9a618d1..7cc8689c 100644
--- a/sbr/m_getfld2.c
+++ b/sbr/m_getfld2.c
@@ -74,7 +74,7 @@ m_getfld2(enum state s, struct field *f, FILE *msg)
 		if (ret == FLD2 && is_separator(tmpline)) {
 			/* header/body separator found */
 			free(tmpline);
-			return m_getfld2(BODY2, f, msg);
+			return EOH2;
 		}
 
 		f->namelen = copyname(f->name, tmpline);
@@ -129,6 +129,9 @@ m_getfld2(enum state s, struct field *f, FILE *msg)
 		free(tmpline);
 		return ret;
 
+	case EOH2:
+		ret = BODY2;
+		/* FALL */
 	case BODY2:
 		*f->name = '\0';
 		f->namelen = 0;
diff --git a/sbr/readconfig.c b/sbr/readconfig.c
index 91d2b4ed..431530ef 100644
--- a/sbr/readconfig.c
+++ b/sbr/readconfig.c
@@ -75,6 +75,7 @@ readconfig(struct node **npp, FILE *ib, char *file, int ctx)
 			advise(NULL, "%s is poorly formatted", file);
 			state = FLD2;
 			continue;
+		case EOH2:
 		case BODY2:
 			adios(EX_CONFIG, NULL, "no blank lines are permitted in %s",
 	file);
diff --git a/sbr/seq_read.c b/sbr/seq_read.c
index 041f6327..ce56ebca 100644
--- a/sbr/seq_read.c
+++ b/sbr/seq_read.c
@@ -83,6 +83,7 @@ seq_public(struct msgs *mp)
 			seq_init(mp, mh_xstrdup(f.name), trimcpy(f.value));
 			continue;
 
+		case EOH2:
 		case BODY2:
 			adios(EX_CONFIG, NULL, "no blank lines are permitted in %s",
 	seqfile);
diff --git a/uip/distsbr.c b/uip/distsbr.c
index 2662c02d..913ade01 100644
--- a/uip/distsbr.c
+++ b/uip/distsbr.c
@@ -67,6 +67,8 @@ distout(char *drft, char *msgnam, char *backup)
 			fprintf(ofp, "%s: %s", f.name, f.value);
 			break;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			for (dp = f.value; *dp; dp++) {
 if (!isspace(*dp)) {
@@ -171,6 +173,8 @@ ready_msg(char *msgnam)
 			fprintf(ofp, "%s: %s", f.name, f.value);
 			break;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			fclose(ofp);
 
diff --git a/uip/mhbuild.c b/uip/mhbuild.c
index a233c685..13dc8510 100644
--- a/uip/mhbuild.c
+++ b/uip/mhbuild.c
@@ -376,6 +376,8 @@ build_mime(char *infile)
 
 			continue;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			fseek(in, (long) (-strlen(f.value)), SEEK_CUR);
 			break;
diff --git a/uip/mhl.c b/uip/mhl.c
index fa83602d..a09cd28b 100644
--- a/uip/mhl.c
+++ b/uip/mhl.c
@@ -683,6 +683,8 @@ mhlfile(FILE *fp, char *mname, int ofilen, int ofilec)
 			}
 			continue;
 
+		case EOH2:
+			continue;
 		case BODY2:
 		case FILEEOF2:
 			column = 0;
diff --git a/uip/mhparse.c b/uip/mhparse.c
index 2b72e7bc..2d8e8ead 100644
--- a/uip/mhparse.c
+++ b/uip/mhparse.c
@@ -275,6 +275,8 @@ get_content(FILE *in, char *file, int toplevel)
 			ct->c_begin = ftell(in) + 1;
 			continue;
 
+		case EOH2:
+			continue;
 		case BODY2:
 			ct->c_begin = ftell(in) - strlen(f.value);
 			break;
diff --git a/uip/new.c b/uip/new.c
index 21435b34..cd206e18 100644
--- a/uip/new.c
+++ b/uip/new.c
@@ -137,6 +137,7 @@ get_msgnums(char *folder, char *sequences[])
 			}
 			continue;
 
+		case EOH2:
 		case BODY2:
 			adios(EX_DATAERR, NULL, "no blank lines are permitted in %s", seqfile);
 			/* FALL */
diff --git a/uip/pick.c b/uip/pick.c
index 8f28f354..a2a03cfb 100644
--- a/uip/

Re: [mmh] multipart/related inside multipart/alternative

[Philipp Takacs]

Hi

I have pushed the mentioned commit.

Philipp

[2019-09-04 10:53] Philipp Takacs 
> Hi
>
> I have recied a mail with folowing mime strukture:
>
> msg part  type/subtype  size description 
>  987   multipart/alternative1648K
>  1 multipart/related1646K
>  1.1   text/html  10K
>  1.2   image/png 464K
>  1.3   image/png 413K
>  1.4   image/png 331K
>  2 text/plain1095
>
> I wanted to see the part 1.1 and used ``show -part 1.1'' and got
> the folowing error message:
>
> > show: don't know how to display any of the contents
> >   (content multipart/alternative in message 987)
>
> The problem is in show_multi() of uip/mhshowsbr line 567. Show
> there checks if it's inside a multipart and if the multipart is
> known. But RFC 2046 clearly says: 
>
> > Any "multipart" subtypes that an implementation does not recognize
> > must be treated as being of subtype "mixed".
>
> The problem is fixed in nmh with commit 298062b03.
> I would cherry pick this commit.
>
> Philipp

[mmh] multipart/related inside multipart/alternative

[Philipp Takacs]

Hi

I have recied a mail with folowing mime strukture:

msg part  type/subtype  size description 
 987   multipart/alternative1648K
 1 multipart/related1646K
 1.1   text/html  10K
 1.2   image/png 464K
 1.3   image/png 413K
 1.4   image/png 331K
 2 text/plain1095

I wanted to see the part 1.1 and used ``show -part 1.1'' and got
the folowing error message:

> show: don't know how to display any of the contents
>   (content multipart/alternative in message 987)

The problem is in show_multi() of uip/mhshowsbr line 567. Show
there checks if it's inside a multipart and if the multipart is
known. But RFC 2046 clearly says: 

> Any "multipart" subtypes that an implementation does not recognize
> must be treated as being of subtype "mixed".

The problem is fixed in nmh with commit 298062b03.
I would cherry pick this commit.

Philipp

Re: [mmh] pick rework the secound

[Philipp Takacs]

Hi

[2019-07-09 19:47] markus schnalke 
> [2019-07-09 16:04] Philipp Takacs 
> >
> > So if you find a test case is to little bug me to add one ;-).
>
> Some suggestions:
>
> - `pick -subject foo' for lower, upper and mixed case ``foo'' in
>   the subject
>
> - `pick -subject übung' for encoded header
>
> - `pick -search übung' for encoded body (does fail currently)
>
> - `pick -search foo' for some messages have ``foo'' only in the
>   subject and some have ``foo'' only in the body (It doesn't so
>   much matter here where `-search' should search, but just that
>   we can detect regression bugs.)

Thanks for the testcases. With them I found some bugs in my
implementation. A new version is attach, witch the testcases.

> > [2019-07-08 16:15] markus schnalke 
> > Also I think about adding regex also to the header name search.
>
> What usecases do you have in mind for that?

I'm not shoure, this was just an idea I have, because it's easy to
implement. So if I find it realy worth to implement, I'll write an
other mail with the reasons.

> > Also the search switch could search in all field values.
>
> It would be nice if `-search' would search in about everywhere in
> the messages, so that you don't have to think about it ... like
> grepping through the files.
>
> If we want specific only body searching, we rather add a `-body'
> switch.

Sounds good. I'll look at it.

> > But this isn't
> > well-thought-out, so maybe my plans how to implement this change.
>
> See how it goes during implementation. See what makes sense from
> the implementation view.

This was more the view from the feature set. Do we realy need this
features and in this exact form or a bit diffrent. Implementing looks
like it is possible just straight forword.

Philipp
From f5e37993d288c40b1d6a02d4ac0faed57d021561 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Mon, 10 Jun 2019 03:17:36 +0200
Subject: [PATCH 1/3] pick matching rework

the last rewrite of the matching implementation was a bit overcomplex.
Now the structures are better to read and without function pointers.

Also now the -not switch is working again.
The early return true in the last version causes this.
Now the leafs of the matching tree remeber, if the message has matched.
---
 test/tests/pick/test-case |  48 +++
 test/tests/pick/test-complex  |  14 +
 test/tests/pick/test-encoded-body |  40 +++
 test/tests/pick/test-normal-output|  15 +-
 test/tests/pick/test-output-on-error  |  18 +-
 test/tests/pick/test-rfc2047  |  41 ++-
 test/tests/pick/test-search   |  39 +++
 test/tests/pick/test-stderr   |  15 +-
 test/tests/pick/test-thread-without-msgid |  19 +-
 uip/pick.c| 558 +-
 10 files changed, 417 insertions(+), 390 deletions(-)
 create mode 100644 test/tests/pick/test-case
 create mode 100644 test/tests/pick/test-complex
 create mode 100644 test/tests/pick/test-encoded-body
 create mode 100644 test/tests/pick/test-search

diff --git a/test/tests/pick/test-case b/test/tests/pick/test-case
new file mode 100644
index ..fc06ef2d
--- /dev/null
+++ b/test/tests/pick/test-case
@@ -0,0 +1,48 @@
+#!/bin/sh
+##
+#
+# Test that the -thread option works.
+#
+##
+
+. "$MH_TEST_COMMON"
+
+#lower
+cat > `mhpath b` << '!'
+From: al...@example.org
+subject: fo
+!
+lower=$(pick l)
+
+#upper
+cat > `mhpath b` << '!'
+From: al...@example.org
+SUBJECT: FO
+!
+upper=$(pick l)
+
+
+#mixed
+cat > `mhpath b` << '!'
+From: al...@example.org
+sUbJEcT: FOooOo
+!
+mixed=$(pick l)
+
+runandcheck 'pick --subject fo' < $msgfile < $expected_out < $expected_out < $expected_err
-
-pick > $actual_out 2> $actual_err
-diff -u $expected_err $actual_err
-diff -u $expected_out $actual_out
+!
diff --git a/test/tests/pick/test-output-on-error b/test/tests/pick/test-output-on-error
index 8d962240..07231f04 100644
--- a/test/tests/pick/test-output-on-error
+++ b/test/tests/pick/test-output-on-error
@@ -5,20 +5,10 @@
 #
 ##
 
-expected_err=$MH_TEST_DIR/$$.expected_err
-expected_out=$MH_TEST_DIR/$$.expected_out
-actual_err=$MH_TEST_DIR/$$.actual_err
-actual_out=$MH_TEST_DIR/$$.actual_out
+. "$MH_TEST_COMMON"
 
 # A zero should go to standard out to protect other programms
-cat > $expected_out < $expected_err < $actual_out 2> $actual_err
-diff -u $expected_err $actual_err
-diff -u $expected_out $actual_out
+0
+!
diff --git a/test/tests/pick/test-rfc2047 b/test/tests/pick/test-rfc2047
index b89a71ab..3eab2535 100644
--- a/test/tests/pick/test-rfc2047
+++ b/test/tests/pick/test-rfc2047
@@ -5,28 +5,3

Re: [mmh] pick rework the secound

[Philipp Takacs]

[2019-07-08 16:15] markus schnalke 
> Unfortunately, I haven't had a closer look, no testing and probably
> won't find the time to do so. What you wrote sounds good. I
> appreciate that you added a testcase to catch regression bugs!

I should add more tests before I rewrite a whole tool or add a
feature. Then I can find bugs before I commit. So if you find a
test case is to little bug me to add one ;-).

> One thing I haven't noticed up to now, regarding this now deleted
> comment in the old version of uip/pick.c:
>
>   ** Unfortunately,
>   ** pick advertises that lowercase characters matches characters of both
>   ** cases.  Since re_exec() doesn't exhibit this behavior, we are stuck
>   ** with this version.
>
> As you couldn't keep this case-sensitivity behavior with regex(3),
> you switched to case-insensitive matching only. That was probably
> the best choice. But we need to document this in the HISTORY section
> of the pick manpage.
>
>
> I think we also need to document there that `-search' does no longer
> search the header fields. AFAICS there is no longer a possibility to
> search all header values and the body. You have to list each header
> field explicitely. You neither can cover it with grep(1) as the
> header values and body text are likely to be encoded. But, well, we
> can live without it, I'd say. -- Did I understand the situation
> corrently?

Yes, this is the current situation. Aditional: Encoding wasn't handled
at all before my rewrite. The search switch still don't handle
encoding. I have plans to add this.

Also I think about adding regex also to the header name search. Also
the search switch could search in all field values. But this isn't
well-thought-out, so maybe my plans how to implement this change.

Philipp

[mmh] New build system

[Philipp Takacs]

Hi

As said in an other mail I'm currently working on a build system
replacement. The idea is that we have a Makefile and a config.mk. The
Makefile defines all the defaults and does the build and installation.
In the config.mk a user can set some config like compiler, prefix, ...

Currently I have a problem with the config.h file. This file is
auto generated by autoconf. I'm not sure if we really need this. It
would be nice if all users could send me there config.h file and on
which system this is generated. Then I can see which options we really
need and which are on all systems the same.

The current version of my Makefile is attached. This is just that you
see the idea. Feel free to send feedback and report problems, but have
in mind this is work in progress.

Philipp
.PHONY: all clean version.o

AWK?=awk
SIGNAL_H?=/dev/null

VERSION=`sed q VERSION`
LORDER?= lorder
RANLIB?= ranlib
TSORT?= tsort
INSTALL?= install
LN?= ln
SED?= sed

PREFIX?=/usr/local/mmh
BINDIR?=$(PREFIX)/bin
ETCDIR?=$(PREFIX)/etc
SBINDIR?=$(PREFIX)/sbin
MANDIR?=$(PREFIX)/share/man
SENDMAILPATH?=/usr/sbin/sendmail
MAILSPOOL?=/var/spool/mail

manext1 = 1$(MANSECTION)
manext5 = 5$(MANSECTION)
manext7 = 7$(MANSECTION)
manext8 = 8$(MANSECTION)

.man1.$(manext1):
	$(SED) -f man/man.sed $< > $@

.man5.$(manext5):
	$(SED) -f man/man.sed $< > $@

.man7.$(manext7):
	$(SED) -f man/man.sed $< > $@

.man8.$(manext8):
	$(SED) -f man/man.sed $< > $@

MAN1SRC = ali. anno. burst. comp. dist. flist. flists. folder. folders. \
   forw. inc. mark. mhbuild. mhl. mhlist. mhsign. mhpgp. mmh. mmhwrap. \
   mhmail. mhparam. mhpath. mhstore. new. fnext. \
   fprev. unseen. next. packf. pick. prev. prompter. rcvdist. rcvpack. \
   rcvstore. refile. repl. rmf. rmm. scan. send. sendfiles. \
   show. slocal. sortm. whatnow. whom. whatnow2.

MAN5SRC = mh-alias. mh-format. mh-mail. mh-profile.

MAN7SRC = mmh-intro. mh-chart. mh-draft. mh-sequence.

MAN8SRC = spost. ap. dp. fmtdump.

MAN1 = $(MAN1SRC:.=.$(manext1))
MAN5 = $(MAN5SRC:.=.$(manext5))
MAN7 = $(MAN7SRC:.=.$(manext7))
MAN8 = $(MAN8SRC:.=.$(manext8))

CFLAGS=-I. $(EXTRACFLAGS) -DVERSION=\"$(VERSION)\" -DNMHETCDIR='"$(ETCDIR)"' -DSENDMAILPATH='"$(SENDMAILPATH)"' -DMAILSPOOL='"$(MAILSPOOL)"' -DHAVE_CONFIG_H

LIBS=-lmh -ltermcap
LDFLAGS=$(EXTRALDFLAGS) -L./sbr -Wl,--as-needed

.o:
	$(CC) $(LDFLAGS) $^$> $(LIBS) -o $@
.c.o:
	$(CC) -c $(CFLAGS) $< -o $@

LIBSRC = sbr/dtimep.c sbr/charset.c sbr/readconfig.c sbr/folder_realloc.c sbr/fmt_def.c sbr/ambigsw.c sbr/getarguments.c sbr/m_atoi.c sbr/lock_file.c sbr/getanswer.c sbr/cpydata.c sbr/fmt_new.c sbr/uprf.c sbr/pidwait.c sbr/print_help.c sbr/trim.c sbr/seq_read.c sbr/folder_addmsg.c sbr/seq_setcur.c sbr/m_getfld2.c sbr/concat.c sbr/seq_setunseen.c sbr/signals.c sbr/m_gmprot.c sbr/seq_save.c sbr/utils.c sbr/gans.c sbr/folder_delmsgs.c sbr/fmt_rfc2047.c sbr/seq_bits.c sbr/putenv.c sbr/strindex.c sbr/encode_rfc2047.c sbr/brkstring.c sbr/strcasecmp.c sbr/fmt_addr.c sbr/getans.c sbr/folder_read.c sbr/norm_charmap.c sbr/folder_free.c sbr/dtime.c sbr/parse_msgs.c sbr/trimcpy.c sbr/m_name.c sbr/error.c sbr/mts.c sbr/execprog.c sbr/makedir.c sbr/context_del.c sbr/seq_del.c sbr/vfgets.c sbr/smatch.c sbr/seq_msgstats.c sbr/crawl_folders.c sbr/print_sw.c sbr/print_version.c sbr/seq_setprev.c sbr/pidstatus.c sbr/fmt_scan.c sbr/m_convert.c sbr/snprintb.c sbr/getthreadid.c sbr/unquote.c sbr/seq_print.c sbr/m_mktemp.c sbr/mf.c sbr/context_find.c sbr/context_read.c sbr/fmt_compile.c sbr/path.c sbr/mhbasename.c sbr/context_replace.c sbr/seq_list.c sbr/m_draft.c sbr/seq_add.c sbr/ext_hook.c sbr/context_save.c sbr/addrsbr.c sbr/seq_nameok.c sbr/seq_getnum.c config/config.c

LIBOBJ = $(LIBSRC:.c=.o)

SRC = uip/aliasbr.c uip/fmtdump.c uip/mhmisc.c uip/pick.c uip/slocal.c uip/ali.c uip/folder.c uip/mhoutsbr.c uip/prompter.c uip/sortm.c uip/anno.c uip/forw.c uip/mhparam.c uip/rcvdist.c uip/spost.c uip/ap.c uip/inc.c uip/mhparse.c uip/rcvpack.c uip/termsbr.c uip/burst.c uip/mark.c uip/mhpath.c uip/rcvstore.c uip/whatnow.c uip/comp.c uip/mhbuild.c uip/mhshow.c uip/refile.c uip/whatnowproc.c uip/dist.c uip/mhfree.c uip/mhshowsbr.c uip/repl.c uip/whom.c uip/distsbr.c uip/mhl.c uip/mhstore.c uip/rmf.c uip/dp.c uip/mhlist.c uip/mhtest.c uip/rmm.c uip/dropsbr.c uip/mhlistsbr.c uip/new.c uip/scansbr.c uip/flist.c uip/mhmail.c uip/packf.c uip/send.c

OBJ = $(SRC:.c=.o)

CMDS = uip/ali uip/anno uip/burst uip/comp uip/dist uip/flist uip/folder uip/forw uip/mmh uip/mark uip/mhbuild uip/mhl uip/mhsign uip/mhpgp uip/mhlist uip/mhmail uip/mhparam uip/mhpath uip/mhstore uip/new uip/packf uip/pick uip/print-mimetype uip/prompter uip/rcvdist uip/rcvpack uip/rcvstore uip/refile uip/repl uip/rmf uip/rmm uip/send uip/sendfiles uip/show uip/slocal uip/sortm uip/spost uip/whatnow uip/whatnow2 uip/whom 

LCMDS = uip/flists uip/folders uip/next uip/prev uip/fnext uip/fprev uip/unseen uip/scan

MISC = uip/ap uip/dp uip/fmtdump uip/mhtest uip/mmhwrap

all: $(CMDS)

$(LIBOBJ): h/mh.h
$(OBJ): 

Re: [mmh] [PATCH] Remove redundant check

[Philipp Takacs]

[2019-02-02 10:26] Dmitry Bogatov 
>   * sbr/pidstatus.c(pidstatus): signum variable, returned by WTERMSIG macro
> is always positive.

The problem is WTERMSIG don't promise this. So if a bad implementation
return a negative number we have undefined behavior. The test is not
expensive. So I would like to keep this test.

Philipp

Re: [mmh] mmh-0.4 is released! (meillo's mail handler)

[Philipp Takacs]

[2019-02-01 14:26] Dmitry Bogatov 
> [2019-01-31 21:47] Philipp Takacs 
> > > 2. Hyphen usage in manpage
> >
> > I don't understand this patch. You change the etc/mhl.reply which
> > is a config file for reply not a manpage.
>
> It gets included into repl(1mh) during build.

I did't see this. The patch is applied.

Philipp

Re: [mmh] mmh-0.4 is released! (meillo's mail handler)

[Philipp Takacs]

[2019-01-08 12:37] Dmitry Bogatov 
> [2019-01-06 20:40] Philipp Takacs 
> > After more then a year I can announce the release of mmh-0.4
> > 
> > http://marmaro.de/prog/mmh/files/mmh-0.4.tar.gz
>
> Good news. You seems to forgot to upload signature (mmh-0.4.tar.gz.asc).
> Would you be so kind to do it?

Hi sorry for the delay. The signature is attached. My new pgp key is
availibel at the keyservers and on my website:

https://www.bureaucracy.de/philipp/key.asc

Philipp


mmh-0.4.tar.gz.asc
Description: mmh-0.4.tar.gz.asc

[mmh] mmh-0.4 is released! (meillo's mail handler)

[Philipp Takacs]

Hi everyone

After more then a year I can announce the release of mmh-0.4

http://marmaro.de/prog/mmh/files/mmh-0.4.tar.gz

The important changes are:
- pick now has thread support
- pick and scan merged into one binary
- mhl globing support for filter out some header fields
- mhl trim (ported from nmh)
- whatnow2 is finished for replacing whatnow
- whom supports message argument

Have a look at the NEWS file in the tarball for more information about
the changes. The full changelog is in the ChangeLog file in the tarball.
All changes and the current developer version can be found in the git
at http://git.marmaro.de/mmh

mmh 0.4 has in total 33,072 SLOC. This is a bit more then then mmh 0.3
which has 32,645 SLOC.

SLOCChange  Directory   SLOC-by-Language (Sorted)
19159   (+100)  uip ansic=18320,sh=839
7061(+102)  sbr ansic=6987,awk=74
3236(+219)  testsh=3236
2753top_dir sh=2753
764 h   ansic=764
60  config  ansic=60
38  man sh=38

The most new code comes from new features in mhl and the scan/pick
merge. On the other side the whole scan.c file is removed. Also whatnow
is still in the count. (All numbers calculated by sloccount.)

Thanks to everyone who contributed to this release. Some of the work is
ported from nmh. The commits:

47  Philipp Takacs 
22  markus schnalke 
 7  Dmitry Bogatov 
 5  c_14 
 4  Vasilii Kolobkov 
 2  Larry Hynes 
 1  David Levine 
 1  mor0i...@gmail.com 

Further contributions on the mailing list by: Jan/subpath, and Link
Satonaka.

Philipp

Re: [mmh] mmh 0.4 whatnow2 and pick

[Philipp Takacs]

[2018-12-30 04:22] Philipp Takacs 
> I'm a bit after my schedule, but I think we still can handle
> the release at 06.01.2018.

So my last featers are commted and pushed (a bit late I know). So we are
ready to test. The Testsuit for the new features is a bit small, so
feel free to write some more tests.

I have changed my mind about the deprecation of whatnow. The best
option I see is to add a warning to whatnow and the manpage mention
whatnow2. The warning could look somithing like this.

> whatnow is deprecated and will removed in mmh-0.5. Consider switching to 
> whatnow2.

If you have some comment to the documantation let me know.

I'll also do some test today, but see no problem with the relase at
06.01.2019.

Philipp

Ps: I hope you all had some freetime during the hollydays
and I wish you all a happy new year.

Re: [mmh] mmh 0.4 whatnow2 and pick

[Philipp Takacs]

[2019-01-02 15:22] c_14 
> [2018-12-30 04:22] Philipp Takacs 
> > I have merged the pick branch. Please test it. Also some aditinonal test
> > would be nice.
> pick -thread aborts with "free(): invalid pointer"

Thanks for the report. There were two problems in the -thread switch.
The one you found was a free of static memory. The -thread switch has
set the header field of the struct grep_data to "message-id" and
"references". This is now copied. Maybe I change after the release
while rewriting the argument parsing.

An other segfault has happend, if the message has no massage-id or
references field. The getthreadid() has then accessed uniniziliced
data. This is also fixed now.

Your attached test has still not the expected result. The problem is
the test messages don't have a message-id. Then pick the -thread switch
was allways true so all messages are printed. pick now fails if a message
selected by the -thread switch has no thread-id.

Thanks also for the tests and the gdb output this was helpfull.

Philipp

Re: [mmh] mmh 0.4 whatnow2 and pick

[Philipp Takacs]

Hi

I'm a bit after my schedule, but I think we still can handle
the release at 06.01.2018.

[2018-12-17 01:40] Philipp Takacs 
> pick is not merged at the moment, because of the documentation. I work
> on this at the moment. The branch is finished to work with so feel free
> to merge it with the master and test it. I've been working with the new
> pick for a few weeks now and have not seen any problems.

I have merged the pick branch. Please test it. Also some aditinonal test
would be nice.

> So whatnow2 is mostly done. Markus asked for a way to store the meta
> information (like last editor, ...) in the draft itself. I also have
> a plan to implement this for mmh 0.4. Then only replace whatnow with
> whatnow2 as default and add a deprecated notice to whatnow.

The mosts thinks are done only the filter for whatnow2 is missing.
I had to add a raw and a globbing option for mhl.

Philipp

Re: [mmh] whatnow, whatnow2, draft metadata

[Philipp Takacs]

Hi

[2018-12-17 10:06] markus schnalke 
> Philipp brought the whatnow topic up again (in private
> conversation). I realized that I still don't use whatnow2, which
> is a bad thing when we would want to drop whatnow in the next
> release. Our last discussions on the topic are a year or so ago.

Have you tested it now and what do you think about it now?
Also to other users(are there any ;-)) what do you think about
whatnow2?

> Bear with me that I start afresh and try to summarize the
> situation and think aload about ways to go ...
>
>
> Philipp wrote that in his eyes, the old whatnow would be dead and
> we should switch to whatnow2 or something similar.
>
>
> If we want to go this way, I need to drop whatnow from my workflow
> and start using whatnow2 or alternatives, now. ;-)
>
> Thus I had a(nother) look at whatnow2. AFAICS, it currently works
> on the current message in the draft folder only. This collides
> with my typical work scheme of drafting multiple messages in
> different tmux shells in parallel. Hence I would need to switch
> the current message in the draft folder explicitely all the time.
>
> (It might be possible to solve that problem with different
> contexts and private sequences, but I'm not sure, as I've never
> used private messages.)

It should work with a simple ``export MMHC=$(mktemp)'' in your bashrc.
I'll test this in the next days. Maybe we can add a way to allow
whatnow2 to change the current message in +drafts.

> In the core of the problem lies the question: How does whatnow2
> know which draft to work on?

This is the same question as how does next know which is the next
message. We have a context and I don't see a reason to don't use
it.

> The answer is: It cannot know it, unless we explicitely tell it.
> The current message is the only automatic hint we have. This
> however collides with parallel working on multiple drafts.

I would say every tool in mmh relays on the context why should the tool
for drafthandling work diffrent.

> But let us look on the topic from a different angle: Let's think
> without having any sort of whatnow/whatnow2. Comp/repl/forw/dist
> would invoke the first editor on their own and after it exited
> just leave the draft in the draft folder and exit to the shell.
>
> Then we need replacements to these whatnow commands:
>
> - edit already covered by `comp -use', we just need to care
>for last-editor handling
>
> - list already covered by `show' (no separate listproc
>necessary, i'd say)
> - send already covered by `send'
> - delete   already covered by `rmm +drafts'
> - refile   already covered by `refile -src +drafts'
>
> - display  new wrapper necessary for `show', which honors
>mhaltmsg (i.e. soon a header in the draft)
> - attach   new wrapper necessary for `anno'
> - alistnew wrapper necessary for `anno'
> - detach   new wrapper necessary for `anno'
>
>
> Thus the question: Why do we want whatnow2 at all? Wouldn't it be
> much better to go for the above changes right away?

All this tools would also solve the problem witch draft is the current
draft.

But more importend to avoid complexity in the mmh tools. I don't know
how complex your solution would be, but I belive it would be more then
271 lines of shell. I see your idear but I belive this would be more
complex then a simple wrapper for draft handling.

An other problem I see is that we already have 49 binaries in the
path I would like to avoid add more.

Don't get me wrong I'm not in generale against this idea, but I see
some problem.

> First, we should to move the draft environment variables into the
> draft. Currently, Philipp uses a separate meta file for that in
> whatnow2. I think we already had consensus for putting it as
> mmh-* headers (or similar) in the draft. I'd say that this is
> what we should do at first, because this change is conceptionally
> worthwhile anyways, independent of any whatnow rework.

I think you missunderstand me in this point. I like the cozept of meta
files. But I have no problem to write this in a way that the meta file
is the mail itself or add some meta information in the mail itself.

> @Philipp: Are you already working on this or should I start with
> it? [This I wrote on 2018-11-25 -- I was really keen to start
> working on it ... but personal life made it impossible just at
> that time. :-/ ]

I'm working on a solution to manage the meta data in the draft itself,
but it's a bit more complex then I thoug. Because mhl didn't allow to
just filter some header fields out and leave the rest of the message
as it is. This problems are now solved I only need to implement the
filter in whatnow2.

> Second, cover the yet uncovered tasks: display, attach, detach,
> and alist. Two ways come to mind to do so:

I'm not shoure about this. On the one hand it sound simple to use.
But on the outher hand: Add more commands to the path is problaly
not the solution we want.

> 1) Add each of those commands as a tool 

[mmh] mmh 0.4 whatnow2 and pick

[Philipp Takacs]

Hi all

The last release of mmh is a bit old and some things have changed.
Therefore I plan a new release of mmh. The big changes for mmh 0.4 are
whatnow2, the merge pick and scan and the redesign of the pick matching.
Also some other features and bug fixes have been done since mmh 0.3.
Talking about features and bug fixes, I want thanks to all for the work
on mmh.

pick is not merged at the moment, because of the documentation. I work
on this at the moment. The branch is finished to work with so feel free
to merge it with the master and test it. I've been working with the new
pick for a few weeks now and have not seen any problems.

So whatnow2 is mostly done. Markus asked for a way to store the meta
information (like last editor, ...) in the draft itself. I also have
a plan to implement this for mmh 0.4. Then only replace whatnow with
whatnow2 as default and add a deprecated notice to whatnow.

These are the features I have on my list for mmh 0.4. Do you have
any features which should be committed or bugs[0] which aren't fixed
now?

I have looked at the calendar and unfortunately this year is not
possible anymore. My release plan is the following:

23.12.2018 finish whatnow2 and pick
30.12.2018 feature freeze
06.01.2019 release

Philipp

[0] And I have missed (sorry for that). I'll look through my mails, but
feel free to write a mail to remember me about a bug/feature request.

Re: [mmh] mhl trailing white space

[Philipp Takacs]

[2018-11-22 08:35] markus schnalke 
> [2018-11-19 03:41] Philipp Takacs 
> >
> > Hi all
> > 
> > After I have looked at the ali bug, I have looked at the other tests
> > which are skipped. I have noticed the mhl with space feature is not this
> > hard to port from nmh. So I have done this. A patch for this is attached.
>
> Seems to be very useful to have that feature. I've always been
> dissatisfied with lines containing only "> " in my replies.
>
> Is there a need to update some files in etc/ to use the rtrim
> function by default?
>
> Some comments on the patch, see below.

I have implemented your comments and the updated patches are
attached.

Philipp
From 69e999074b30eb34178a7647582cdc740631813c Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Mon, 19 Nov 2018 03:04:12 +0100
Subject: [PATCH 1/2] Trailing withspace handling in mhl

If a component has trailing whitespace, e.g., body:component="> ", mhl
now trims that whitespace off when filtering blank text lines. Also add
a rtrim flag to mhl. This flag removes trailing whitespace in filtert text.

The implementation was done by David Levine  for nmh
commit ea0a6d8112a918809bd03d8126411040d22f2bb8
commit 1aa8f3e11e6d83ee4806abaa132ab9466f02ca5f
---
 h/prototypes.h|  1 +
 man/mhl.man1  |  2 ++
 sbr/trim.c| 12 
 test/tests/mhl/test-mhl-flags |  1 -
 uip/mhl.c | 33 +
 5 files changed, 40 insertions(+), 9 deletions(-)

diff --git a/h/prototypes.h b/h/prototypes.h
index 69556f94..65265822 100644
--- a/h/prototypes.h
+++ b/h/prototypes.h
@@ -107,6 +107,7 @@ char *snprintb(char *, size_t, unsigned, char *);
 int stringdex(char *, char *);
 char *toabsdir(char *);
 char *trim(unsigned char *);
+char *rtrim(char *);
 char *trimcpy(unsigned char *);
 int unputenv(char *);
 void unquote_string(const char *input, char *output);
diff --git a/man/mhl.man1 b/man/mhl.man1
index a80d6bf0..9785fe27 100644
--- a/man/mhl.man1
+++ b/man/mhl.man1
@@ -136,6 +136,8 @@ leftadjust	flag	strip off leading whitespace on each
 noleftadjust	flag	don't leftadjust
 compress	flag	change newlines in text to spaces
 nocompress	flag	don't compress
+rtrim	flag	trim whitespace at end of text lines
+nortrim	flag	retain whitespace at end of text lines (default)
 split	flag	don't combine multiple fields into
 		a single field
 nosplit	flag	combine multiple fields into
diff --git a/sbr/trim.c b/sbr/trim.c
index 249faf9c..cbf90984 100644
--- a/sbr/trim.c
+++ b/sbr/trim.c
@@ -31,3 +31,15 @@ trim(unsigned char *cp)
 
 	return cp;
 }
+
+char *
+rtrim(char *cp)
+{
+	char *sp = cp + strlen(cp) - 1;
+
+	while (sp >= cp && isspace(*sp)) {
+		sp--;
+	}
+	*++sp = '\0';
+	return cp;
+}
diff --git a/test/tests/mhl/test-mhl-flags b/test/tests/mhl/test-mhl-flags
index 7220e6cb..578fd37c 100755
--- a/test/tests/mhl/test-mhl-flags
+++ b/test/tests/mhl/test-mhl-flags
@@ -4,7 +4,6 @@
 
 . "$MH_TEST_COMMON"
 
-test_skip "not implemented yet"
 
 
 cat >`mhpath b` <c_text ? c1->c_text : c1->c_name, sizeof(trimmed_prefix) - 1);
+	rtrim(trimmed_prefix);
 	cchdr = 0;
 	lm = 0;
 	wid = c1->c_width ? c1->c_width : global.c_width;
@@ -897,7 +903,7 @@ putcomp(struct mcomp *c1, struct mcomp *c2, int flag)
 	onelp = NULL;
 
 	if (c1->c_flags & CLEARTEXT) {
-		putstr(c1->c_text);
+		putstr((c1->c_flags & RTRIM) ? rtrim(c1->c_text) : c1->c_text);
 		putstr("\n");
 		return;
 	}
@@ -922,7 +928,11 @@ putcomp(struct mcomp *c1, struct mcomp *c2, int flag)
 			for (cp = (c1->c_text ? c1->c_text : c1->c_name); *cp; cp++)
 if (islower(*cp))
 	*cp = toupper(*cp);
-		putstr(c1->c_text ? c1->c_text : c1->c_name);
+		if (*c2->c_text && *c2->c_text != '\n' && *c2->c_text != '\r') {
+			putstr(c1->c_text ? c1->c_text : c1->c_name);
+		} else {
+			putstr(trimmed_prefix);
+		}
 		if (flag != BODYCOMP) {
 			putstr(": ");
 			if (!(c1->c_flags & SPLIT))
@@ -970,16 +980,23 @@ putcomp(struct mcomp *c1, struct mcomp *c2, int flag)
 	}
 	count += c1->c_offset;
 
-	if ((cp = oneline(c2->c_text, c1->c_flags)))
-	   putstr(cp);
+	if ((cp = oneline(c2->c_text, c1->c_flags))) {
+	  putstr((c1->c_flags & RTRIM) ? rtrim(cp) : cp);
+	}
 	if (term == '\n')
 		putstr("\n");
 	while ((cp = oneline(c2->c_text, c1->c_flags))) {
 		lm = count;
-		if (flag == BODYCOMP && !(c1->c_flags & NOCOMPONENT))
-			putstr(c1->c_text ? c1->c_text : c1->c_name);
+		if (flag == BODYCOMP && !(c1->c_flags & NOCOMPONENT)) {
+			if (*cp) {
+putstr(c1->c_text ? c1->c_text : c1->c_name);
+			} else {
+putstr(trimmed_prefix);
+			}
+		}
 		if (*cp)
-			putstr(cp);
+			putstr((c1->c_flags & RTRIM) ? rtrim(cp) : cp);
+

[mmh] whatnow2 and mhl globbing

[Philipp Takacs]

Hi

Meillo has mentiont he would like the metadat from whatnow2 in the
draft. I would implement this by using mhl to remove the all mmh-*
fields. For this I have implemented a globing option for the ignore
argument. The implementation for whatnow2 is not done but is on my list.

Speaking of whatnow2 I have also two patches for whatnow2. One ist only
cleanup to strict use "" around variables the other handles $mh_use.
Now whatnow2 detects, if ``comp -use'' is used and don't override the
metadata.

Comments are, like on all patches, welcome.

Philipp
From e3b841e72e0e3d2c3607881c19bc2f0011dfa743 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Mon, 10 Jul 2017 16:34:45 +0200
Subject: [PATCH] mhl implement simple globbing

ignores now supports simple globbing
---
 man/mhl.man1 |  5 +
 uip/mhl.c| 30 +-
 2 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/man/mhl.man1 b/man/mhl.man1
index a80d6bf0..f38d0d84 100644
--- a/man/mhl.man1
+++ b/man/mhl.man1
@@ -166,6 +166,11 @@ ignores=component,...
 .RE
 .PP
 specifies a list of components which are never output.
+This option supports some simple globbing,
+so a '*' at the end of a component will match
+for all components which start wich the string.
+When you want to match a component which ends with
+a '*', you can escape the '*' with a '\\'.
 .PP
 The component `MessageName' (case\-insensitive) will output the
 message file name as a one-line header, similar to
diff --git a/uip/mhl.c b/uip/mhl.c
index a84703b8..a46b5a9b 100644
--- a/uip/mhl.c
+++ b/uip/mhl.c
@@ -595,6 +595,34 @@ process(char *fname, int ofilen, int ofilec)
 		c1->c_flags &= ~HDROUTPUT;
 }
 
+static boolean
+simplematch(char *pattern, char *b)
+{
+	char *match = strrchr(pattern, '*');
+	char repl;
+	boolean ret;
+
+	/* check if pattern ends with a * and is not escaped witch a \ */
+	if (!match || match[1] || (match > pattern && match[-1] == '\\')) {
+		if (!match || match[1]) {
+			return mh_strcasecmp(pattern, b) == 0;
+		}
+		match[0] = '\0';
+		match[-1] = '*';
+		ret = mh_strcasecmp(pattern, b)==0;
+		match[-1] = '\\';
+		match[0] = '*';
+		return ret;
+	}
+
+	repl = b[match-pattern];
+	b[match-pattern] = '\0';
+	*match = '\0';
+	ret = (mh_strcasecmp(pattern, b) == 0);
+	b[match-pattern] = repl;
+	*match = '*';
+	return ret;
+}
 
 static void
 mhlfile(FILE *fp, char *mname, int ofilen, int ofilec)
@@ -623,7 +651,7 @@ mhlfile(FILE *fp, char *mname, int ofilen, int ofilec)
 		switch (state = m_getfld2(state, , fp)) {
 		case FLD2:
 			for (ip = ignores; *ip; ip++)
-if (mh_strcasecmp(f.name, *ip)==0) {
+if (simplematch(*ip, f.name)) {
 	break;
 }
 			if (*ip) {
-- 
2.11.0

From 9c104c5550e5f8caa858d3282b8a338be8ef7759 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 21 Nov 2018 02:26:49 +0100
Subject: [PATCH 1/2] strict use "" in whatnow2

---
 uip/whatnow2.sh | 54 +++---
 1 file changed, 27 insertions(+), 27 deletions(-)

diff --git a/uip/whatnow2.sh b/uip/whatnow2.sh
index 59e91b7a..4e767663 100755
--- a/uip/whatnow2.sh
+++ b/uip/whatnow2.sh
@@ -92,7 +92,7 @@ create()
 	fi
 	mhmetafile=$mhdraft.meta
 	touch $mhmetafile
-	if [ -z $mheditor ]
+	if [ -z "$mheditor" ]
 	then
 		get_editor
 	fi
@@ -145,23 +145,23 @@ list()
 
 sendfunktion()
 {
-	export mhaltmsg=`anno -list -component 'mhaltmsg' $mhmetafile`
-	export mhdist=`anno -list -component 'mhdist' $mhmetafile`
-	export mhuse=`anno -list -component 'mhuse' $mhmetafile`
-	export mhfolder=`anno -list -component 'mhfolder' $mhmetafile`
-	export mhmessages=`anno -list -component 'mhmessages' $mhmetafile`
-	export mhannotate=`anno -list -component 'mhannotate' $mhmetafile`
-	send "$@" $mhdraft || exit $?
-	rm -f $mhmetafile
+	export mhaltmsg=`anno -list -component 'mhaltmsg' "$mhmetafile"`
+	export mhdist=`anno -list -component 'mhdist' "$mhmetafile"`
+	export mhuse=`anno -list -component 'mhuse' "$mhmetafile"`
+	export mhfolder=`anno -list -component 'mhfolder' "$mhmetafile"`
+	export mhmessages=`anno -list -component 'mhmessages' "$mhmetafile"`
+	export mhannotate=`anno -list -component 'mhannotate' "$mhmetafile"`
+	send "$@" "$mhdraft" || exit $?
+	rm -f "$mhmetafile"
 	exit 0
 }
 
 delete()
 {
-	folder -push $draftfolder >/dev/null 2>&1
-	rmm $draftfolder c
+	folder -push "$draftfolder" >/dev/null 2>&1
+	rmm "$draftfolder" c
 	folder -pop >/dev/null 2>&1
-	rm $mhmetafile
+	rm "$mhmetafile"
 }
 
 attach()
@@ -179,7 +179,7 @@ attach()
 			exit 1
 		fi
 		file=`get_realpath "$1"`
-		anno -nodate -append -component $header -text "$file" $mhdraft
+		anno -nodate -append -component "$header" -text "$file" "$mhdraft"
 		shift
 	done
 }
@@ -187,7 +187,7 @@ attach()
 alist()
 

[mmh] pick rework

[Philipp Takacs]

Hi

I have looked again at my pick branch. There is a free of static memory.
Also I have found a logic error in the GREPaction. Then I have
implemented rfc2046 support. The patches are attached. I need to rework
the commit message before pushing.

I have also started to rework the command parsing for pick, but this
is still unfinished.

Philipp
From b7df0ea22b417220d96485c84d5d0a0b2e8e86ea Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 21 Nov 2018 22:06:22 +0100
Subject: [PATCH 1/3] remove free of static memory

---
 uip/pick.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/uip/pick.c b/uip/pick.c
index 25bd6b6a..12b2bd70 100644
--- a/uip/pick.c
+++ b/uip/pick.c
@@ -1282,7 +1282,6 @@ DATEfree(struct nexus **n)
 	}
 	dd = (*n)->data;
 
-	mh_free0(>datef);
 	mh_free0(n);
 }
 
-- 
2.11.0

From f826c136f3bee802c63cef0510bcd8de71ccbe84 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 21 Nov 2018 23:23:10 +0100
Subject: [PATCH 2/3] fix logic bug in pick GREPaction

GREPaction now check the header field correct
---
 uip/pick.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/uip/pick.c b/uip/pick.c
index 12b2bd70..7f0d3a9a 100644
--- a/uip/pick.c
+++ b/uip/pick.c
@@ -1095,7 +1095,7 @@ GREPaction(struct field *f, int msgnum, void *data)
 	}
 
 	/* check for the right field */
-	if (g->header && *g->header && mh_strcasecmp(g->header, f->name)==0) {
+	if (!(g->header && *g->header && mh_strcasecmp(g->header, f->name)==0)) {
 		return FALSE;
 	}
 
-- 
2.11.0

From 20597014bd0a3a6afa0244e5ee6531c59bc8fceb Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Wed, 21 Nov 2018 23:25:54 +0100
Subject: [PATCH 3/3] pick implement rfc2047

---
 uip/pick.c | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/uip/pick.c b/uip/pick.c
index 7f0d3a9a..aba0c028 100644
--- a/uip/pick.c
+++ b/uip/pick.c
@@ -1088,7 +1088,7 @@ GREPaction(struct field *f, int msgnum, void *data)
 {
 	struct grep_data *g = data;
 	int ret;
-	char *buf;
+	char buf[BUFSIZ];
 
 	if (!g->header && *f->name) {
 		return FALSE;
@@ -1099,15 +1099,18 @@ GREPaction(struct field *f, int msgnum, void *data)
 		return FALSE;
 	}
 
-	ret = regexec(g->preg, f->value, 0, NULL, 0) == REG_NOMATCH;
+	if(decode_rfc2047(f->value, buf, sizeof(buf))) {
+		ret = regexec(g->preg, buf, 0, NULL, 0);
+	} else {
+		ret = regexec(g->preg, f->value, 0, NULL, 0);
+	}
 	switch (ret) {
 	case 0:
 		return TRUE;
 	case REG_NOMATCH:
 		return FALSE;
 	default:
-		buf = mh_xcalloc(BUFSIZ, sizeof(char));
-		regerror(ret, g->preg, buf, BUFSIZ*sizeof(char));
+		regerror(ret, g->preg, buf, sizeof(buf));
 		fprintf(stderr, "%s\n", buf);
 		return FALSE;
 	}
-- 
2.11.0

Re: [mmh] mhl trailing white space

[Philipp Takacs]

[2018-11-22 08:35] markus schnalke 
> [2018-11-19 03:41] Philipp Takacs 
> >
> > Hi all
> > 
> > After I have looked at the ali bug, I have looked at the other tests
> > which are skipped. I have noticed the mhl with space feature is not this
> > hard to port from nmh. So I have done this. A patch for this is attached.
>
> Seems to be very useful to have that feature. I've always been
> dissatisfied with lines containing only "> " in my replies.
>
> Is there a need to update some files in etc/ to use the rtrim
> function by default?

I don't think so. The rtrim for lines with only "> " is done automatic.
only the rtrim of contant have to be requested by the component settings.

> > Ps: the RFC 2231 support is on the todo list but not on top
>
> Okay.

To my todo list: I have some features and patches I'll send the next
days to the list.

> > From b9d4b0dd7841c1b038beca6d47216b20b9d82ba0 Mon Sep 17 00:00:00 2001
> > From: Philipp Takacs 
> > Date: Mon, 19 Nov 2018 03:04:12 +0100
> > Subject: [PATCH] Trailing withspace handling in mhl
> > 
> > If a component has trailing whitespace, e.g., body:component="> ", mhl
> > now trims that whitespace off when filtering blank text lines. Also add
> > a rtrim flag to mhl. This flag removes trailing whitespace in filtert text.
> > 
> > The implementation was done by David Levine  for nmh
> > commit ea0a6d8112a918809bd03d8126411040d22f2bb8
> > commit 1aa8f3e11e6d83ee4806abaa132ab9466f02ca5f
> > ---
>
> > diff --git a/sbr/trim.c b/sbr/trim.c
> > index 249faf9c..ec41c70c 100644
> > --- a/sbr/trim.c
> > +++ b/sbr/trim.c
> > @@ -31,3 +31,13 @@ trim(unsigned char *cp)
> >  
> > return cp;
> >  }
> > +
> > +char *
> > +rtrim(char *cp)
> > +{
> > +   char *sp = cp + strlen(cp) - 1;
> > +
> > +   for (; sp >= cp && isspace(*sp); sp--) {}
> > +   *++sp = '\0';
>
> First I thought, there would be a wrong indentation here. Only
> after that I saw the braces at the end of the line.
>
> IMO there are exactly two good forms for empty loop bodies:
>
>   for (...) {
>   }
>
> And:
>   for (...) {
>   continue;
>   }
>
> All others lead to bad readability, IMO.

I wasn't shure which version I should use. Yes this two version
are way better to read.

> But in this case, the better code might as well be:
>
>   while (sp >= cp && isspace(*sp)) {
>   sp--;
>   }

Thanks, I have been blind.

> > +   return cp;
> > +}
>
>
> > diff --git a/uip/mhl.c b/uip/mhl.c
> > index a84703b8..b0ba02ed 100644
> > --- a/uip/mhl.c
> > +++ b/uip/mhl.c
>
> > @@ -884,7 +887,10 @@ putcomp(struct mcomp *c1, struct mcomp *c2, int flag)
>
> > if (c1->c_flags & CLEARTEXT) {
> > -   putstr(c1->c_text);
> > +   putstr(c1->c_flags & RTRIM ? rtrim(c1->c_text) : c1->c_text);
>
> For those bit-check-plus-ternary-operator cases, parenthesis
> around the bit check should be used to clarify the precedence. I'm
> no fan of using too many parenthesis but in cases like this one,
> I think they improve the readability. Thus:
>
>   putstr((c1->c_flags & RTRIM) ? rtrim(c1->c_text) : c1->c_text);
>
> There are some more occurences of this pattern in the patch ...

As I said this patch is only ported from the nmh code, but I can add
parenthesis.

> > @@ -970,16 +980,23 @@ putcomp(struct mcomp *c1, struct mcomp *c2, int flag)
> > }
> > count += c1->c_offset;
> >  
> > -   if ((cp = oneline(c2->c_text, c1->c_flags)))
> > -  putstr(cp);
> > +   if ((cp = oneline(c2->c_text, c1->c_flags))) {
> > +  putstr(c1->c_flags & RTRIM ? rtrim(cp) : cp);
>
> Here's an indent with spaces. (That's the mixed tabs and spaces
> indent style of nmh. In mmh all indents should be tabs only.)

I have also noticed this, but the indention is already in our code.
I wanted to write a secound patch to fix the style. But I can include
this in the patch as well.

> (Note: We should write an indent checker script ...)
>
>
> Hope my comments help.

Yes. I'll work them in and commit/push on the weekend.

Philipp

[mmh] inbox and defaultfolder

[Philipp Takacs]

Hi

I have noticed that mmh creates everytime a +inbox folder, reagardless
how many times I delete the folder. I have set "Inbox" to "INBOX"
to match the imap INBOX.

I have looked at the code and the funktion context_read() creates
the inbox with "defaultfolder". I have written a patch which uses
getdeffol(). I'm also not shure if we should fail if the default
folder can't create.

Philipp
From 8a1fa1771c1b345cfdb96171149e5429dfc706fc Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Fri, 26 Oct 2018 21:53:36 +0200
Subject: [PATCH] read the defaultfolder from the profile

context_read now creates the inbox specified in the profile,
instand of create the compiletime default.
---
 sbr/context_read.c   |  2 +-
 test/tests/config/test-inbox | 32 
 2 files changed, 33 insertions(+), 1 deletion(-)
 create mode 100644 test/tests/config/test-inbox

diff --git a/sbr/context_read.c b/sbr/context_read.c
index cfdcff27..51592a91 100644
--- a/sbr/context_read.c
+++ b/sbr/context_read.c
@@ -137,7 +137,7 @@ context_read(void)
 	/*
 	** Create the default folder (inbox)
 	*/
-	cp = toabsdir(defaultfolder);
+	cp = toabsdir(getdeffol());
 	if (stat(cp, ) == -1) {
 		if (!makedir(cp)) {
 			adios(EX_CANTCREAT, cp, "Unable to create the default folder");
diff --git a/test/tests/config/test-inbox b/test/tests/config/test-inbox
new file mode 100644
index ..997e52e6
--- /dev/null
+++ b/test/tests/config/test-inbox
@@ -0,0 +1,32 @@
+#!/bin/sh
+##
+#
+# Test the creation of the inbox
+#
+##
+
+export MMHP="$MH_TEST_DIR/profile2"
+rm -rf "$MH_TEST_DIR/Mail2"
+printf "Path: %s\n" "$MH_TEST_DIR/Mail2" > "$MMHP"
+
+< /dev/null | folder > /dev/null
+if [ ! -d "$MH_TEST_DIR/Mail2" ]; then
+	exit 1
+fi
+
+if [ ! -d "$MH_TEST_DIR/Mail2/inbox" ]; then
+	exit 1
+fi
+
+folder -create +test > /dev/null
+rmdir "$MH_TEST_DIR/Mail2/inbox"
+
+printf "inbox: testinbox\n" >> $MMHP
+< /dev/null | folder > /dev/null
+if [ ! -d "$MH_TEST_DIR/Mail2/testinbox" ]; then
+	exit 1
+fi
+
+if [ -d "$MH_TEST_DIR/Mail2/inbox" ]; then
+	exit 1
+fi
-- 
2.11.0

Re: [mmh] Accept header lines of exactly 998 characters

[Philipp Takacs]

Hi

[2018-07-18 23:13] Vasilii Kolobkov 
> I'm having the tests green... well technically they are black, but
> function much better to their end on OpenBSD :)

Thanks.

> Yet two of them do fail, but that'd be another patch. The ali tests
> for the blind lists fail - it's the first time i came to know of
> this feature and frankly don't know if i'd want to work on fixing
> that. Does anybody use them?

If you talk about the test which fails on markus nightly tests[0],
it's a known bug, but down on the priority list. I don't know if
someone use this feature[1], but I would like to have this sane
implemented.

> The other broken part is second to last case in test/send/test-mimeify.
> The sizes reported for multipart/... types differ from expected
> values. I'll be looking further into it, but wonder if it's broken
> on your systems as well?

This also a known bug and behaves realy strange, because sometimes
the numbers differ. I have looked at this some time ago, but because
the mime implementation is realy complex I coldn't find it.

In general I like your patch, but I have some comments:

> p.s. i prefixed all the portable functions with p* so to not shadown
> the system ones just in case.
> 
> ---8<---
> From 204ae88524838063493242f012a1b1fe2fcf2d05 Mon Sep 17 00:00:00 2001
> From: Vasilii Kolobkov 
> Date: Sun, 15 Jul 2018 23:05:09 +0200
> Subject: [PATCH] Fix tests to run on OpenBSD
> 
>   For there's no seq(1) in OpenBSD, Use pseq function from common.sh
>   for generating sequence numbers. Credits for the function goes
>   to Markus Schnalke.
> 
>   He also had a one-liner, which having less features is so freaking
>   nice to not mention it here:
> < /dev/zero tr \\000 \\n | sed 10q  | nl -b a
> 
>   awk(1) doesn't support interval regular expression syntax ({n,m})
>   on some BSDs.
> 
>   hostname(1) has no -f flag on OpenBSD and, while invocation without
>   flags is supposed to return FQDN allright. Use the same phostname
>   function in mhsign(1).
> 
>   mktemp(1) is limited to at least 6 placeholder symbols on OpenBSD.
> ---
>  test/common.sh| 18 ++
>  test/runtest  |  4 +++-
>  test/tests/inc/test-eom-align |  2 +-
>  test/tests/mhsign/test-mhsign | 11 +--
>  test/tests/mhstore/test-filenames |  2 +-
>  test/tests/show/test-longlines|  6 ++
>  uip/mhsign.sh | 11 ++-
>  7 files changed, 44 insertions(+), 10 deletions(-)
> 
> diff --git a/test/common.sh b/test/common.sh
> index 805afea..056eac2 100644
> --- a/test/common.sh
> +++ b/test/common.sh
> @@ -105,7 +105,25 @@ runandcheck()
>  
>   if [ "$diff" ]; then
>   echo "$0: $1 failed"
>   echo "$diff"
>   failed=`expr "${failed:-0}" + 1`
>   fi
>  }
> +
> +pseq() {
> + start=1
> + inc=1
> +
> + case $# in
> + 1) end="$1" ;;
> + 2) start="$1"; end="$2" ;;
> + 3) start="$1"; inc="$2" ; end="$3" ;;
> + *) echo "Usage: pseq end | start end | start step end" >&2; return 1 ;;
> + esac
> +
> + awk 'BEGIN{
> + for (i='"$start"'; i<='"$end"'; i+='"$inc"') {
> + print i
> + }
> + }'
> +}

I'm not sure, if we realy nead this funktion, because it's used on
two places. In the one place it's easy to remove. On the other place
I would remove the hole test.

> diff --git a/test/runtest b/test/runtest
> index 9f35ade..37a23ac 100755
> --- a/test/runtest
> +++ b/test/runtest
> @@ -1,13 +1,15 @@
>  #!/bin/sh
>  
>  set -e
>  
>  export MH_TEST_COMMON="$PWD/common.sh"
>  
> +. ${MH_TEST_COMMON}
> +

Would be nice if this patch would also remove the source in
the single tests.

>  if [ ! -f test-temp-dir ]; then
>   echo "test-temp-dir not found: running setup-test"
>   ./setup-test
>  fi
>  
>  export MH_TEST_DIR=`cat test-temp-dir`
>  
> @@ -32,15 +34,15 @@ cat >"$MMH/profile" <<-!
>   Inbox: +inbox
>  !
>  folder -create `mhparam inbox` >/dev/null
>  folder -create `mhparam trashfolder` >/dev/null
>  folder -create `mhparam draftfolder` >/dev/null
>  
>  # create 10 basic messages
> -for i in `seq 1 10`;
> +for i in `pseq 10`;

Markus also mentiont to just use "1 2 3 4 5 6 7 8 9 10". I would
say this is absolutly ok for this usage.

>  do
>   cat >"$MAILDIR/inbox/$i" <<-!
>   From: Test$i 
>   To: Some User 
>   Date: Fri, 29 Sep 2006 00:00:00
>   Subject: Testing message $i
>  
> diff --git a/test/tests/inc/test-eom-align b/test/tests/inc/test-eom-align
> index 2b6afb4..0e9c1dd 100644
> --- a/test/tests/inc/test-eom-align
> +++ b/test/tests/inc/test-eom-align
> @@ -111,13 +111,13 @@ do_one_test_B () {
>  
>  # Cover a decent range around the stdio buffer size to make sure we catch
>  # any corner cases whether they relate to total message size equal to
>  # buffer size or to body size equal to buffer size.
>  START=$(($STDIO_BUFSZ - 16))

Re: [mmh] Accept header lines of exactly 998 characters

[Philipp Takacs]

[2018-07-16 09:20] Vasilii Kolobkov 
> [2018-07-16 03:18 +0200] Philipp Takacs 
> > What a strange BSD are you using, maybe OpenBSD ;-)? seq is part of Net,
> > Free and DragonflyBSD. Ok, the awk bug/missing feature is also in FreeBSD.
> 
> There's no hiding from you, Philipp!
> 
> I thought those issues to be rather common in BSD-land and am sorry
> for badmouthing all the good ones. Can you please do a s/BSDs/OpenBSD/g
> on the patches?

This was more a joke, then a real comment.

Philipp

Re: [mmh] Accept header lines of exactly 998 characters

[Philipp Takacs]

[2018-07-15 23:29] Vasilii Kolobkov 
> [2018-07-15 13:50 +0200] markus schnalke 
> > Fortunately you don't have to slap me. *phew* ;-)
> 
> Seems like i missed out on some fun :) Still managed to get spanked
> - i didn't run tests until now, just to find out that there are
> some issues running them on BSD.

What a strange BSD are you using, maybe OpenBSD ;-)? seq is part of Net,
Free and DragonflyBSD. Ok, the awk bug/missing feature is also in FreeBSD.

> Here's a patch that fixes most of the show(1) tests, and i'll be
> fixing the rest should you deem it useful.

Thanks I'll commit this tomorrow. More fixes for the test suite are
welcome.

Philipp

Re: [mmh] Accept header lines of exactly 998 characters

[Philipp Takacs]

[2018-07-14 14:53] Vasilii Kolobkov 
> [2018-07-14 14:16 +0200] Philipp Takacs 
> > By the way, the body handling in spost is a bit odd.
> 
> Mind sharing what's the odd bits? I might take a stab at it in spare
> time.

It's the inner loop, so first time m_getfld2() returns BODY2
finish_header() is called the the body/header filler is printed
and then a body loop prints the rest off the body.

This is quit common in mmh. The problem I see with this pattern is,
that the first line of the body is handled a bit diffrent then the
other lines. This patter is used in some programms and I don't like
it. I would like to have a bool for this.

Philipp

Re: [mmh] Accept header lines of exactly 998 characters

[Philipp Takacs]

[2018-07-12 13:46] markus schnalke 
> [2018-07-11 13:49] Vasilii Kolobkov 
> > Lately I've been getting issues with messages containing header
> > lines of 998 characters. This patch fixes it.
> 
> I'm gonna apply it (plus add a test case to detect this situation).
> 
> If it's not done till Sunday night, slap me and do it yourself. ;-P

I have two follow up patches. The first one fixes crlf handling for
long lines in m_getfld2. The secound fix the simular length checking
bug in sposts.

By the way, the body handling in spost is a bit odd.

Philipp
From 4a271b7cef4270e0079ab8903e0de6850d6e8710 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Sat, 14 Jul 2018 11:24:36 +0200
Subject: [PATCH 1/2] check for crlf in m_getfld2

now header fields witch lines 998 chars and crlf
are accepted.
---
 h/mh.h  | 1 +
 sbr/m_getfld2.c | 6 --
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/h/mh.h b/h/mh.h
index 53b301e5..20346b6b 100644
--- a/h/mh.h
+++ b/h/mh.h
@@ -210,6 +210,7 @@ struct field {
 	char *value;
 	size_t valuelen;
 	size_t alloclen;
+	boolean crlf;
 };
 
 /* m_getfld2() states */
diff --git a/sbr/m_getfld2.c b/sbr/m_getfld2.c
index 954ed331..b9a618d1 100644
--- a/sbr/m_getfld2.c
+++ b/sbr/m_getfld2.c
@@ -62,7 +62,8 @@ m_getfld2(enum state s, struct field *f, FILE *msg)
 			}
 		}
 
-		if (nchars > NAMESZ) {
+		f->crlf = (nchars > 2 && tmpline[nchars-2] == '\r');
+		if (nchars > NAMESZ+1 || (!f->crlf && nchars > NAMESZ)) {
 			ret = LENERR2;
 		}
 
@@ -103,7 +104,7 @@ m_getfld2(enum state s, struct field *f, FILE *msg)
 return IOERR2;
 			}
 
-			if (nchars > NAMESZ) {
+			if (nchars > NAMESZ+1 || (!f->crlf && nchars > NAMESZ)) {
 ret = LENERR2;
 			}
 
@@ -145,6 +146,7 @@ m_getfld2(enum state s, struct field *f, FILE *msg)
 			}
 		}
 
+		f->crlf = (nchars > 2 && tmpline[nchars-2] == '\r');
 		free(f->value);
 		f->value = tmpline;
 		f->valuelen = nchars;
-- 
2.18.0

From 15739c47c966ceba74c4b645ffac8751f6c75a83 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Sat, 14 Jul 2018 13:41:09 +0200
Subject: [PATCH 2/2] spost accept body lines with 998 characters

---
 uip/spost.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/uip/spost.c b/uip/spost.c
index 39d8ada5..fd0cbcc9 100644
--- a/uip/spost.c
+++ b/uip/spost.c
@@ -245,7 +245,7 @@ main(int argc, char **argv)
 			finish_headers(out);
 			fprintf(out, "\n%s", f.value);
 			while ((state = m_getfld2(state, , in)) == BODY2) {
-if (f.valuelen >= NAMESZ) {
+if (f.valuelen > NAMESZ+1 || (!f.crlf && f.valuelen > NAMESZ)) {
 	adios(EX_DATAERR, NULL, "Body contains a to long line");
 }
 fputs(f.value, out);
-- 
2.18.0

Re: [mmh] Re: Comments on the Debian package

[Philipp Takacs]

Hi

[2018-02-16 23:16] kact...@gnu.org
> 
> Hello. Sorry for late reply, was away from keyboard for very long time.

Nice to hear from you again.

> [2017-04-17 15:36] markus schnalke 
> > Hoi Dmitry and all,
> > 
> > first of all, thanks for maintaining the Debian package! It's great to
> > have it. I have some comments on it.
> 
> You are welcome!
> 
> > Should I file Debian bug reports or is it okay to tell it to you this
> > way?
> 
> It is okay to send it privately to me, but BTS never loses emails. I
> do, sometimes.
> 
> > 1) The package description has ``nmh'' instead of ``mmh'' in
> > the second sentece.
> 
> Fixed. Copied from README, btw.
>  
> > 2) This is a bit harsh: ``[...] rather mmh breaks compatibility
> > to nmh in order to modernize and simplify it.'' To some ears it
> > could sound like we would not care for compatibility at all.
> > (Compared to the nmh-situation in 2010 this might look correct.)
> > But we rather are willing to sacrifice compatibility for (in
> > our eyes) higher goals like modernity and simplicity, for
> > instance. Maybe you could reword this a bit.
> 
> This description is from README, too. So, probably I am not right person
> to reword.

Thanks for pointing this out. I'll look for a better wording.

> By the way, you are welcome to release more often. With
> better description, hm?

Oh...
Yes we should release more often. Sadly I don't have much
time at the moment. I plan to put more time in mmh mid/end of
May. So maybe end of June[0] a new release is there.

> > 3) The license information is not exactly correct. The license
> > included in debian/copyright is not the license of mmh but only
> > the closest one of the generic licenses.
> > [...]
> 
> Fine. I will change 'debian/changelog' and insert content of COPYRIGHT
> verbatim.
>  
> > 4) The mmh package is still unreproducible (same problem as nmh).
> > This comes from compiling the compile time into the executables.
> > This way, it cannot be reproducible. I have no strong oppinion
> > either way. Just want to mention it here, for the record.
> 
> Thank you for warning. I will patch out source of non-reproducibility.

The time bug should be fixed. If not, contact me. But there
is a problem with the hostname[1]. I haven't understand yet,
so if you find and fix this, please let me know.
 
Philipp

[0] I realy hope I can manage this
[1] see 
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/mmh.html

Re: [mmh] archiving encrypted mail

[Philipp Takacs]

[2017-09-01 11:00] Vasilii Kolobkov 
> Hello mmh folks!

Hi

> Does anybody here save their outgoing encrypted mail unencrypted?
> Any ideas on how would be best to handle this?

After looking into this I have seen that send refile the draft
to the +trash folder, but this don't include attachments. There
is a more or less similar ``bug'' in mhsign, I'll look at this
today.

So if you want to safe the file with the attachments, you
can add

rcvstore +sent < $FILE.orig

at the end of mhsign. This would have the some problems. First
it would also save sing only mails without the signing. You could
fix this with a stringcompare of $function. Also the Fcc field is
ignored. You could read and delete the field with anno. The last
problem I see is that the mail is saved in the drafts folder,
regardless of the delivery result.

Philipp

Re: [mmh] pick(1) putting zero at the end

[Philipp Takacs]

Hi

[2017-07-08 15:15] markus schnalke <mei...@marmaro.de>
> [2017-07-08 14:51] Philipp Takacs <phil...@bureaucracy.de>
> > [2017-07-07 11:58] markus schnalke <mei...@marmaro.de>
> > > [2017-07-07 11:32] Vasily Kolobkov <polezaivs...@openmailbox.org>
> > > > [2017-07-07 10:30 +0200] Philipp Takacs <phil...@bureaucracy.de>
> > > > > While doing this we could also remove the -[no]list, -sequence
> > > > > and the -[no]zero switches.
> > > > 
> > > > You mean dropping writing to sequences aspect altogether? I haven't
> > > > ever used this function once. If others find it of no use as well,
> > > > i'm all for slaying some code :)
> > > 
> > > I do use -seq always! I think we can remove -list when we the
> > > pick/scan merge is done, but -seq and -zero have to stay.
> > 
> > I think this three switches work only together. I'll explain this
> > later.
> > 
> > > Mark(1)
> > > and pick(1) are the two tools that provide marking sequences,
> > 
> > This is why I want to remove the -seq switch from pick, I don't
> > think we need two tools for the same task.
> 
> I generally agree. But I don't see how mark(1) could cover the
> pick(1) task.
> 
> The scenario is such:
> 
> I want to handle all messages from bob as a group (show them,
> refile them, remove them, etc.) Thus I use a sequence for them.
> But as they are scattered over the whole folder I cannot simply
> `mark l:20' but rather have to `pick -from bob'.
> 
> Now, how do I get them into the sequence if pick(1) doesn't do
> it for me? (Keep in mind, I don't necessarily want to show them,
> but rather refile them.)
> 
> If we can cover this scenario without pick(1) needing the -seq
> switch, then we can consider dropping it.

What about using pick in backticks, like with anno/show/...

> Currently, however, I rather see the situation this way: Pick(1)
> sets sequences, just like mark(1) but by conditions. Listing the
> results (`pick -list') and showing them (scan/pick merge) are
> structurally rather special tasks, which we provide for
> convenience.
> 
> Well, maybe we should first get the relations between mark(1),
> pick(1), and scan(1) clear. As I think about then, I realize
> that don't have them as clear as I thought I would.

I see the situation a bit diffrent: Pick is like find, to search
messages. The merge of scan/pick is adding a ``-form'' switch to
pick and realising scan is now obsulet. I wouldn't have sugested
the ``-form'' swtich for pick, if there wasn't the benifit of
removing the hole scan tool.

The situation with pick and mark is a bit diffrent. Mark is the
main tool to handle sequences and pick has one of its feature
implemented. I don't see a clean way to merge these two tools.

So why do we need the code/feature duplication?

As I said, I don't see any benifit of this feature. You can use
pick in backticks with every tool. Why shouldn't you use it with
mark?

> > I would say remove all this sequence handling from pick, because we don't
> > need two times the same code. I really don't see any benefits by having
> > these switches.
> 
> Maybe we should collect some scenarios.
> 
> My one is:
> 
>   pick -from bob -seq P +inbox
>   refile P +bob

I asume you use the default switches, then what about:

$ refile `pick -from bob +inbox` +bob

If you use ``-nozero'' or you want to scan/anno/... before
refile, how about:

$ mark -add [-nozero] -seq P `pick -from bob +inbox`
$ refile P +bob

Philipp

[mmh] Re: Building with musl libc

[Philipp Takacs]

[2016-12-28 17:47] m...@arascio.xyz
> | > hostname: sethostname: Operation not permitted
> | > hostname: sethostname: Operation not permitted
> | > tests/mhsign/test-mhsign: mhsign -enc 
> | /tmp/mmh-test-XXFnPJDc/Mail/drafts/1 f
> | > ailed
> | > --- - 2016-12-25 12:13:43.492162240 -0700
> | > +++ /tmp/tmp.dDaEJc   2016-12-25 12:13:43.49100 -0700
> | > @@ -1,5 +1,7 @@
> | > +hostname: sethostname: Operation not permitted
> | >  Could not find key for 
> | >  Could not find key for 
> | > +hostname: sethostname: Operation not permitted
> | >  Could not find key for 
> | >  Could not find key for 
> | >  Could not find key for 
> | > Test tests/mhsign/test-mhsign 
> | FAIL
> |
> | This is a problem of mhsign and the tests, we use ``hostname''
> | to get the hostname, if it isn't in the address. The problem
> | is that hostname is not part of posix.
> |
> | The problem is, I haven't found a way to get the FQDN with
> | pure posix. There is ``uname -n'', but the output of uname
> | isn't nececary a FQDN.
> 
> If this functionality is needed then maybe we could provide it; for
> example, consider the attached program "mmhhostname".

I wouldn't like to add more binaries, also this would be used just
for one special usecase.

I would say we write this in the manpage. This is a small issue,
because this feature is for using mmh on the local mashien with
pgp. This is a rare usecase.

Philipp

Re: [mmh] scan and pick (was: How "modern" are you willing to go?)

[Philipp Takacs]

[2016-09-20 16:56] Dmitry Bogatov <kact...@gnu.org>
> Seems, symlink replacement of 'scan' is not part of this patch, isn't it?

No, I have written this patch to have some code to discuss
and get a response from markus what he thinks now about merging
scan and pick. Thats why I have focused on the actuall code not
on the buildsystem changes.

> > [2016-09-09 19:45] Philipp Takacs <phil...@bureaucracy.de>
> > --- a/uip/pick.c
> > +++ b/uip/pick.c
> > @@ -83,10 +89,12 @@ static int listsw = -1;
> >
> >  void putzero_done();
> >
> > +void printmsg(FILE *f, struct msgs *mp, int msgnum, char *fmtstr, int wid
> th);
> 
> static?

Yes.

> 
> > -if (atexit(putzero_done) != 0) {
> > +   if (atexit(putzero_done) != 0) {
> > adios(EX_OSERR, NULL, "atexit failed");
> > }
> 
> Seems to fix spacing error, but it probably belong to another patch.

Yes this will be part of an other commit, I have just seen it and fixed it.

> > @@ -271,8 +299,10 @@ main(int argc, char **argv)
> > if (msgnum > hi)
> > hi = msgnum;
> >
> > -   if (listsw)
> > -   printf("%s\n", m_name(msgnum));
> > +   if (listsw) {
> > +   printmsg(fp, mp, msgnum, fmtstr, width
> );
> > +   /* printf("%s\n", m_name(msgnum)); */
> > +   }
> 
> Commented code?

Will be removed on the final commit.

> > +void printmsg(FILE *f, struct msgs *mp, int msgnum, char *fmtstr, int wid
> th)
> > +{
> > +   int seqnum;
> > +   int state;
> > +   boolean unseen = FALSE;
> > +
> > +   fseek(f, 0L, SEEK_SET);
> > +
> > +   seqnum = seq_getnum(mp, seq_unseen);
> > +   unseen = in_sequence(mp, seqnum, msgnum);
> > +
> > +   switch (state = scan(f, msgnum, SCN_FOLD, fmtstr,
> > +   width, msgnum==mp->curmsg, unseen)) {
> > +   case SCNMSG:
> > +   case SCNERR:
> > +   break;
> 
> Why cases
> 
>   #define SCNMSG  1 /* message just fine*/
> #define SCNERR  (-1)  /* error message*/
> 
> are merged?

I have more or less just copied the code from uip/scan.c (line 209).
The cases are merged there, too. I have looked a bit into it now.
It's because, if scan() fails there is nothing we can do about it.

Philipp

Re: [mmh] Re: Building with musl libc

[Philipp Takacs]

[2016-12-25 12:18] m...@arascio.xyz
>
> part 1 text/plain 303
> | [2016-07-29 00:26] markus schnalke 
> | >
> | > @all: Has anyone verified that the latest version runs fine with
> | > musl libc?
> 
> mmh builds without a problem on Void Linux with gcc 6.3 and
> musl 1.1.15.
> 
> Output from configuration, from compilation and from testing is
> attached.

Thanks for the tests.


> ==1665== Invalid free() / delete / delete[] / realloc()
> ==1665==at 0x4CABE00: free (in /usr/lib/valgrind/vgpreload_memcheck-amd6
> 4-linux.so)
> ==1665==by 0x406EACE: reclaim_gaps (dynlink.c:488)
> ==1665==by 0x406F042: map_library (dynlink.c:708)
> ==1665==by 0x406FC4D: load_library (dynlink.c:1014)
> ==1665==by 0x4070978: load_preload (dynlink.c:1112)
> ==1665==by 0x4070978: __dls3 (dynlink.c:1581)
> ==1665==by 0x40705C5: __dls2 (dynlink.c:1383)
> ==1665==by 0x4072C27: ??? (in /usr/lib/libc.so)
> ==1665==by 0x3: ???
> ==1665==by 0xFFF000D86: ???
> ==1665==by 0xFFF000D8A: ???
> ==1665==by 0xFFF000D92: ???
> ==1665==by 0xFFF000D98: ???
> ==1665==  Address 0x4eb6200 is in a rw- mapped file /usr/lib/valgrind/vgprel
> oad_memcheck-amd64-linux.so segment
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x406314C: strlen (strlen.c:15)
> ==1665==by 0x406FC6E: load_library (dynlink.c:1023)
> ==1665==by 0x407012B: load_deps (dynlink.c:1085)
> ==1665==by 0x4070989: __dls3 (dynlink.c:1582)
> ==1665==by 0x40705C5: __dls2 (dynlink.c:1383)
> ==1665==by 0x4072C27: ??? (in /usr/lib/libc.so)
> ==1665==by 0x3: ???
> ==1665==by 0xFFF000D86: ???
> ==1665==by 0xFFF000D8A: ???
> ==1665==by 0xFFF000D92: ???
> ==1665==by 0xFFF000D98: ???
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x40629AF: stpcpy (stpcpy.c:20)
> ==1665==by 0x4062D48: strcpy (strcpy.c:8)
> ==1665==by 0x406FD49: load_library (dynlink.c:1043)
> ==1665==by 0x407012B: load_deps (dynlink.c:1085)
> ==1665==by 0x4070989: __dls3 (dynlink.c:1582)
> ==1665==by 0x40705C5: __dls2 (dynlink.c:1383)
> ==1665==by 0x4072C27: ??? (in /usr/lib/libc.so)
> ==1665==by 0x3: ???
> ==1665==by 0xFFF000D86: ???
> ==1665==by 0xFFF000D8A: ???
> ==1665==by 0xFFF000D92: ???
> ==1665==by 0xFFF000D98: ???
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x406314C: strlen (strlen.c:15)
> ==1665==by 0x40633BF: strrchr (strrchr.c:7)
> ==1665==by 0x406FD5C: load_library (dynlink.c:1045)
> ==1665==by 0x407012B: load_deps (dynlink.c:1085)
> ==1665==by 0x4070989: __dls3 (dynlink.c:1582)
> ==1665==by 0x40705C5: __dls2 (dynlink.c:1383)
> ==1665==by 0x4072C27: ??? (in /usr/lib/libc.so)
> ==1665==by 0x3: ???
> ==1665==by 0xFFF000D86: ???
> ==1665==by 0xFFF000D8A: ???
> ==1665==by 0xFFF000D92: ???
> ==1665==by 0xFFF000D98: ???
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x406314C: strlen (strlen.c:15)
> ==1665==by 0x402797A: setlocale (setlocale.c:55)
> ==1665==by 0x40249F: main (inc.c:174)
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x406314C: strlen (strlen.c:15)
> ==1665==by 0x404001: concat (concat.c:33)
> ==1665==by 0x4044CF: context_read (context_read.c:97)
> ==1665==by 0x4024BD: main (inc.c:177)
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x40629AF: stpcpy (stpcpy.c:20)
> ==1665==by 0x4062D48: strcpy (strcpy.c:8)
> ==1665==by 0x40B639: toabsdir (path.c:296)
> ==1665==by 0x40472D: context_read (context_read.c:140)
> ==1665==by 0x4024BD: main (inc.c:177)
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x406314C: strlen (strlen.c:15)
> ==1665==by 0x404001: concat (concat.c:33)
> ==1665==by 0x404898: context_read (context_read.c:174)
> ==1665==by 0x4024BD: main (inc.c:177)
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x406314C: strlen (strlen.c:15)
> ==1665==by 0x4062E6D: strdup (strdup.c:7)
> ==1665==by 0x4029B8: main (inc.c:299)
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x40629AF: stpcpy (stpcpy.c:20)
> ==1665==by 0x4062D48: strcpy (strcpy.c:8)
> ==1665==by 0x40B639: toabsdir (path.c:296)
> ==1665==by 0x402A03: main (inc.c:307)
> ==1665== 
> ==1665== Conditional jump or move depends on uninitialised value(s)
> ==1665==at 0x40629AF: stpcpy (stpcpy.c:20)
> ==1665==by 0x4062D48: strcpy (strcpy.c:8)
> ==1665==by 0x40B639: toabsdir (path.c:296)
> ==1665==by 0x405CB2: folder_read (folder_read.c:36)
> ==1665==by 0x402A96: main (inc.c:317)
> ==1665== 

Re: [mmh] Changes to ali.man1

[Philipp Takacs]

[2016-12-15 12:20] markus schnalke 
> [2016-12-13 20:14] Larry Hynes 
> >
> > Following is a start on the man pages, beginning with ali.man1.
> 
> Great, thanks!
>
> [...]
> 
> Apart from that, your patches are fine. I'm gonna apply them.

I have applied them now. Thanks for your work and sorry for
the long delay.

I have seen you have also send a punch of patches to the
nmh-wrokers mailing list. Are these patches only relevant
for nmh? If not, do you plan to port them to mmh and send
them to our mailing list?

Philipp

Re: [mmh] better thread support

[Philipp Takacs]

[2016-11-28 19:27] markus schnalke 
> What I think about for a long time is the use of shell
> scripts as mmh commands. I see the great leverage we get
> from shell scripting, but OTOH I see the inconsistency
> that creeps into our so consistent C framework. All C tools
> behave just the same. The shell scripts behave differently
> and if we want to change that, then we need to build up a
> new sbr/ library but now in sh. -- I see the problem but I
> don't see the answer.
> 
> Some things, however, are clear to me: Shell scripting is
> powerful, we should use it in cases like this threading
> prototype. Our C code base is wonderful, as well, and we
> should try to preserve it's good quality and high consistency.
> Maybe we should consider rewriting the shell scripts in C
> when they are established (like mhsign(1) and mhpgp(1), for
> instance).

I use shell scripts to get a feature which is not (yet) part of
mmh. If we want this features in mmh, we should get a good C
implementation of it. The benefit of this is, if we don't want
this be part of mmh, other can use it without patch it on every
release.

There are exceptions, like whatnow2.sh or mmhwrap.sh, because this
are wrapper scripts, which perfect match the shell.

So I want a ``-thread msgs'' switch for pick with a similar
behavior like this script. The problem is I don't have the
time to implement it, maybe after Christmas.

> > part 2 text/x-shellscript 364 pickthread
> >   
> > #!/bin/sh
> > 
> > getthread()
> > {
> 
> Have you used jwz's threading algorithm here?
>   https://www.jwz.org/doc/threading.html
> If so, please document it. If not, then why not? ;-)
> AFAIK it's the best way to do it.

I don't use this, because I didn't know it.

I'll look at it, but just for this feature it's a bit
to much, but for the long run we can use this.

> > if [ -z "$id" ];then
> > pick $1
> > fi
> > pick --References "$id" -or --Message-ID "$id"
> > }
> 
> You should keep in mind, that pick(1) is an mmh tool for which
> most users have default switches in the profile. Thus, you
> should explicitly set -[no]list and -sequence.
> 
> This of course is part of a larger problem. Actually we should
> set all the default switches to ensure correct behavior. Maybe
> we should add some -noprofile (or -nodefaults or -usedefaults)
> option to all tools ...

Yes I missed this. But a ``-noprofile'' is not the solution, what
is about the switches I want to have set like -[no]public or -form
in show?

> > for msg in `pick "$msgs"`;do
> 
> You probably meant  `mhpath "$msgs"`  here?
> 
> > getthread $msg
> > done
> 
> 
> Actually, what's the (intended) output of your command?

The intended output is all messages which are in the same
thread as the given massage. For example if you get a new
mail on a thread and you want to know all the mails before
you can use ``show `pickthread`'' (or ``pick -thread c'')
to see the full thread.

> Maybe, the algorithm should be implemented as an option to
> sortm(1):  sortm -thread
> 
> Maybe we could add some fmt function to scan(1) to draw
> such little branches before the subject, if we're on the
> same thread as the message before.

This are features we should implement, too.

Philipp

Re: [mmh] scan and pick (was: How "modern" are you willing to go?)

[Philipp Takacs]

[2016-09-09 19:45] Philipp Takacs <phil...@bureaucracy.de>
> It wouldn't be that complicate. We would to replace the
> 
> > printf("%s\n", m_name(msgnum));
> 
> with a call to scan() and add the ``-format'' switch.
> 
> In other words pick supports only the format '=%(msg)' at
> the moment. If we have a format switch in pick we could replace
> scan by a symlink to pick.

I have implemented this to have a code example we can discouse
not only an idea. The patch is attached.

Philipp
diff --git a/uip/Makefile.in b/uip/Makefile.in
index e7e7b46..cd43aaa 100644
--- a/uip/Makefile.in
+++ b/uip/Makefile.in
@@ -172,8 +172,8 @@ new: new.o $(LOCALLIBS)
 packf: packf.o dropsbr.o $(LOCALLIBS)
 	$(LINK) packf.o dropsbr.o $(LINKLIBS)
 
-pick: pick.o $(LOCALLIBS)
-	$(LINK) pick.o $(LINKLIBS)
+pick: pick.o scansbr.o termsbr.o $(LOCALLIBS)
+	$(LINK) pick.o scansbr.o termsbr.o $(LINKLIBS) $(TERMLIB)
 
 print-mimetype: print-mimetype.sh
 	cp $(srcdir)/print-mimetype.sh print-mimetype
diff --git a/uip/pick.c b/uip/pick.c
index 4cb33fb..10f79ef 100644
--- a/uip/pick.c
+++ b/uip/pick.c
@@ -12,6 +12,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 
 #ifdef HAVE_SYS_TIME_H
 # include 
@@ -63,9 +65,13 @@ static struct swit switches[] = {
 	{ "list", 0 },
 #define NLISTSW  21
 	{ "nolist", 2 },
-#define VERSIONSW  22
+#define FORMATSW  22
+	{ "format format", 0 },
+#define WIDTHSW  23
+{ "width columns", 0 },
+#define VERSIONSW  24
 	{ "Version", 0 },
-#define HELPSW  23
+#define HELPSW  25
 	{ "help", 0 },
 	{ NULL, 0 }
 };
@@ -83,10 +89,12 @@ static int listsw = -1;
 
 void putzero_done();
 
+void printmsg(FILE *f, struct msgs *mp, int msgnum, char *fmtstr, int width);
+
 int
 main(int argc, char **argv)
 {
-	int publicsw = -1, zerosw = 1, vecp = 0;
+	int publicsw = -1, zerosw = 1, vecp = 0, width = 0;
 	unsigned int seqp = 0;
 	int lo, hi, msgnum;
 	char *maildir, *folder = NULL, buf[100];
@@ -94,9 +102,11 @@ main(int argc, char **argv)
 	char *seqs[NUMATTRS + 1], *vec[MAXARGS];
 	struct msgs_array msgs = { 0, 0, NULL };
 	struct msgs *mp;
+	char *form = NULL;
+	char *fmtstr;
 	FILE *fp;
 
-if (atexit(putzero_done) != 0) {
+	if (atexit(putzero_done) != 0) {
 		adios(EX_OSERR, NULL, "atexit failed");
 	}
 
@@ -109,6 +119,10 @@ main(int argc, char **argv)
 	arguments = getarguments(invo_name, argc, argv, 1);
 	argp = arguments;
 
+	if (strcmp(invo_name,"scan")==0) {
+		form = scanformat;
+	}
+
 	while ((cp = *argp++)) {
 		if (*cp == '-') {
 			if (*++cp == '-') {
@@ -193,6 +207,18 @@ main(int argc, char **argv)
 			case NLISTSW:
 listsw = 0;
 continue;
+			case FORMATSW:
+if (!(form = *argp++) || *form == '-') {
+	adios(EX_USAGE, NULL, "missing argument to %s", argp[-2]);
+}
+continue;
+			case WIDTHSW:
+if (!(cp = *argp++) || *cp == '-') {
+	adios(EX_USAGE, NULL, "missing argument to %s",
+			argp[-2]);
+}
+width = atoi(cp);
+continue;
 			}
 		}
 		if (*cp == '+' || *cp == '@') {
@@ -205,6 +231,8 @@ main(int argc, char **argv)
 	}
 	vec[vecp] = NULL;
 
+	fmtstr = new_fs(form, "pick.default");
+
 	/*
 	** If we didn't specify which messages to search,
 	** then search the whole folder.
@@ -271,8 +299,10 @@ main(int argc, char **argv)
 if (msgnum > hi)
 	hi = msgnum;
 
-if (listsw)
-	printf("%s\n", m_name(msgnum));
+if (listsw) {
+	printmsg(fp, mp, msgnum, fmtstr, width);
+	/* printf("%s\n", m_name(msgnum)); */
+}
 			} else {
 /* if it doesn't match, then unselect it */
 unset_selected(mp, msgnum);
@@ -320,6 +350,30 @@ putzero_done()
 		printf("0\n");
 }
 
+void printmsg(FILE *f, struct msgs *mp, int msgnum, char *fmtstr, int width)
+{
+	int seqnum;
+	int state;
+	boolean unseen = FALSE;
+
+	fseek(f, 0L, SEEK_SET);
+
+	seqnum = seq_getnum(mp, seq_unseen);
+	unseen = in_sequence(mp, seqnum, msgnum);
+
+	switch (state = scan(f, msgnum, SCN_FOLD, fmtstr,
+			width, msgnum==mp->curmsg, unseen)) {
+	case SCNMSG:
+	case SCNERR:
+		break;
+	case SCNEOF:
+		advise(NULL, "message %d: empty", msgnum);
+		break;
+	default:
+		adios(EX_SOFTWARE, NULL, "scan() botch(%d)", state);
+	}
+}
+
 
 static struct swit parswit[] = {
 #define PRAND   0

Re: [mmh] [PATCH 4/4] whatnow2: improve attach action

[Philipp Takacs]

[2016-09-16 15:22] Dmitry Bogatov 
>
> part   text/plain1232
> 
>  * avoid bogus Attach header, containing directory of
>file being attached

Can you explain what bug.

>  * simplify code, taking advantage of `readlink -f'.
> ---
> 
> Notes:
> This patch fixes bug, it is fact.

Claim down explain the bug.

If you mean that symlinks arn't followed, this is expected.

> Another fact is that there is no
> `readlink` utility in POSIX-2008. So questions are:
> 
>  * Are there any worthy systems, which are still in interactive use,
>and are lacking `readlink' utility?

Any System implementing POSIX.

>  * Does `readlink' utility on systems other then GNU/Linux support
>option -f?

The ``-f'' option in FreeBSD specifies the output format[0]. This is the
problem, we can't point to a standard and say follow this and everything
works.

>  * Why do we use deprecated `command substitution` syntax instead of
>modern $(syntax), which is in POSIX-2008?

I don't want to start this discousion now.

Philipp

[0]: this is acourding to the manpage, the actual code does something
diffrent.

Re: [mmh] [PATCH 1/4] Factor out draft file creation from replout()

[Philipp Takacs]

[2016-09-16 15:22] Dmitry Bogatov 
> + if (!out) {
> + adios(EX_CANTCREAT, filepath, "unable to create");
> + }

I don't think it's good to add more exit/adios calls in the library.

Philipp

[mmh] scan and pick (was: How "modern" are you willing to go?)

[Philipp Takacs]

[2016-09-09 09:35] Link Satonaka 
> > [meillo]
> > > [Link]
> > > Another oddity to me as a new user is the separation of ‘pick’ and
> > > ‘scan’. How much sense does it actually make that these are two
> > > separate utilities?
> > 
> > How much sense does it make that ls(1) and grep(1)/find(1) are
> > separate utilities? ;-)

I find this idea not that bad. As far as I see wold this be 50 extra
lines in pick and we could drop scan.c.

> > [meillo]
> > It would complicate the code and limit the possibilities. (This
> > is a question of Unix style.) But what I can think of is adding
> > a `-scan' switch to pick, which would automatically invoke scan
> > with the result. I think it would be worth considering that. If
> > the added complexity is small and the usability increases greatly
> > (you can add such a switch to the profile or we could even enable
> > it by default) ...

It wouldn't be that complicate. We would to replace the

> printf("%s\n", m_name(msgnum));

with a call to scan() and add the ``-format'' switch.

In other words pick supports only the format '=%(msg)' at
the moment. If we have a format switch in pick we could replace
scan by a symlink to pick.

Philipp

Ps: yes I have read the other parts of this thread and I'll write another
mail to this thread, but not now.

[mmh] build in diffrent directory don't work

[Philipp Takacs]

Hi

Since the commit 94ac6bb build in a diffrent directory don't work.
This means the test suite can't be used. I have a patch attached
and will push this today.

@Markus
For what is the gettitle() funktion in man/gettitles.sh?

Philipp
diff --git a/man/Makefile.in b/man/Makefile.in
index adb0e89..abfcd41 100644
--- a/man/Makefile.in
+++ b/man/Makefile.in
@@ -77,10 +77,10 @@ mh-chart.man7: $(ALLPROGS)
 	$(srcdir)/mh-chart-gen.sh $(ALLPROGS) >$@
 
 titles:
-	$(srcdir)/gettitles.sh 1 >titles1.temp
-	$(srcdir)/gettitles.sh 5 >titles5.temp
-	$(srcdir)/gettitles.sh 7 >titles7.temp
-	$(srcdir)/gettitles.sh 8 >titles8.temp
+	$(srcdir)/gettitles.sh $(srcdir) 1 >titles1.temp
+	$(srcdir)/gettitles.sh $(srcdir) 5 >titles5.temp
+	$(srcdir)/gettitles.sh $(srcdir) 7 >titles7.temp
+	$(srcdir)/gettitles.sh $(srcdir) 8 >titles8.temp
 
 $(MAN1) $(MAN5) $(MAN7) $(MAN8): man.sed
 
diff --git a/man/gettitles.sh b/man/gettitles.sh
index 6585edd..1c4abaf 100755
--- a/man/gettitles.sh
+++ b/man/gettitles.sh
@@ -4,6 +4,9 @@ gettitle() {
 	sed -n '/^.SH NAME/{n;p;q}' "$1"
 }
 
+cd $1 || exit 1
+shift
+
 for i in *.man"$1" ; do
 	sed -n '
 		/^.SH NAME/{

Re: [mmh] m_convert refactor

[Philipp Takacs]

Hi

[2016-09-02 23:11] Dmitry Bogatov 
> I am considering refactor m_convert function. It parses messages
> specification, like `1-c' or `15' and set corresponding messages
> selected.

In general this is a good idea, but Markus mentioned some time ago that
he wants to redesign the messages specification. So before you start we
should discuss following questions:

1) Do we still want a redesign?

2) Should it be backward compatible?

3) What features do we want?

4) How should it look?

We don't need to answer all questions in detail, but we should
have a rough idea about what we want.

> It is rather complicated, and accepts message specification string as
> non-const, and it's non-trivial to make it const. (And there is FIXME
> in header of file!) While it is possible to write better parsing code,
> I propose to take advantage of re2c tool[1].
> 
> It is like flex or bison, but, unlike them, which read from file, re2c
> generates parser for C string (const char *). As far as I know, it is
> only string parser for C language.

What speaks against using a standard lex (i.e. flex or bison) or what is
the advantage of re2c?

Philipp


pgpTgAmk2EB_K.pgp
Description: PGP signature

Re: [mmh] [PATCH] Fix unreproducible build

[Philipp Takacs]

Thanks it's commited.

Philipp

Re: [mmh] [PATCH] Fix spelling errors, including binaries ones

[Philipp Takacs]

Thanks it's commited

Philipp

Re: [mmh] [PATCH] Fix unreproducible build

[Philipp Takacs]

[2016-08-27 11:24] kact...@gnu.org
> +echo "char *version_str = \"mmh-$VERSION\";"

This is a little less information. Maybe add the git commit hash.

Philipp

Re: [mmh] [PATCH] Fix out-of-bounds error when incorporating email from stdin

[Philipp Takacs]

[2016-08-26 21:14] kact...@gnu.org
> Before this patch, if +inbox is empty, following error happened:
> 
>   $ inc -file - < /dev/null
> 
>   Incorporating new mail into inbox...
> 
>   inc: no messages incorporated, continuing...
>   inc: Bug: message out of bounds
> 
> This happened due improper call to `seq_setunseen', which implicitly
> assumed that there is at least one message (it does not handle case
> where both mp->hghmsg and mh->lowmsg are 0).

Thanks for the report.
  
> - seq_setunseen(mp, 1);
> + if (mp->lowmsg) { /* seq_setunseen() assumes at least one message */
> + seq_setunseen(mp, 1);
> + }

I don't think this is the best way to fix this problem. I would rather fix
seq_setunseen(), something like:

diff --git a/sbr/seq_setunseen.c b/sbr/seq_setunseen.c
index d833650..a2ed157 100644
--- a/sbr/seq_setunseen.c
+++ b/sbr/seq_setunseen.c
@@ -23,6 +23,10 @@ seq_setunseen(struct msgs *mp, int doadd)
int n;
char **ap, *cp, *dp;
 
+   if (mp->numsel == 0) {
+   return;
+   }
+
/*
** Get the list of sequences for Unseen-Sequence
** and split them.

This would be my soulution for the problem.

Philipp


pgpPJoYKsKxva.pgp
Description: PGP signature

Re: [mmh] [PATCH] Enable configuration of sendmail program

[Philipp Takacs]

[2016-08-23 17:59] Dmitry Bogatov 
> But what about mentioning it in spost(8)? Manpage (btw, why it is in section 
> 8?)
> mentions following configuration, and does not mention anything else:
> 
>   Aliasfile:   For default alias files
>   Default-From:The default From header
>   Alternate-Mailboxes: The user's addresses
> 
> It creates false sense of completeness. WDYT?

Yes it's missing there and we should add it.

I think spost is section 8 because you should call it only over send(1).

Philipp

Re: [mmh] [PATCH] Enable configuration of sendmail program

[Philipp Takacs]

[2016-08-23 17:04] kact...@gnu.org
> This patch introduces new profile option 'Sendmail', which is name of
> program used to actually send email. Using this option, user can
> specify other mail-sending program, like `msmtp' without installing it
> globally as `/usr/sbin/sendmail'.

If you look at mh-profile(5), you can see we already have this config option.

For your information, this config is done in readconfig(), in the loop of
line 66.

Philipp

Re: [mmh] Testcases

[Philipp Takacs]

[2016-08-11 03:48] Philipp Takacs <phil...@bureaucracy.de>
> [2016-08-10 09:39] markus schnalke <mei...@marmaro.de>
> > Besides the ali test, this is the one failing (as Philipp already
> > mentioned):
> > 
> > tests/repl/test-decode-addr: cat `mhpath +drafts l` failed
> > --- -   2016-08-10 09:12:35.858863138 +0200
> > +++ /tmp/tmp.RxsUilrI1m 2016-08-10 09:12:35.856917739 +0200
> > @@ -1,13 +1,13 @@
> > -To: J?rgen <juer...@example.net>
> > -Cc: b...@example.com, K?the <kae...@example.org>
> > +To: J?rgen
> > +Cc: b...@example.com, K?the
> >  Fcc: +sent
> >  Subject: Re: repl addr decode
> >  In-reply-to: <1as2n8-1q9...@deseo.home.schnalke.org>
> >  References: <1as2n8-1q9...@deseo.home.schnalke.org>
> > -Comments: In-reply-to J?rgen <juer...@example.net>
> > +Comments: In-reply-to J?rgen
> > message dated "Mon, 18 Apr 2016 08:36:14 +0200."
> >  
> > -[2016-04-18 08:36] J?rgen <juer...@example.net>
> > +[2016-04-18 08:36] J?rgen
> >  >
> >  > part   text/plain   4
> >  > foo
> > Test tests/repl/test-decode-addr
> > FAIL
> > 
> > Why are the email addresses not output? It might be related to
> > using
> > %(void(width))%(void(decode))%(putaddr To: )
> > in the replcomps (and similar in replgroupcomps), introduced by
> > 09f632a7c9e4eea0
> 
> Yes and no, the real problem is the funktion decode_rfc2047().
> I have looked a bit deeper in this and also have a smaller test
> program for debuging attached[0].
> 
> I found two thinks so far. First the problem only occur only, if
> mmh is build with gcc. With clang the bug is not there. This means
> either we have undefined/implementationdefined behavior or we found
> a bug in gcc.
> 
> Secound is that the full output is there, but there is a '\0' after the
> encoded part. This means the string looks like this:
> 
> > J?rgen\0 <juer...@example.net>
> 
> This is a real strange bug and I want to know whats the problem. It
> would also be nice to know, if the bug ocours only with UTF-8 to ASCII
> or with diffrent encoding compinations, too.

I have found and fixed it, it's an off by one:

> if (fromutf8) {
>   for (start++;(start < q) && ((*start & 192) == 128);start++)
>   inbytes--;
>   }

start is increased one more then inbytes in decreased. This causes iconv()
to write a extra '\0' to saveq (this is dst).

The problem with gcc and clang was, that I have two different iconv()
implementations on my System. One is from the libc and one is the gnu
version in the ports. If I compile with gcc the gnu version is preferred
with clang the libc-version.

The gnu version of iconv has a bug. If a character of the input string can't
be converted in the dst encoding (e.g. 'ü' in ASCII), then gnu-iconv returns
``(size_t)-1'' instand of setting a placeholder char in the destination.

Philipp

Re: [mmh] Testcases

[Philipp Takacs]

[2016-08-11 03:48] Philipp Takacs <phil...@bureaucracy.de>
> Yes and no, the real problem is the funktion decode_rfc2047().
> I have looked a bit deeper in this and also have a smaller test
> program for debuging attached[0].

I found a bug in my test new version is attached.
#include 
#include 
#include 
#include 

int main() {
	char *in = "=?UTF-8?Q?J=C3=BCrgen?= <juer...@example.net>";
	char out[BUFSIZ];
	char *out2;
	char *in2 = calloc(sizeof(in), sizeof(char *));

	setlocale(LC_ALL, "");

	strcpy(in2, in);

	decode_rfc2047(in2, out, BUFSIZ);
	puts(out);
	out2 = strchr(out, '\0');
	out2++;
	puts(out2);
	return 0;
}

Re: [mmh] Testcases

[Philipp Takacs]

[2016-08-10 09:39] markus schnalke 
> Hoi.
> 
> Philipp, thanks for your work in the test suite. The result is
> much better now.
> 
> Great, you did a start with tests/send/test-mimeify! I'll add some
> more cases.
> 
> 
> Besides the ali test, this is the one failing (as Philipp already
> mentioned):
> 
>   tests/repl/test-decode-addr: cat `mhpath +drafts l` failed
>   --- -   2016-08-10 09:12:35.858863138 +0200
>   +++ /tmp/tmp.RxsUilrI1m 2016-08-10 09:12:35.856917739 +0200
>   @@ -1,13 +1,13 @@
>   -To: J?rgen 
>   -Cc: b...@example.com, K?the 
>   +To: J?rgen
>   +Cc: b...@example.com, K?the
>Fcc: +sent
>Subject: Re: repl addr decode
>In-reply-to: <1as2n8-1q9...@deseo.home.schnalke.org>
>References: <1as2n8-1q9...@deseo.home.schnalke.org>
>   -Comments: In-reply-to J?rgen 
>   +Comments: In-reply-to J?rgen
>   message dated "Mon, 18 Apr 2016 08:36:14 +0200."
>
>   -[2016-04-18 08:36] J?rgen 
>   +[2016-04-18 08:36] J?rgen
>>
>> part   text/plain   4
>> foo
>   Test tests/repl/test-decode-addr
> FAIL
> 
> Why are the email addresses not output? It might be related to
> using
>   %(void(width))%(void(decode))%(putaddr To: )
> in the replcomps (and similar in replgroupcomps), introduced by
> 09f632a7c9e4eea0

Yes and no, the real problem is the funktion decode_rfc2047().
I have looked a bit deeper in this and also have a smaller test
program for debuging attached[0].

I found two thinks so far. First the problem only occur only, if
mmh is build with gcc. With clang the bug is not there. This means
either we have undefined/implementationdefined behavior or we found
a bug in gcc.

Secound is that the full output is there, but there is a '\0' after the
encoded part. This means the string looks like this:

> J?rgen\0 

This is a real strange bug and I want to know whats the problem. It
would also be nice to know, if the bug ocours only with UTF-8 to ASCII
or with diffrent encoding compinations, too.

> If I run the whole testsuite with LC_ALL=C (as is the case for
> the nightly test run (but should be changed to UTF-8 I think))
> this other failure appears:
> 
>   tests/mhbuild/test-linebreak: sed "/^Content-ID:/s/:.*/: /" 
> "$draft" failed
>   --- -   2016-08-10 09:12:29.060968585 +0200
>   +++ /tmp/tmp.s3rYrn73OO 2016-08-10 09:12:29.058794252 +0200
>   @@ -3,7 +3,7 @@
>Date: Thursday, 28 Sep 2006 00:02:00
>Subject: mhbuild line breaking with quoted-printable
>MIME-Version: 1.0
>   -Content-Type: text/plain; charset="UTF-8"
>   +Content-Type: text/plain; charset="US-ASCII"
>Content-ID: 
>Content-Transfer-Encoding: quoted-printable
>
>   Test tests/mhbuild/test-linebreak   
> FAIL
> 
> The draft contains non-ASCII characters, thus US-ASCII is not
> correct, but in the C locale one is just not able to handle the
> characters in the draft correctly, AFAIK.
> 
> What do you think that the right solution is?

The problem is, mhbuild trust the users envirement, The right solution
would be to check the encoding, if it isn't set manual in the mhbuild
draft. Also check the encoding for the body in the attachment code.

I don't think this is realy a bugfix we should do in the releas.

> Is the test only valid for UTF-8 locales? And thus add
> 
>   require_locale en_US.utf-8 en_US.utf8
>   LC_ALL=en_US.UTF-8
>   export LC_ALL
> 
> to those testcases?

For the moment this is the best solution.

Philipp

[0]: compile: $CC -o t -I. -Ih -Lsbr t.c -lmh
 maybe a -liconv is need
#include 
#include 
#include 

int main() {
	char *in = "=?UTF-8?Q?J=C3=BCrgen?= ";
	char out[BUFSIZ];
	char *out2;
	char *in2 = calloc(sizeof(in), sizeof(char *));
	strcpy(in2, in);

	decode_rfc2047(in2, out, BUFSIZ);
	puts(out);
	out2 = strchr(out, '\0');
	out2++;
	puts(out2);
	return 0;
}


pgp4NvTKS2on9.pgp
Description: PGP signature

Re: [mmh] Fix typo in whatnow2 function "display"

[Philipp Takacs]

[2016-08-08 16:14] m...@arascio.xyz
> The attached patch fixes a typo in the whatnow2's "display" function.

Thanks, it's pushed.

Philipp

Re: [mmh] Fix typo in whatnow2.sh

[Philipp Takacs]

[2016-08-06 14:13] m...@arascio.xyz
> Hello,
> 
> The attached patch corrects a typo in uip/whatnow2.sh, introduced in
> commit 8a5c714.

Thanks it's pushed.

Philipp


pgpkAnWufNFR8.pgp
Description: PGP signature

Re: [mmh] Release plan for mmh-0.3

[Philipp Takacs]

[2016-08-04 18:45] markus schnalke 
>
> part   text/plain1060
> [2016-08-01 07:13] markus schnalke 
> > 
> > Okay, so let's freeze next Sunday.
> 
> Work still to do:
> 
> - Always MIMEify patch: I'm working on it. (The If-has-MIME-Version
>   handling is not yet implemented.)

This is done, but no tests are written jet.

> - broken mhsign(1): Philipp is working on it

This is done.

> - musl libc: Can someone help with testing the compatibility?
>   Maybe there's a LiveCD that bases on musl, which we could use
>   to compile and run the testsuite.

I have run the tests on a VM[0]. There are two problems I can't explain:
 - tests/inc/test-deb359167: Is this a bug in musl or valgrind or
   a bug we cause? The nmh version of this test also produces this
   output.

 - tests/scan/test-scan-multibyte: Why is there a '*' instant of
   a '?'. I don't think we need to fix, nmh has the same bug.
   It would be interesting who this happens. I would guess this
   is caused by the iconv implementation of muls/alphine

> - There are some tests that fail. See:
>   http://marmaro.de/prog/mmh/nightly/tests-log/runalltests.log

>   * tests/bad-input/test-header   adjust the testcase or m_getfld2()?

I have adjust this test to the current behavior of mmh.

>   * tests/mhparam/test-mhparam?
>   * tests/mhsign/test-mhsign  (see above)

These test are also fixed.

>   In the release, only the ali(1) test should still fail.

On my System this is the case so this as far as I see everything
for the release is done. Except the tests for send.

This is the last call, if somebody wants a feature be part of mmh-0.3,
you should send it now. I can't promise it will be part of mmh-0.3,
but, if it is to late I promise it won't.

If anyone don't know what to do know here is a list of tasks, which
can be done:
- Test: Let the test suite run and share the result, if something fails.
- Test: install the git-version and test the current version in your
  daily work flow.
- Migrate Test: The nmh guys have a lot of tests,
- Write new Tests

[0]: http://ix.io/1bsC

Re: [mmh] Release plan for mmh-0.3

[Philipp Takacs]

[2016-08-04 19:15] m...@arascio.xyz
> scrīpsit markus schnalke :
> 
> | Work still to do:
> |
> | - There are some tests that fail. See:
> | http://marmaro.de/prog/mmh/nightly/tests-log/runalltests.log
> |   * tests/ali/test-ali  that's okay, we need to port code from nmh
> |   * tests/bad-input/test-header adjust the testcase or m_getfld2()?
> |   * tests/mhparam/test-mhparam  ?
> |   * tests/mhsign/test-mhsign(see above)
> |   In the release, only the ali(1) test should still fail.
> 
> With the attached patch, mhparam passes tests/mhparam/test-mhparam.

Thanks for the patch and the explanation. It is pushed.

Philipp

Re: [mmh] Always MIMEify?

[Philipp Takacs]

[2016-07-28 23:45] markus schnalke 
> I see no reason why this should be problematic. (At least the
> nmh guys didn't wanted to create MIME messages if possible to
> avoid ... but if the MIMEification differs just by these three
> header lines, I can hardly imagine why one would want to prevent
> that.)

Just for the recourd the nmh guys have this already, for the rfc 2047
support.

> The patch is more of a first step for discussion, because I
> think we should have an additional check for a MIME-Version
> header. Currently, if such a header is present, mhbuild(1)
> (which does the MIMEification) would refuse to work. Now that
> we invoke mhbuild(1) on each draft, we are no longer able to
> pass a draft around it. Hence I think that if a MIME-Version
> header is present, we assume to have an already MIMEified
> draft, thus the mhbuild(1) invocation should be omited.

The nmh guys have done this with a [no]auto swith for mhbuild.
I don't know how, if this is easy to port to mmh. So I have written
a patch for send to check if mime-version is set. The attached patch
is based on markus patch.

Philipp
diff --git a/uip/send.c b/uip/send.c
index 4eb8ddf..9cfb156 100644
--- a/uip/send.c
+++ b/uip/send.c
@@ -361,6 +361,10 @@ attach(char *draft_file_name)
 field[length] != ':') {
 			fprintf(composition_file, "%s\n", field);
 		}
+		if (strncasecmp(field, VRSN_FIELD, sizeof(VRSN_FIELD))==0) {
+			clean_up_temporary_files();
+			return DONE;
+		}
 	}
 	fputs("\n", composition_file);
 

[mmh] mhsign.sh

[Philipp Takacs]

Hi

The current version of mhsign isn't working, I have a patch for it.
I haven't fully tested it, but it's on my todo list.

Philipp

Re: [mmh] Release plan for mmh-0.3

[Philipp Takacs]

[2016-07-29 00:26] markus schnalke 
> More concrete: Start to freeze this Sunday or start to freeze in
> a week? What do you think?

Because it's already Sunday I would say in a week.

> Stuff currently in the pipe (in decreasing priority for me):
> 
> - Allow inc(1) to read from stdin (recent topic)
> - Allow send(1) to send from other than draftfolder (not yet announced)
> - Always MIMEify patch (message of today)
> - Whatnow2 as a prototype/preview (message of today)

I would say, we get all this features in.

> - Abort if mkstemp() fails (message of 2015-09-20 14:24:11)

Oh, this again. Has anyone looked at it and tested?

> - Booleans (message of 2016-04-21)

I send a mail on this tomorrow (monday)

> - Solving the part line question (message of 2016-03-06 00:04)

I think this is a big issue, but I don't know how to fix this without
big changes. So I would say we discous this a bit more.

> - Error conditions in m_getfld2 (messages of 2016-05)

AFAIK this is already fixed.

> - From_ quoting and un-quoting (message of 2015-11-12 23:25:55)

This is also an bigger change, I'll write a mail to this tomorrow.

> @Philipp: Is your scan problem of the message you sent at
> 2015-09-19 14:43:56 solved in the meantime?

This was fixed by not using the m_getfld-patches from nmh

Philipp

Re: [mmh] new whatnowproc

[Philipp Takacs]

[2016-07-30 23:34] markus schnalke 
> I just discovered this typo:
> 
> > draftfolder=`mhparam Draft-Folder`
> > if [ -z "$dratffolder" ]
> > then
> > draftfolder="+drafts"
> > fi
> 
> Should be ``draftfolder'' instead of ``dratffolder'' in the
> is-empty test.

Thanks, a fix in the pipe.

> Unfortunately, the profile entry is ``Draft-Folder'' and the
> compile-time value is ``draftfolder''. I'm not sure if they
> should match. All those values in the debugging output were
> the result of ``Let's print the full list of those compile-time
> values from h/mh.h i.e. config/config.c and use the variable
> names''. I'm not really satisfied with the result ist was just
> better than the situation before.

The problem I see hear is that mhparam. I look at this tomorrow.

Philipp

Re: [mmh] new whatnowproc

[Philipp Takacs]

[2016-07-29 00:42] markus schnalke <mei...@marmaro.de>
> [2016-02-27 16:40] Philipp Takacs <phil...@bureaucracy.de>
> > Some meta-info (i.e. last-editor) are stored in a metafile so this is
> > available over different sessions.
> 

> How should your new whatnow behave?
> 
> The next question is: Which one of the drafts is the one we
> are currently working on? You use the current message in the
> draft folder. This is connected to the folder, which is not
> what I would want, because I couldn't work on multiple drafts
> in parallel. I think the ``current'' draft should be
> connected to the context or to the shell session (env var).
> This would allow editing multiple drafts in parallel. Using
> the current message does not allow concurrent editing of
> drafts but only alternated editing of drafts, by switching
> the current message each time. Seems as if I would want to
> have concurrent editing ...

You can work on multiple drafts in parallel, by using multiple
context. This is the default behavior of every other mmh tool
I don't think we shouldn't change that only for drafts.

I want not to have to many check in whatnow2. Most features
are direct provided by the mmh-tools so I don't have to handle
it.

Maybe we can implement this in general: first environment variables,
then a set context, then the .mmh/context. Then mhpath and other
tools could provide this.

> > Some features that I want to implement in the next time:
> 
> > - attach msgs: this is actually a new feature. I think it's useful
> 
> It is no new feature, we already have it. Just use forw(1) and
> have a look at the draft!
> 
> You do it this way:
> 
>   Attach: +folder 123 124
> 
> ... or do you only mean your attach() function below? Maybe
> you could use mhpath(1) to check if the parameters form
> valid messages in mmh notation. Either they do or they all
> have to be valid files in the filesystem. Otherwise it's an
> error.

I mean my attach() function. I wold like to have this, because
sometimes I need to attach other mails to replies. I know I
can write the attachment header by hand, but whatnow can also
do this for me.

> > part 1.2   text/x-shellscript4185 mywhatnow
> 
> I think you should rename it to whatnow2 and add it to the
> VCS ... maybe installing it into lib/ (like mhtest) until
> we settled on your version and it has a man page.

It's pushed.

> > create()
> > {
> > if [ -z "$mhdraft" ]
> > then
> > usage 1
> > fi
> > mhmetafile=$mhdraft.meta
> 
> We can get into trouble if someone uses sortm(1) on +drafts. And
> in cases when mywhatnow is not involved in modifications in the
> drafts folder.
> 
> ... but still, your whatnow proposal is a step forward, thus
> let's not get entangled into such ``details''.
> 
> Generally, I think we need some place to store metadata for drafts
> anyway. There's this problem of annotating the replied-to message.
> Its location is stored in an env var. We annotate at the time of
> sending the draft out, but if we have refiled or sortm'd the
> original message, then the wrong message or none is annotated! I
> don't know how to solve that, yet. Maybe hardlinks or referencing
> Message-Ids could help. Yet, it is a problem similar to the draft
> metadata.

I see this the other way around. I would like to have a meta-data
file in the hole mmh environment, but this is another discussion.

> To be compatible to all other mmh tools, I wonder if we should
> not just store this metadata in the draft itself, e.g. as
> Mmh-Draft-* headers. Of course, it would blow up the draft, but
> if we usually only need a few headers, then it seems to be worth
> to have it all in one place. Fcc and Sign/Enc are of this kind
> already! It appears to be so much more consistent!
> 
> Send(1) would just need to filter those draft headers out. It
> would be compatible to all other tools automatically.
> 
> And all this would be just the same ... only that we'd do it on
> the draft and not on the metafile:
> 
> > send()
> > {
> > export mhaltmsg=`anno -list -component 'mhaltmsg' $mhmetafile`
> > export mhdist=`anno -list -component 'mhdist' $mhmetafile`
> > export mhuse=`anno -list -component 'mhuse' $mhmetafile`
> > export mhfolder=`anno -list -component 'mhfolder' $mhmetafile`
> > export mhmessages=`anno -list -component 'mhmessages' $mhmetafile`
> > export mhannotate=`anno -list -component 'mhannotate' $mhmetafile`
> 
> Send(1) could then get all this information from the draft itself.
> No more need to pass env vars.
> 
> And no more need to call send via whatnow. Just invoke send(1), if
> this information is part

Re: [mmh] mmh and IMAP

[Philipp Takacs]

[2016-07-25 19:02] j...@ssh.bio
> As you see, I am getting started with mmh.  I have mail with gandi.net,
> and have a few device I wish I can read my mail from.
> 
> I've often seen users (maybe all here) have their own server.
> Pulling them all locally, and acessing them via ssh.
> 
> But I do not own a server, at least not yet, so I need something like
> IMAP to preserve the mail on the server, if I'm not wrong...

An shell access on the mailserver should be enough, but this is probably
not the case.

> To do so with mmh, I have to pull the mail to mbox format in
> '/var/mail/sshbio' (or directly to MH format [1]?).  Then 'inc' pulls
> them to '~/Mail' and I can read, and manage the mails.
> 
> But how to get the mail go back to the server, and sync all the changes?
>  
>    __ scan
> ||   IMAP mbox sync > |  |   inc   > || / 
> | Server || mbox |   | mh |<- show
> || < IMAP mbox sync   |__| < packf   || \ 

The problem I see with this setup, is that inc don't know which mails
are new and which not.

> I fear that whatever 'IMAP mbox sync' will have to download a huge mbox
> file each time, is it true?  If so, I will keep using mbsync, which
> works well, but would require this workflow:
> 
>    _  __ 
> ||   mbsync > | |   converter1 > |  |   inc   > ||
> | Server || maildir || mbox |   | mh |- 
> ...
> || < mbsync   |_| < converter2   |__| < packf   ||

This setup has the same problem as the other setup and is more complex.

> This starts to become a bit heavy...  Can mmh read maildir format
> directly?  I get a read error when I do 'mmh -file ~/path-to/maildir/'.

mmh has no maildir implementation and never gets one.

> Finally, would it be possible to organize the mails into multiple folders
> and pack them this way afterward, or would the organization be lost
> by packf?

packf produce mbox, so any organization get lost. You could produce one mbox for
every directory.

> What is your workflow?  I would be glad to hear it, whatever it is,
> and particularly if involves multiple servers

I use mailsync[0] as IMAP-Client, this syncs between IMAP and an mh-dir

  _
 ||  | |
 | Server | < mailsync > | mh  |
 ||  |_|

The advantage of this setup is that there is only mailsync between mh and the 
Server.

The disadvantage is, that mailsync is a horrible software, but as far as I know 
the
only stand alone imap-client with mh-support. Also mailsync has no daemon mode 
so
you have to start it to get your mails synced.

If you want more info about my setup you can look at my old mail[1] where I 
descrype my
setup.

You can also use another mail client with mh support like mutt or claws. I have 
no experiments
with this, but it should work.

A friend of me is writing a imap-client with mh-support. I'll write about this 
on the
mailinglist, when I have some time.

Philipp

[0]: http://mailsync.sourceforge.net/
[1]: http://article.gmane.org/gmane.mail.mmh/83

Re: [mmh] Compilation error for 0.2 ang git

[Philipp Takacs]

[2016-07-25 01:17] j...@ssh.bio
> Commands I ran to get compilation error, from either the git 0.2 tar
> version:
> 
> ~~~
> $ ./autogen.sh  # For git version only
> [...]
> 
> $ ./configure   # I skipped all the 'yes' and the default/normal values
> [...]
> 
> $ make
> [...]
> gcc -s -o forw forw.o whatnowproc.o ../sbr/libmh.a
> ./sbr/libmh.a(fmt_scan.o): In function `fmt_scan':
> fmt_scan.c:(.text+0x6b9): undefined reference to `dparsetime'
> collect2: error: ld returned 1 exit status
> Makefile:116: recipe for target 'forw' failed
> make[1]: *** [forw] Error 1
> make[1]: Leaving directory '/home/sshbio/mmh-0.2/uip'
> Makefile:81: recipe for target 'all-recursive' failed
> make: *** [all-recursive] Error 1
> ~~~

Looks like you don't have a lexer like flex. On the git version you need
one, on the tarball only, if you call make clean.

> My system info:
> 
> ~~~
> $ uname -a
> Linux bunsenlabs 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt25-2 (2016-04-08) 
> x86_64 GNU/Linux
> 
> $ cc --version
> gcc (Debian 5.3.1-20) 5.3.1 20160519
> 
> $ make --version
> GNU Make 4.1
> ~~~

what does `flex --version` say? If the command is not found, you can solve the 
problem
on two ways:

1. Take 'sbr/dtimep.c' from the tarball and copy it to your build dir
2. Install flex

Philipp

Re: [mmh] [PATCH 1/4] whatnow.c/main(): factor out interactive subcommand `display'

[Philipp Takacs]

[2016-07-22 12:42] Dmitry Bogatov 
> 
> > Concerning your criticism of the whatnow-specific problems, the
> > short answer is: Whatnow(1) is supposed to die. There's no sense
> > in improving the implementation of a structurally bad concept.
> > Whatnow is a shell in the shell. It tries to simulate the user's
> > real shell but will never succeed. The problems you describe are
> > the necessary consequences. Your proprosals will improve the
> > worst problems but further problems will pop up right away. The
> > solution is: Dropping whatnow(1) altogether! This is already
> > planned.
> 
> Sad I did not knew about such plans. What do you think about
> ultimately-minimalistic alternative to whatnow(1), like follow?
> 
>  * comp,forw and other commands, that now invoke whatnow(1) just
>write communication variables into ~/.mmh/env.$ppid and propose
>user to source it.
> 
>  * user will source that file and invokes other commands, that will
>use communication variables. It will require some changes to commands, like
>to send(1), which should respect `mhdraft` variable.
> 
>  * it is optional, whether implement such functionality as replacement
>whatnowproc or to -whatnowproc option.
> 
> I like it. Do you? If not, will you accept patches, that support such
> workflow (like send(1) respecting `mhdraft` variable).

It should be avoid to have a to complicated interface. My implementation
is a drop in replacement. comp,forw and other commands call mywhatnow and
mywhatnow is holds the state a metafile.

Also I don't think it's useful to change the interface of other tools,
because I want to be compatible with nmh.

If you are interested in a whatnow replacement the current version of mywhatnow
is attached.

Philipp
#!/bin/sh

printhelp()
{
	echo "Usage: $0 command"
	echo "  commands are:"
	echo "  edit [editor]"
	echo "  list"
	echo "  send [sendargs]"
	echo "  delete"
	echo "  display"
	echo "  attach files"
	echo "  alist"
	echo "  detach anum"
	echo "  refile +folder"
	echo "  -help"
	echo "  -Version"
}

version()
{
	if [ $# -eq 0 ]
	then
		echo "$0 has no own version number, thus this instead:"
		folder -Version
		exit 0
	fi
	echo "$0 has no own version number, thus this instead:" 1>&2
	folder -Version 1>&2
	exit 1
}

usage()
{
	if [ $1 -eq 0 ]
	then
		printhelp
		exit 0
	fi
	printhelp 1>&2
	exit $1
}

get_editor()
{
	if [ -f "$mhmetafile" ]
	then
		lasteditor=`anno -list -component 'last-editor' $mhmetafile`
		if [ -n "$lasteditor" ]
		then
			editor=`echo $lasteditor | cut -d ' ' -f 1`
			mheditor=`mhparam "$editor-next"`
			[ -n "$mheditor" ] && return
			mheditor=$lasteditor
			return
		fi
	fi
	if [ -n "$MMHEDITOR" ]
	then
		mheditor=$MMHEDITOR
		return
	fi
	mheditor=`mhparam 'Editor'`
	if [ -n "$mheditor" ]
	then
		return
	fi
	if [ -n "$VISUAL" ]
	then
		mheditor=$VISUAL
		return
	fi
	if [ -n "$EDITOR" ]
	then
		mheditor=$EDITOR
		return
	fi
	mheditor=vi
}

get_showproc()
{
	mhshowproc=`mhparam 'listproc'`
	return
}

get_realpath()
{
	reldir=`dirname $1`
	filename=`basename $1`
	cd $reldir
	echo "$PWD/$filename"
	cd -
}

get_attachmentheader()
{
	header=`mhparam 'Attachment-Header'`
}

set_lasteditor()
{
	anno -delete -number all -component 'last-editor' $mhmetafile
	anno -nodate -component 'last-editor' -text "$1" $mhmetafile
}

create()
{
	if [ -z "$mhdraft" ]
	then
		usage 1
	fi
	mhmetafile=$mhdraft.meta
	touch $mhmetafile
	if [ -z $mheditor ]
	then
		get_editor
	fi
	if [ -n "$mhaltmsg" ]
	then
		anno -nodate -component 'mhaltmsg' -text "$mhaltmsg" $mhmetafile
	fi
	if [ -n "$mhdist" ]
	then
		anno -nodate -component 'mhdist' -text "$mhdist" $mhmetafile
	fi
	if [ -n "$mhdist" ]
	then
		anno -nodate -component 'mhuse' -text "$mhuse" $mhmetafile
	fi
	if [ -n "$mhfolder" ]
	then
		anno -nodate -component 'mhfolder' -text "$mhfolder" $mhmetafile
	fi
	if [ -n "$mhmessages" ]
	then
		anno -nodate -component 'mhmessages' -text "$mhmessages" $mhmetafile
	fi
	if [ -n "$mhannotate" ]
	then
		anno -nodate -component 'mhannotate' -text "$mhannotate" $mhmetafile
	fi
	set_lasteditor "$mheditor"
	exec $mheditor $mhdraft
}

edit()
{
	if [ $# -eq 0 ]
	then
		get_editor
	else
		mheditor="$@"
	fi

	set_lasteditor "$mheditor"
	exec $mheditor $mhdraft
}

list()
{
	get_showproc
	exec $mhshowproc -file $mhdraft
}

send()
{
	export mhaltmsg=`anno -list -component 'mhaltmsg' $mhmetafile`
	export mhdist=`anno -list -component 'mhdist' $mhmetafile`
	export mhuse=`anno -list -component 'mhuse' $mhmetafile`
	export mhfolder=`anno -list -component 'mhfolder' $mhmetafile`
	export mhmessages=`anno -list -component 'mhmessages' $mhmetafile`
	export mhannotate=`anno -list -component 'mhannotate' $mhmetafile`
	rm -f $mhmetafile
	exec send $@ $mhdraft
}

delete()
{
	rm $mhdraft
	rm $mhmetafile
}

attach()
{
	get_attachmentheader
	while [ -n "$1" ]
	do
		if [ ! -f $1 ]
		then
			echo "file not found: $1" 1>&2
			shift
			echo -n "folloing files are not attached: " 1>&2
			echo -n "$1" 1>&2
			echo "$@" 

Re: [mmh] [PATCH 4/4] Refine function signatures with const

[Philipp Takacs]

[2016-07-21 19:46] markus schnalke 
>
> part   text/plain2069
> [2016-07-21 15:50] Dmitry Bogatov 
> >
> > > I agree that giving this const hint is valuable in these cases.
> > > But I disagree to apply this patch.
> > >
> > > Our code base currently contains almost no const indicators at
> > > all. (Grep for it!) By introducing them at just one location,
> > > we introduce an inconsistency which will irritate the reader.
> > > (At least all of the ad* functions, i.e. adios, admonish,
> > > advertise, advise, ..., should be changed in one go.)
> > >
> > > I'm unsure what consequences we might have to deal with if we
> > > start with const. It would be bad if it leads to compiler
> > > warnings or requiring casts for non-const-to-const conversion.
> > > If we clutter the code with const dealing then we lose as much
> > > as we gain, but if there are few ``costs'' and more clarity,
> > > then we should convert the code in a larger degree.
> > 

A function taking a const argument only effects the function itself.
So the function is not allow to change the value of the variable.
We don't change the format string in adios and advise so this wouldn't
be a problem.

const to noconst casts would produce a warning, this would show bugs we have
made in the implementation of the function.

So all in all, I don't have a problem with this particular changes, when they
finished (adios, advertise, admonish). But only because this variable is more
or less directly passed to printf.

> > Okay. What if I propose patch (big one), which introduces as much
> > const as possible without casts and is gcc-4.9.2 warning free. (I do
> > not have other compilers)
> 
> Philipp (and possibly others) what do you think?

I don't want a big patch to change all possible variables to cost, because
it would be hard to check, if it correct and reasonable.

To the gcc-4.9.2 warning free: If you want to add such big changes, you
should precheck also with other compilers like clang or icc. It shouldn't
be that hard to add them to a common Linux/UNIX.

> > The most important properties of const is following:
> > 
> >  * it makes clear to programmer, whether function argument is
> >input or output.

IMHO this isn't the point of const, just nice side effect.

> ... if he can trust that the actual code does not do things
> differently. This is what I meant with it being a hint: The
> programmer can indicate parameters which he intends to be inputs
> only, but this is not enforced. If one cannot rely on const
> being really used as const, then the situation is even worse.
> But well, this is a matter of rigor and care of the programmer ...

Yes const is only a promise that the value isn't changed.  The
function itself can still change the value, because this can only
insured with read only memory, but then changes produce segfaults.

More or less this is the reason why printf takes const format strings,
to avoid accidentally change the format string, which is in many cases
a string constant.

Another reason for const is, that the caller want to reuse the value.
A good example for this is open(). If the caller want to reuse the
value of the path-argument, he can do this without an expensive copy.
He can do this because he trust the api.

On the other side, for the implementer a const lable can indicate bugs
in the code. If you accidentally change a const value, the compiler
fails or warns you.

So all in all const would be really nice for every argument, if it is
reasonable. But it would be a lot of work. Not only for the one who
implement it, also for us (markus and me), because we need to check
the changes.

For the functions in error.c, it's not really hard so I don't have a problem
with adding the patch.

In general we should focus on other problems first, before focus on const.

Philipp

Re: [mmh] A pair of smalltime mmh fixes

[Philipp Takacs]

Hi Vasily

[2016-07-13 21:04] Vasily Kolobkov 
> I found some minor issues during setting up mmh and thought you
> might want to have a look.

Thanks for your patches. The changes you make look good.

I have two small issue with the patches:

1. The patches are not against master, can you change this?
   The problem is the patch 
0001-Fix-header-lookup-table-setup-in-slocal-1.patch.

2. The patches are Debian-patches. The problem with this is, that
   I don't know how to apply this patches. Can you send raw patches
   or explain how I add this patches to our git?

Philipp

Re: [mmh] better error handling

[Philipp Takacs]

[2016-05-11 13:26] markus schnalke <mei...@marmaro.de>
>
> part   text/plain 492
> [2016-05-08 18:53] Philipp Takacs <phil...@bureaucracy.de>
> >
> > I have added this output, because I like it and not valid mails
> > indicates spammer.
>
> Better write a new program that checks for validity.

I doubt I would use this in my daily workflow, but this is an other
discussion.

> Btw: Why is it necessary to set `state' here?
>
>   case LENERR2:
>   state = FLD2;
>   /* FALL */
>   case FLD2:

It's necessary, because m_getfld2 with something else then FLD2 or BODY2 as
sate is a noop.

>   ...
>
> It would be nicer to be able to write:
>
>   case LENERR2:
>   case FLD2:
>   ...
>
> But that change can be done later.

I'm unsure of this. We discuss this after the release.

> Please push your changes. Refinements can be done later.

Done.

Philipp


pgpn2BVqz9Qm7.pgp
Description: PGP signature

[mmh] better error handling

[Philipp Takacs]

Hi

I have written a patch to handle the error messages of m_getfld2 better.

This patch consider LENERR2 and FMTERR2 not fatal in every case, so a bad
message can be used. IOERR2 is still handle as fatal, because this indicates
an error in fgets or malloc (over getline).

What do you think about this patch?

Philipp
diff --git a/sbr/readconfig.c b/sbr/readconfig.c
index 6975d72..45e1cb6 100644
--- a/sbr/readconfig.c
+++ b/sbr/readconfig.c
@@ -48,6 +48,9 @@ readconfig(struct node **npp, FILE *ib, char *file, int ctx)
 
 	for (state = FLD2;;) {
 		switch (state = m_getfld2(state, , ib)) {
+		case LENERR2:
+			state = FLD2;
+			/* FALL */
 		case FLD2:
 			np = mh_xcalloc(1, sizeof(*np));
 			*npp = np;
@@ -68,7 +71,10 @@ readconfig(struct node **npp, FILE *ib, char *file, int ctx)
 }
 			}
 			continue;
-
+		case FMTERR2:
+			advise(NULL, "%s is poorly formated", file);
+			state = FLD2;
+			continue;
 		case BODY2:
 			adios(EX_CONFIG, NULL, "no blank lines are permitted in %s",
 	file);
@@ -76,6 +82,10 @@ readconfig(struct node **npp, FILE *ib, char *file, int ctx)
 		case FILEEOF2:
 			break;
 
+		case IOERR2:
+			adios(EX_IOERR, NULL, "m_getfld2", "some error happend");
+			break;
+
 		default:
 			adios(EX_CONFIG, NULL, "%s is poorly formatted", file);
 		}
diff --git a/uip/mhparse.c b/uip/mhparse.c
index 67d769c..c250675 100644
--- a/uip/mhparse.c
+++ b/uip/mhparse.c
@@ -250,6 +250,10 @@ get_content(FILE *in, char *file, int toplevel)
 	*/
 	for (compnum = 1, state = FLD2;;) {
 		switch (state = m_getfld2(state, , in)) {
+		case LENERR2:
+			advise(NULL, "To long field");
+			state = FLD2;
+			/* FALL */
 		case FLD2:
 			compnum++;
 
@@ -267,11 +271,13 @@ get_content(FILE *in, char *file, int toplevel)
 			ct->c_begin = ftell(in);
 			break;
 
-		case LENERR2:
 		case FMTERR2:
+			advise(NULL, "message format error in component #%d", compnum);
+			state = FLD2;
+			continue;
+
 		case IOERR2:
-			adios(EX_DATAERR, NULL, "message format error in component #%d",
-	compnum);
+			adios(EX_IOERR, "m_getfld2", "io error");
 
 		default:
 			adios(EX_SOFTWARE, NULL, "getfld() returned %d", state);
diff --git a/uip/new.c b/uip/new.c
index 4335892..efbea04 100644
--- a/uip/new.c
+++ b/uip/new.c
@@ -108,6 +108,10 @@ get_msgnums(char *folder, char *sequences[])
 
 	for (state = FLD2;;) {
 		switch (state = m_getfld2(state, , fp)) {
+		case LENERR2:
+			state = FLD2;
+			/* FALL */
+
 		case FLD2:
 			/*
 			** if it's in a sequence we want,
diff --git a/uip/pick.c b/uip/pick.c
index 5df2861..b0205fd 100644
--- a/uip/pick.c
+++ b/uip/pick.c
@@ -1261,8 +1261,12 @@ plist
 
 		case LENERR2:
 		case FMTERR2:
-		case IOERR2:
 			advise(NULL, "format error in message %d", msgnum);
+			state = FLD2;
+			continue;
+
+		case IOERR2:
+			adios(EX_IOERR, "m_getfld2", "io error on message %d", msgnum);
 			/* FALL */
 
 		case BODY2:
diff --git a/uip/prompter.c b/uip/prompter.c
index c7eb957..bc7cd43 100644
--- a/uip/prompter.c
+++ b/uip/prompter.c
@@ -140,6 +140,10 @@ main(int argc, char **argv)
 	*/
 	for (state = FLD2;;) {
 		switch (state = m_getfld2(state, , in)) {
+		case LENERR2:
+			advise(NULL, "Header to long");
+			state = FLD2;
+			/* FALL */
 		case FLD2:
 			/*
 			** Check if the value of field contains
@@ -212,6 +216,9 @@ abort:
 	}
 } while ((state = m_getfld2(state, , in))
 		==BODY2);
+if (state != FILEEOF2) {
+	adios(EX_IOERR, "m_getfld2", "io error");
+}
 			}
 
 			if (prepend || !qbody) {
@@ -230,8 +237,11 @@ has_no_body:
 			}
 			break;
 
+		case FMTERR2:
+			advise(NULL, "skeleton is poorly formatted");
+			continue;
 		default:
-			adios(EX_DATAERR, NULL, "skeleton is poorly formatted");
+			adios(EX_IOERR, "m_getfld2", "io error");
 		}
 		break;
 	}
diff --git a/uip/slocal.c b/uip/slocal.c
index f8f9d95..26cad52 100644
--- a/uip/slocal.c
+++ b/uip/slocal.c
@@ -758,6 +758,11 @@ parse(int fd)
 	*/
 	for (i = 0, state = FLD2;;) {
 		switch (state = m_getfld2(state, , in)) {
+		case LENERR2:
+			advise(NULL, "format error in message");
+			state = FLD2;
+			/* FALL */
+
 		case FLD2:
 			lp = mh_xstrdup(f.value);
 			for (p = hdrs; p->p_name; p++) {
@@ -793,7 +798,6 @@ parse(int fd)
 		case FILEEOF2:
 			break;
 
-		case LENERR2:
 		case FMTERR2:
 		case IOERR2:
 			advise(NULL, "format error in message");
diff --git a/uip/sortm.c b/uip/sortm.c
index 56fa7cb..061ae5a 100644
--- a/uip/sortm.c
+++ b/uip/sortm.c
@@ -331,6 +331,10 @@ get_fields(char *datesw, int msg, struct smsg *smsg)
 	}
 	for (compnum = 1, state = FLD2;; compnum++) {
 		switch (state = m_getfld2(state, , in)) {
+		case LENERR2:
+			admonish(NULL, "To long header field in message %d (header %s, #%d)", msg, f.name, compnum);
+			state = FLD2;
+			/* FALL */
 		case FLD2:
 			if (mh_strcasecmp(f.name, datesw)==0) {
 datecomp = mh_xstrdup(f.value);
@@ -350,7 +354,6 @@ get_fields(char *datesw, int msg, struct smsg *smsg)
 		case FILEEOF2:
 			break;
 
-		case LENERR2:
 		case FMTERR2:
 		case 

Re: [mmh] Issues following the m_getfld merge

[Philipp Takacs]

[2016-04-23 19:02] markus schnalke <mei...@marmaro.de>
> Thanks for your reply.
>
> [2016-04-23 17:51] Philipp Takacs <phil...@bureaucracy.de>
> > [2016-04-23 14:38] markus schnalke <mei...@marmaro.de>
> > >
> > > 1) mh_sequences is poorly formatted
> > >
> > >   :-Q flists
> > >   flists: /home/meillo/Mail/l/tuhs/.mh_sequences is poorly formatted
> > >
> > >   :-Q awk '{print length()}' /home/meillo/Mail/l/tuhs/.mh_sequences
> > >   7
> > >   263
> > >   1041
> > >   17
> > >
> > > Seems to be the long line number three.
> > >
> > > Even if RFC 822 does not support such long lines, mark(1) does
> > > generate them in mh_sequences files.
> >
> > I think this should also fixed,
>
> Maybe and maybe not. See the explanation of the reason for the
> length limit in RFC 2822:
>
>The 998 character limit is due to limitations in many implementations
>which send, receive, or store Internet Message Format messages that
>simply cannot handle more than 998 characters on a line.
>
> As mh_sequences files will never enter the mail transfer system,
> there is no need for them to stick to that limit. If mmh can
> handle longer lines, mh_sequences files may well have longer
> lines. Of course, we could fold them nonetheless, but the RFC's
> line length limitation goes in a different direction, unrelated
> to mh_sequences files.

Yes, this is right. I'll remove the warning from the patch.

> > > I think, m_getfld2 should tolerate such long lines.
> > >
> > > The mh_sequences file is attached. We should make a testcase out of
> > > it.
> >
> > m_getfld2 does support this long lines, but it returns a LENERR2. So the
> > caller can choose how to handle this. I have written a small patch for
> > m_getfld2() now the hole field is read, if a line is to long. The patch
> > also adjust seq_read(), now to long lines are not a case to fail.
>
> Again some sentences from the RFC (section 2.1.1):
>
>Receiving
>implementations would do well to handle an arbitrarily large number
>of characters in a line for robustness sake.
>
>Again, even though this limitation is put on
>messages, it is encumbant upon implementations which display messages
>to handle an arbitrarily large number of characters in a line
>(certainly at least up to the 998 character limit) for the sake of
>robustness.
>
> Given these words, I think that long lines should not generally
> be considered errors (LENERR2). In message reading they should just
> be considered to be valid input! Even more, ``encumbant'' appears
> to be misspelled for ``incumbent'', which means ``necessary'' --
> .. necessary for robustness to accept an arbitrary lange number
> of characters in a line! Thus no error messages but normal input
> if a line has 9000 chars.
>
> Only concerning the generation of messages we MUST stick to the
> 998 char limit and SHOULD stick to the 78 char limit:
>
>Each line of characters MUST be no more than
>998 characters, and SHOULD be no more than 78 characters, excluding
>the CRLF.
>
>However, there are so
>many implementations which (in compliance with the transport
>requirements of [RFC2821]) do not accept messages containing more
>than 1000 character including the CR and LF per line, it is important
>for implementations not to create such messages.
>
> In conclusion, I think that m_getfld2 should not care about too
> long lines, because they are perfectly valid in almost all cases.
> The only case where they are not valid is on message generation
> .. or maybe more exact: Messages that leave the MH system. I see
> two places where messages leave MH: send/spost and packf (maybe
> there are more). Only these should care for folding too long
> lines.
>
> Opinions?

Yes for the mmh-parts to long lines aren't a problem, but you miss some
points.

1) Inside the private mh-dir doesn't mean it is only available for mmh.
I use mailsync to synchronize my mail over imap, and there are other
possibility's other software access this mail, like using mutt/emacs
as second MUA.

2) It isn't easy to check for a length error after m_getfld2 returned
in the header, because it's gives every line of a field. m_getfld2
can easy detect a length error, because it handle every line separate.

3) m_getfld2 doesn't really care[0] about too long lines, it just notify
the caller this field contains a to long line or this line in the
body is to long. The problem is most mmh-parts consider a LENERR2 as
a cause to fail hard.

I would say we can drop the length check in the body, because the
caller can do this without a problem, but keep it in the header.

Philipp

[0] I imply my patch for m_getfld2. Before this patch some problems
could appear.


pgpuNoAoJv0KV.pgp
Description: PGP signature

Re: [mmh] Fwd: Replace getcpy() and strdup() with mh_xstrdup()

[Philipp Takacs]

[2016-04-03 21:33] m...@arascio.xyz
> The attached patch replaces getcpy() and strdup() with mh_xstrdup() as
> discussed previously.  Specifically,

Thanks for your work I have added it with two little adjustment.

>
>   - mh_xstrdup() calls adios() in case of memory allocation error;

I have changed the adios call a bit, errno is checked now.

>   - getcpy() is replaced by strdup() if the return value is checked
> for an error.

The two places (uip/mhparse.c) don't check the return value for NULL.
The code dereferenc the returned pointer and check if this is '\0'. So this
would cause a segmentation fault, if strdup fails.

Philipp


pgpcOVGxBeIDQ.pgp
Description: PGP signature

Re: Replace free() with mh_free0() (previously "Replace mh_xmalloc() with mh_xcalloc()", "[mmh] Future work")

[Philipp Takacs]

[2016-03-21 20:28] m...@arascio.xyz
>
> The attached patch replaces all calls to free() with the discussed
> function mh_free0(), which I've added to sbr/utils.c.

Thanks for this patch. I have added a little detail. In some cases a
pointer is set to NULL write after the free, I removed this, because
it's redundant now.

Philipp


pgpiiCLw_m6I8.pgp
Description: PGP signature

Re: [mmh] Replace getcpy() and strdup() with mh_xstrdup().

[Philipp Takacs]

Hi

[2016-03-16 09:05] m...@arascio.xyz
> One item in the list found in docs/TODO was replace getcpy() with
> strdup().  So I've added a subroutine to sbr/utils.c called "mh_xstrdup"
> with which I've replaced all calls to getcpy() and to strdup(), except
> those in sbr/mf.c; the file sbr/mf.c defines its own getcpy(), so I've
> left it alone.  See the attached patch.

Thanks for your patch. But your patch doesn't cover all the problems.

getcpy() internal use mh_xcalloc, which calls exit() if it fail, but
strdup and mh_xstrdup does not. The problem is in the libary[0] (sbr-dir)
it's a bad idea to have a function that exit the hole programm.

In the programm this is just a nice feature, because you can asume, if
malloc fail the programm exit. Therefor you can just use it without
checking the return value and call exit() manual.

Maybe you add an boolean argument to mh_xstrdump() to choos, if the
caller checks the return value or not.

The call of advise should also set the ``what'' argument, this causes
advertise to check errno and print the accourding error message.

Philipp

[0]: I know there are a bunch of getcpy/mh_xcalloc/adios calls in the
libary, but we shouldn't add more.


pgpueQnnuwAhv.pgp
Description: PGP signature

Re: Replace mh_xmalloc() with mh_xcalloc() (previously "[mmh] Future work")

[Philipp Takacs]

[2016-03-14 09:29] m...@arascio.xyz
> scrīpsit markus schnalke :
> 
> > [2016-03-13 12:46] m...@arascio.xyz
> > > 
> > > As suggested, the attached patch changes calls to mh_xmalloc() into calls
> > > to mh_xcalloc().
> > 
> > If feasable, I would suggest, to apply the changes in two commits:
> > 
> > 1) replace mh_xmalloc() with mh_xcalloc()
> > 2) remove the casts
> 
> Thanks for the feedback.  The attached patch accomplishes the first of
> these two suggested commits.  It leaves existing casts alone and it leaves
> the parentheses used with sizeof alone.

Thanks for the work, it's committed.

Do you have a patch, which removes the casts?

[2016-03-13 12:46] m...@arascio.xyz
>
> But I don't understand the idea of an mh_xfree(void **).  Would you mind
> explaining the idea further?  Would the mh_xfree() be used everywhere
> instead of free()?  If I understand more, I may be able to complete the task.

After a discussion with Markus we have decided to call this function better
mh_free0(void *). The idea is to have a function which not only frees the
memory, it should also set the pointer to NULL. An example implementation
is attached.

With this function following code should run without a problem:

char *ptr = mh_xcalloc(42, sizeof(char *));
//do work
mh_free0();
assert(ptr == NULL);

Philipp
#include 

void
mh_free0(void *ptr)
{
	void **pointer = ptr;
	free(*pointer);
	*pointer = NULL;
}


pgptbNr9ULvPB.pgp
Description: PGP signature

Re: [mmh] reply and MIME

[Philipp Takacs]

[2016-03-01 20:51] markus schnalke 
> Maybe try `show -part 1'. It's enough to quote the first MIME part;
> strange attachments come later.

Actually I like this behavior. Sometimes I want to use parts of the
attachments in my answer. For example if I reply to a message with
a patch attached, or some source code.

> One issue I encounter is the ``part'' line that is printed before
> the body text. I have to remove that each time from the quoted
> text.

Yes the ``part'' line is a bit annoying, but less pain then before. The problem
here is this default config (actually it's more a fallback):

> mhshow-show-text/plain: %liconv -f %c

The ``%l'' creates the ``part'' line.

I have two idea to remove this:

First we could add an extra switch to show like ``-nopartline'' which
could suppress the ``part'' line.

Second we could extend the mhshow-show-/ config option with
the arg0. So that we check mhshow--/. Then call show
with an other arg0 (i.e. replshow). Now we could say the default for
replshow is without the ``%l''.

The second approach could also solve the problem with the attachments, the
user could set such defaults for replshow.

What do you think?

Philipp


pgp7qdE0XdfYF.pgp
Description: PGP signature

[mmh] reply and MIME

[Philipp Takacs]

Hi

I have recently looked at the repl problem with MIME-mails and had
a simple idea: Just use show.

To explain at the moment the body of an answer is produced the following way:

$ < $mail | mhl -form $filter >> $newdraft

The problem now is, mhl can't decode MIME. But we have already a tool to decode
MIME: show. So my idea is just produce the new body in the following way:

$ < $mail | show -file - | mhl -form $filter >> $newdraft

I have attached a patch which does this. What do you think about this?

Philipp
diff --git a/uip/repl.c b/uip/repl.c
index f5df9ec..71ae98e 100644
--- a/uip/repl.c
+++ b/uip/repl.c
@@ -740,7 +740,8 @@ insert(struct mailname *np)
 static void
 replfilter(FILE *in, FILE *out, char *filter)
 {
-	int pid, n;
+	int pid, pid_show, n;
+	int mailpipe[2];
 	char *errstr;

 	if (filter == NULL)
@@ -752,12 +753,32 @@ replfilter(FILE *in, FILE *out, char *filter)
 	rewind(in);
 	lseek(fileno(in), (off_t) 0, SEEK_SET);

-	switch (pid = fork()) {
+	if (pipe(mailpipe) == -1) {
+		adios(EX_OSERR, "pipe", "can't create pipe");
+	}
+
+	switch (pid_show = fork()) {
 	case NOTOK:
 		adios(EX_OSERR, "fork", "unable to");

 	case OK:
 		dup2(fileno(in), fileno(stdin));
+		dup2(mailpipe[1], fileno(stdout));
+		for (n=3; n

Re: [mmh] [PATCH] Add existing "From:" to Bcc-ed messages.

[Philipp Takacs]

[2016-01-13 15:15] markus schnalke 
> @Philipp: Have you looked at the code already?

Yes and I have a few comments:

Is it the best way to copy the hole from-field? Wouldn't it be better
to find the mailaddress in the same way as for the sender? In most cases
this would be the same. I'm unsusre about that.

At the moment the from is set multible times, if you have multible own
addresse in the from-field. This should be done only once.

At the point where the from-field is put in the bcc-draft you should
use brackets for the ``if''.

Philipp


pgp6nXQaBEWtv.pgp
Description: PGP signature