Re: mhbuild and long header fields
[2023-09-17 21:51] David Levine
> > A format-patch is attached.
>
> Thanks. I think that the commit message should note that only header field
> bodies are folded. And because folding can only occur a
> t whitespace, it is possible to end up with more than 78 bytes in a line.
> How about this?
>
> mhbuild now folds header field bodies to avoid lines with more then 78
> bytes.
>
> This is required by RFC 5322 2.1.1. Line Length Limits.
>
> I'll add the first line of the commit message to docs/pending-release-notes.
> And add the year to the copyright notice in fold.[hc].
Done, updated version is attached.
Philipp
From 3157510d821db9bf62b051aeb1dcc94fac1dffcd Mon Sep 17 00:00:00 2001
From: Philipp Takacs
Date: Fri, 25 Aug 2023 09:29:42 +0200
Subject: [PATCH] mhbuild implement header folding
mhbuild now folds header field bodies to avoid lines with more then 78 bytes.
This is required by RFC 5322 2.1.1. Line Length Limits.
---
Makefile.am | 2 ++
sbr/fold.c| 63 +++
sbr/fold.h| 7 +
test/mhbuild/test-mhbuild | 53
uip/mhoutsbr.c| 7 -
5 files changed, 131 insertions(+), 1 deletion(-)
create mode 100644 sbr/fold.c
create mode 100644 sbr/fold.h
diff --git a/Makefile.am b/Makefile.am
index 4fc84c1d..168d9fe6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -378,6 +378,7 @@ noinst_HEADERS = \
sbr/fmt_new.h \
sbr/fmt_rfc2047.h \
sbr/fmt_scan.h \
+sbr/fold.h \
sbr/folder_addmsg.h \
sbr/folder_delmsgs.h \
sbr/folder_free.h \
@@ -1106,6 +1107,7 @@ sbr_libmh_a_SOURCES = \
sbr/fmt_new.c \
sbr/fmt_rfc2047.c \
sbr/fmt_scan.c \
+sbr/fold.c \
sbr/folder_addmsg.c \
sbr/folder_delmsgs.c \
sbr/folder_free.c \
diff --git a/sbr/fold.c b/sbr/fold.c
new file mode 100644
index ..f6047703
--- /dev/null
+++ b/sbr/fold.c
@@ -0,0 +1,63 @@
+/* fold.c -- fold a mail header field
+ *
+ * This code is Copyright (c) 2023, by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+#include "h/mh.h"
+#include "h/mime.h"
+#include "sbr/charstring.h"
+#include "fold.h"
+
+void
+fold(charstring_t dst, size_t namelen, const char *restrict body)
+{
+ const char *restrict body_next;
+ const char *restrict wsp;
+ const char *restrict wsp_next;
+ const bool crlf = strchr(body, '\r');
+ charstring_clear(dst);
+ namelen++;
+
+ while (*body) {
+ body_next = strchr(body, '\n');
+ if ((unsigned long) (body_next - body) <= MAXTEXTPERLN - namelen) {
+ charstring_push_back_chars(dst, body, body_next - body + 1, body_next - body + 1);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+ wsp = body;
+ while (namelen == 0 && (*wsp == ' ' || *wsp == '\t')) {
+ wsp++;
+ }
+ wsp = wsp_next = strpbrk(wsp, " \t");
+
+ /* if now whitespace is in the current line just print the curret line as is */
+ if (!wsp_next || wsp_next > body_next) {
+ charstring_push_back_chars(dst, body, body_next - body + 1, body_next - body + 1);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+
+ while ((unsigned long)(wsp_next - body) <= MAXTEXTPERLN - namelen) {
+ wsp = wsp_next;
+ wsp_next = strpbrk(wsp+1, " \t");
+ if (!wsp_next) {
+break;
+ }
+ if (wsp_next > body_next) {
+break;
+ }
+ }
+
+ charstring_push_back_chars(dst, body, wsp - body, wsp - body);
+ if (crlf) {
+ charstring_push_back(dst, '\r');
+ }
+ charstring_push_back(dst, '\n');
+ namelen = 0;
+ body = wsp;
+ }
+}
diff --git a/sbr/fold.h b/sbr/fold.h
new file mode 100644
index ..40bf6a49
--- /dev/null
+++ b/sbr/fold.h
@@ -0,0 +1,7 @@
+/* fold.h -- fold a mail header field
+ *
+ * This code is Copyright (c) 2023, by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+void fold(charstring_t dst, size_t namelen, const char *restrict body);
diff --git a/test/mhbuild/test-mhbuild b/test/mhbuild/test-mhbuild
index 706a804a..f8d4992b 100755
--- a/test/mhbuild/test-mhbuild
+++ b/test/mhbuild/test-mhbuild
@@ -221,5 +221,58 @@ run_test "mhbuild $f" \
check "$f" "$expected"
+start_test "Checking for correct header folding"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+MIME-Version: 1.0
+Content-Type: text/plain; charset="us-ascii"
+
+This is a test
+E
+
+run_test "mhbuild -auto `mhpath last`"
+check "`mhpath last`" "$expected"
+
+star
Re: mhbuild and long header fields
[2023-09-17 18:32] David Levine
> Philipp wrote:
>
> > Good to hear. Do you want a format-patch of the final version?
>
> Sure, that would be great. Or, all I need is the commit message. I just
> verified that I'm using your last code submission with no
> changes.
A format-patch is attached.
Thanks for good feedback.
Philipp
From c2fce54f46b3a393457b242c65b0e0af95ecfa21 Mon Sep 17 00:00:00 2001
From: Philipp Takacs
Date: Fri, 25 Aug 2023 09:29:42 +0200
Subject: [PATCH] mhbuild implement header folding
mhbuild now folds header fields to prevent lines with more then 78
bytes. This is required by RFC 5322 2.1.1. Line Length Limits.
---
Makefile.am | 2 ++
sbr/fold.c| 63 +++
sbr/fold.h| 7 +
test/mhbuild/test-mhbuild | 53
uip/mhoutsbr.c| 7 -
5 files changed, 131 insertions(+), 1 deletion(-)
create mode 100644 sbr/fold.c
create mode 100644 sbr/fold.h
diff --git a/Makefile.am b/Makefile.am
index 4fc84c1d..168d9fe6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -378,6 +378,7 @@ noinst_HEADERS = \
sbr/fmt_new.h \
sbr/fmt_rfc2047.h \
sbr/fmt_scan.h \
+sbr/fold.h \
sbr/folder_addmsg.h \
sbr/folder_delmsgs.h \
sbr/folder_free.h \
@@ -1106,6 +1107,7 @@ sbr_libmh_a_SOURCES = \
sbr/fmt_new.c \
sbr/fmt_rfc2047.c \
sbr/fmt_scan.c \
+sbr/fold.c \
sbr/folder_addmsg.c \
sbr/folder_delmsgs.c \
sbr/folder_free.c \
diff --git a/sbr/fold.c b/sbr/fold.c
new file mode 100644
index ..d80cfe2d
--- /dev/null
+++ b/sbr/fold.c
@@ -0,0 +1,63 @@
+/* fold.c -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+#include "h/mh.h"
+#include "h/mime.h"
+#include "sbr/charstring.h"
+#include "fold.h"
+
+void
+fold(charstring_t dst, size_t namelen, const char *restrict body)
+{
+ const char *restrict body_next;
+ const char *restrict wsp;
+ const char *restrict wsp_next;
+ const bool crlf = strchr(body, '\r');
+ charstring_clear(dst);
+ namelen++;
+
+ while (*body) {
+ body_next = strchr(body, '\n');
+ if ((unsigned long) (body_next - body) <= MAXTEXTPERLN - namelen) {
+ charstring_push_back_chars(dst, body, body_next - body + 1, body_next - body + 1);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+ wsp = body;
+ while (namelen == 0 && (*wsp == ' ' || *wsp == '\t')) {
+ wsp++;
+ }
+ wsp = wsp_next = strpbrk(wsp, " \t");
+
+ /* if now whitespace is in the current line just print the curret line as is */
+ if (!wsp_next || wsp_next > body_next) {
+ charstring_push_back_chars(dst, body, body_next - body + 1, body_next - body + 1);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+
+ while ((unsigned long)(wsp_next - body) <= MAXTEXTPERLN - namelen) {
+ wsp = wsp_next;
+ wsp_next = strpbrk(wsp+1, " \t");
+ if (!wsp_next) {
+break;
+ }
+ if (wsp_next > body_next) {
+break;
+ }
+ }
+
+ charstring_push_back_chars(dst, body, wsp - body, wsp - body);
+ if (crlf) {
+ charstring_push_back(dst, '\r');
+ }
+ charstring_push_back(dst, '\n');
+ namelen = 0;
+ body = wsp;
+ }
+}
diff --git a/sbr/fold.h b/sbr/fold.h
new file mode 100644
index ..8d0fd6d7
--- /dev/null
+++ b/sbr/fold.h
@@ -0,0 +1,7 @@
+/* fold.h -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+void fold(charstring_t dst, size_t namelen, const char *restrict body);
diff --git a/test/mhbuild/test-mhbuild b/test/mhbuild/test-mhbuild
index 706a804a..f8d4992b 100755
--- a/test/mhbuild/test-mhbuild
+++ b/test/mhbuild/test-mhbuild
@@ -221,5 +221,58 @@ run_test "mhbuild $f" \
check "$f" "$expected"
+start_test "Checking for correct header folding"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+MIME-Version: 1.0
+Content-Type: text/plain; charset="us-ascii"
+
+This is a test
+E
+
+run_test "mhbuild -auto `mhpath last`"
+check "`mhpath last`" "$expected"
+
+start_test "Checking header folding with a to long line"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+
+MIME-Ve
Re: mhbuild and long header fields
[2023-09-16 12:36] David Levine > Phillipp, I've been using your header field folding for a few weeks and I > really like it. I think that it should be committed, it's big step forward. Good to hear. Do you want a format-patch of the final version? > One issue to resolve first: the References header field at the end of this > message ends with: > <16f13de3df80f62de768420a3cc3a2fe.phil...@bureaucracy.de> > <14298-1693614066.188834@mIBq.BRD6.6M0V> > > There's always an extra space in front of the Message-ID that was appended > last. Is that a relic of repl? That's strange and I can't reproduce this. Also I can't find code which would explain this in repl and the default configuration. Maybe in the convert interface of mhbuild? > Or, should the folding code collapse multiple white space characters at the > beginning of a line? No, fold() should not change the body of the field. Depending on the context multible white space characters might have a meaning or are used to increase readability without a MUA. Philipp
Re: some thought on m_getfld
[2023-09-02 14:41] Ken Hornstein > >I'm not sure if it's a good idea to start this topic, but it bugs me a > >bit. Also as a disclaimer my view on this is highly influenced by my own > >m_getfld variant[0]. > > > >Because of the missunderstanding of when LENERR is used I have looked > >at m_getfld.c and I see some problems with the interface and the > >implemetation. > >[...] > > First, my apologies for not contributing more on the discussion regarding > header folding; my life has been busy as of late. But I have some thoughts > on this. > > I think you are thinking at the wrong scale. The problem with m_getfld() > is not the interface, it is that it should not exist at all. Actually not quite, I think step by step. First get a good function parsing the fields. Then look how this function can be used to parse the complete header. > As you note, there's a ton of duplicated code when dealing with m_getfld(). > This has led me to believe that the number of programs that actually need > a m_getfld()-style interface is close to zero. I think programs tend to > fall into two styles of header access: > > - One where they process each header field at a time (like the config file > parser). > - One where they need to slurp in all header fields and then just extract > the ones they care about (I think this is the most common). I would split up your second case in two: - One where all fields are read and processed, but some fields are processed in a special way (like post) - One where only some fields are needed (like pick) > So this suggests to me that really in most cases the whole header should > just be read and then you get some accessor functions to get out the > headers you care about. > [...] > I realize that this is a larger project and if someone just wants to > improve m_getfld() that is fine with me. A function to parse the complete header would be nice, but as you mention this is a larger project. The problem header parsing is required everywhere in nmh. Implementing such a function direct would required to either rewrite every m_getfld user at once or having two parser at the same time. So to separate this first improve the field parser and build a high level API based on this parser. This way there would only one parse and the single programs could be updated step by step. > In terms of implementation, as you note header fields have a well-defined > structure so honestly my preference would be to write a flex lexer to > process header fields. Because I have not that much experience with flex, I have handcrafted my m_getfld. But using flex (and yacc) sounds like a good idea. > But really I think there should be a higher level > API to deal with header fields rather than have nmh code call m_getfld(). Some time ago I have started a getfields() function for this. This function collect either all fields or the one specified by a filter. When you are interested in it, I can share it. Philipp > --Ken >
some thought on m_getfld
Hi I'm not sure if it's a good idea to start this topic, but it bugs me a bit. Also as a disclaimer my view on this is highly influenced by my own m_getfld variant[0]. Because of the missunderstanding of when LENERR is used I have looked at m_getfld.c and I see some problems with the interface and the implemetation. Let's start with the interface: m_getfld takes a char *buf and an int *bufsz for the field body and length. The bufsz is used as input and output parameter. If a field exceeds the legth of the of the buffer FLDPLUS is returned and the caller is responsible to call m_getfld again and concat the field body. Most callers use fmt_appendcomp in a loop to do this. This leads to a huge amount of duplicated code. I would say m_getfld should dynamic allocate the buffer. This way m_getfld could returen exact one field per call. For the body m_getfld should return one line per call. Next let look at LENERR and FMTERR. FMTERR is to indecate a format error and LENERR indicates that the field name is to long. I don't see why to distinguish these errors and most of the code handle them the same way[1]. Speaking of length, when the caller would like to know if a field contains a line with more then 998 bytes he has to check it on it's own. But for m_getfld this would be simple to check. I know this would be ignored in most cases. But for mhbuild and post this could be usefull. m_getfld also supports mbox and MMDF. There are only a few places where this is necesary. With a field/line based API it would be quite easy to implement this on top of m_getlfd. Next lets look at the implementation: For fields m_getlfd reads byte by byte from it's internal state and checks how to handle it depending on the current situation. A header is quite good structured so in most cases something like strchr could be used. There is this complex state struct which mimics some parts of buffered IO. Is this realy necesary? I have avoided implementing buffered IO myself by using getline(3). There might be other ways to implemet this without a so complex state. I don't want to tell you how you have to implement m_getfld. I just hope my view on this helps you to improve m_getfld. Philipp [0] http://git.marmaro.de/?p=mmh;a=blob;f=sbr/m_getfld2.c;h=b9a618d166e245a37854003610f5a9cb1576ef0c;hb=HEAD [1] at the cases I have looked at there is not even an indication in the error message which sate caused the error
Re: mhbuild and long header fields
[2023-08-31 22:14] David Levine > Philipp wrote: > > > I have a version with charstring_t attached. I'm unsure if it's better > > to only fold the body or include the field name. The version attached > > only fold the body. > > RFC 5322 §2.2.3 only mentions folding the body. And field names > can't have whitespace. So I think that only a body can be folded. This was to vague asked by me. I was woundering about the API. Should fold construct a string of the complete field or only the body. Philipp
Re: mhbuild and long header fields
[2023-08-28 11:53] Philipp
> [2023-08-27 22:00] David Levine
> > Philipp wrote:
> >
> > > [2023-08-27 09:29] David Levine
> > > >
> > > > My only comment on the code itself is that I prefer functions that
> > > > just do one thing. So I would implement a fold function that just
> > > > modifies a string, and leave the fprintf/fwrite to output_headers().
> > >
> > > I have thought about this, but this would require fold() to allocate
> > > memory.
> > > Because the plan was to use fold() in output_headers() I though this could
> > > be avoided.
> >
> > I don't think that allocating memory is a drawback.
>
> I first wrote a long text about the missing string abstraction, then
> I found charstring_t. I'll adjust this a few days.
I have a version with charstring_t attached. I'm unsure if it's better
to only fold the body or include the field name. The version attached
only fold the body.
Philipp
diff --git a/Makefile.am b/Makefile.am
index 4fc84c1d..168d9fe6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -378,6 +378,7 @@ noinst_HEADERS = \
sbr/fmt_new.h \
sbr/fmt_rfc2047.h \
sbr/fmt_scan.h \
+sbr/fold.h \
sbr/folder_addmsg.h \
sbr/folder_delmsgs.h \
sbr/folder_free.h \
@@ -1106,6 +1107,7 @@ sbr_libmh_a_SOURCES = \
sbr/fmt_new.c \
sbr/fmt_rfc2047.c \
sbr/fmt_scan.c \
+sbr/fold.c \
sbr/folder_addmsg.c \
sbr/folder_delmsgs.c \
sbr/folder_free.c \
diff --git a/sbr/fold.c b/sbr/fold.c
new file mode 100644
index ..d80cfe2d
--- /dev/null
+++ b/sbr/fold.c
@@ -0,0 +1,63 @@
+/* fold.c -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+#include "h/mh.h"
+#include "h/mime.h"
+#include "sbr/charstring.h"
+#include "fold.h"
+
+void
+fold(charstring_t dst, size_t namelen, const char *restrict body)
+{
+ const char *restrict body_next;
+ const char *restrict wsp;
+ const char *restrict wsp_next;
+ const bool crlf = strchr(body, '\r');
+ charstring_clear(dst);
+ namelen++;
+
+ while (*body) {
+ body_next = strchr(body, '\n');
+ if ((unsigned long) (body_next - body) <= MAXTEXTPERLN - namelen) {
+ charstring_push_back_chars(dst, body, body_next - body + 1, body_next - body + 1);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+ wsp = body;
+ while (namelen == 0 && (*wsp == ' ' || *wsp == '\t')) {
+ wsp++;
+ }
+ wsp = wsp_next = strpbrk(wsp, " \t");
+
+ /* if now whitespace is in the current line just print the curret line as is */
+ if (!wsp_next || wsp_next > body_next) {
+ charstring_push_back_chars(dst, body, body_next - body + 1, body_next - body + 1);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+
+ while ((unsigned long)(wsp_next - body) <= MAXTEXTPERLN - namelen) {
+ wsp = wsp_next;
+ wsp_next = strpbrk(wsp+1, " \t");
+ if (!wsp_next) {
+break;
+ }
+ if (wsp_next > body_next) {
+break;
+ }
+ }
+
+ charstring_push_back_chars(dst, body, wsp - body, wsp - body);
+ if (crlf) {
+ charstring_push_back(dst, '\r');
+ }
+ charstring_push_back(dst, '\n');
+ namelen = 0;
+ body = wsp;
+ }
+}
diff --git a/sbr/fold.h b/sbr/fold.h
new file mode 100644
index ..8d0fd6d7
--- /dev/null
+++ b/sbr/fold.h
@@ -0,0 +1,7 @@
+/* fold.h -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+void fold(charstring_t dst, size_t namelen, const char *restrict body);
diff --git a/test/mhbuild/test-mhbuild b/test/mhbuild/test-mhbuild
index 706a804a..f8d4992b 100755
--- a/test/mhbuild/test-mhbuild
+++ b/test/mhbuild/test-mhbuild
@@ -221,5 +221,58 @@ run_test "mhbuild $f" \
check "$f" "$expected"
+start_test "Checking for correct header folding"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+MIME-Version: 1.0
+Content-Type: text/plain; charset="us-ascii"
+
+This is a test
+E
+
+run_test "mhbuild -auto `mhpath last`"
+check "`mhpath last`" "$expected"
+
+start_test "Checking header folding with a to long line"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+
+
Re: mhbuild and long header fields
[2023-08-27 22:00] David Levine > Philipp wrote: > > > [2023-08-27 09:29] David Levine > > > > > > My only comment on the code itself is that I prefer functions that > > > just do one thing. So I would implement a fold function that just > > > modifies a string, and leave the fprintf/fwrite to output_headers(). > > > > I have thought about this, but this would require fold() to allocate memory. > > Because the plan was to use fold() in output_headers() I though this could > > be avoided. > > I don't think that allocating memory is a drawback. I first wrote a long text about the missing string abstraction, then I found charstring_t. I'll adjust this a few days. One question about charstring_t: Why does charstring_push_back_chars() not use memcpy? Philipp
Re: mhbuild and long header fields
[2023-08-27 22:00] David Levine
> > For the blank lines you mentioned: Can you test the attached version of
> > my patch. If this don't fix your problem, can you provide a sample mail?
>
> The patch didn't fix it. The attached sample shows the added line
> when run through mhfixmsg:
>
> $ uip/mhfixmsg -file fold_adds_blank_line.txt -out -
> From: sen...@example.com
> DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=example.com;
>
>
> b=0123456789012345678901234567890123456789012345678901234567890123456789012345
Ah, I forgott the early continue. The attached version sould work.
Philipp
From fe971fea754f79a1dfe874e64ad157f2b294076f Mon Sep 17 00:00:00 2001
From: Philipp Takacs
Date: Fri, 25 Aug 2023 09:29:42 +0200
Subject: [PATCH] mhbuild implement header folding
---
Makefile.am | 2 ++
sbr/fold.c| 63 +++
sbr/fold.h| 7 +
test/mhbuild/test-mhbuild | 53
uip/mhoutsbr.c| 3 +-
5 files changed, 127 insertions(+), 1 deletion(-)
create mode 100644 sbr/fold.c
create mode 100644 sbr/fold.h
diff --git a/Makefile.am b/Makefile.am
index 4fc84c1d..168d9fe6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -378,6 +378,7 @@ noinst_HEADERS = \
sbr/fmt_new.h \
sbr/fmt_rfc2047.h \
sbr/fmt_scan.h \
+sbr/fold.h \
sbr/folder_addmsg.h \
sbr/folder_delmsgs.h \
sbr/folder_free.h \
@@ -1106,6 +1107,7 @@ sbr_libmh_a_SOURCES = \
sbr/fmt_new.c \
sbr/fmt_rfc2047.c \
sbr/fmt_scan.c \
+sbr/fold.c \
sbr/folder_addmsg.c \
sbr/folder_delmsgs.c \
sbr/folder_free.c \
diff --git a/sbr/fold.c b/sbr/fold.c
new file mode 100644
index ..95553782
--- /dev/null
+++ b/sbr/fold.c
@@ -0,0 +1,63 @@
+/* fold.c -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+#include "h/mh.h"
+#include "h/mime.h"
+#include "fold.h"
+#include
+
+void
+fold(FILE *restrict stream, const char *restrict name, const char *restrict body)
+{
+ const char *restrict body_next;
+ const char *restrict wsp;
+ const char *restrict wsp_next;
+ const bool crlf = strchr(body, '\r');
+ size_t namelen = fprintf(stream, "%s:", name);
+
+ while (*body) {
+ body_next = strchr(body, '\n');
+ if ((unsigned long) (body_next - body) <= MAXTEXTPERLN - namelen) {
+ fwrite(body, 1, body_next - body + 1, stream);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+ wsp = body;
+ while (namelen == 0 && (*wsp == ' ' || *wsp == '\t')) {
+ wsp++;
+ }
+ wsp = wsp_next = strpbrk(wsp, " \t");
+
+ /* if now whitespace is in the current line just print the curret line as is */
+ if (!wsp_next || wsp_next > body_next) {
+ fwrite(body, 1, body_next - body + 1, stream);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+
+ while ((unsigned long)(wsp_next - body) <= MAXTEXTPERLN - namelen) {
+ wsp = wsp_next;
+ wsp_next = strpbrk(wsp+1, " \t");
+ if (!wsp_next) {
+break;
+ }
+ if (wsp_next > body_next) {
+break;
+ }
+ }
+
+ fwrite(body, 1, wsp - body, stream);
+ if (crlf) {
+ fputs("\r\n", stream);
+ } else {
+ fputs("\n", stream);
+ }
+ namelen = 0;
+ body = wsp;
+ }
+}
diff --git a/sbr/fold.h b/sbr/fold.h
new file mode 100644
index ..185f1758
--- /dev/null
+++ b/sbr/fold.h
@@ -0,0 +1,7 @@
+/* fold.h -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+void fold(FILE *restrict stream, const char *restrict name, const char *restrict body);
diff --git a/test/mhbuild/test-mhbuild b/test/mhbuild/test-mhbuild
index 706a804a..f8d4992b 100755
--- a/test/mhbuild/test-mhbuild
+++ b/test/mhbuild/test-mhbuild
@@ -221,5 +221,58 @@ run_test "mhbuild $f" \
check "$f" "$expected"
+start_test "Checking for correct header folding"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+MIME-Version: 1.0
+Content-Type: text/plain; charset="us-ascii"
+
+This is a test
+E
+
+run_test "mhbuild -auto `mhpath last`"
+check "`mhpath last`" "$expected"
+
+start_test "Checking header folding with a to long line"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" &
Re: mhbuild and long header fields
[2023-08-27 09:29] David Levine
> Philipp wrote:
>
> > Do you only test the patch or have you looked at the code? It would
> > be nice to get some feedback on the implementation.
>
> My only comment on the code itself is that I prefer functions that
> just do one thing. So I would implement a fold function that just
> modifies a string, and leave the fprintf/fwrite to output_headers().
I have thought about this, but this would require fold() to allocate memory.
Because the plan was to use fold() in output_headers() I though this could
be avoided.
> There is a bigger issue, however. output_headers() gets called by
> other code, such as mhfixmsg via output_message_fp(). So the change
> as-is interferes badly with it. I haven't looked for a better place
> to fold the header fiels, but we'll have to find one.
As far as I see output_message_fp() is currently only used by mhfixmsg
and mhbuild. For mhfixmsg I though it would be ok to also fold the field.
For the blank lines you mentioned: Can you test the attached version of
my patch. If this don't fix your problem, can you provide a sample mail?
Philipp
From fb921a3fe127dade93b56b1c98c9f8cfc95da722 Mon Sep 17 00:00:00 2001
From: Philipp Takacs
Date: Fri, 25 Aug 2023 09:29:42 +0200
Subject: [PATCH] mhbuild implement header folding
---
Makefile.am | 2 ++
sbr/fold.c| 64 +++
sbr/fold.h| 7 +
test/mhbuild/test-mhbuild | 53
uip/mhoutsbr.c| 3 +-
5 files changed, 128 insertions(+), 1 deletion(-)
create mode 100644 sbr/fold.c
create mode 100644 sbr/fold.h
diff --git a/Makefile.am b/Makefile.am
index 4fc84c1d..168d9fe6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -378,6 +378,7 @@ noinst_HEADERS = \
sbr/fmt_new.h \
sbr/fmt_rfc2047.h \
sbr/fmt_scan.h \
+sbr/fold.h \
sbr/folder_addmsg.h \
sbr/folder_delmsgs.h \
sbr/folder_free.h \
@@ -1106,6 +1107,7 @@ sbr_libmh_a_SOURCES = \
sbr/fmt_new.c \
sbr/fmt_rfc2047.c \
sbr/fmt_scan.c \
+sbr/fold.c \
sbr/folder_addmsg.c \
sbr/folder_delmsgs.c \
sbr/folder_free.c \
diff --git a/sbr/fold.c b/sbr/fold.c
new file mode 100644
index ..e7ae108f
--- /dev/null
+++ b/sbr/fold.c
@@ -0,0 +1,64 @@
+/* fold.c -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+#include "h/mh.h"
+#include "h/mime.h"
+#include "fold.h"
+#include
+
+void
+fold(FILE *restrict stream, const char *restrict name, const char *restrict body)
+{
+ const char *restrict body_next;
+ const char *restrict wsp;
+ const char *restrict wsp_next;
+ const bool crlf = strchr(body, '\r');
+ size_t namelen = fprintf(stream, "%s:", name);
+
+ while (*body) {
+ body_next = strchr(body, '\n');
+ if ((unsigned long) (body_next - body) <= MAXTEXTPERLN - namelen) {
+ fwrite(body, 1, body_next - body + 1, stream);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+ wsp = wsp_next = strpbrk(body, " \t");
+
+ /* if now whitespace is in the current line just print the curret line as is */
+ if (!wsp_next || wsp_next > body_next) {
+ fwrite(body, 1, body_next - body + 1, stream);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+
+ while ((unsigned long)(wsp_next - body) <= MAXTEXTPERLN - namelen) {
+ wsp = wsp_next;
+ wsp_next = strpbrk(wsp+1, " \t");
+ if (!wsp_next) {
+break;
+ }
+ if (wsp_next > body_next) {
+break;
+ }
+ }
+
+ fwrite(body, 1, wsp - body, stream);
+ if (crlf) {
+ fputs("\r\n", stream);
+ } else {
+ fputs("\n", stream);
+ }
+ namelen = 0;
+ while (*wsp == ' ' || *wsp == '\t') {
+ fputc(*wsp, stream);
+ namelen++;
+ wsp++;
+ }
+ body = wsp;
+ }
+}
diff --git a/sbr/fold.h b/sbr/fold.h
new file mode 100644
index ..185f1758
--- /dev/null
+++ b/sbr/fold.h
@@ -0,0 +1,7 @@
+/* fold.h -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+void fold(FILE *restrict stream, const char *restrict name, const char *restrict body);
diff --git a/test/mhbuild/test-mhbuild b/test/mhbuild/test-mhbuild
index 706a804a..f8d4992b 100755
--- a/test/mhbuild/test-mhbuild
+++ b/test/mhbuild/test-mhbuild
@@ -221,5 +221,58 @@ run_test "mhbuild $f" \
check "$f" "$expected"
+start_test "Checking for correct header folding"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From:
Re: mhbuild and long header fields
[2023-08-26 17:42] David Levine > Philipp wrote: > I agree. I'm sending this message using your patch. I'd like to > exercise it for a few days before committing. No problem, take your time. Do you only test the patch or have you looked at the code? It would be nice to get some feedback on the implementation. Philipp
Re: test-mhfixmsg and test-binary fails on debian stable
[2023-08-26 17:39] David Levine > Philipp wrote: > > > [2023-08-26 15:28] David Levine > > > > > > The NUL byte is output as \x00: > > > > I found the problem: The build in printf of dash don't write a NUL. > > Using printf from path to generate the test and expected file works as > > expected. > > Thank you! I'll update the test. An other hint for this test. Posix printf specifies an octal escape sequences[0]. Replacing the '\x00' with '\000' fixes the test. Philipp [0] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html
Re: test-mhfixmsg and test-binary fails on debian stable
[2023-08-26 15:28] David Levine > Philipp wrote: > > > This test now passes. > > Thank you for reporting the test failure and for confirming the fix. > > > Here you go: > > $ od -ax /tmp/nmh/test/testdir/test-binary302898.actua > > Thanks. > > The NUL byte is output as \x00: I found the problem: The build in printf of dash don't write a NUL. Using printf from path to generate the test and expected file works as expected. Philipp > > 260 y t e : sp \ x 0 0 . nl > >74793a655c2030782e30000a > > The content in the message is identified as binary so I'm not sure why. > For me, it is output as NUL: > > 260 y t e : sp nul . nl >74793a6500200a2e > > David >
Re: test-mhfixmsg and test-binary fails on debian stable
[2023-08-26 14:33] David Levine > Philipp wrote: > > > LANG=en_US.UTF-8 > > LC_ALL= > > > I have "#define HAVE_ICONV 1" in config.h. I just run autogen.sh and > > configure without any arguments. > > Thanks, those match what I use. > > I just committed a fix to test-mhfixmsg. Different html renderers > produced different output. I replace a non-breakable space with a > space so the output is now the same from w3m, lynx, and elinks. This test now passes. > I don't know why your test_binary would count 26 bytes instead of 23. > Can you post or send me your /tmp/nmh/test/testdir/test-binary.actual > file? Or better, run od -ax on it and post that? Here you go: $ od -ax /tmp/nmh/test/testdir/test-binary302898.actua 000 [ sp M e s s a g e sp i n b o x : 205b654d7373676120656e696f623a78 020 1 1 sp ] nl T o : sp sp sp sp sp sp r e 31315d20540a3a6f2020202020206572 040 c i p i e n t @ e x a m p l e . 696369706e65407478656d616c702e65 060 e d u nl F r o m : sp sp sp sp s e n 64650a7572466d6f203a202073206e65 100 d e r @ e x a m p l e . e d u nl 6564407278656d616c702e6564650a75 120 S u b j e c t : sp t e s t sp b i 75536a6263653a747420736520746962 140 n a r y sp c o n t e n t nl nl M I 616e797263206e6f6574746e0a0a494d 160 M E - V e r s i o n : sp 1 . 0 nl 454d562d726569736e6f203a2e310a30 200 nl [ sp p a r t sp sp - sp t e x t / 5b0a7020726120742d20742078652f74 220 p l a i n sp - sp sp sp 2 6 B sp sp ] 6c706961206e202d2020363220425d20 240 nl H e r e ' s sp a sp n u l l sp b 480a7265276520732061756e6c6c6220 260 y t e : sp \ x 0 0 . nl 74793a655c2030782e30000a 273 Philipp > > David >
Re: test-mhfixmsg and test-binary fails on debian stable
[2023-08-26 08:44] David Levine > Philipp wrote: > > > I have noticed that two tests (test-mhfixmsg and test-binary) fail > > on my computer. I use debian stable and configure outputs folloing: > > What is your locale? $ locale LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= > Do your nmh build use iconv? I have "#define HAVE_ICONV 1" in config.h. I just run autogen.sh and configure without any arguments. Philipp > > David >
test-mhfixmsg and test-binary fails on debian stable
Hi I have noticed that two tests (test-mhfixmsg and test-binary) fail on my computer. I use debian stable and configure outputs folloing: nmh configuration - nmh version: 1.8+dev host os: x86_64-pc-linux-gnu compiler : cc compiler flags : -g -O2 -Wall -Wextra linker flags : preprocessor flags : -D_GNU_SOURCE source code location : . binary install path: /usr/local/nmh/bin libexec install path : /usr/local/nmh/libexec/nmh config files install path : /usr/local/nmh/etc/nmh man page install path : /usr/local/nmh/share/man docs install path : /usr/local/nmh/share/doc/nmh RPM build root : ./RPM backup prefix : , transport system : smtp spool default locking type : fcntl default smtp server: localhost SASL support : no TLS support: yes OAuth support : no cc is gcc 12.2.0 output test-mhfixmsg: *** /tmp/nmh/test/testdir/test-mhfixmsg83068.expected 2023-08-26 09:48:51.891484073 + --- /tmp/nmh/test/testdir/test-mhfixmsg83068.actual 2023-08-26 09:48:51.923483955 + *** *** 4,22 From: Subject: body requires binary encoding Content-Type: multipart/alternative; boundary="=_nmh-multipart" --=_nmh-multipart Content-Type: text/plain; charset="ISO-8859-1" ! Content-Transfer-Encoding: 7bit ! Mile $0.00 ! Time$78.71 ! State Tax$5.90 ! Vehicle License Fee $1.84 ! State Txn Tax$6.00 --=_nmh-multipart Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable - --- 4,22 From: Subject: body requires binary encoding Content-Type: multipart/alternative; boundary="=_nmh-multipart" + Content-Transfer-Encoding: 8bit --=_nmh-multipart Content-Type: text/plain; charset="ISO-8859-1" ! Content-Transfer-Encoding: 8bit ! Mile $0.00 ! Time $78.71 ! State Tax $5.90 ! Vehicle License Fee $1.84 ! State Txn Tax $6.00 --=_nmh-multipart Content-Type: text/html; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable ./test/mhfixmsg/test-mhfixmsg: test failed, outputs are in /tmp/nmh/test/testdir/test-mhfixmsg83068.expected and /tmp/nmh/test/testdir/test-mhfixmsg83068.actual. first named test failure: -reformat succeeds when decode of binary text fails output test-binary: *** /tmp/nmh/test/testdir/test-binary85135.expected Sat Aug 26 09:48:54 2023 --- /tmp/nmh/test/testdir/test-binary85135.actual Sat Aug 26 09:48:54 2023 *** *** 5,9 MIME-Version: 1.0 ! [ part - text/plain - 23B ] Here's a null byte: \x00. --- 5,9 MIME-Version: 1.0 ! [ part - text/plain - 26B ] Here's a null byte: \x00. ./test/mhshow/test-binary: test failed, outputs are in /tmp/nmh/test/testdir/test-binary85135.expected and /tmp/nmh/test/testdir/test-binary85135.actual. Philipp
Re: mhbuild and long header fields
[2023-08-25 23:46] Steffen Nurpmeso
> Steffen Nurpmeso wrote in
> <20230825213130.xthnk%stef...@sdaoden.eu>:
> |Philipp wrote in
> | <93c190b84a23e620d07b9504f619b1f2.phil...@bureaucracy.de>:
> ||[2023-08-24 21:07] Philipp Takacs
> ||> [2023-08-23 14:29] Philipp
> ||>> [2023-08-20 22:14] David Levine
> ||>>> Ken Hornstein wrote:
> | ...
> ||I have tested this and fixed some bugs. A commit with a test is attached.
> |
> |I have not looked at your code (for real), but .. i have a test
> |for the MUA i maintain like
> |
> | i=$(awk 'BEGIN{for(i=0; i<92; ++i) printf "0123456789_"}')
> |
> |and then i use "$i" as a subject.
> |The problem with RFC 5322 is that artificial spaces may have to be
> |introduced to break a line without whitespace. (That is, you have
> |to break somewhere, and have to start the next line with
> |a whitespace, but that is included then, modifying the original
> |data.)
>
> That is: you should switch to MIME RFC 2047 then for the purpose
> of uniting maybe even. Or however nmh wants to deal with this.
Thanks for your feedback, but I don't think general encoding every field
with a line longer the 78 (998) bytes is the way to go. Because fields
which are to long are uncommen. Also fields which tend to be to long have
most of the time contain white space[0]. So folding is possible.
That said, my code detects lines without WSP (and are longer then 78
bytes). Some rfc 2047 encoding could be added there. But first the
folding should be implemented.
Philipp
[0] I speak of fields you would include raw in a draft file, not binary
data like a dkim signature. Such data is mostly direct rfc 2047 encoded.
Re: mhbuild and long header fields
[2023-08-24 21:07] Philipp Takacs
> [2023-08-23 14:29] Philipp
> > [2023-08-20 22:14] David Levine
> > > Ken Hornstein wrote:
> > >
> > > > [Phillip wrote:]
> > > > >Or in output_headers(). I'm not sure if there an extra options would be
> > > > >required.
> > > >
> > > > That is one option. Another is that repl(1) could do a better job; I
> > > > suppose that is the fault of mhl.
> > >
> > > [...]
> > >
> > > I think that output_headers() is the right place. It's the only
> > > place where header fields are output.
> > >
> > > [...]
> >
> > [...]
> >
> > I'll try to implement a corrent fold funktion on the weekend.
>
> I found a bit time to implement this today. Just the fold() fuction
> not the integration in output_headers().
I have tested this and fixed some bugs. A commit with a test is attached.
Philipp
From 2342446107a415e35f7895a2a1fe2d4833420e80 Mon Sep 17 00:00:00 2001
From: Philipp Takacs
Date: Fri, 25 Aug 2023 09:29:42 +0200
Subject: [PATCH] mhbuild implement header folding
---
Makefile.am | 2 ++
sbr/fold.c| 60 +++
sbr/fold.h| 7 +
test/mhbuild/test-mhbuild | 26 +
uip/mhoutsbr.c| 3 +-
5 files changed, 97 insertions(+), 1 deletion(-)
create mode 100644 sbr/fold.c
create mode 100644 sbr/fold.h
diff --git a/Makefile.am b/Makefile.am
index 4fc84c1d..168d9fe6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -378,6 +378,7 @@ noinst_HEADERS = \
sbr/fmt_new.h \
sbr/fmt_rfc2047.h \
sbr/fmt_scan.h \
+sbr/fold.h \
sbr/folder_addmsg.h \
sbr/folder_delmsgs.h \
sbr/folder_free.h \
@@ -1106,6 +1107,7 @@ sbr_libmh_a_SOURCES = \
sbr/fmt_new.c \
sbr/fmt_rfc2047.c \
sbr/fmt_scan.c \
+sbr/fold.c \
sbr/folder_addmsg.c \
sbr/folder_delmsgs.c \
sbr/folder_free.c \
diff --git a/sbr/fold.c b/sbr/fold.c
new file mode 100644
index ..023deed8
--- /dev/null
+++ b/sbr/fold.c
@@ -0,0 +1,60 @@
+/* fold.c -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+#include "h/mh.h"
+#include "h/mime.h"
+#include "fold.h"
+#include
+
+void
+fold(FILE *restrict stream, const char *restrict name, const char *restrict body)
+{
+ const char *restrict body_next;
+ const char *restrict wsp;
+ const char *restrict wsp_next;
+ const bool crlf = strchr(body, '\r');
+ size_t namelen = fprintf(stream, "%s:", name);
+
+ while (*body) {
+ body_next = strchr(body, '\n');
+ if ((unsigned long) (body_next - body) <= MAXTEXTPERLN - namelen) {
+ fwrite(body, 1, body_next - body + 1, stream);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+ wsp = wsp_next = strpbrk(body, " \t");
+
+ /* if now whitespace is in the current line just print the curret line as is */
+ if (!wsp_next || wsp_next > body_next) {
+ fwrite(body, 1, body_next - body + 1, stream);
+ namelen = 0;
+ body = body_next + 1;
+ continue;
+ }
+
+ while ((unsigned long)(wsp_next - body) <= MAXTEXTPERLN - namelen) {
+ wsp = wsp_next;
+ wsp_next = strpbrk(wsp+1, " \t");
+ if (!wsp_next) {
+break;
+ }
+ if (wsp_next > body_next) {
+break;
+ }
+ }
+
+ fwrite(body, 1, wsp - body, stream);
+ if (crlf) {
+ fputs("\r\n", stream);
+ } else {
+ fputs("\n", stream);
+ }
+ fputc(*wsp, stream);
+ namelen = 1;
+ body = wsp + 1;
+ }
+}
diff --git a/sbr/fold.h b/sbr/fold.h
new file mode 100644
index ..185f1758
--- /dev/null
+++ b/sbr/fold.h
@@ -0,0 +1,7 @@
+/* fold.h -- fold a mail header field
+ *
+ * This code is Copyright (c), by the authors of nmh. See the
+ * COPYRIGHT file in the root directory of the nmh distribution for
+ * complete copyright information. */
+
+void fold(FILE *restrict stream, const char *restrict name, const char *restrict body);
diff --git a/test/mhbuild/test-mhbuild b/test/mhbuild/test-mhbuild
index 706a804a..464ad309 100755
--- a/test/mhbuild/test-mhbuild
+++ b/test/mhbuild/test-mhbuild
@@ -221,5 +221,31 @@ run_test "mhbuild $f" \
check "$f" "$expected"
+start_test "Checking for correct header folding"
+
+cat >"`mhpath new`" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+This is a test
+E
+
+cat > "$expected" <<\E
+From: Somebody
+To: Nobody
+Subject: Test message
+References:
+
+MIME-Version: 1.0
+Content-Type: text/plain; charset="us-ascii"
+
+This is a test
+E
+
+run_test "mhbuild -auto `mhpath last`"
+check &q
Re: mhbuild and long header fields
[2023-08-23 14:29] Philipp
> [2023-08-20 22:14] David Levine
> > Ken Hornstein wrote:
> >
> > > [Phillip wrote:]
> > > >Or in output_headers(). I'm not sure if there an extra options would be
> > > >required.
> > >
> > > That is one option. Another is that repl(1) could do a better job; I
> > > suppose that is the fault of mhl.
> >
> > [...]
> >
> > I think that output_headers() is the right place. It's the only
> > place where header fields are output.
> >
> > [...]
>
> [...]
>
> I'll try to implement a corrent fold funktion on the weekend.
I found a bit time to implement this today. Just the fold() fuction
not the integration in output_headers().
Philipp
> Philipp
>
> [0] I don't know if this is an issue in the context of ical
>
> > David
>
#include "h/mh.h"
#include "h/mime.h"
#include
void fold(FILE *restrict stream, const char *restrict name, const char *restrict body)
{
const char *restrict body_next;
const char *restrict wsp;
const char *restrict wsp_next;
const bool crlf = strchr(body, '\r');
size_t namelen = fprintf(stream, "%s:", name);
if (namelen >= MAXTEXTPERLN) {
if (crlf) {
fputs("\r\n ", stream);
} else {
fputs("\n ", stream);
}
namelen = 0;
}
while (*body) {
body_next = strchr(body, '\n');
if ((unsigned long) (body_next - body) <= MAXTEXTPERLN - namelen) {
fwrite(body, 1, body_next - body + 1, stream);
namelen = 0;
body = body_next + 1;
continue;
}
wsp = strpbrk(body, " \t");
/* if now whitespace is in the current line just print the curret line as is */
if (!wsp || wsp > body_next) {
fwrite(body, 1, body_next - body + 1, stream);
namelen = 0;
body = body_next + 1;
continue;
}
while ((unsigned long)(wsp - body) <= MAXTEXTPERLN - namelen) {
wsp_next = strpbrk(wsp, " \t");
if (!wsp_next) {
break;
}
if (wsp_next > body_next) {
break;
}
wsp = wsp_next;
}
fwrite(body, 1, wsp - body, stream);
if (crlf) {
fputs("\r\n", stream);
} else {
fputs("\n", stream);
}
fputc(*wsp, stream);
namelen = 1;
body = wsp + 1;
}
}
Re: mhbuild and long header fields
[2023-08-20 22:14] David Levine > Ken Hornstein wrote: > > > [Phillip wrote:] > > >Or in output_headers(). I'm not sure if there an extra options would be > > >required. > > > > That is one option. Another is that repl(1) could do a better job; I > > suppose that is the fault of mhl. > > mhl is used for display. And a user can substitute their own > filter for it. We need to fold header fields when producing a > message. > > I think that output_headers() is the right place. It's the only > place where header fields are output. > > There is a fold() function in sbr/mhical.c, maybe it could be > moved to sbr and called from output_headers() for message and > content part headers. Those shouldn't need multibyte character > support, though that wouldn't hurt. I have had a quick look at fold() and just calling fold() would produce incorrect results. fold() just insert '\n ' after 76 characters[0]. But to correct fold a field you must insert a '\n' befor a whitespace. I'll try to implement a corrent fold funktion on the weekend. Philipp [0] I don't know if this is an issue in the context of ical > David
Re: mhbuild and long header fields
[2023-08-18 21:48] David Levine
> > I have noticed that mhbuild don't implement header field folding.
> > Specialy for the References field this might cause problems, when you
> > use it to feed a mail build by mhbuild to a tool which checks the
> > line length.
>
> Thank you for pointing that out. Header field folding does need to
> be properly implemented. It would be a great contribution if someone
> has the bandwidth.
I have looked a bit at it. I found two places where this could be
implemented:
Either as part of encode_rfc2047(), this function does this allready,
but only if a non ascii char is in the field body.
Or in output_headers(). I'm not sure if there an extra options would
be required.
> > Also I don't get the code. In uip/mhbuildsbr.c:359 a LENERR is handled
> > with die(), but I can't create a testcase for this.
>
> A draft with a header field name exceeding 997 bytes in length should
> trigger LENERR. The only code that sets a LENERR is in m_getfld().
> So the draft would trigger is in any nmh command that reads a message
> header, such as scan(1):
Thanks for pointing this out, but now I'm more confused. Whats the point
of LENERR (in contrast to FMTERR)? But this is a diffrent discussion.
Philipp
> $ (printf '%.sX' {1..997}; printf ': too-long\n') | scan -file -
> scan: field name
> "XX
>
>
>
>
>
>
> XXX:"
> exceeds 997 bytes
> ??Format error (message -1) in component 1
>1 08/18/23*
>
> David
>
mhbuild and long header fields
Hi I have noticed that mhbuild don't implement header field folding. Specialy for the References field this might cause problems, when you use it to feed a mail build by mhbuild to a tool which checks the line length. Also I don't get the code. In uip/mhbuildsbr.c:359 a LENERR is handled with die(), but I can't create a testcase for this. Philipp
Re: mhl nocomponent
[2023-04-02 14:04] Michael Richardson > I use MH-E, which does it's header display/hiding outside of MH. > In general, I like to see extra headers, but they have gotten way out of hand > from the MS/Outlook space. > We probably need a better list of common "this is junk" header list that > probably > has to have wildcards in it. I have a patch for simple globbing in ignores. If you are intrested, I can port it. Philipp
Re: Export/sync nmh folders to IMAP server
[2022-06-29 21:07] Steffen Nurpmeso > (I personally like a nice and so on, append-only MIME MBOX the > most, and do not understand why people do not like it. Because mbox is a horrible file format. First of all it's only one file so you have to build a folder stucture around it. Next it's hard to delete (or move) mails if they not at the end of the file. These problems are not that bad depending on for what you want to use the mbox and can solved by writing more and complex code. The main problem is that there is not the mbox format, there are several variants. Some variants don't allow arbatrary mails as content and most (probably all) of them aren't realy specified. It's nearly impossible to determ which variant you have. Therfor most software just implement one variant and handle other variants incorrect. Jamie Zawinski explains the details of the problems quite good[0]. So please stop using mbox. Philipp Ps: what bugs me the most is when mbox is used to store exactly one mail Pps: Sorry for the OT rant, but this bugs me every time I get a mbox [0] https://www.jwz.org/doc/content-length.html
Re: Export/sync nmh folders to IMAP server
Hi [2022-06-27 12:50] Wolfgang Denk > tldr: How can I export (and keep in sync) MH folders to an IMAP > account; direction is only MH -> IMAP. I currently use netoric[0] a small imap sync programm written by a friend of mine. Before that I have used mailsync[1]. Philipp [0] https://git.c-14.de/netoric.git/ [1] http://mailsync.sourceforge.net/
Re: mhl linewrapping
Hi [2022-05-22 12:17] Ralph Corderoy > > First of all it depends on the terminalsize, if the size is not given > > by the arguments. This leads to diffrent linewrapping on reply > > depending on the size of the terminal. This could be fixed by going > > to some default width when stdout is not a tty. > > It would seem right to only use the terminal to set the width when the > output is a TTY. There is already a default width of 80, mentioned in > mhl(1), if the terminal width can't be found so my initial thought it > that would do. I'm currently looking at the code and for me it looks like if the size can't detected it default to 1. I might have missed something. > > Next it only supports hard linewrapping. Therefor sometimes words and > > links get split. Some support for softwrapping would be nice. So mhl > > could search for the last whitespace befor the selected width. > > Are you just talking about the body component or all of them? Are you > aware of -fmtproc, etc? Some versions of fmt(1), for example, do > other nice things like trying to avoid the word ‘I’ at the end of > the line. Other formatters might re-introduce two spaces after the end > of a sentence. I'm talking about all components. Of course most importend is the body. I'm aware that you can do powerfull formating with an external fmtproc. I just think it would be better to have a basic softwrapping in nmh itself, because nmh should be able to create sane replies without dependencies. For the other components there is already a comment in mhl explaining that putstr should know about atomics of addresses and dates. I would start with working on this. Philipp
mhl linewrapping
Hi The linewrapping in mhl is not realy good and I would like to implement some improvements. First of all it depends on the terminalsize, if the size is not given by the arguments. This leads to diffrent linewrapping on reply depending on the size of the terminal. This could be fixed by going to some default width when stdout is not a tty. Next it only supports hard linewrapping. Therefor sometimes words and links get split. Some support for softwrapping would be nice. So mhl could search for the last whitespace befor the selected width. What do you think about this? Philipp
Re: Failing to extract text/html part of message with show
Hi again [2022-05-17 11:43] Greg Davidson > When I list my message with > mhlist -file mobile-tabs.eml -part 1 > I get > mhlist: bogus multipart content in message .../mobile-tabs.eml Sorry, I have overseen this output. > Looking at the message, I see that the part in question begins with > --5f31ec05defff85c > Content-Type: text/html; charset="UTF-8" > Content-Transfer-Encoding: quoted-printable > > Virtue Ethics (Stanford Encyclopedia > of= > Philosophy), Phronesis - Wikipedia, Stanford Encyclopedia of Philosophy > - = > > which looks reasonable as far as I can tell. As far as I see the problem is at the end of the mail. There should be a line like: --5f31ec05defff85c-- This line indicates the end of the mime-part. This might be a sign that you haven't got the full mail. If only this line is missing you can add it or ignore this output. nmh has also mhfixmsg(1) to handle sutch errors automaticaly. The things I have written in my other mail is still true. Philipp
Re: Failing to extract text/html part of message with show
Hi Greg First of all, it looks like you using _m_mh not _n_mh. [2022-05-17 11:43] Greg Davidson > I'm trying to extract part of a multipart message sent to me > by Firefox Mobile in order to save my mobile tabs. I would > like to extract the text/html part as that is the part which has > the tab titles as well as the urls. Note: I've indented quoted > I/O with 4 spaces and used ellipses ... to replace some bits. > > When I list my message with > mhlist -file mobile-tabs.eml -part 1 > I get > mhlist: bogus multipart content in message .../mobile-tabs.eml > msg part type/subtype size description >0 multipart/alternative 11K > 1 text/html 6861 > but when I try to extract it with > show -file mobile-tabs.eml -part 1 Do you want to store the content on disk or display it? For storing there is mhstore(1). You can simple use: mhstore -file mobile-tabs.eml -part 1 show (mhshow on nmh) is to display the mail. > I get > show: bogus multipart content in message .../mobile-tabs.eml > Date:... > From:... > To: ... > Subject: Mobile tabs > > show: don't know how to display any of the contents > (content multipart/alternative in message .../mobile-tabs.eml) > > Following the mmh documentation I added > mhshow-suffix-text/html: .html > to my > ~/.mmh/profile > as suggested by the documentation but it didn't help. I also tried to > direct it to use cat for such content type, also with no change. To display text/html parts you need to add a mhshow-show-/ entry to your profile. For example to display html rendered with elinks add somethig like: mhshow-show-text/html: elinks -dump That the fallback to just print the text for text/* mimetypes fail when selecting the part directly is a known bug in mmh. Philipp
Re: merge pick and scan
[2022-04-21 14:49] Eric Gillespie > And it gets even worse: first you have to wait for pick to slowly > search ALL THE FILES (within limiting message range you may have > given it if you have any idea and often I did not), and then you > wait for scan to slowly readdir everything, and THEN you finally > get your results. What I really want (and I doubt I'm alone) is > scan lines as soon as a message is a HIT, so I can interrupt when > the message I'm looking for comes across, without waiting for any > further work. It's even worse, because pick don't read a message once, it reads the message for eatch action. There is a optimization to not perform eatch action on the message, but this depends on the given actions and the messages. So for example have the following command: > pick -from bla -and -subject blub All mails from bla will read twice. So if you know you have only a few messages with the subject blub and many messages from bla, switching the actions will increase the performance of pick. To avoid this pick could read the message per field[0] and check for a match. Yes the actions do early return, but this also depend on the messages and the libc/fs could do some read optimication. But in general pick does many IO-operations which aren't necesary. In general there could be a pattern to read a mail/header before processing and store the required fields. All the follow up processing could be done without IO operation. > On a modern filesystem on SSD in 2022 maybe nobody cares anymore. > But over NFS in 2010, this mattered a lot. I think this still matters, not in the way that we must optimice every operation, but nmh sould avoid wasting time. Philipp [0] I only think of header search, because body search don't work good on most mails I get (MIME).
Re: merge pick and scan
Hi Ralph [2022-04-01 13:41] Ralph Corderoy > > Ralph wrote: > > Ok I see this more the other way around, pick shoundn't write > > sequences, there is mark(1) for this. > > Yes, if pick always listed by default then it could skip the writing of > sequences by passing them as arguments to mark. But bear in mind the > limit on the number of arguments to a process and their total size in > characters, especially some decades ago. The thing is some decades ago is some decades ago ;-) So is this limitation still relevant? And are there other solutions like let mark read from stdin or use xargs? > > To get these you can currently choose between "pick seq", > > Only with ‘-list’ which you may have as a default in your ~/.mh_profile. Just a litle nitpick: "-list" is the default if no sequence is given. > > > > If this would be added to pick, then scan would be obsolet. > > > > > > If all commands grew message-set expressions, like pick's, then pick > > > would be obsolete. ;-) mark(1) would do when the sequence only > > > needs modifying without display. > > > > Yes, but would it be reasable to add message-set expressions to all > > tools? As you pointed out they where removed some time ago. > > No, they weren't. I didn't point that out. They have only ever been > arguments to pick. Yes sorry I mixed this up a bit. I still don't think is reasonable to add message-set to all tools. > > With leads to the question: Why does pick only print out message > > numbers and not a mh-format(5). The obvious awnser is: Because there > > is scan for this. But you could also ask: Is scan (as an extra tool) > > required at all? A -form switch on pick would do the same with the > > same interface. > > You're missing the essence of what each command is intended to do. > > What is the one thing which currently make pick distinct from all the > other nmh commands? It is the ‘matcher’ arguments like ‘-subject’. > > With the Unix ‘do one thing well’ outlook, > > - pick's single purpose is to search emails with its ‘matcher’ > operators. The point where we disagree is what should pick do with the matched emails. > - scan's single purpose is a one-line summary of a message, perhaps just > the message number. > - show's single purpose is to display a message over multiple lines. > - mark's single purpose is to manipulate sequences with set-like > operations. > > I should break down what I want to do so I can cover the operations with > nmh's commands and then they compose. > > Moving the ‘matcher’ arguments from pick to all the other commands would > improve the UI and they'd be consistent in that ‘-subject’ could be > given to all of them. Then there would be no tool to match/search emails. Yes, with my suggestion there would be no tool to display a one-line summary. Also as said earlier this would require all tools to be able read and parse the mails. Yes this would be hidden in srb/, but still be there. So all tools would match and do there purpose. > Moving the ‘matcher’ arguments to just scan would > be a wart. Yes, I know you think of it as scan's one-line formatting > moving to pick, but it's the same thing looked at the wrong way around > simply because of how you see to implement it. I see it this way, because of the way I use it. Most of the time I use the matchers of pick, I search for the next mail to handle and therefor use a combination of scan and pick. I don't use sequences that much. > pick and scan overlap, but that's due to the error of adding -list to > pick and isn't a reason to merge the two. I still see the error in pick in the -sequence switch. This may was different at the time of implementing pick, but I look at it from a today's perspective. > Your suggestion is one interpretation and I don't think it's the best > one. Anointing your suggestion by making your change would further > muddle the two distinct operations of scan and pick. I think here we are at a point we can agree to disagree. Philipp
Re: merge pick and scan
Hi David [2022-03-30 08:39] Ralph Corderoy > > David wrote: > > > I don't think that departure from the Unix philosophy can be > > > considered good for nmh. > > > > I don't quit understand this argument. From my perspective the current > > interface don't match the Unix philosophy. To explain this lets look > > what scan and pick do: > > > > Pick: > > > > 1. selected messages from a folder > > 2. unselect messages with don't match the given pattern > > 3. Print out all selected messages > > I don't think pick(1) does item 2, unless you mean by virtual of item 1 > leaving some messages out. Yes I mean this more virtual, the implementation details are not realy importend for my point. > And I think it shouldn't do item 3. When sequences were added to MH and > pick altered to define them, scan(1) could have been used to get the > message numbers, similarly to how you suggested earlier. Ok I see this more the other way around, pick shoundn't write sequences, there is mark(1) for this. I don't know whitch suggestion you mean. I guess you mean something to get the message numbers of a sequence. To get these you can currently choose between "pick seq", "scan -format '%(msg)' seq" and "mark -list -sequence seq". The last one prints the contet of .mh_sequences and is special for sequence management. But the scan and pick version do more or less the same. By implementing my sugguestion this would be also the same code. > > Scan: > > > > 1. select messages from a folder > > 2. Print out all selected messages > > > > The function of the two tools looks overlapping. > > Agreed, but compounding the wrongs doesn't make a right. :-) Are they realy that wrong or is this a question of the perspective? >From my point of view pick does message selection and printing. The sequence handling is kind of an odd feature. If I understand you correct you see pick more as a tool to add messages to a sequence and the -list switch is kind of odd. So in both views pick can do something which don't realy match. Also there are other tools (scan and mark) with can do parts of the functions better. So it's reasonable to ask what is the purpose of pick. My suggestion would clarify the purpose. > > If this would be added to pick, then scan would be obsolet. > > If all commands grew message-set expressions, like pick's, then pick > would be obsolete. ;-) mark(1) would do when the sequence only needs > modifying without display. Yes, but would it be reasable to add message-set expressions to all tools? As you pointed out they where removed some time ago. This would also require all tools to read and parse the messages. Currently not all tools (i.e. rmm) need to do this. But looking at pick: pick already reads and (somehow) parses the messages. A -form/-format switch would only add a way to print this information. With leads to the question: Why does pick only print out message numbers and not a mh-format(5). The obvious awnser is: Because there is scan for this. But you could also ask: Is scan (as an extra tool) required at all? A -form switch on pick would do the same with the same interface. The implementation of this is not big and only affects one tool. It don't change the general input interface, only the output interface of pick. > > > There needs to be really good justification for intentional > > > breakage. > > > > I hate this argument. Yes changing the user interface shouldn't be > > done just because it feels good. But To improve the user interface it > > must change sometimes. > > But changing the user interface doesn't have to cause breakage. Changing the user interface itself doesn't break only if you add a new interface for an already existing feature. So in this case adding -form/-format switches to pick without removing scan. This is possible, but I don't see this as a good option. Philipp
Re: merge pick and scan
[2022-03-30 10:41] Conrad Hughes > I'm somewhat neutral on this — can see the convenience etc. — but here's > a case that is currently unambiguous (because MH commands have side > effects) that might cause complications for general switch parsing: > > refile +foo `pick ... +bar` > > .. if pick's args were merged into refile's, then order of arguments > becomes even more significant: I belive I there is a big missunderstanding. I don't want to add pick's args to scan (or an other tool). I want to add scan's -form/-format to pick. So no other tool should be affected. I don't think it's reasonable to add the pick args to another tool. As David already pointed out this was seen as `inverted logic' and therefor removed. I don't want to bring it back. Philipp
Re: merge pick and scan
[2022-03-29 05:05] David Levine > > In order to do this we could add the -form/-format switches to pick and > > make scan a link to pick. To avoid most breakage pick would use "%(msg)" > > format by default and scan.default if called as scan. This way only some > > scan aliases would break (if -form/-format is not explicitly set for them). > > I don't think that departure from the Unix philosophy can be > considered good for nmh. I don't quit understand this argument. From my perspective the current interface don't match the Unix philosophy. To explain this lets look what scan and pick do: Pick: 1. selected messages from a folder 2. unselect messages with don't match the given pattern 3. Print out all selected messages Scan: 1. select messages from a folder 2. Print out all selected messages The function of the two tools looks overlapping. There are two diffrence: First the secound step of the message selection on pick is not there in scan. Secound the print out of pick only prints a hardcoded format. So my suggestion is basicly to extend the print function of pick to allow printing in a user defined format. The data for the print format is already read by pick[0]. If this would be added to pick, then scan would be obsolet. > There needs to be really good justification for intentional breakage. I hate this argument. Yes changing the user interface shouldn't be done just because it feels good. But To improve the user interface it must change sometimes. Also I don't sugguest to completle redesigne the interface from scratch and the breakage would not affect many cases. The most affected case would probably be something like ``scanf: first'' and it can be fixed by adding ``-form scan.default'' to the config. Philipp [0] I have some other improvements in mind which would allow to better handle the mail header and avoid double parsing.
merge pick and scan
Hi To scan selected messages currently a combination of scan and pick is used. It would be nice to be able to write something like: > scan -from bob In order to do this we could add the -form/-format switches to pick and make scan a link to pick. To avoid most breakage pick would use "%(msg)" format by default and scan.default if called as scan. This way only some scan aliases would break (if -form/-format is not explicitly set for them). All other usages would still work. It would still be possible to call: > scan `pick -from bob` I have implemented this for mmh and could port it to nmh. What do you think of the idea? Philipp
proposal docs/contrib/whatnow2.sh
Hi
I have written a small whatnow replacement[0]. This might be intresting
for some of you. The main benefit from it, is that it works without be
a shell. You allways work from your own shell.
It is a drop in replacement for whatnow. So all you need to do is to
put this script in your path and set whatnowproc or -whatnowproc
appropriate. To work with the current draft just call "whatnow2 [...]".
There are some problems for nmh users and changes between this version
and the mmh version:
This works only with the drafts folder, because it was written without
draftfile in mind. Also it currently works only with one drafts folder.
I don't know how hard is it to change this.
There is no "mime" command, because mmh don't have one. It's easy to add
one.
This version stores the state in a seperate file next to the draft. The
suffix of this file can be configured by the Metafile-Extension option
and falls back to '.meta'. In the mmh version is it also posible to
store the state in the draft itself. Therefor some changes in mhl are
required.
There are some options which aren't necesary set on nmh. I have added
sane[1] fallbacks, if they are not set. The following options and
fallbacks are used:
* draftfolder: +drafts
* Attachment-Header: Nmh-Attachment
* listproc: show
I hope I haven't missed some problems. If you have problems with it let
me know.
Philipp
[0] I have written this for mmh and never tested it with nmh but it should
just work in most cases
[1] I hope there are sane
#!/bin/sh
printhelp()
{
echo "Usage: $0 command"
echo " commands are:"
echo " edit [editor]"
echo " list"
echo " send [sendargs]"
echo " delete"
echo " display"
echo " attach files"
echo " alist"
echo " detach anum"
echo " refile +folder"
echo " -help"
echo " -version"
}
version()
{
if [ $1 -eq 0 ]
then
echo "$0 has no own version number, thus this instead:"
folder -version
exit 0
fi
echo "$0 has no own version number, thus this instead:" 1>&2
folder -version 1>&2
exit 1
}
usage()
{
if [ $1 -eq 0 ]
then
printhelp
exit 0
fi
printhelp 1>&2
exit 1
}
get_editor()
{
if [ -f "$mhmetafile" ]
then
lasteditor=`anno -list -component 'nmh-last-editor' "$mhmetafile"`
if [ -n "$lasteditor" ]
then
editor=`echo $lasteditor | cut -d ' ' -f 1`
mheditor=`mhparam "$editor-next"`
[ -n "$mheditor" ] && return
mheditor=$lasteditor
return
fi
fi
if [ -n "$MMHEDITOR" ]
then
mheditor=$MMHEDITOR
return
fi
mheditor=`mhparam 'Editor'`
}
save_config()
{
component="$1"
newtext="$2"
anno -delete -number all -component "$component" "$mhmetafile"
anno -nodate -component "$component" -text "$newtext" "$mhmetafile"
}
get_showproc()
{
mhshowproc=`mhparam 'listproc'`
mhshowproc=${mhshowproc:-show}
return
}
get_realpath()
{
reldir=`dirname "$1"`
filename=`basename "$1"`
cd "$reldir"
echo "$PWD/$filename"
cd "$OLDPWD"
}
create()
{
if [ -z "$mhdraft" ]
then
usage 1
fi
mhext=`mhparam Metafile-Extension`
mhext=${mhext:-.meta}
mhmetafile="$mhdraft""$mhext"
touch "$mhmetafile"
if [ -z "$mheditor" ]
then
get_editor
fi
if [ "$mhuse" -eq 1 ]
then
exec $mheditor $mhdraft
return
fi
save_config nmh-mhaltmsg "$mhaltmsg"
save_config nmh-mhdist "$mhdist"
save_config nmh-mhfolder "$mhfolder"
save_config nmh-mhmessages "$mhmessages"
save_config nmh-mhannotate "$mhannotate"
save_config nmh-last-editor "$mheditor"
exec $mheditor "$mhdraft"
}
edit()
{
if [ $# -eq 0 ]
then
get_editor
else
mheditor="$@"
fi
save_config nmh-last-editor "$mheditor"
exec $mheditor "$mhdraft"
}
list()
{
get_showproc
exec $mhshowproc -file $mhdraft
}
sendfunktion()
{
export mhaltmsg=`anno -list -component 'nmh-mhaltmsg' "$mhmetafile"`
export mhdist=`anno -list -component 'nmh-mhdist' "$mhmetafile"`
export mhfolder=`anno -list -component 'nmh-mhfolder' "$mhmetafile"`
export mhmessages=`anno -list -component 'nmh-mhmessages' "$mhmetafile"`
export mhannotate=`anno -list -component 'nmh-mhannotate' "$mhmetafile"`
send "$@" "$mhdraft" || exit $?
rm -f "$mhmetafile"
exit 0
}
delete()
{
folder -push "$draftfolder" >/dev/null 2>&1
rmm "$draftfolder" c
folder -pop >/dev/null 2>&1
rm "$mhmetafile"
}
attach()
{
header=`mhparam 'Attachment-Header'`
while [ -n "$1" ]
do
if [ ! -f "
Re: automatic decode mime in repl
[2022-02-13 13:09] David Levine > Thank you for submitting your patch, Philipp. I have these > reservations with it: > > 2. For existing users, it may require changes to their existing nmh >environments. As I explained, for most cases this is not true, even if this is enabled by default. For users with the convertargs workaround this don't affect them because they use mhl.replywithoutbody. For other workarounds this will akt like a mail with no Content-Transfer-Encodin. So I don't see a risk of breaking good workarounds. Some may break, but then I would suspect the workaround will also break in some cases without my patch. If you still have conserns, the switch can also be disabled by default. I still belive there is no requirement for a switch at all, but I don't realy care. > 3. It does not add new functionality to nmh. Its functionality is a >proper subset of a pre-existing feature. > > 4. Given 3, if implemented within nmh, an alternative could be to >internally simply wrap a suitable use of the pre-existing feature. >From the user point of view, the result would be the same. I think you miss the point of my patch[0]. The functionality is there it has just no easy way to use it. My patch adds the easy way to use it. Think the idea forward this could allow to remove the convertargs switch of repl and the convert interface of mhbuild. Because all convertargs can do, can also be done by mhshow. As you said it so beautiful: From the user point of view, the result would be the same. > I have additional though lesser concerns, such as the name of the > switch The name of the switch is not importend to me, it was just the first think that come in my mind. So feel free to suggest a better name. > and the support effort required, In order to reduce the overall support effort required you could drop convertargs and the convert interface. This would reduce the overall code size, make the codeflow better to understand and help users and developers to better understand how repl works. And before you acuse me of want to break user setup, I realy don't care if you want to keep convertargs. This is only how I see what would be the best way to reduce support effort. Philipp [0] Aktuall I belive you full understand this and the implication
Bug: repl breaks urls
Hi
repl will break urls with the format switch, depending on the size of
your terminal. A test case is attached. To reproduce run the test in a
terminal with less then 300 chars width.
Philipp
#!/bin/sh
##
#
# Test repl -convertarg
#
##
set -e
if test -z "${MH_OBJ_DIR}"; then
srcdir=`dirname $0`/../..
MH_OBJ_DIR=`cd $srcdir && pwd`; export MH_OBJ_DIR
fi
. "${srcdir}/test/post/test-post-common.sh"
expected="$MH_TEST_DIR/test-url$$.expected"
actual=`mhpath +`/draft
printf 'Local-Mailbox: recipi...@example.com\n' >>"$MH"
Make sure that this works with 7-bit encoding.
LC_ALL=C; export LC_ALL
# check if urls get broken
start_test 'long urls get broken on reply'
cat >"$expected" <<'EOF'
From: recipi...@example.com
To: sen...@example.com
cc:
Fcc: +outbox
Subject: Re: test
Comments: In-reply-to sen...@example.com
message dated "Thu, 11 Dec 2014 08:19:02 -0600."
sen...@example.com writes:
> https://example.com/realy/long/path/to/some/importent/document/with/i/need/to/reference/in/the/reply/because/it/is/realy/importent/and/has/also/some/random/string/in/it/a//but/is/not/longer/than/900/chars/because/it/is/still/a/valide/mail.html
EOF
cat >`mhpath new` <<'EOF'
From: sen...@example.com
To: recipi...@example.com
Subject: test
Date: Thu, 11 Dec 2014 08:19:02 -0600
https://example.com/realy/long/path/to/some/importent/document/with/i/need/to/reference/in/the/reply/because/it/is/realy/importent/and/has/also/some/random/string/in/it/a//but/is/not/to/long/because/it/is/still/a/valide/mail.html
EOF
repl -editor true -nowhatnowproc -format last
check "$actual" "$expected"
Make sure that this works with 8-bit encoding.
finish_test
exit ${failed:-0}
Re: automatic decode mime in repl
[2022-02-11 18:25] David Levine > Philipp wrote: > > As a _m_mh user I don't have the -convertargs switch. > > You submitted a patch to nmh. I'm not sure what mmh has to do with > any of this. Are you asking for an nmh enhancement to support mmh? No, I try to explain why thinks looks diffrent to me, but you akt like you don't want to listen. I'm full aware I have submitted a patch to nmh. I think some of your users can benefit from it. With every mail from you I regent it more. I though there will be some discussion, but not this empty defence of an complex feature in order to reject my patch. So back to topic. You asked me if it's possible to improve my patch by using the convertargs and convert interface: No because it is an alternativ solution to convertargs. Your example of how this could be done doesn't even get the idea of my patch. So you might go back and look again what my patch does before complaining about it. If you have complaines or problems with my patch explain them proper. If you just claim there are other problems, I will asume you have somehow brok your setup. But stick to stuff which is introduced by my patch, not bugs which are there since forever and you just currently don't see because you don't use the default config. If you still like convertargs, stick with it. I couldn't care less. But then go away and let other users have this feature if they want to. > I would rather do that then add new code to nmh that doesn't seem > like a complete solution. You say: A complex copy of the mhshow interface is good to keep, but a small patch with calls mhshow out of repl is bad. The users should better add all the code to there config. Or did I missunderstand you? Philipp
Re: automatic decode mime in repl
[2022-02-11 02:55] David Levine > Philipp wrote: > > > Hi David > > > > I don't understand why do you try to convince me from convertargs. > > I'm not trying to convince you of anything. I'm trying to understand > how nmh could benefit from your patch. And whether your approach could > be improved by tying it to features that are already in repl (and that > you weren't aware of when you wrote the patch). And whether your patch > interferes with existing repl features. And what are both intended and > unintended uses of the new feature. Ok sorry then I missunderstand your request. > To clear up some things: [ ... ] > > * Your nmh installation is broken, as you observed. I expect that the > cause was deleting par after nmh was installed. The test suite did > the right thing by alerting you to the deficiency. We could guide > you through (easy) repair of your installation, but I would instead > suggest a complete rebuild, if you built from sources, or re-install > if you obtained a pre-built package. I think you miss a big point point, I don't have nmh. I have _m_mh. All I did was clone the repo, build it and run ``make check''. The hole time ``par'' wasn't installed. The build creates etc/mhn.defaults with the wrong mhbuild-convert-text entry. > And whether your approach could be improved by tying it to features > that are already in repl (and that you weren't aware of when you wrote > the patch). As a _m_mh user I don't have the -convertargs switch. And I don't see why I need it. It looks like the -convertargs switch and the convert interface are only there to allow repl decoding mime. But to decode mime messages there is a feature in nmh which was there before convertargs: mhshow. All my patch does is call mhshow and pipe the mail through it. Nothing more and nothing less. By this way you get mhshow to decode mime and all of it's features. convertargs is a diffrent aproatch, therefor I don't see a way to improve my patch by using convertargs. > And whether your patch interferes with existing repl features. The -convertargs switch still works, because it is used with the filter mhl.replywithoutbody. With the -mime switch you wont even hit the coresponding code path. I don't see any possible interferes with other repl features. With other workarounds there may some problems. I would asume most will just work, because they should also work on non mime mails. > And what are both intended and unintended uses of the new feature. Intendet is to make repl -fixmime be able to reply to a mime message. Unintendet is that repl may start a videoplayer, if there is a video in the message (or start some other graphical programms). > I'm trying to understand how nmh could benefit from your patch. Have an easy to use, enable by default and sane way to reply to mime messages. Under a easy to use and enable by default solution I understand something the user needs at most one switch to use this in a sane way. No aliases or editing of the profile, no requirement to manuall call mhbuild (or anything else) and no convenience shell file with aliases. > > The question is: Do you want it? > > Not with its current handling of URLs. How do you handle URLs in > messages that you reply to? This patch doesn't handle URLs at all. mhshow will print them as is to stdout. After that mhl will add the '> ' and linebreaks. As said in the other mail, this is the default behavior of repl. This has nothing to do with my patch. There where already two ways mentioned in this thread how this could be fixed. Philipp
Re: automatic decode mime in repl
Hi David
I don't understand why do you try to convince me from convertargs. As
said before: I have a working solution. Also convertargs will not work
in my setup. No I don't say convertargs is bad, I say my aproatch is
better. It has it own drawbacks, but in comparson they are smaller and
better to fix or workaround. Maybe I'm biased on this, but thats my view
on this. But lets look at it a bit deeper:
[2022-02-09 22:34] David Levine
> Philipp wrote:
>
> > The problem I see is that convertargs looks complex.
>
> If you use a Bourne-compatible shell, would you try this, please:
>
> source $(mhparam docdir)/contrib/replaliases
> rtm [msg]
This is not that easy, because I use mmh. So convertargs is not there.
But I can give you my experience I have got from the nmh tests. The
test repl/test-convert fails with following error:
> /bin/sh: 1: par: not found
i> charset=; iconv -f ${charset:-us-ascii} -t utf-8
'/home/satanist/src/nmh/test/testdir/Mail/mhbuildm4m2Df' | sed 's/^\(.\)/> \1/;
s/^$/>/;' | par 64 >/home/satanist/src/nmh/test/testdir/Mail/mhbuildwpUMcf
"$@": exited 127
> mhbuild: store of text/plain content failed, continuing...
> /bin/sh: 1: par: not found
> charset=; iconv -f ${charset:-us-ascii} -t utf-8
> '/home/satanist/src/nmh/test/testdir/Mail/mhbuildgcLmQf' | sed 's/^\(.\)/>
> \1/; s/^$/>/;' | par 64
> >/home/satanist/src/nmh/test/testdir/Mail/mhbuildXaOu6d "$@": exited 127
> mhbuild: store of text/plain content failed, continuing...
This is probaly a bug in etc/mhn.defaults.sh and install par fixed it.
But this is a bad first experience.
> > The problem I see is that convertargs looks complex. The goal of my
> > patch is to have an easy to use solution which is enabled by default.
>
> Easy for developers or easy for users?
First for the user, but also for the developers. For a user I see a few
problems:
What needs to be done that this is enabled by default? Yes I poke on this
point, because I belive this is importend. Wheter you like it or not:
MIME messages are common and nmh is not able to reply to one in the
default configuration.
How do I create alternativ repl configurations? Like have a `replwork'
which has diffrent default switches and forms. I just see the solution
to write an alias. But I like the argv0 configuration methode and would
miss it.
How can I use it from other things then a Bourne-compatible shell? Like
fish shell, powershell, ipython or a java process. As far as I see you
would need to port contrib/replaliases to all these interfaces or use a
sh instand of calling repl directly.
As a developer I see also a few problems:
For me both arpoces are code changes[0]. convertargs changes two
programms and needs a wrapper which provides an easy interface. My
aproatch changes only repl and depends with the rest on already
implemented features.
The required wrapper also needs some maintainership. Fixing bugs,
changed interface, interfaces for other shells. Yes the wrapper is
small, but it moves parts of a core feature to a contributer script.
So in genaral convertargs looks like a stopgap solution. I wanted to
have a solution and I have this solution since 2016. Yes it's not the
best solution, but a good one. The question is: Do you want it?
Philipp
[0] Or at least my arpoatch was 2016 when I intruduced it to mmh.
Re: automatic decode mime in repl
[2022-02-08 18:51] David Levine > Philipp wrote: > > > Thanks for all the discoussion. I have an updated version of this patch. > > I tried the patch. It works, but it produces text content of fixed > line lengths. So it can break URLs and other text that shouldn't be > arbitrarily modified. This is true, but not introduced by my patch. The mail is still piped through mhl which creates the fixed line length. As far as I see to change this there are some smaller changes in mhl necesary. > I haven't deciphered how it works, but I would think the same result > could be obtained using repl's -convertargs. That relies on the > mhbuild(1) Convert Interface. That was intended to be flexible, at > the expense of simplicity. Could it be used instead of adding new > code? Till your mail I didn't know convertargs, so I can't say if its possible. The problem I see is that convertargs looks complex. The goal of my patch is to have an easy to use solution which is enabled by default. The patch automaticly decodes the configured mime types. For common mime types the defaults work. To add extra mime types the already known config for mhshow is used, so there is no new interface to learn. Philipp
Re: automatic decode mime in repl
Hi
[2022-02-05 16:53] Philipp Takacs
> Reply to mime messages is an old problem. I have an idea to workaround
> or fix this. Instand of adding mime to mhl, repl could also first decode
> mime before pipe the mail to mhl. This can be done by mhshow.
>
> I have attached a patch for this.
Thanks for all the discoussion. I have an updated version of this patch.
I belive it's clear that you want a switch for this so I have implemented
it. Currently on by default. Also I have an other patch which implements
Ralphs suggestion.
I'm not sure if the manpage updates are good. Also the tests should be
updated. I'm also look for a way to autodetect if -nofixmime should be
set. I think the best place would be in nmh_init().
As a disclousure: I work with this patch since 2016. If you implement it
or not will not affect me. I thought this was clear from the beginning,
but it looks for me that this wasn't clear for everyone in the discussion.
Sorry for that. I belive this patch is good and you will benefit from it.
Thats is why I have send it to you and discoused why I belive this should
be enable by default. But in the end you can decide if and in which way
you want this patch.
Philipp
From a587df65eafb93bea5cc74053823fd07bc13ea8b Mon Sep 17 00:00:00 2001
From: Philipp Takacs
Date: Sat, 5 Feb 2022 12:21:35 +0100
Subject: [PATCH 1/2] repl: pipe mail through mhshow
When reply to a MIME message the mhshow will decode the mime message
before mhl will get it to create a reply draft. This makes it more
convinient to reply to a MIME message.
Problems:
* mhshow can be configured to display parts in a graphical programm. This
can lead to unexpected behavior on repl (i.e. start plaing musik).
* mhshow display string can contain '%l' which leads to a listing prior
to displaying content. This listing is contained in the draft.
---
man/repl.man | 22
uip/repl.c| 11 +++-
uip/replsbr.c | 137 +-
uip/replsbr.h | 2 +-
4 files changed, 134 insertions(+), 38 deletions(-)
diff --git a/man/repl.man b/man/repl.man
index 790e19b5..fc1935fc 100644
--- a/man/repl.man
+++ b/man/repl.man
@@ -51,6 +51,7 @@ all/to/cc/me]
.RB [ \-build ]
.RB [ \-file
.IR msgfile ]
+.RB [ \-fixmime " | " \-nofixmime ]
.ad
.SH DESCRIPTION
.B repl
@@ -71,6 +72,14 @@ format file (see
.IR mh\-format (5)
for details).
.PP
+To decode MIME messages
+.B repl
+pipe the message through
+.IR showmimeproc .
+This can be disabled with the
+.B \-nofixmime
+switch.
+.PP
If the switch
.B \-nogroup
is given (it is on by default), then
@@ -516,6 +525,7 @@ is checked.
^Editor:~^To override the default editor
^Msg\-Protect:~^To set mode when creating a new message (draft)
^fileproc:~^Program to refile the message
+^showmimeproc:~^Program to show non-text (MIME) messages
^mhlproc:~^Program to filter message being replied-to
^whatnowproc:~^Program to ask the \*(lqWhat now?\*(rq questions
.fi
@@ -542,6 +552,7 @@ is checked.
.RB ` \-noquery '
.RB ` \-noatfile '
.RB ` "\-width\ 72" '
+.RB ` \-fixmime '
.fi
.SH CONTEXT
If a folder is given, it will become the current folder. The message
@@ -578,3 +589,14 @@ don't call it
since
.B repl
won't run it.
+.PP
+The
+.RB \-fixmime
+switch causes repl to pipe the message through
+.IR showmimeproc .
+If the
+.IR showmimeproc
+is configured to display listing prior a mime part, these listing will end up in the draft.
+There might also be some unexpected behaviors, when the
+.IR showmimeproc
+is configured to display the content externel (i.e. open a browser to display html).
diff --git a/uip/repl.c b/uip/repl.c
index b1ceb1f6..8ed22709 100644
--- a/uip/repl.c
+++ b/uip/repl.c
@@ -72,6 +72,8 @@
X("noatfile", 0, NOATFILESW) \
X("fmtproc program", 0, FMTPROCSW) \
X("nofmtproc", 0, NFMTPROCSW) \
+X("fixmime", 0, FIXMIMESW) \
+X("nofixmime", 0, NFIXMIMESW) \
#define X(sw, minchars, id) id,
DEFINE_SWITCH_ENUM(REPL);
@@ -143,6 +145,7 @@ main (int argc, char **argv)
bool nedit = false;
bool nwhat = false;
bool atfile = false;
+bool fixmime = true;
int fmtproc = -1;
char *cp, *cwd, *dp, *maildir, *file = NULL;
char *folder = NULL, *msg = NULL, *dfolder = NULL;
@@ -354,6 +357,12 @@ main (int argc, char **argv)
case NFMTPROCSW:
fmtproc = 0;
continue;
+ case FIXMIMESW:
+ fixmime = true;
+ continue;
+ case NFIXMIMESW:
+ fixmime = false;
+ continue;
}
}
if (*cp == '+' || *cp == '@') {
@@ -469,7 +478,7 @@ try_it_again:
}
replout (in, msg, drft, mp, outputlinelen, mime, form, filter,
- fcc, fmtproc);
+ fcc, fmtproc, fixmime);
fclose (in);
{
diff --git a/uip/replsbr.c b/uip/replsbr.c
index ef4925ae..d2bab72f 100644
--- a/uip/replsbr.c
+++ b/uip/replsbr.c
@@ -61,7 +61,7 @@ static char *addrcomps[] = {
* static prototypes
*/
static
Re: automatic decode mime in repl
Hi [2022-02-07 16:54] Ralph Corderoy > > Instand of only depending on the programm name the config can be > > extended to also depend on the calling programm. So you can config > > diffrent options depending on which nmh programm called another nmh > > programm. This could be done by (ab)using argv[0]. > > Adding yet another style of interface between nmh's parts, which would > have to be documented and users would have to understand and use, seems > wrong. What's your reason against running mhshow as ‘mhreplyfmt’, or > something? Is it the need to duplicate all MIME-configuration entries > from ‘mhshow-*’ to ‘mhreplyfmt-*’? The duplication is the one thing. The other thing is I would like to extend the existing configuration system. Yes you can run mhshow as `mhrelyfmt', but there is no logic behind the choice of argv[0]. The idea was to define such a logic and use it to make the config more powerfull. As an exaple it could be possible to have diffrent showmimeproc config for a `replA' and a `replB'. Or what about a diffrent sendmail or even another path? Yes this is a big change and way beound this patch. This might be as far in the future as implementing MIME for mhl. Start with smaller fix like setting argv[0] to `mhreplyfmt' is perfect fine. > > * mhshow display string can contain '%l' which leads to a listing > > prior to displaying content. This listing is contained in the draft. > > Again, the mhreplyfmt-... could choose whether to use it. I think > I would sometimes want it as part of the reply text, e.g. it's a > one-line summary of a PDF and I want to quote that line for context > about my following chunk of reply. I have never thought[0] about solving this in this way. Yes this will also work. In general I'm not a big fan of the '%l' and would prefer one switch instand of '%l', but this is another discussion. Philipp [0] Mostly because of small diffrences between mmh and nmh.
Re: automatic decode mime in repl
[2022-02-06 08:26] Paul Fox > philipp wrote: > > [2022-02-05 22:19] Ken Hornstein > > > > > > My only concern with this patch is that it unilaterally overrides the > > > existing behavior, and one thing I've seen over the years is that there > > > are a LOT of users wedded to existing behavior. So I'd rather make it > > > selectable (I am neutral about it being default or not). I'd welcome > > > discussion on this issue. > > > > I can understand that a changed behavior is a problem. But in this case > > I would argue there is no other way. Every patch witch tries to fix this > > will change the existing behavior, because the existing behavior is the > > problem. > > > > So I would prefer no option. But if you want one, I can add an option. > > I would prefer the new behavior be the default. Also I would prefer a > > build time option over a runtime option. I'm bad at naming, has someone > > an idea for the name of such an option. > > I haven't tried the patch, but I think any change to basic repl > behavior should a) be optional, b) be controlled by a runtime option. > And it should probably have a lot of testing by a lot of users before > being made the default. > > I can't be alone in having customized scripts for doing replies which > already run mhshow in crafting the reply. Perhaps everything would > continue to just work, or perhaps not. I understand your reasaning. I just fear then this will get just an other stopgap solution. I would assume most users with a workaround for the problem will try this once. If it don't work with there workaround, they will just disable it. If someone brings up the idea of change the default they would vote against it. I don't want to blame such behavior. This makes perfect sense from the standpoint of a user with a workaround. It just don't make nmh better. I belive reply to a mime message should just work. The best solution would be implementing mime support in mhl. But this will not happen for a long time. So the goal should be add a solution[0] and make it the default. I don't see a reason for an option[1], because I consider the current behavior as a bug. There shouldn't be an option to enable/disable a bug. As said in my previous mail, I'm not complete against an option. I would just prefere not having one. Philipp [0] I don't care if this is my solution or any other. I just don't see another solution. [1] You still can change the showmimeproc to disable this.
Re: automatic decode mime in repl
[2022-02-05 22:19] Ken Hornstein > >Reply to mime messages is an old problem. I have an idea to workaround > >or fix this. Instand of adding mime to mhl, repl could also first decode > >mime before pipe the mail to mhl. This can be done by mhshow. > > > >I have attached a patch for this. > > From a purely architectural perspective, I think that having mhl be > MIME-aware is the correct solution. But I must acknowledge the reality > on the ground: > > - This has been a problem for a very long time > > - Stopgap solutions (like replyfilter) have not made much headway, because > they require end-user configuration. > > - Getting a MIME-aware mhl (really, all of nmh) is a huge job. > > - I believe "perfect is the enemy of the good". > > My only concern with this patch is that it unilaterally overrides the > existing behavior, and one thing I've seen over the years is that there > are a LOT of users wedded to existing behavior. So I'd rather make it > selectable (I am neutral about it being default or not). I'd welcome > discussion on this issue. I can understand that a changed behavior is a problem. But in this case I would argue there is no other way. Every patch witch tries to fix this will change the existing behavior, because the existing behavior is the problem. So I would prefer no option. But if you want one, I can add an option. I would prefer the new behavior be the default. Also I would prefer a build time option over a runtime option. I'm bad at naming, has someone an idea for the name of such an option. An alternativ would be to depend on fmtproc. When fmtproc is set don't pipe through mhshow. When it is not set pipe through mhshow. Philipp > --Ken > > >This approatch has currently two problems: > > > >* mhshow can be configured to display parts in a graphical programm. This > > can lead to unexpected behavior on repl (i.e. start plaing musik). > > > >I belive there are two options to fixed this problem. First the -textonly > >swtich can be used. In some probably uncommon setups (i.e. text to speech) > >this still can lead to problems. > > > >The secound option is a bit more complex. Instand of only depending on > >the programm name the config can be extended to also depend on the > >calling programm. So you can config diffrent options depending on which > >nmh programm called another nmh programm. This could be done by (ab)using > >argv[0]. > > > >* mhshow display string can contain '%l' which leads to a listing prior > > to displaying content. This listing is contained in the draft. > > > >To fix this I would suggest to remove the '%l' from the display string > >escapes and create a switch for that. Then you can't enable/disable this > >per part. > > > >What do you think about this patch? Did I missed some problems? > > > >Philipp > > > >>> text/x-diff content
automatic decode mime in repl
Hi
Reply to mime messages is an old problem. I have an idea to workaround
or fix this. Instand of adding mime to mhl, repl could also first decode
mime before pipe the mail to mhl. This can be done by mhshow.
I have attached a patch for this.
This approatch has currently two problems:
* mhshow can be configured to display parts in a graphical programm. This
can lead to unexpected behavior on repl (i.e. start plaing musik).
I belive there are two options to fixed this problem. First the -textonly
swtich can be used. In some probably uncommon setups (i.e. text to speech)
this still can lead to problems.
The secound option is a bit more complex. Instand of only depending on
the programm name the config can be extended to also depend on the
calling programm. So you can config diffrent options depending on which
nmh programm called another nmh programm. This could be done by (ab)using
argv[0].
* mhshow display string can contain '%l' which leads to a listing prior
to displaying content. This listing is contained in the draft.
To fix this I would suggest to remove the '%l' from the display string
escapes and create a switch for that. Then you can't enable/disable this
per part.
What do you think about this patch? Did I missed some problems?
Philipp
From 34d2763d73edf12e4b4cfb45e063e1ffa19519c3 Mon Sep 17 00:00:00 2001
From: Philipp Takacs
Date: Sat, 5 Feb 2022 12:21:35 +0100
Subject: [PATCH] repl: pipe mail through mhshow
When reply to a MIME message the mhshow will decode the mime message
before mhl will get it to create a reply draft. This makes it more
convinient to reply to a MIME message.
Problems:
* mhshow can be configured to display parts in a graphical programm. This
can lead to unexpected behavior on repl (i.e. start plaing musik).
* mhshow display string can contain '%l' which leads to a listing prior
to displaying content. This listing is contained in the draft.
---
uip/replsbr.c | 106 --
1 file changed, 76 insertions(+), 30 deletions(-)
diff --git a/uip/replsbr.c b/uip/replsbr.c
index ef4925ae..2dce085e 100644
--- a/uip/replsbr.c
+++ b/uip/replsbr.c
@@ -436,6 +436,10 @@ replfilter (FILE *in, FILE *out, char *filter, int fmtproc)
char *errstr;
char **arglist;
int argnum;
+int mailpipe[2];
+char *mhshow;
+char **mhshowargs;
+int mshowargnum;
if (filter == NULL)
return;
@@ -447,48 +451,90 @@ replfilter (FILE *in, FILE *out, char *filter, int fmtproc)
lseek(fileno(in), 0, SEEK_SET);
arglist = argsplit(mhlproc, , );
+mhshowargs = argsplit(showmimeproc, , );
switch (pid = fork()) {
case NOTOK:
adios ("fork", "unable to");
case OK:
- dup2 (fileno (in), fileno (stdin));
- dup2 (fileno (out), fileno (stdout));
-
- /*
- * We're not allocating the memory for the extra arguments,
- * because we never call arglist_free(). But if we ever change
- * that be sure to use getcpy() for the extra arguments.
- */
- arglist[argnum++] = "-form";
- arglist[argnum++] = filter;
- arglist[argnum++] = "-noclear";
-
- switch (fmtproc) {
- case 1:
- arglist[argnum++] = "-fmtproc";
- arglist[argnum++] = formatproc;
- break;
- case 0:
- arglist[argnum++] = "-nofmtproc";
- break;
+
+ if (pipe(mailpipe) == -1) {
+ adios("pipe", "can't create pipe");
}
- arglist[argnum++] = NULL;
+ switch (pid = fork()) {
+ case NOTOK:
+ adios ("fork", "unable to");
+
+ case OK:
+ dup2 (fileno (in), fileno (stdin));
+ dup2 (mailpipe[1], fileno (stdout));
+ close(fileno(in));
+ close(fileno(out));
+ close(mailpipe[0]);
+ close(mailpipe[1]);
+
+ mhshowargs[argnum++] = "-file";
+ mhshowargs[argnum++] = "-";
+ mhshowargs[argnum++] = "-concat";
+ mhshowargs[argnum++] = NULL;
+
+ execvp (mhshow, mhshowargs);
+ errstr = strerror(errno);
+ if (write(2, "unable to exec ", 15) < 0 ||
+ write(2, mhshow, strlen(mhshow)) < 0 ||
+ write(2, ": ", 2) < 0 ||
+ write(2, errstr, strlen(errstr)) < 0 ||
+ write(2, "\n", 1) < 0) {
+ advise ("stderr", "write");
+ }
+ _exit(1);
+ default:
+ dup2 (mailpipe[0], fileno (stdin));
+ dup2 (fileno (out), fileno (stdout));
+ close(fileno(in));
+ close(fileno(out));
+ close(mailpipe[0]);
+ close(mailpipe[1]);
+
+ /*
+ * We're not allocating the memory for the extra arguments,
+ * because we never call arglist_free(). But if we ever change
+ * that be sure to use getcpy() for the extra arguments.
+ */
+ arglist[argnum++] = "-nomoreproc";
+ arglist[argnum++] = "-form";
+ arglist[argnum++] = filter;
+ arglist[argnum++] =
Re: How to give Thomas Dupond the date in his replies.
Hi [2022-01-29 12:41] Thomas Dupond > > Philipp Takacs wrote: > > > [2022-01-28 18:18] Ken Hornstein > > > >However when I do that and quote some email which is not strictly ANSI I > > > >get these weird =C3=A9 instead of é. Some other characters are > > > >affected. > > > > > > > >Do you have any idea on how to resolve that? > > > > > > In a perfect world mhl would be MIME-aware and just take care of that for > > > you. We don't yet live in that world. I wrote "replyfilter" to mostly > > > deal with that; it's a perl script that lives in the nmh contrib directory > > > and the comments in it explain how to use it. > > > > In mmh we have a workaround for this problem. To decode MIME we pipe the > > mail through show. I know this has a few downsides, but if you are > > intrested I can help to port this to nmh. > > Never heard of mmh before. Is it another port of MH? It's a fork of nmh. Currently not realy maintained, because of lack of time and motivation. See http://marmaro.de/prog/mmh/ Philipp
Re: How to give Thomas Dupond the date in his replies.
[2022-01-28 18:18] Ken Hornstein > >However when I do that and quote some email which is not strictly ANSI I > >get these weird =C3=A9 instead of é. Some other characters are > >affected. > > > >Do you have any idea on how to resolve that? > > In a perfect world mhl would be MIME-aware and just take care of that for > you. We don't yet live in that world. I wrote "replyfilter" to mostly > deal with that; it's a perl script that lives in the nmh contrib directory > and the comments in it explain how to use it. In mmh we have a workaround for this problem. To decode MIME we pipe the mail through show. I know this has a few downsides, but if you are intrested I can help to port this to nmh. Philipp
Re: Message header formatting
Hi [2021-06-05 15:16] Jon Steinhart > Is there any incantation for "show only the headers explicitly listed > in mhl.format" so that new and uninteresting headers from everybody's > latest spam filter, mailing list manager, and internal tracking don't > fill the screen. You can remove the ``extras'' component from your format file. See mhl(1): > The component "Extras" will output all of the components of the > message which were not matched by explicit components, or included in > the ignore list. Philipp
Re: questions about the Previous-Sequence
[2020-03-22 21:13] Conrad Hughes > One of the things I use it for is "unseeing" a message. Let's say I see > someone has emailed me, I'm about to go out, want to take a peek at the > message but don't want to mark it as seen — or alternatively I look at a > message, think "gosh that's gonna take a lot of work to deal with" so > want to mark it as unseen again, so I come back to it. My > unseen-sequence is "un", and my previous-sequence is "ditto", so I have > this alias to "unsee" whatever I just looked at: > > mark -sequence un -add ditto I have following in my profile: marku: -sequence u -add -nozero So if I want to unsee a mail I have just read, I just type marku. This only works if I read only one mail at the time or reuse the sequence/pick. Philipp
Re: questions about the Previous-Sequence
[2020-03-22 14:53] Ken Hornstein > >We think currently about removing the Previous-Sequence support for > >mmh. But because we don't use it we are not sure, if we missed some > >aspect of it. Therefor I would like to ask some questions. > > I personally find the previous-sequence rather useful myself (when you > find, for example, the results of "pick" were rather larger than you > expected and didn't put it in a sequence). But that's up to you. May I ask, why do you prefer it over the history and command line editing? > >When and why was the Previous-Sequence introduced to mh or nmh? > > Looks like it has been around for a while. From mh4/MHCHANGES: > > Mon Jul 16 00:21:52 1984 Rand MH mail system (agent: Marshall Rose) > > > Install the new Previous-Sequence mechanism. Introduce the SEQMOD > flag to the msgs structure. Just about every MH program now calls > m_setseq () upon parsing the messages and calls m_sync() prior to > exiting. > > Before my time, so I can't really answer the "why". Thanks for this info. Philipp Ps: Please don't put me in to or cc. I'm subcribed to the list and use the list-id field for scan and filtering.
Re: questions about the Previous-Sequence
[2020-03-22 14:58] Ralph Corderoy > > 1. in the shell using the same messages again and spare some typing > ... > > The first completely covered with the shell history. > > If the arguments given to the last nmh command don't have a side effect > then shell history may be useful, but > > show next:3 > scan !$ > > doesn't achieve the same as Previous-Sequence. > > Not arguing you shouldn't remove it, just spotting a case you may not > have considered. Yes we have overseen this case, thanks. Philipp
questions about the Previous-Sequence
Hi all Disclaimer first: I'm a mmh-developer and user We think currently about removing the Previous-Sequence support for mmh. But because we don't use it we are not sure, if we missed some aspect of it. Therefor I would like to ask some questions. When and why was the Previous-Sequence introduced to mh or nmh? What are the use cases? We found two: 1. in the shell using the same messages again and spare some typing 2. put messages in a sequence after a refile[0] The first completely covered with the shell history. The second one can don with the new sequence or the -retainsequences switch. Did we miss some use cases? Are there other reasons to keep this feature? Philipp Ps: really nice work you do. [0] https://lists.nongnu.org/archive/html/nmh-workers/2005-12/msg00086.html
Re: [nmh-workers] FSF is changing Mailman list settings unless you opt out (fwd)
[2019-09-29 18:16] Michael Richardson
> So I've added:
>
>%-16(putstrf{List-Id})
>
> to my scan arguments, and it's close, but not yet what I want.
> Ideally, I just want what's in <> of the List-Id:, and I'd be happy with the
> first 16 characters only. But I can't see a way to get that. So I'd settle
> for the last 16 bytes of the entire header, but I can't get that either.
You could just use:
%(addr{list-id})
I know list-id isn't realy an address, but it works.
Philipp
--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
[nmh-workers] mmh-0.4 is released! (meillo's mail handler)
Hi everyone After more then a year I can announce the release of mmh-0.4 http://marmaro.de/prog/mmh/files/mmh-0.4.tar.gz The important changes are: - pick now has thread support - pick and scan merged into one binary - mhl globing support for filter out some header fields - mhl trim (ported from nmh) - whatnow2 is finished for replacing whatnow - whom supports message argument Have a look at the NEWS file in the tarball for more information about the changes. The full changelog is in the ChangeLog file in the tarball. All changes and the current developer version can be found in the git at http://git.marmaro.de/mmh mmh 0.4 has in total 33,072 SLOC. This is a bit more then then mmh 0.3 which has 32,645 SLOC. SLOCChange Directory SLOC-by-Language (Sorted) 19159 (+100) uip ansic=18320,sh=839 7061(+102) sbr ansic=6987,awk=74 3236(+219) testsh=3236 2753top_dir sh=2753 764 h ansic=764 60 config ansic=60 38 man sh=38 The most new code comes from new features in mhl and the scan/pick merge. On the other side the whole scan.c file is removed. Also whatnow is still in the count. (All numbers calculated by sloccount.) Thanks to everyone who contributed to this release. Some of the work is ported from nmh. The commits: 47 Philipp Takacs 22 markus schnalke 7 Dmitry Bogatov 5 c_14 4 Vasilii Kolobkov 2 Larry Hynes 1 David Levine 1 mor0i...@gmail.com Further contributions on the mailing list by: Jan/subpath, and Link Satonaka. Philipp -- nmh-workers https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Attachments question
Hi Stewart I just used to composed messages of the form To: ... cc: ... From: ... Fcc: +outbox Subject: ... Bla, bla, bla ... #application/pdf; name=Abiturzeugnis.pdf Abiturzeugnis.pdf #image/jpeg; name=Hotel_Walnut-Creek.jpg Hotel_Walnut-Creek.jpg #application/rtf; name=LetterRecom.rtf /Users/phil/LetterRecom.rtf ... Best regards Philipp On 29 Aug 2007, at 23:26 , Stewart W. Wilson wrote: Hi, How do you do attachments with nmh? I've googled for an hour and can't find anything that works. Suppose I have, say, a tar file that I want to attach. What do I do and how do I have to be set up? My .mh_profile is as follows (I have nmh-1.1-RC3): -- Path: Mail Current-folder: inbox Draft-folder: outbox Editor: vi Signature: Stewart Wilson ali: -alias /home/wilson/.mh_aliases send: -alias /home/wilson/.mh_aliases whom: -alias /home/wilson/.mh_aliases unseen-sequence: unseen -- Thanks for your help! Stewart ___ Nmh-workers mailing list Nmh-workers@nongnu.org http://lists.nongnu.org/mailman/listinfo/nmh-workers ___ Nmh-workers mailing list Nmh-workers@nongnu.org http://lists.nongnu.org/mailman/listinfo/nmh-workers
[Nmh-workers] send (From: ) problem
When trying to send mails via smtp I get the following response:
$ send -snoop -verbose -server mail.gmx.net -user 1281346 -sasl -
saslmech LOGIN
...
= MAIL FROM:[EMAIL PROTECTED]
= 550 5.1.8 {mp040} Cannot resolve your domain
= RSET
= 250 2.0.0 {mp040} Flushed
= QUIT
= 221 2.0.0 {mp040} GMX Mailservices
post: problem initializing server; [RPLY] 550 5.1.8 {mp040} Cannot
resolve your domain
send: message not delivered to anyone
MAIL FROM shouldn't be [EMAIL PROTECTED], which is my local
login and machine name, but my email address for that account (like
[EMAIL PROTECTED]). How can I force nmh to use that?
Philipp
___
Nmh-workers mailing list
Nmh-workers@nongnu.org
http://lists.nongnu.org/mailman/listinfo/nmh-workers
[Nmh-workers] Re: send (smtp) problems
Philipp Roessler [EMAIL PROTECTED] writes: I have no .netrc (why would I?).To avoid typing the password every time... Ah, ok, have a .netrc now.I haven't tried SASL yet,so can't help you there. However, send should be prompting you for apassword but does not seem to be doing so. Bug?Not a bug, I figured out what went wrong in the first place: I had to configure with --with-cyrus-sasl to get SASL support. One more general question: how can I inspect (in plain text) the messages sent back and forth between my machine and the smtp server?I would expect that "send -watch" would do this for you, but itdoesn't. Try the undocumented argument "-snoop" instead.Cool, this looks interesting ... though I don't understand it. Can anyone tell from the following what goes wrong (I'm pretty sure it's not the wrong password, I don't mistype 100 times ;-) )?$ send -snoop -verbose -server mail.gmx.net -user 1281346 -sasl -saslmech LOGINUse "/Users/phil/Mail/draft"? y -- Posting for All Recipients --= 220 {mp036} GMX Mailservices ESMTP= EHLO organtin.local= 250-{mp036} GMX Mailservices= 250-8BITMIME= 250-ENHANCEDSTATUSCODES= 250-AUTH=LOGIN CRAM-MD5 PLAIN= 250-AUTH CRAM-MD5 LOGIN PLAIN= 250 STARTTLS= AUTH LOGIN= 334 WXRajy5hbW3= nGopbA=== 334 KNPzc3dasC6= B3ObjkdpZWRbhuiYwNjA2= 535 5.7.0 {mp036} Incorrect username or password= RSET= 250 2.0.0 {mp036} Flushed= QUIT= 221 2.0.0 {mp036} GMX Mailservicespost: problem initializing server; [BHST] 5.7.0 {mp036} Incorrect username or passwordsend: message not delivered to anyone-- Bill Wohler [EMAIL PROTECTED] http://www.newt.com/wohler/ GnuPG ID:610BD9ADOn 05 Jul 2006, at 21:06 , Philipp Roessler wrote:I'm new to nmh and are having troubles sending mail.First try (first smtp server):$ send -verbose -server mx.freenet.de -user der_wachtmeister -saslUse "/Users/phil/Mail/draft"? y-- Posting for All Recipients -- -- Network Recipients -- phroessler at gmx.de: loses; [USER] 550 authentication requiredpost: 1 addressee undeliverablesend: message not delivered to anyone~/Mail/draft: "From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] cc: Fcc: +outbox Subject: test nmh 20060705 ---- Hi Phil Let's see how this works... Philipp"Second try (another smtp server):$ send -verbose -server mail.gmx.net -user 1281346 -saslUse "/Users/phil/Mail/draft"? y-- Posting for All Recipients --post: problem initializing server; [RPLY] 550 5.7.0 {mp010} Need to authenticate via SMTP-AUTH-Loginsend: message not delivered to anyoneWhat am I missing in each case?Here are my configuration files:~/.mh-profile: Path: Mail~/hier/etc/mts.conf: mts: smtp hostable: /Users/phil/hier/etc/hosts localname: mail.gmx.net masquerade: draft_from mmdfldir: /var/mail mmdflfil: pophost: pop.gmx.net servers: mail.gmx.net~/hier/etc/hosts: mail.gmx.net mx.freenet.deI have no .netrc (why would I?).One more general question: how can I inspect (in plain text) the messages sent back and forth between my machine and the smtp server?Philipp___
Nmh-workers mailing list
Nmh-workers@nongnu.org
http://lists.nongnu.org/mailman/listinfo/nmh-workers
[Nmh-workers] send (smtp) problems
I'm new to nmh and are having troubles sending mail.
First try (first smtp server):
$ send -verbose -server mx.freenet.de -user der_wachtmeister -sasl
Use /Users/phil/Mail/draft? y
-- Posting for All Recipients --
-- Network Recipients --
phroessler at gmx.de: loses; [USER] 550 authentication required
post: 1 addressee undeliverable
send: message not delivered to anyone
~/Mail/draft:
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
cc:
Fcc: +outbox
Subject: test nmh 20060705
Hi Phil
Let's see how this works...
Philipp
Second try (another smtp server):
$ send -verbose -server mail.gmx.net -user 1281346 -sasl
Use /Users/phil/Mail/draft? y
-- Posting for All Recipients --
post: problem initializing server; [RPLY] 550 5.7.0 {mp010} Need to
authenticate via SMTP-AUTH-Login
send: message not delivered to anyone
What am I missing in each case?
Here are my configuration files:
~/.mh-profile:
Path: Mail
~/hier/etc/mts.conf:
mts: smtp
hostable: /Users/phil/hier/etc/hosts
localname: mail.gmx.net
masquerade: draft_from
mmdfldir: /var/mail
mmdflfil:
pophost: pop.gmx.net
servers: mail.gmx.net
~/hier/etc/hosts:
mail.gmx.net
mx.freenet.de
I have no .netrc (why would I?).
One more general question: how can I inspect (in plain text) the
messages sent back and forth between my machine and the smtp server?
Philipp
On 05 Jul 2006, at 18:06 , [EMAIL PROTECTED] wrote:
Send Nmh-workers mailing list submissions to
nmh-workers@nongnu.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.nongnu.org/mailman/listinfo/nmh-workers
or, via email, send a message with subject or body 'help' to
[EMAIL PROTECTED]
You can reach the person managing the list at
[EMAIL PROTECTED]
When replying, please edit your Subject line so it is more specific
than Re: Contents of Nmh-workers digest...
Today's Topics:
1. Mac OS X Tiger [Was: compile nmh on Mac OS X breaks on
slocal.c] (Philipp Roessler)
--
Message: 1
Date: Wed, 5 Jul 2006 13:45:23 +0200
From: Philipp Roessler [EMAIL PROTECTED]
Subject: [Nmh-workers] Mac OS X Tiger [Was: compile nmh on Mac OS X
breaks on slocal.c]
To: nmh-workers@nongnu.org
Message-ID: [EMAIL PROTECTED]
Content-Type: text/plain; charset=us-ascii
Here is how I was able to install nmh on my ibook (using the rc
shell):
$ cvs -z3 -d:pserver:[EMAIL PROTECTED]:/sources/nmh
co nmh
$ cd nmh
$ autoconf
$ autoheader
$ LDFLAGS = -dynamic configure --enable-debug --enable-pop --with-
editor'='$EDITOR --prefix'='$HOME/hier
$ make
$ sudo make install
Good day
Philipp
-- next part --
An HTML attachment was scrubbed...
URL: http://lists.gnu.org/pipermail/nmh-workers/attachments/
20060705/48821831/attachment.html
--
___
Nmh-workers mailing list
Nmh-workers@nongnu.org
http://lists.nongnu.org/mailman/listinfo/nmh-workers
End of Nmh-workers Digest, Vol 32, Issue 1
**
___
Nmh-workers mailing list
Nmh-workers@nongnu.org
http://lists.nongnu.org/mailman/listinfo/nmh-workers
