Re: automatic decode mime in repl

[Philipp Takacs]

[2022-02-13 13:09] David Levine 
> Thank you for submitting your patch, Philipp.  I have these
> reservations with it:
>
> 2. For existing users, it may require changes to their existing nmh
>environments.

As I explained, for most cases this is not true, even if this is enabled
by default. For users with the convertargs workaround this don't affect
them because they use mhl.replywithoutbody. For other workarounds this
will akt like a mail with no Content-Transfer-Encodin. So I don't see a
risk of breaking good workarounds. Some may break, but then I would
suspect the workaround will also break in some cases without my patch.

If you still have conserns, the switch can also be disabled by default.
I still belive there is no requirement for a switch at all, but I don't
realy care.

> 3. It does not add new functionality to nmh.  Its functionality is a
>proper subset of a pre-existing feature.
>
> 4. Given 3, if implemented within nmh, an alternative could be to
>internally simply wrap a suitable use of the pre-existing feature.
>From the user point of view, the result would be the same.

I think you miss the point of my patch[0]. The functionality is there it
has just no easy way to use it. My patch adds the easy way to use it.
Think the idea forward this could allow to remove the convertargs switch
of repl and the convert interface of mhbuild. Because all convertargs
can do, can also be done by mhshow. As you said it so beautiful: From
the user point of view, the result would be the same.

> I have additional though lesser concerns, such as the name of the
> switch

The name of the switch is not importend to me, it was just the first
think that come in my mind. So feel free to suggest a better name.

> and the support effort required,

In order to reduce the overall support effort required you could drop
convertargs and the convert interface. This would reduce the overall
code size, make the codeflow better to understand and help users and
developers to better understand how repl works.

And before you acuse me of want to break user setup, I realy don't care
if you want to keep convertargs. This is only how I see what would be
the best way to reduce support effort.

Philipp

[0] Aktuall I belive you full understand this and the implication

Re: automatic decode mime in repl

[David Levine]

Thank you for submitting your patch, Philipp.  I have these
reservations with it:

1. For all users, it can add newlines in text that cause, for example,
   URLs to break.

2. For existing users, it may require changes to their existing nmh
   environments.

3. It does not add new functionality to nmh.  Its functionality is a
   proper subset of a pre-existing feature.

4. Given 3, if implemented within nmh, an alternative could be to
   internally simply wrap a suitable use of the pre-existing feature.
   From the user point of view, the result would be the same.

I have additional though lesser concerns, such as the name of the
switch and the support effort required, mostly related to 1. and 2.
above.

David

Re: automatic decode mime in repl

[Philipp Takacs]

[2022-02-11 18:25] David Levine 
> Philipp wrote:
> > As a _m_mh user I don't have the -convertargs switch.
>
> You submitted a patch to nmh.  I'm not sure what mmh has to do with
> any of this.  Are you asking for an nmh enhancement to support mmh?

No, I try to explain why thinks looks diffrent to me, but you akt like
you don't want to listen.

I'm full aware I have submitted a patch to nmh. I think some of your
users can benefit from it. With every mail from you I regent it more. I
though there will be some discussion, but not this empty defence of
an complex feature in order to reject my patch.

So back to topic. You asked me if it's possible to improve my patch by
using the convertargs and convert interface: No because it is an
alternativ solution to convertargs. Your example of how this could be
done doesn't even get the idea of my patch. So you might go back and
look again what my patch does before complaining about it.

If you have complaines or problems with my patch explain them proper. If
you just claim there are other problems, I will asume you have somehow
brok your setup. But stick to stuff which is introduced by my patch, not
bugs which are there since forever and you just currently don't see
because you don't use the default config.

If you still like convertargs, stick with it. I couldn't care less. But
then go away and let other users have this feature if they want to.

> I would rather do that then add new code to nmh that doesn't seem
> like a complete solution.

You say: A complex copy of the mhshow interface is good to keep, but a
small patch with calls mhshow out of repl is bad. The users should
better add all the code to there config. Or did I missunderstand you?

Philipp

Re: automatic decode mime in repl

[David Levine]

Laura wrote:

> It's
> the people who send me mail as pdf and mail as encoded base64 that
> are hard to reply to.  But it's not a common occurrance these days,

I would hope so.  It should be easy to add support for pdf as long as
it originated from text rather than an image.  And it should be easy
to support base64 directly, or try piping it through mhfixmsg.

David

Re: automatic decode mime in repl

[David Levine]

Ralph wrote:

> It's better than what I normally do.  I need to read the fine manual,
> understand what's going on, and then unwind my ancient methods and
> adjust to trying this.

The most important piece is mhbuild-convert-text/html in your profile
or mhn.defaults.

David

Re: automatic decode mime in repl

[Laura Creighton]

In a message of Fri, 11 Feb 2022 18:37:28 +, David Levine writes:
>Laura wrote:
>
>> I would really like to have this.  The workaround I have made is really > 
>> ugly, and doesn't always work.
>
>Does this produce a result that seems at least as good?
>
>  \repl -filter mhl.replywithoutbody -convertargs text/html ''
>-convertargs text/plain '' -editor mhbuild [msg]
>
>Where [msg] is the usual optional message argument to repl.
>
>David

I will have to find a suitable piece of mail to check it on, but
I already have a working solution for replying to html mail.  It's
the people who send me mail as pdf and mail as encoded base64 that
are hard to reply to.  But it's not a common occurrance these days,
which is why I don't have something handy.  Will look tomorrow after
I have had some sleep.

Laura

Re: automatic decode mime in repl

[Ralph Corderoy]

Hi David,

> Does this produce a result that seems at least as good?

It's better than what I normally do.  I need to read the fine manual,
understand what's going on, and then unwind my ancient methods and
adjust to trying this.

-- 
Cheers, Ralph.

Re: automatic decode mime in repl

[David Levine]

Laura wrote:

> I would really like to have this.  The workaround I have made is really > 
> ugly, and doesn't always work.

Does this produce a result that seems at least as good?

  \repl -filter mhl.replywithoutbody -convertargs text/html ''
-convertargs text/plain '' -editor mhbuild [msg]

Where [msg] is the usual optional message argument to repl.

David

Re: automatic decode mime in repl

[David Levine]

Philipp wrote:

> All I did was clone the repo, build it and run ``make check''. The
> hole time ``par'' wasn't installed. The build creates etc/mhn.defaults
> with the wrong mhbuild-convert-text entry.

Here is the mhn.defaults.sh code that puts par in mhn.defaults:
if [ -n `$SEARCHPROG "$SEARCHPATH" par` ]; then
textfmt=' | par 64'
replfmt=" | sed 's/^\(.\)/> \1/; s/^$/>/;' | par 64"
SEARCHPROG is etc/mhn.find.sh.  SEARCHPATH is your PATH.  If you
want to help us track down this issue, would you please:
  1. cd to your cloned nmh directory
  2. execute etc/mhn.defaults.sh "$PATH" etc/mhn.find.sh and pipe
 the output through:  grep 'par '
using whatever commands are appropriate for your shell, e.g., for bash:
  cd (your nmh directory)
  etc/mhn.defaults "$PATH" etc/mh.find.sh | grep 'par '
And let us (or me) know the result.  Those commands will not modify
anything in your nmh directory hierarchy.

> As a _m_mh user I don't have the -convertargs switch.

You submitted a patch to nmh.  I'm not sure what mmh has to do with
any of this.  Are you asking for an nmh enhancement to support mmh?

> It looks like the -convertargs switch and the convert
> interface are only there to allow repl decoding mime.

More than just decoding mime content, convertargs provides an
interface to support conversion of any content when replying.

> But to decode mime messages there is a feature in nmh which was there
> before convertargs: mhshow. All my patch does is call mhshow and pipe
> the mail through it. Nothing more and nothing less. By this way you
> get mhshow to decode mime and all of it's features.

And all of its drawbacks.

> > I'm trying to understand how nmh could benefit from your patch.
>
> Have an easy to use, enable by default and sane way to reply to mime
> messages.

It doesn't work for me.  There are other issues, likely to do with
other configuration in my setup.  But if I run into issues, I should
expect that other people will, too.

> This patch doesn't handle URLs at all. mhshow will print them as is to
> stdout. After that mhl will add the '> ' and linebreaks. As said in the
> other mail, this is the default behavior of repl. This has nothing to do
> with my patch.

With your patch, URLs break for me.  I can look at fixing that somewhere
else, but then this isn't true for me:

> Under a easy to use and enable by default solution I understand something
> the user needs at most one switch to use this in a sane way. No aliases
> or editing of the profile, no requirement to manuall call mhbuild (or
> anything else) and no convenience shell file with aliases.

. . .

> There where already two ways mentioned in this thread how
> this could be fixed.

I missed them, and just went and searched and didn't find them.  Would
you summarize for me, please?

As far as using convertargs instead of your patch, doing this seems to
get close to the same result for me:

  1. add to my profile:
  mhbuild-convert-text/html: charset="%{charset}"; mhl -nomoreproc
-noclear -width 99 %F | mhshow -file - -concat | /bin/w3m -dump
${charset:+-I} ${charset:+"$charset"} -T text/html | sed 's/^\(.\)/>
\1/; s/^$/>/;' | par 64
  2. rtm [msg]

I would rather do that then add new code to nmh that doesn't seem
like a complete solution.

David

Re: automatic decode mime in repl

[Laura Creighton]

In a message of Fri, 11 Feb 2022 16:52:34 +0100, Philipp Takacs writes:
>[2022-02-11 02:55] David Levine 

>But to decode mime messages there is a feature in nmh which was there
>before convertargs: mhshow. All my patch does is call mhshow and pipe
>the mail through it. Nothing more and nothing less. By this way you
>get mhshow to decode mime and all of it's features.

I would really like to have this.  The workaround I have made is really ugly, 
and
doesn't always work.

>This patch doesn't handle URLs at all. mhshow will print them as is to
>stdout. After that mhl will add the '> ' and linebreaks. As said in the
>other mail, this is the default behavior of repl. This has nothing to do
>with my patch. There where already two ways mentioned in this thread how
>this could be fixed.

and I don't care about urls.

>Philipp

Laura Creighton

Re: automatic decode mime in repl

[Philipp Takacs]

[2022-02-11 02:55] David Levine 
> Philipp wrote:
>
> > Hi David
> >
> > I don't understand why do you try to convince me from convertargs.
>
> I'm not trying to convince you of anything.  I'm trying to understand
> how nmh could benefit from your patch.  And whether your approach could
> be improved by tying it to features that are already in repl (and that
> you weren't aware of when you wrote the patch).  And whether your patch
> interferes with existing repl features.  And what are both intended and
> unintended uses of the new feature.

Ok sorry then I missunderstand your request.

> To clear up some things:
[ ... ]
>
> * Your nmh installation is broken, as you observed.  I expect that the
>   cause was deleting par after nmh was installed.  The test suite did
>   the right thing by alerting you to the deficiency.  We could guide
>   you through (easy) repair of your installation, but I would instead
>   suggest a complete rebuild, if you built from sources, or re-install
>   if you obtained a pre-built package.

I think you miss a big point point, I don't have nmh. I have _m_mh.

All I did was clone the repo, build it and run ``make check''. The
hole time ``par'' wasn't installed. The build creates etc/mhn.defaults
with the wrong mhbuild-convert-text entry.

> And whether your approach could be improved by tying it to features
> that are already in repl (and that you weren't aware of when you wrote
> the patch).

As a _m_mh user I don't have the -convertargs switch. And I don't see
why I need it. It looks like the -convertargs switch and the convert
interface are only there to allow repl decoding mime.

But to decode mime messages there is a feature in nmh which was there
before convertargs: mhshow. All my patch does is call mhshow and pipe
the mail through it. Nothing more and nothing less. By this way you
get mhshow to decode mime and all of it's features.

convertargs is a diffrent aproatch, therefor I don't see a way to
improve my patch by using convertargs.

> And whether your patch interferes with existing repl features.

The -convertargs switch still works, because it is used with the filter
mhl.replywithoutbody. With the -mime switch you wont even hit the
coresponding code path. I don't see any possible interferes with other
repl features.

With other workarounds there may some problems. I would asume most will
just work, because they should also work on non mime mails.

> And what are both intended and unintended uses of the new feature.

Intendet is to make repl -fixmime be able to reply to a mime message.

Unintendet is that repl may start a videoplayer, if there is a video in
the message (or start some other graphical programms).

> I'm trying to understand how nmh could benefit from your patch.

Have an easy to use, enable by default and sane way to reply to mime
messages.

Under a easy to use and enable by default solution I understand something
the user needs at most one switch to use this in a sane way. No aliases
or editing of the profile, no requirement to manuall call mhbuild (or
anything else) and no convenience shell file with aliases.

> > The question is: Do you want it?
>
> Not with its current handling of URLs.  How do you handle URLs in
> messages that you reply to?

This patch doesn't handle URLs at all. mhshow will print them as is to
stdout. After that mhl will add the '> ' and linebreaks. As said in the
other mail, this is the default behavior of repl. This has nothing to do
with my patch. There where already two ways mentioned in this thread how
this could be fixed.

Philipp

Re: automatic decode mime in repl

[David Levine]

Philipp wrote:

> Hi David
>
> I don't understand why do you try to convince me from convertargs.

I'm not trying to convince you of anything.  I'm trying to understand
how nmh could benefit from your patch.  And whether your approach could
be improved by tying it to features that are already in repl (and that
you weren't aware of when you wrote the patch).  And whether your patch
interferes with existing repl features.  And what are both intended and
unintended uses of the new feature.

To clear up some things:
* replyaliases is not required to use convertargs.  The replyaliases
  shell functions can boil down to this:
repl -filter mhl.replywithoutbody -convertargs text/html ''
  That can be entered on the command line, of course.  And of course,
  it's much easier to put that in an alias or function.

* We could just as easily make that convertargs behavior the default,
  as we could make the behavior of your patch the default.  But as
  noted previously, there are good reasons for not doing that.

* Your nmh installation is broken, as you observed.  I expect that the
  cause was deleting par after nmh was installed.  The test suite did
  the right thing by alerting you to the deficiency.  We could guide
  you through (easy) repair of your installation, but I would instead
  suggest a complete rebuild, if you built from sources, or re-install
  if you obtained a pre-built package.

> How can I use it from other things then a Bourne-compatible shell?

The same way all nmh commands are used from other than a Bourne-
compatible shell.

> As far as I see you
> would need to port contrib/replaliases to all these interfaces

There's no need.  replaliases contains Bourne-shell functions for
convenience, that's all.

Alternatively, a user can add those switches to their profile.  Or,
a user can rely on whatever method they'd like to add arguments to a
command line.  That's one of the beauties of MH/nmh.

> So in genaral convertargs looks like a stopgap solution. I wanted to
> have a solution and I have this solution since 2016.

I've been using convertargs since 2014.

> The question is: Do you want it?

Not with its current handling of URLs.  How do you handle URLs in
messages that you reply to?

David

Re: automatic decode mime in repl

[Philipp Takacs]

Hi David

I don't understand why do you try to convince me from convertargs. As
said before: I have a working solution. Also convertargs will not work
in my setup. No I don't say convertargs is bad, I say my aproatch is
better. It has it own drawbacks, but in comparson they are smaller and
better to fix or workaround. Maybe I'm biased on this, but thats my view
on this. But lets look at it a bit deeper:

[2022-02-09 22:34] David Levine 
> Philipp wrote:
>
> > The problem I see is that convertargs looks complex.
>
> If you use a Bourne-compatible shell, would you try this, please:
>
> source $(mhparam docdir)/contrib/replaliases
> rtm [msg]

This is not that easy, because I use mmh. So convertargs is not there.
But I can give you my experience I have got from the nmh tests. The
test repl/test-convert fails with following error:

> /bin/sh: 1: par: not found
i> charset=; iconv -f ${charset:-us-ascii} -t utf-8 
'/home/satanist/src/nmh/test/testdir/Mail/mhbuildm4m2Df' | sed 's/^\(.\)/> \1/; 
s/^$/>/;' | par 64  >/home/satanist/src/nmh/test/testdir/Mail/mhbuildwpUMcf 
"$@": exited 127
> mhbuild: store of text/plain content failed, continuing...
> /bin/sh: 1: par: not found
> charset=; iconv -f ${charset:-us-ascii} -t utf-8 
> '/home/satanist/src/nmh/test/testdir/Mail/mhbuildgcLmQf' | sed 's/^\(.\)/> 
> \1/; s/^$/>/;' | par 64  
> >/home/satanist/src/nmh/test/testdir/Mail/mhbuildXaOu6d "$@": exited 127
> mhbuild: store of text/plain content failed, continuing...

This is probaly a bug in etc/mhn.defaults.sh and install par fixed it.
But this is a bad first experience.

> > The problem I see is that convertargs looks complex. The goal of my
> > patch is to have an easy to use solution which is enabled by default.
>
> Easy for developers or easy for users?

First for the user, but also for the developers. For a user I see a few
problems:

What needs to be done that this is enabled by default? Yes I poke on this
point, because I belive this is importend. Wheter you like it or not:
MIME messages are common and nmh is not able to reply to one in the
default configuration.

How do I create alternativ repl configurations? Like have a `replwork'
which has diffrent default switches and forms. I just see the solution
to write an alias. But I like the argv0 configuration methode and would
miss it.

How can I use it from other things then a Bourne-compatible shell? Like
fish shell, powershell, ipython or a java process. As far as I see you
would need to port contrib/replaliases to all these interfaces or use a
sh instand of calling repl directly.

As a developer I see also a few problems:

For me both arpoces are code changes[0]. convertargs changes two
programms and needs a wrapper which provides an easy interface. My
aproatch changes only repl and depends with the rest on already
implemented features.

The required wrapper also needs some maintainership. Fixing bugs,
changed interface, interfaces for other shells. Yes the wrapper is
small, but it moves parts of a core feature to a contributer script.

So in genaral convertargs looks like a stopgap solution. I wanted to
have a solution and I have this solution since 2016. Yes it's not the
best solution, but a good one. The question is: Do you want it?

Philipp

[0] Or at least my arpoatch was 2016 when I intruduced it to mmh.

Re: automatic decode mime in repl

[Robert Elz]

Date:Wed, 9 Feb 2022 22:34:07 +
From:David Levine 
Message-ID:  


  | If you use a Bourne-compatible shell, would you try this, please:
  |
  | source $(mhparam docdir)/contrib/replaliases

That means bash - or perhaps one or two others, for general
shell compat, use . instead of source:

. $(mhparam docdir)/contrib/replaliases

  | Where [msg] is the usual optional message argument to repl.  All
  | other common repl arguments are supported as well because rtm is an
  | alias that eventually relies on repl.

Really a function, not an alias, which is a much better idea.

kre

Re: automatic decode mime in repl

[David Levine]

Philipp wrote:

> The problem I see is that convertargs looks complex.

If you use a Bourne-compatible shell, would you try this, please:

source $(mhparam docdir)/contrib/replaliases
rtm [msg]

Where [msg] is the usual optional message argument to repl.  All
other common repl arguments are supported as well because rtm is an
alias that eventually relies on repl.

(If you want to easily dispose of the aliases after the above test,
do it in a new shell and simply exit the shell when done.)

> The problem I see is that convertargs looks complex. The goal of my
> patch is to have an easy to use solution which is enabled by default.

Easy for developers or easy for users?

David

Re: automatic decode mime in repl

[Philipp Takacs]

[2022-02-08 18:51] David Levine 
> Philipp wrote:
>
> > Thanks for all the discoussion. I have an updated version of this patch.
>
> I tried the patch.  It works, but it produces text content of fixed
> line lengths.  So it can break URLs and other text that shouldn't be
> arbitrarily modified.

This is true, but not introduced by my patch. The mail is still piped
through mhl which creates the fixed line length. As far as I see to
change this there are some smaller changes in mhl necesary.

> I haven't deciphered how it works, but I would think the same result
> could be obtained using repl's -convertargs.  That relies on the
> mhbuild(1) Convert Interface.  That was intended to be flexible, at
> the expense of simplicity.  Could it be used instead of adding new
> code?

Till your mail I didn't know convertargs, so I can't say if its possible.

The problem I see is that convertargs looks complex. The goal of my
patch is to have an easy to use solution which is enabled by default.
The patch automaticly decodes the configured mime types. For common
mime types the defaults work. To add extra mime types the already known
config for mhshow is used, so there is no new interface to learn.

Philipp

Re: automatic decode mime in repl

[David Levine]

Philipp wrote:

> Thanks for all the discoussion. I have an updated version of this patch.

I tried the patch.  It works, but it produces text content of fixed
line lengths.  So it can break URLs and other text that shouldn't be
arbitrarily modified.

I haven't deciphered how it works, but I would think the same result
could be obtained using repl's -convertargs.  That relies on the
mhbuild(1) Convert Interface.  That was intended to be flexible, at
the expense of simplicity.  Could it be used instead of adding new
code?

David

Re: automatic decode mime in repl

[Philipp Takacs]

Hi

[2022-02-05 16:53] Philipp Takacs 
> Reply to mime messages is an old problem. I have an idea to workaround
> or fix this. Instand of adding mime to mhl, repl could also first decode
> mime before pipe the mail to mhl. This can be done by mhshow.
>
> I have attached a patch for this.

Thanks for all the discoussion. I have an updated version of this patch.
I belive it's clear that you want a switch for this so I have implemented
it. Currently on by default. Also I have an other patch which implements
Ralphs suggestion.

I'm not sure if the manpage updates are good. Also the tests should be
updated. I'm also look for a way to autodetect if -nofixmime should be
set. I think the best place would be in nmh_init().

As a disclousure: I work with this patch since 2016. If you implement it
or not will not affect me. I thought this was clear from the beginning,
but it looks for me that this wasn't clear for everyone in the discussion.
Sorry for that. I belive this patch is good and you will benefit from it.
Thats is why I have send it to you and discoused why I belive this should
be enable by default. But in the end you can decide if and in which way
you want this patch.

Philipp
From a587df65eafb93bea5cc74053823fd07bc13ea8b Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Sat, 5 Feb 2022 12:21:35 +0100
Subject: [PATCH 1/2] repl: pipe mail through mhshow

When reply to a MIME message the mhshow will decode the mime message
before mhl will get it to create a reply draft. This makes it more
convinient to reply to a MIME message.

Problems:

* mhshow can be configured to display parts in a graphical programm. This
  can lead to unexpected behavior on repl (i.e. start plaing musik).

* mhshow display string can contain '%l' which leads to a listing prior
  to displaying content. This listing is contained in the draft.
---
 man/repl.man  |  22 
 uip/repl.c|  11 +++-
 uip/replsbr.c | 137 +-
 uip/replsbr.h |   2 +-
 4 files changed, 134 insertions(+), 38 deletions(-)

diff --git a/man/repl.man b/man/repl.man
index 790e19b5..fc1935fc 100644
--- a/man/repl.man
+++ b/man/repl.man
@@ -51,6 +51,7 @@ all/to/cc/me]
 .RB [ \-build ]
 .RB [ \-file
 .IR msgfile ]
+.RB [ \-fixmime " | " \-nofixmime ]
 .ad
 .SH DESCRIPTION
 .B repl
@@ -71,6 +72,14 @@ format file (see
 .IR mh\-format (5)
 for details).
 .PP
+To decode MIME messages
+.B repl
+pipe the message through
+.IR showmimeproc .
+This can be disabled with the
+.B \-nofixmime
+switch.
+.PP
 If the switch
 .B \-nogroup
 is given (it is on by default), then
@@ -516,6 +525,7 @@ is checked.
 ^Editor:~^To override the default editor
 ^Msg\-Protect:~^To set mode when creating a new message (draft)
 ^fileproc:~^Program to refile the message
+^showmimeproc:~^Program to show non-text (MIME) messages
 ^mhlproc:~^Program to filter message being replied-to
 ^whatnowproc:~^Program to ask the \*(lqWhat now?\*(rq questions
 .fi
@@ -542,6 +552,7 @@ is checked.
 .RB ` \-noquery '
 .RB ` \-noatfile '
 .RB ` "\-width\ 72" '
+.RB ` \-fixmime '
 .fi
 .SH CONTEXT
 If a folder is given, it will become the current folder.  The message
@@ -578,3 +589,14 @@ don't call it
 since
 .B repl
 won't run it.
+.PP
+The
+.RB \-fixmime
+switch causes repl to pipe the message through
+.IR showmimeproc .
+If the
+.IR showmimeproc
+is configured to display listing prior a mime part, these listing will end up in the draft.
+There might also be some unexpected behaviors, when the
+.IR showmimeproc
+is configured to display the content externel (i.e. open a browser to display html).
diff --git a/uip/repl.c b/uip/repl.c
index b1ceb1f6..8ed22709 100644
--- a/uip/repl.c
+++ b/uip/repl.c
@@ -72,6 +72,8 @@
 X("noatfile", 0, NOATFILESW) \
 X("fmtproc program", 0, FMTPROCSW) \
 X("nofmtproc", 0, NFMTPROCSW) \
+X("fixmime", 0, FIXMIMESW) \
+X("nofixmime", 0, NFIXMIMESW) \
 
 #define X(sw, minchars, id) id,
 DEFINE_SWITCH_ENUM(REPL);
@@ -143,6 +145,7 @@ main (int argc, char **argv)
 bool nedit = false;
 bool nwhat = false;
 bool atfile = false;
+bool fixmime = true;
 int fmtproc = -1;
 char *cp, *cwd, *dp, *maildir, *file = NULL;
 char *folder = NULL, *msg = NULL, *dfolder = NULL;
@@ -354,6 +357,12 @@ main (int argc, char **argv)
 		case NFMTPROCSW:
 		fmtproc = 0;
 		continue;
+		case FIXMIMESW:
+		fixmime = true;
+		continue;
+		case NFIXMIMESW:
+		fixmime = false;
+		continue;
 	}
 	}
 	if (*cp == '+' || *cp == '@') {
@@ -469,7 +478,7 @@ try_it_again:
 }
 
 replout (in, msg, drft, mp, outputlinelen, mime, form, filter,
-	 fcc, fmtproc);
+	 fcc, fmtproc, fixmime);
 fclose (in);
 
 {
diff --git a/uip/replsbr.c b/uip/replsbr.c
index ef4925ae..d2bab72f 100644
--- a/uip/replsbr.c
+++ b/uip/replsbr.c
@@ -61,7 +61,7 @@ static char *addrcomps[] = {
  * static prototypes
  */
 static int insert (struct mailname *);
-static void replfilter (FILE *, FILE *, char *, 

Re: automatic decode mime in repl

[Philipp Takacs]

Hi

[2022-02-07 16:54] Ralph Corderoy 
> > Instand of only depending on the programm name the config can be
> > extended to also depend on the calling programm. So you can config
> > diffrent options depending on which nmh programm called another nmh
> > programm. This could be done by (ab)using argv[0].
>
> Adding yet another style of interface between nmh's parts, which would
> have to be documented and users would have to understand and use, seems
> wrong.  What's your reason against running mhshow as ‘mhreplyfmt’, or
> something?  Is it the need to duplicate all MIME-configuration entries
> from ‘mhshow-*’ to ‘mhreplyfmt-*’?

The duplication is the one thing. The other thing is I would like to
extend the existing configuration system. Yes you can run mhshow as
`mhrelyfmt', but there is no logic behind the choice of argv[0]. The
idea was to define such a logic and use it to make the config more
powerfull.

As an exaple it could be possible to have diffrent showmimeproc config
for a `replA' and a `replB'. Or what about a diffrent sendmail or even
another path?

Yes this is a big change and way beound this patch. This might be as far
in the future as implementing MIME for mhl. Start with smaller fix like
setting argv[0] to `mhreplyfmt' is perfect fine.

> > * mhshow display string can contain '%l' which leads to a listing
> > prior to displaying content. This listing is contained in the draft.
>
> Again, the mhreplyfmt-... could choose whether to use it.  I think
> I would sometimes want it as part of the reply text, e.g. it's a
> one-line summary of a PDF and I want to quote that line for context
> about my following chunk of reply.

I have never thought[0] about solving this in this way. Yes this will
also work. In general I'm not a big fan of the '%l' and would prefer
one switch instand of '%l', but this is another discussion.

Philipp

[0] Mostly because of small diffrences between mmh and nmh.

Re: automatic decode mime in repl

[Ralph Corderoy]

Hi Philipp,

> Instand of only depending on the programm name the config can be
> extended to also depend on the calling programm. So you can config
> diffrent options depending on which nmh programm called another nmh
> programm. This could be done by (ab)using argv[0].

Adding yet another style of interface between nmh's parts, which would
have to be documented and users would have to understand and use, seems
wrong.  What's your reason against running mhshow as ‘mhreplyfmt’, or
something?  Is it the need to duplicate all MIME-configuration entries
from ‘mhshow-*’ to ‘mhreplyfmt-*’?

> * mhshow display string can contain '%l' which leads to a listing
> prior to displaying content. This listing is contained in the draft.

Again, the mhreplyfmt-... could choose whether to use it.  I think
I would sometimes want it as part of the reply text, e.g. it's a
one-line summary of a PDF and I want to quote that line for context
about my following chunk of reply.

-- 
Cheers, Ralph.

Re: automatic decode mime in repl

[Ralph Corderoy]

Hi Ken,

> Also, just thinking out loud; should the default maybe be
> "mhshow | sed -e 's/^/>/'"?

Probably more like

/./s/^/ /
s/^/>/

so you get output like

$ seq 10 | sed '2~3s/.//' | sed '/./s/^/ /; s/^/>/' | sed -n l
> 1$
>$
> 3$
> 4$
>$
> 6$
> 7$
>$
> 9$
> 10$
$

-- 
Cheers, Ralph.

Re: automatic decode mime in repl

[David Levine]

Philipp wrote:

> I don't see a reason for an option[1], because I consider the
> current behavior as a bug. There shouldn't be an option to enable/disable
> a bug.

That's a fair point, but I'd be more reluctant to change default
behavior in case users depend on it.

Have you tried nmh/share/doc/nmh/contrib/replaliases?  (Or whereever
it's installed on your system.)  It think it could be configured to
do what your patch does.  In any case, it works as-is with recent
nmh.  Of course, it has it's own drawbacks, primarily with the user
editing quoted-printable.  That should be solvable though I haven't
been motivatee enough to try.

David

Re: automatic decode mime in repl

[Ken Hornstein]

>I can understand that a changed behavior is a problem. But in this case
>I would argue there is no other way. Every patch witch tries to fix this
>will change the existing behavior, because the existing behavior is the
>problem.

Right, I get that.  But as Paul Fox points out, plenty of people have
come up with workarounds for this behavior and I'm reluctant to break
that.  The eternal problem, sigh.

>So I would prefer no option. But if you want one, I can add an option.
>I would prefer the new behavior be the default. Also I would prefer a
>build time option over a runtime option. I'm bad at naming, has someone
>an idea for the name of such an option.

Oof, well, if we've learned ONE thing after all of these years, it's
that build time options are terrible, TERRIBLE ideas.  The big reason
is that most users use a nmh install from a distribution, so whatever
configuration they end up is based on the person who created the
distribution build.  I think build time options are fine for things
that require external libraries (e.g., TLS or Cyrus-SASL support)
but to change user behavior I think this is definitely sub-optimal.

So in general, I think the options are:

1) Make the new behavior the default, and require that anyone who wants
   the old behavior add a switch (that will will allow existing setups
   to work but it will require that people who have those setups to
   change something)

2) Make new behavior optional with a switch

3) Make new behavior the default but try to auto-detect if a user has
   a customized reply configuration.

I understand where you're coming from in that the current behavior is a bug
and the default should just work; I agree!  So to me that completely
eliminates #2 as an option.  Is #3 feasable?  Not sure; I'll have to think
about it.

Also, just thinking out loud; should the default maybe be
"mhshow | sed -e 's/^/>/'"?

--Ken

Re: automatic decode mime in repl

[Philipp Takacs]

[2022-02-06 08:26] Paul Fox 
> philipp wrote:
>  > [2022-02-05 22:19] Ken Hornstein 
>  > >
>  > > My only concern with this patch is that it unilaterally overrides the
>  > > existing behavior, and one thing I've seen over the years is that there
>  > > are a LOT of users wedded to existing behavior.  So I'd rather make it
>  > > selectable (I am neutral about it being default or not).  I'd welcome
>  > > discussion on this issue.
>  > 
>  > I can understand that a changed behavior is a problem. But in this case
>  > I would argue there is no other way. Every patch witch tries to fix this
>  > will change the existing behavior, because the existing behavior is the
>  > problem.
>  > 
>  > So I would prefer no option. But if you want one, I can add an option.
>  > I would prefer the new behavior be the default. Also I would prefer a
>  > build time option over a runtime option. I'm bad at naming, has someone
>  > an idea for the name of such an option.
>
> I haven't tried the patch, but I think any change to basic repl
> behavior should a) be optional, b) be controlled by a runtime option. 
> And it should probably have a lot of testing by a lot of users before
> being made the default.
>
> I can't be alone in having customized scripts for doing replies which
> already run mhshow in crafting the reply.  Perhaps everything would
> continue to just work, or perhaps not.

I understand your reasaning. I just fear then this will get just an other
stopgap solution. I would assume most users with a workaround for the
problem will try this once. If it don't work with there workaround, they
will just disable it. If someone brings up the idea of change the default
they would vote against it. I don't want to blame such behavior. This
makes perfect sense from the standpoint of a user with a workaround. It
just don't make nmh better.

I belive reply to a mime message should just work. The best solution
would be implementing mime support in mhl. But this will not happen
for a long time. So the goal should be add a solution[0] and make it the
default. I don't see a reason for an option[1], because I consider the
current behavior as a bug. There shouldn't be an option to enable/disable
a bug.

As said in my previous mail, I'm not complete against an option. I would
just prefere not having one.

Philipp

[0] I don't care if this is my solution or any other. I just don't see
another solution.

[1] You still can change the showmimeproc to disable this.

Re: automatic decode mime in repl

[Paul Fox]

philipp wrote:
 > [2022-02-05 22:19] Ken Hornstein 
 > >
 > > My only concern with this patch is that it unilaterally overrides the
 > > existing behavior, and one thing I've seen over the years is that there
 > > are a LOT of users wedded to existing behavior.  So I'd rather make it
 > > selectable (I am neutral about it being default or not).  I'd welcome
 > > discussion on this issue.
 > 
 > I can understand that a changed behavior is a problem. But in this case
 > I would argue there is no other way. Every patch witch tries to fix this
 > will change the existing behavior, because the existing behavior is the
 > problem.
 > 
 > So I would prefer no option. But if you want one, I can add an option.
 > I would prefer the new behavior be the default. Also I would prefer a
 > build time option over a runtime option. I'm bad at naming, has someone
 > an idea for the name of such an option.

I haven't tried the patch, but I think any change to basic repl
behavior should a) be optional, b) be controlled by a runtime option. 
And it should probably have a lot of testing by a lot of users before
being made the default.

I can't be alone in having customized scripts for doing replies which
already run mhshow in crafting the reply.  Perhaps everything would
continue to just work, or perhaps not.

paul
=--
paul fox, p...@foxharp.boston.ma.us (arlington, ma)

Re: automatic decode mime in repl

[Philipp Takacs]

[2022-02-05 22:19] Ken Hornstein 
> >Reply to mime messages is an old problem. I have an idea to workaround
> >or fix this. Instand of adding mime to mhl, repl could also first decode
> >mime before pipe the mail to mhl. This can be done by mhshow.
> >
> >I have attached a patch for this.
>
> From a purely architectural perspective, I think that having mhl be
> MIME-aware is the correct solution.  But I must acknowledge the reality
> on the ground:
>
> - This has been a problem for a very long time
>
> - Stopgap solutions (like replyfilter) have not made much headway, because
>   they require end-user configuration.
>
> - Getting a MIME-aware mhl (really, all of nmh) is a huge job.
>
> - I believe "perfect is the enemy of the good".
>
> My only concern with this patch is that it unilaterally overrides the
> existing behavior, and one thing I've seen over the years is that there
> are a LOT of users wedded to existing behavior.  So I'd rather make it
> selectable (I am neutral about it being default or not).  I'd welcome
> discussion on this issue.

I can understand that a changed behavior is a problem. But in this case
I would argue there is no other way. Every patch witch tries to fix this
will change the existing behavior, because the existing behavior is the
problem.

So I would prefer no option. But if you want one, I can add an option.
I would prefer the new behavior be the default. Also I would prefer a
build time option over a runtime option. I'm bad at naming, has someone
an idea for the name of such an option.

An alternativ would be to depend on fmtproc. When fmtproc is set don't
pipe through mhshow. When it is not set pipe through mhshow.

Philipp

> --Ken
>
> >This approatch has currently two problems:
> >
> >* mhshow can be configured to display parts in a graphical programm. This
> >  can lead to unexpected behavior on repl (i.e. start plaing musik).
> >
> >I belive there are two options to fixed this problem. First the -textonly
> >swtich can be used. In some probably uncommon setups (i.e. text to speech)
> >this still can lead to problems.
> >
> >The secound option is a bit more complex. Instand of only depending on
> >the programm name the config can be extended to also depend on the
> >calling programm. So you can config diffrent options depending on which
> >nmh programm called another nmh programm. This could be done by (ab)using
> >argv[0].
> >
> >* mhshow display string can contain '%l' which leads to a listing prior
> >  to displaying content. This listing is contained in the draft.
> >
> >To fix this I would suggest to remove the '%l' from the display string
> >escapes and create a switch for that. Then you can't enable/disable this
> >per part.
> >
> >What do you think about this patch? Did I missed some problems?
> >
> >Philipp
> >
> >>> text/x-diff content

Re: automatic decode mime in repl

[Ken Hornstein]

>Reply to mime messages is an old problem. I have an idea to workaround
>or fix this. Instand of adding mime to mhl, repl could also first decode
>mime before pipe the mail to mhl. This can be done by mhshow.
>
>I have attached a patch for this.

>From a purely architectural perspective, I think that having mhl be
MIME-aware is the correct solution.  But I must acknowledge the reality
on the ground:

- This has been a problem for a very long time

- Stopgap solutions (like replyfilter) have not made much headway, because
  they require end-user configuration.

- Getting a MIME-aware mhl (really, all of nmh) is a huge job.

- I believe "perfect is the enemy of the good".

My only concern with this patch is that it unilaterally overrides the
existing behavior, and one thing I've seen over the years is that there
are a LOT of users wedded to existing behavior.  So I'd rather make it
selectable (I am neutral about it being default or not).  I'd welcome
discussion on this issue.

--Ken

>This approatch has currently two problems:
>
>* mhshow can be configured to display parts in a graphical programm. This
>  can lead to unexpected behavior on repl (i.e. start plaing musik).
>
>I belive there are two options to fixed this problem. First the -textonly
>swtich can be used. In some probably uncommon setups (i.e. text to speech)
>this still can lead to problems.
>
>The secound option is a bit more complex. Instand of only depending on
>the programm name the config can be extended to also depend on the
>calling programm. So you can config diffrent options depending on which
>nmh programm called another nmh programm. This could be done by (ab)using
>argv[0].
>
>* mhshow display string can contain '%l' which leads to a listing prior
>  to displaying content. This listing is contained in the draft.
>
>To fix this I would suggest to remove the '%l' from the display string
>escapes and create a switch for that. Then you can't enable/disable this
>per part.
>
>What do you think about this patch? Did I missed some problems?
>
>Philipp
>
>>> text/x-diff content

automatic decode mime in repl

[Philipp Takacs]

Hi

Reply to mime messages is an old problem. I have an idea to workaround
or fix this. Instand of adding mime to mhl, repl could also first decode
mime before pipe the mail to mhl. This can be done by mhshow.

I have attached a patch for this.

This approatch has currently two problems:

* mhshow can be configured to display parts in a graphical programm. This
  can lead to unexpected behavior on repl (i.e. start plaing musik).

I belive there are two options to fixed this problem. First the -textonly
swtich can be used. In some probably uncommon setups (i.e. text to speech)
this still can lead to problems.

The secound option is a bit more complex. Instand of only depending on
the programm name the config can be extended to also depend on the
calling programm. So you can config diffrent options depending on which
nmh programm called another nmh programm. This could be done by (ab)using
argv[0].

* mhshow display string can contain '%l' which leads to a listing prior
  to displaying content. This listing is contained in the draft.

To fix this I would suggest to remove the '%l' from the display string
escapes and create a switch for that. Then you can't enable/disable this
per part.

What do you think about this patch? Did I missed some problems?

Philipp
From 34d2763d73edf12e4b4cfb45e063e1ffa19519c3 Mon Sep 17 00:00:00 2001
From: Philipp Takacs 
Date: Sat, 5 Feb 2022 12:21:35 +0100
Subject: [PATCH] repl: pipe mail through mhshow

When reply to a MIME message the mhshow will decode the mime message
before mhl will get it to create a reply draft. This makes it more
convinient to reply to a MIME message.

Problems:

* mhshow can be configured to display parts in a graphical programm. This
  can lead to unexpected behavior on repl (i.e. start plaing musik).

* mhshow display string can contain '%l' which leads to a listing prior
  to displaying content. This listing is contained in the draft.
---
 uip/replsbr.c | 106 --
 1 file changed, 76 insertions(+), 30 deletions(-)

diff --git a/uip/replsbr.c b/uip/replsbr.c
index ef4925ae..2dce085e 100644
--- a/uip/replsbr.c
+++ b/uip/replsbr.c
@@ -436,6 +436,10 @@ replfilter (FILE *in, FILE *out, char *filter, int fmtproc)
 char *errstr;
 char **arglist;
 int argnum;
+int mailpipe[2];
+char *mhshow;
+char **mhshowargs;
+int mshowargnum;
 
 if (filter == NULL)
 	return;
@@ -447,48 +451,90 @@ replfilter (FILE *in, FILE *out, char *filter, int fmtproc)
 lseek(fileno(in), 0, SEEK_SET);
 
 arglist = argsplit(mhlproc, , );
+mhshowargs = argsplit(showmimeproc, , );
 
 switch (pid = fork()) {
 	case NOTOK:
 	adios ("fork", "unable to");
 
 	case OK:
-	dup2 (fileno (in), fileno (stdin));
-	dup2 (fileno (out), fileno (stdout));
-
-	/*
-	 * We're not allocating the memory for the extra arguments,
-	 * because we never call arglist_free().  But if we ever change
-	 * that be sure to use getcpy() for the extra arguments.
-	 */
-	arglist[argnum++] = "-form";
-	arglist[argnum++] = filter;
-	arglist[argnum++] = "-noclear";
-
-	switch (fmtproc) {
-	case 1:
-		arglist[argnum++] = "-fmtproc";
-		arglist[argnum++] = formatproc;
-		break;
-	case 0:
-		arglist[argnum++] = "-nofmtproc";
-		break;
+
+	if (pipe(mailpipe) == -1) {
+		adios("pipe", "can't create pipe");
 	}
 
-	arglist[argnum++] = NULL;
+	switch (pid = fork()) {
+		case NOTOK:
+		adios ("fork", "unable to");
+
+		case OK:
+		dup2 (fileno (in), fileno (stdin));
+		dup2 (mailpipe[1], fileno (stdout));
+		close(fileno(in));
+		close(fileno(out));
+		close(mailpipe[0]);
+		close(mailpipe[1]);
+
+		mhshowargs[argnum++] = "-file";
+		mhshowargs[argnum++] = "-";
+		mhshowargs[argnum++] = "-concat";
+		mhshowargs[argnum++] = NULL;
+
+		execvp (mhshow, mhshowargs);
+		errstr = strerror(errno);
+		if (write(2, "unable to exec ", 15) < 0  ||
+			write(2, mhshow, strlen(mhshow)) < 0  ||
+			write(2, ": ", 2) < 0  ||
+			write(2, errstr, strlen(errstr)) < 0  ||
+			write(2, "\n", 1) < 0) {
+			advise ("stderr", "write");
+		}
+		_exit(1);
+		default:
+		dup2 (mailpipe[0], fileno (stdin));
+		dup2 (fileno (out), fileno (stdout));
+		close(fileno(in));
+		close(fileno(out));
+		close(mailpipe[0]);
+		close(mailpipe[1]);
+
+		/*
+		 * We're not allocating the memory for the extra arguments,
+		 * because we never call arglist_free().  But if we ever change
+		 * that be sure to use getcpy() for the extra arguments.
+		 */
+		arglist[argnum++] = "-nomoreproc";
+		arglist[argnum++] = "-form";
+		arglist[argnum++] = filter;
+		arglist[argnum++] = "-noclear";
+
+		switch (fmtproc) {
+			case 1:
+			arglist[argnum++] = "-fmtproc";
+			arglist[argnum++] = formatproc;
+			break;
+			case 0:
+			arglist[argnum++] = "-nofmtproc";
+			break;