[Nmh-workers] par
Joel wrote: > I'd be happy to have par as part of nmh, especially since there's no > Fedora package for par right now. (Or, alternatively, I'd be happy for > someone to package par for Fedora.) The par build hints contain a link to a spec file by Volker Kuhlmann. It works very nicely on Fedora. There's also a patch to add support for multibyte charsets. The attached script downloads all that and builds an RPM. Takes just seconds. The par codebase hasn't evolved since 2001, not a typo! The multibyte char patch has been living by itself since Oct 2006. If we decide to rely on it by default, I'll try to contact the author to see if we can make this a little easier. David if [ -f /usr/bin/par ]; then echo 'You already have par. Is it from par2cmdline?'; exit 1 fi [ -x /usr/bin/rpmbuild ] || sudo yum install rpm-build mkdir -p ~/lib/rpmbuild/SOURCES ~/lib/rpmbuild/SPECS cd ~/lib/rpmbuild/SOURCES wget http://www.nicemice.net/par/Par152.tar.gz wget http://www.nicemice.net/par/par_1.52-i18n.3.diff.gz cd ../SPECS wget http://www.nicemice.net/par/par-1.52-1.spec.gz gzip -d par-1.52-1.spec.gz patch -b <___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] The curse of m_getfld()
Wow. What's the coder equivalent of the Purple Heart? Extreme bravery in the case of old crusty code. Rescuing their comrades from vestiges of the VAX. While I'm too busy to participate in the current effort, m_getfld.c has been the thing that's kept me from implementing the attachment changes that I talked about years ago. So it would be really nice to have it do what I want if it's rewritten. For those of you who are new or old and have forgotten, I'd like to add options to nmh where the parts of multi-part messages are treated like sub-messages. I'd like scan to be able to show message 123, 123.1 123.2, ... if message 123 has multiple parts with something sensible in the scan line. I'd like to have -part options to show/next/prev to show the current/next/previous part and to keep track of the current part. Also an option to move on to the next/previous message. I've forgotten what little I knew about getfld so I don't know what to suggest to make this work. Probably the optional ability to scan the entire message to pull out parts. Note that if this all ever gets done it should eliminate mhlist and mhshow. Anyway, please go for it. I should free up in a few months and would be glad to build on the result. Jon ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] The curse of m_getfld()
>i've just looked at m_getfld.c, for the first time since 1994 or so. >this was good code in the pdp11 era. >[...] Wow. Okay, that was actually pretty damn useful; thanks for the tutorial, history lesson, and brief primer on VAX assembly! I hope it wasn't too painful for you! :-) Okay, you've convinced me. I've also got my finger on the "fire photon torpedoes" button. Does anyone want to speak up for m_getfld.c before we decide to relegate it to the dustbin of history? >my sketch would look a lot like "man 3 db" so please start there. Okay, I think I have the basic idea. I think this all sounds great. Does anyone have any objections to this? Paul, would you be willing to cook up some basic function prototypes and brief documentation to at least get things started? That would give people a chance to look it over and comment on the API. I can commit to doing some of the implementation work; I don't know if I can do it all, that's depends on a bunch of things. Perhaps it's a bit early, but getting into the implementation details ... obviously this is going to require a new message parser. I think I'm way too old to build my own. Valdis made the case here: http://lists.gnu.org/archive/html/nmh-workers/2010-12/msg00017.html that a parser based on lex/yacc would make sense. I'm just talking about the headers ... I think the MIME format of the body is simple enough that you would want to do that yourself (in addition, since the MIME content separator is dynamic I don't think you could do it anyway with lex/flex). Do people think that's reasonable? The only thing that concerns me is that reasonable error handling with lex/yacc is tricky; with flex/bison it's a bit easier, but you still have to be careful. Note: I was thinking we should do one final release (call it 1.5, maybe?) or at least branch for a release right before all of this work starts. And yes, I am going to be am optimist and assume that it's GOING to happen, but be a realist and not pretend that it's going to happen anytime soon! :-) --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
On Thu, Jan 26, 2012 at 3:04 PM, Paul Vixie wrote: > On 1/26/2012 7:35 PM, Ken Hornstein wrote: >> ... So what I'm thinking is twofold: - An option to disable any kind >> of line breaking in mhl. - An option to filter mhl content through a >> program. I'm thinking maybe a flag like "format" and using a >> mh_profile entry called "formatproc". Sounds good? --Ken > > +1. Concur. An easy way to do it that doesn't involve me hacking a vi macro or the like would be my preferred solution. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
On 2012-01-26, at 12:06 PM, Jerrad Pierce wrote: > Quoth Lyndon: >> Par isn't part of nmh any more than 'less' is. > My point was that if there's an existing tool we recommend deferring the > heavy lifting to, particularly something that is not part of a standard > *nx kit, maybe it should be included in the distro. nmh is a scriptable application capable of working with an endless number of external programs. You want we should ship the entire Internet with it? :-) ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
On Thu, 26 Jan 2012 14:35:42 EST, Ken Hornstein said: > - An option to disable any kind of line breaking in mhl. > - An option to filter mhl content through a program. I'm thinking maybe > a flag like "format" and using a mh_profile entry called "formatproc". GIven those two, I think I've convinced myself that we can synthesize all the different needed behaviors. pgpvJFZOiMXeQ.pgp Description: PGP signature ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
Quoth Lyndon: >Par isn't part of nmh any more than 'less' is. My point was that if there's an existing tool we recommend deferring the heavy lifting to, particularly something that is not part of a standard *nx kit, maybe it should be included in the distro. Quoth Valdis: >given lproc and moreproc, isn't the MH way to do this something like >parproc? Probably fmtproc (formatproc) if fmt is standard , and then we can simply recommend par over fmt without actually bundling it... Might be worth mirroring though? ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
On 1/26/2012 7:35 PM, Ken Hornstein wrote: > ... So what I'm thinking is twofold: - An option to disable any kind > of line breaking in mhl. - An option to filter mhl content through a > program. I'm thinking maybe a flag like "format" and using a > mh_profile entry called "formatproc". Sounds good? --Ken +1. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] The curse of m_getfld()
On 1/26/2012 6:06 PM, Ken Hornstein wrote: >> my thought is, fire photon torpedoes. m_getfld was the wrong approach >> when it was new but it worked well enough (especially on slower older >> machines). we should not compromise on readability in order to keep the >> couple hundred places that call m_getfld from having to change. > By my count it's actually only 77 calls to m_getfld(). So it's not > insurmountable by any means. looks like i'm not off by even one order of magnitude: close enough for software work :-). > I was thinking that m_getfld() would actually have a couple arguments > added to it in terms of "here's the mime type of whatever data you > have". Internally there would be a lot of rototilling. imagine me backing toward the door, making the sign of the cross at you as i do so. > But okay, > since you've thought about this more than I have ... what is _your_ > vision of an API? i've just looked at m_getfld.c, for the first time since 1994 or so. this was good code in the pdp11 era. this is a stateful iterator which does character level processing from an underlying stdio FILE object. its caller must know internal details of the state machine. opaqueness is nowhere attempted. it digs into the underlying FILE object to effect multi-character "ungetc" which is not supported by POSIX stdio, and it also returns pointers into the underlying FILE object's buffer to avoid character copying, all with #ifdef's for LINUX_STDIO which presumably works differently. it tries hard to give the compiler hints to use the vax MATCH3 instruction for substring searching. its API and its implementation make UTF-8 impossible and by the time this thing has returned it is not possible for the caller to perform any I18N processing on fields like "subject" that can have same. its indentation has several off-by-one shifts. those of you who knew me in 1990 know that i used to write code like this; it was an art; m_getfld.c is high art. > If you'd be willing to sketch something out that sure would be great. my sketch would look a lot like "man 3 db" so please start there. the idea would be to call some function that returned a heap object that had some function pointers (for example, iterating through headers or mime components) and had some private data (like pointers to the underlying storage media, such as a POSIX "FILE" object. it would be high level enough that if someone wanted to reimplement it on top of an imap connection or a postgres connection then the callers would not have to know about it. the implementation would beat the holy crap out of the heap, returning pointerballs that were created with strdup and asprintf. the caller would be obligated to call some kind of "free" like function after consuming data from each iteration, and another "destroy" function for the message object itself. we would feel free to call "strstr", "strchr", "strcasecmp", etc. we would not use any "register" variables. once we had this for message objects we'd clean it up from the lessons learned and then use the same approach for folder objects and mailstore objects. folder objects are hard, since MH message numbers are more permanent than IMAP message numbers but less permanent than IMAP UID's. but the new higher-level "folder" object's interface would be defined with an eye toward supporting non-MH mail stores (such as IMAP or SQL). when complete, the only part of the MH code base that knew about opendir() or fopen() would be behind an API. note that others (bernstein for example, and horton) have attempted this. some user interfaces do include an abstract API to make them mail store independent. i observe that: none of these API's has taken the world by storm; none of them support the "middle level" of message number permanent required by (and loved by!) MH; none of them are even as modern as "db(3)" in their presentation of the object level interface. so, i think this field is still green for us. i'd be happy to go further with this if my vision is attractive to folks. i am not willing to do all of the world or even the lion's share of the work myself -- nor will i do anything more at all if the "core team" doesn't give a thumbs up. paul ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
>I'm more of the opinion that mhl *should* do some basic line wrapping, >but with the option to instead invoke an external app that might be >able to do a better job. I don't have a massively strong objection to >not doing so, but it seems wrong to be shipping mhl in essentially a >broken state. I know we've been doing so since forever, but I'd like >the out of the box behaviour to be good, even if a bit of customization >to call external utilities could make it better. Okay, I understand your point. I'm just thinking ... ew, that code is a mess, and I'm not crazy about making it worse. Specifically, the "line break" happens inside of uip/mhlsbr.c:putch(). The decision to make the line break happens on a per-character basis so reworking that all is going to be some digging. I think I'm going to have to invoke the "someone can gladly tackle that, as long as it's not me" rule. :-) Other people have asked about pulling par into nmh; that feels wrong to me. As Lyndon pointed out "more" isn't part of nmh either. Calling out to another program which already does this is more like the toolset philosophy that makes MH powerful. So what I'm thinking is twofold: - An option to disable any kind of line breaking in mhl. - An option to filter mhl content through a program. I'm thinking maybe a flag like "format" and using a mh_profile entry called "formatproc". Sounds good? --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
On 1/26/2012 6:43 PM, valdis.kletni...@vt.edu wrote: > Probably is correct to not reflow it by default, but you really need > to have a provision to allow it when requested. given lproc and moreproc, isn't the MH way to do this something like parproc? ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] The curse of m_getfld()
>> - Given the above ... do people think there is value in rolling another >> release soon-ish? > >Release early, release often. Okay ... but are people going to try those releases out and give feedback? Bill, I'm looking at you :-) --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
On Thu, 26 Jan 2012 10:22:26 PST, Bill Wohler said: > Definitely need to treat quoted text (> or some supercite format) as > pre-formatted text and not touch it. Debatable. I've had to reply to far too many e-mails that contained > some sort of line that went on for ever and ever off the right side of the > screen, because somebody was using a mail client that didn't tag it as > format=flowed, and the next person's MUA quoted the long line, leaving me > with a quoted long line to deal with. :) Probably is correct to not reflow it by default, but you really need to have a provision to allow it when requested. pgpqknaDKeACz.pgp Description: PGP signature ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] The curse of m_getfld()
Ken Hornstein writes: > - Given the above ... do people think there is value in rolling another > release soon-ish? Release early, release often. -- Bill Wohler aka http://www.newt.com/wohler/ GnuPG ID:610BD9AD ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
Ken Hornstein writes: > Fixing this requires some subtle surgery to mhl, which doesn't look > terrible. But it begs the question ... what's the right behavior? > Break at a space? If so, what should we do about the "next" line? > I confess to being a bit unsure. If we have format=flowed, yes, > the decision is easy (mhl doesn't know about that just yet, but it > could). But it seems right now that modern mail programs assume > that everybody is doing line wrapping and paragraph reflowing, and > I'm wondering if we should just get on board with that. Yes. That way your viewing experience can be as wide or as narrow as you want. We would definitely want to fold with format=flowed. However, I'm not so sure we want to fold if format=flowed is missing. In Emacs, Gnus provides "washing" commands to fix up long lines in messages that are missing the format=flowed attribute. Perhaps we add a -wash option to mhl so the user can use it on the fly without committing all emails to it? Agree with Paul that we don't want to be too fancy; anything that is better than now is good. Definitely need to treat quoted text (> or some supercite format) as pre-formatted text and not touch it. -- Bill Wohler aka http://www.newt.com/wohler/ GnuPG ID:610BD9AD ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] The curse of m_getfld()
>> One of the major wrinkles is ... well, m_getfld() is a complicated hot >> mess. I know some of the people here have been inside of it; if they >> wanted to impart some public knowledge here about it, I for one sure >> would appreciate it. > >my thought is, fire photon torpedoes. m_getfld was the wrong approach >when it was new but it worked well enough (especially on slower older >machines). we should not compromise on readability in order to keep the >couple hundred places that call m_getfld from having to change. By my count it's actually only 77 calls to m_getfld(). So it's not insurmountable by any means. I was thinking that m_getfld() would actually have a couple arguments added to it in terms of "here's the mime type of whatever data you have". Internally there would be a lot of rototilling. But okay, since you've thought about this more than I have ... what is _your_ vision of an API? If you'd be willing to sketch something out that sure would be great. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
On 2012-01-25, at 8:41 PM, Joel Uckelman wrote: > I'd be happy to have par as part of nmh, especially since there's no > Fedora package for par right now. (Or, alternatively, I'd be happy for > someone to package par for Fedora.) Par isn't part of nmh any more than 'less' is. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
Ken Hornstein writes: >People aren't so crazy about adding more text formatting smarts to mhl; >that makes sense to me, I think whatever code I add wouldn't be as good >as something like fmt or par, and that's keeping with the toolset idea >of nmh. I'm more of the opinion that mhl *should* do some basic line wrapping, but with the option to instead invoke an external app that might be able to do a better job. I don't have a massively strong objection to not doing so, but it seems wrong to be shipping mhl in essentially a broken state. I know we've been doing so since forever, but I'd like the out of the box behaviour to be good, even if a bit of customization to call external utilities could make it better. Tet ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] The curse of m_getfld()
On Wed, 25 Jan 2012 22:40:58 -0500 Ken Hornstein wrote: ... > Here's my thinking: the bulk of MIME parsing and translation _has_ > to happen in m_getfld(). So in the New World Order, m_getfld() > reads in a message off of disk, translates anything necessary into > UTF-8, does things like RFC-2047 header parsing, base-64 & > quoted-printable decoding, and returns UTF-8 strings to the calling > functions. i don't think you're cutting deep enough. nmh is simply not that big; we can afford an internal api change to get away from the m_getfld monolith. we don't need to add mime as a layer; we need to change the underlying data structures to be mime-cognizant, and make accessing them and iterating over them abstract operations. ... > One of the major wrinkles is ... well, m_getfld() is a complicated hot > mess. I know some of the people here have been inside of it; if they > wanted to impart some public knowledge here about it, I for one sure > would appreciate it. my thought is, fire photon torpedoes. m_getfld was the wrong approach when it was new but it worked well enough (especially on slower older machines). we should not compromise on readability in order to keep the couple hundred places that call m_getfld from having to change. ... > A few other things: > > - I know work on nmh tends to be bursty ... and in my case, that's > definitely true. I think I am going to have to work on other things > soon, and I don't know if I'll get a chance to get to the MIME work. > > - Given the above ... do people think there is value in rolling > another release soon-ish? We've got a number of a bug fixes, a new > build system, significant improvements to the format strings that let > people (in some cases) select headers based on message contents, and > if I get the repl work done then that means one of my > long-outstanding beefs about replies will be solved (not the biggest > one, sadly, but we can't have everything). yes, i think so, definitely. > - I know Paul Vixie was asking about putting most of nmh in a shared > library, and I think I've done 70% of that work with the Automake > migration. If someone wanted to take that over the finish line, I > think that would be great. A quick glance at the Libtool manual > suggests that it shouldn't be hard. i'd like to emphasize that whether the library is shared or not is the least of our worries. we need a sensible, abstract, modular, data hiding API. even if we go on statically linking everything. and there's no way to get that in the 1.x version stream. the work in 1.4, and since 1.4, deserves release. -- Paul Vixie ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] Line wrapping in mhl
Joel Uckelman writes: >Or, alternatively, I'd be happy for someone to package par for Fedora. Heh. You have no idea how long that one's been on my TODO list. One day I might actually have the spare time to get around to it... Tet ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
