Re: [Nmh-workers] repl and mime handling
ra...@hep.wisc.edu writes: The problem with that approach is that sometimes the text part just says ``There is no text part, use an HTML capable mail reader''. I'm seeing more of them these days. Really? I've been reading text/plain over text/html with EXMH for years and recently with my own mh front-end (mh-v) and this has never been an issue. Yes. I see many of these. Probably because I have exmh configured to show text/plain in preference to any of the other text/whatever that might be present. Tet ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
*Please, no!* Conversion from any charset to utf-8 is possible, but conversion back, according to user preferences, is not. People start to use funny characters like non-breakable space and so on. Unfortunately, we don't have unlimited development resources. Here's my reading of the world: - The general trend (especially in English-speaking countries) is to move toward Unicode (specifically, UTF-8). - People in Eastern Europe aren't crazy about this. - Conversion to Unicode is relatively simple. Conversion from Unicode to some other character set isn't, depending on the character set (although it would seem to me that converting from a non-breaking space to a regular space is a no-brainer, but whatever). - Given the lack of unlimited development resources, I don't really see people willing to change all of the internal APIs to include character set information. That means we pretty much have to choose one character set for an internal representation inside of nmh. - If we have to choose one, the only logical choice is Unicode (which in practice means UTF-8; maybe UTF-16, but a trip through the format code made me think that UTF-8 is probably the only real choice). --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Hi Ken, - Conversion to Unicode is relatively simple. Conversion from Unicode to some other character set isn't, depending on the character set (although it would seem to me that converting from a non-breaking space to a regular space is a no-brainer, but whatever). I assumed nmh would convert to UTF-8, I'd edit a UTF-8 draft, and that's what it would hand over to the MDA. http://en.wikipedia.org/wiki/Space_%28character%29#Spaces_in_Unicode (turn Javascript off to bypass blackout) lists various spaces so I don't think they're a problem for the conversion to Unicode. Cheers, Ralph. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
I assumed nmh would convert to UTF-8, I'd edit a UTF-8 draft, and that's what it would hand over to the MDA. That works great if your locale is UTF-8 ... but what if it isn't? That's where life gets complicated, and I think that means that it's just going to suck for some people. Of course it sucks right now for EVERYONE, and there's not a good solution based on the amount of free time that we have. http://en.wikipedia.org/wiki/Space_%28character%29#Spaces_in_Unicode (turn Javascript off to bypass blackout) lists various spaces so I don't think they're a problem for the conversion to Unicode. I think Max's point was that if you have a non-breaking space in Unicode, what will it be converted to in other character sets? --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
On 01/18/12 10:42, Tethys wrote: Ken Hornstein writes: So, I have some thoughts in this direction, but I'm wondering: what do you want out of repl in terms of better MIME handling? #1 is for base64 and quoted-printable to be decoded into UTF-8 before being included in the reply body. #2 would be for attachments to be optionally included in the reply (as attachments), defaulting to no. #2 is more of a problem with forw than repl (where IMHO it should default to yes), it has to be said. repl -mime sort of achieves #2 but in a very clumsy way. #2 filtering out attachments would be the most useful approach to start with. (Although the question would stay open how to nicely include the attachment in a repl/forw) #1 base64 and quoted-printable conversion into something nicer would be great. Not sure if it needs to be always converted to UTF8 or something like ISO 819x would also be sufficient. Also high priority. If both text/plain and text/html is available drop text/html (maybe configurable ?) Bottom line. Focus on dropping unneeded parts and if possible do some simple base64/quoted printable conversation. But don't get lost into too much details now. Such small improvements would make nmh usable for starters again. Andreas Tet ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers Verizon Deutschland GmbH - Sebrathweg 20, 44149 Dortmund, Germany - Amtsgericht Dortmund, HRB 14952 - Geschäftsführer: Detlef Eppig - Vorsitzender des Aufsichtsrats: Dominique Gaillard ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
If both text/plain and text/html is available drop text/html (maybe configurable ?) You'd have to do this interactively, showing a snippet of each, with the option to specify it on the command line. Too many mismanaged mailing lists have nothing as their text/plain content other than Your mail read is misconfigured, follow this link with tracking info ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Ken Hornstein writes: *Please, no!* Conversion from any charset to utf-8 is possible, but conversion back, according to user preferences, is not. People start to use funny characters like non-breakable space and so on. Unfortunately, we don't have unlimited development resources. Here's my reading of the world: - The general trend (especially in English-speaking countries) is to move toward Unicode (specifically, UTF-8). For English-speaking countries UTF-8, in majority, means ASCII, they can see no difference. As an advantage they can use foreign names like Moebius in original, this makes message more readable. But I'm afraid they wouldn't be happy with message written in Russian, Chinese or Korean. - People in Eastern Europe aren't crazy about this. I know, at least, one exception. =:-) - Given the lack of unlimited development resources, I don't really see people willing to change all of the internal APIs to include character set information. That means we pretty much have to choose one character set for an internal representation inside of nmh. In fact, I know very little about API, so it might be difficult. But restrict the entire nmh to utf-8 charset would cripple system. Beside the information on charset inside API, from my point of view the correct, and too much resource consumig, is move out module for conversion outside, as separate program. The default program would convert to utf-8, but anyone can provide his own program for conversion according to his taste. Suppose an entry in .mh_profile mh-text-convert: prog This (or in similiar way) would also can handle conversion to and from html, not only charset. Max -- -- ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Shouldn't you guys also be talking about pick in connection with messages containing Mime? Norman Shapiro 798 Barron Avenue Palo Alto CA 94306-3109 (650) 565-8215 n...@dad.org ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Ken Hornstein wrote: - If we have to choose one, the only logical choice is Unicode (which in practice means UTF-8; maybe UTF-16, but a trip through the format code made me think that UTF-8 is probably the only real choice). One very reasonable option would be to use the current locale default. This is what the current mh-format decode() does. The current locale is also what you can expect the terminal to display correctly and the user's text editor to handle. The limitations occur where e-mails use characters that can't be displayed in the current locale but we can't do anything about that. We also have fixed files like replcomps and components. It wouldn't be unreasonable to always treat these as UTF-8. \u and \U escapes are an alternative but I'd prefer to be able to specify an alternative for characters that won't work in the current locale. Oliver ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
For English-speaking countries UTF-8, in majority, means ASCII, they can see no difference. As an advantage they can use foreign names like Moebius in original, this makes message more readable. But I'm afraid they wouldn't be happy with message written in Russian, Chinese or Korean. Well, I can't speak for anyone else, but I can handle UTF-8 fine (I got some spam recently in some variant of Arabic and it displayed just fine). In fact, I know very little about API, so it might be difficult. But restrict the entire nmh to utf-8 charset would cripple system. It's not a restriction, per se ... it's more about the internal representation. It's easy enough to call iconv() (or whatever) to convert UTF-8 to and from the native character set. Beside the information on charset inside API, from my point of view the correct, and too much resource consumig, is move out module for conversion outside, as separate program. The default program would convert to utf-8, but anyone can provide his own program for conversion according to his taste. Suppose an entry in .mh_profile mh-text-convert: prog Yeah, that's doesn't really solve the problem in any meaningful way. Conversion isn't the problem (most Unix systems today have access to an API which does that for you). There is a good chunk of code inside of nmh that assumes ASCII (in terms of what is a space, what is a newline, and other things). Given the existing APIs today, handling multiple character sets within this code would require a lot of work. That's why I am proposing making the internal representation UTF-8. Now my plan was to convert from UTF-8 to the native character set, but that conversion won't be perfect. So here's my general feeling: since I'm writing the code, I'm the one who makes the decisions. If you (or anyone else) wants to write the code, then hey, you can make the decisions - you won't hear a peep out of me. Otherwise, you can give me your OPINION, and I'll listen to it, but I'll still be the one making the ultimate decision. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Shouldn't you guys also be talking about pick in connection with messages containing Mime? HOPEFULLY if I do things right, pick should Just Work. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Ken Hornstein k...@pobox.com writes: Shouldn't you guys also be talking about pick in connection with messages containing Mime? HOPEFULLY if I do things right, pick should Just Work. But doesn't the pick API need new primitives, such all messages containing a given mime type, all messages containing more than n mime types, etc? Or do things like -search and --component somehow make those easy? Norman Shapiro 798 Barron Avenue Palo Alto CA 94306-3109 (650) 565-8215 n...@dad.org ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Ken Hornstein k...@pobox.com writes: But doesn't the pick API need new primitives, such all messages containing a given mime type, all messages containing more than n mime types, etc? Or do things like -search and --component somehow make those easy? Sigh. One thing at a time, okay? Yes, pick probably does need some of those additions, but I was thinking more along the lines of you giving -search a string and having it correctly look inside of a base64-encoded UTF-8 message body. Don't mean to be picky. Just trying to understand. Norman Shapiro 798 Barron Avenue Palo Alto CA 94306-3109 (650) 565-8215 n...@dad.org ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Hi Aleksander, For English-speaking countries UTF-8, in majority, means ASCII, they can see no difference. I don't think that's the case. Even North Americans, who have $ in ASCII, still find ‘ ’ “ ” and … cropping up, especially when services automatically convert ` ' and And then there's £ and €. As an advantage they can use foreign names like Moebius in original, this makes message more readable. But I'm afraid they wouldn't be happy with message written in Russian, Chinese or Korean. The UTF-8 fonts on systems like Linux, and I assume Windows and Mac too, handle these just fine; Cyrillic, Chinese, and Japanese spam turns up here daily and mhshow copes. But restrict the entire nmh to utf-8 charset would cripple system. What language/charset/locale is it that you have where UTF-8 causes problems? Cheers, Ralph. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Thus spake Oliver Kiddle: The limitations occur where e-mails use characters that can't be displayed in the current locale but we can't do anything about that. How likely is it that a message containing characters undisplayable in the user's locale will be useful for the user? (This isn't meant rhetorically, it's a serious question.) Speaking for myself, if a message can't be displayed in my chosen locale, then switching to a locale where it can be probably won't help me read it. -- J. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
What do you mean by internal representation? Conversion from any to utf-8, processing by the code and conversion back to the original charset is really internal, transparent for the user. Well, for example ... when you call fmt_scan(), what should the character set be? Just one example. That's what I mean by internal representation. Now my plan was to convert from UTF-8 to the native character set, but that conversion won't be perfect. But such the internall conversion would be perfect, no new characters is introduced (except formatting like newlines, spaces). Depending on what the character set was in the original message, of course. The question is: what charset will be in draft for edition? Original, converted to something (e.g. according to locale) or utf-8. This is no longer internal. It will either be based on the locale or UTF-8. I'll know more when I get into the exact details. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Joel Uckelman writes: Thus spake Oliver Kiddle: The limitations occur where e-mails use characters that can't be displayed in the current locale but we can't do anything about that. How likely is it that a message containing characters undisplayable in the user's locale will be useful for the user? (This isn't meant rhetorically, it's a serious question.) This is not that simple. For years I enforced displaying iso-8859-1 charset on terminal supporting only iso-8859-2 and it works. 1. Charset declared in mail header. Quite a lot of people have incorrectly configured charset. 2. Language of the message. Might be different than charset suggests. For almost any charset basic ASCII is the same, so message writen in English would be readable. 3. Rare non-latin characters (e.g. names, cities) may enforce MUA to switch to another charset, while the almost whole text is readable. On the other end, message written in (say) Japanese would be unreadable even perfectly displayed =:-) But the same would happen in case of message written in foreign language that use the same charset as mine. This is not so strong relation: supported charset = readable message. Max -- -- ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
But restrict the entire nmh to utf-8 charset would cripple system. How so, specifically? Plan9 has run a native UTF8-only mail environment for ages (with a very MH-like mailstore, as well), and it's far from crippled. It stores messages in their native format, and dynamically converts message content to UTF8 whenever the underlying message is read. (There are also interfaces that let you read the message in it's raw (un-translated) format, for actions like message/rfc822 forwarding.) All new content is generated using utf-8. In the 10 or so years I've been using it for production mail, I have never had problem with incoming mail, nor have I had issues with people not being able to read anything I sent to them. These days, pretty much any MIME capable MUA that understands character sets can handle UTF8. As for external tools that cannot handle UTF8 text, they most likely can't handle random ISO* charsets, either. UTF8 is where the world is going. nmh should get on board. --lyndon ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Couple of years ago emacs switch to internal coding in utf-8. I had to stop using emacs and mh-e. See, this is what I'm missing - why, exactly? I assume the problem was not just philsophical. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Guess it will take a while before better mime handling is implemented. Not necessarily. So, I have some thoughts in this direction, but I'm wondering: what do you want out of repl in terms of better MIME handling? --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
Hi Ken, So, I have some thoughts in this direction, but I'm wondering: what do you want out of repl in terms of better MIME handling? All the text parts turned into UTF-8 and quoted would be a good start. I can then trim down in vi as normal. Cheers, Ralph. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
So, I have some thoughts in this direction, but I'm wondering: what do you want out of repl in terms of better MIME handling? All the text parts turned into UTF-8 and quoted would be a good start. I can then trim down in vi as normal. Yeah, to me that would make things 100% better. That's also on my to-do list. I think when I start thinking about better UTF-8 support then this will be an obvious benefit. I wouldn't mind some better intelligence in terms of picking out the good MIME parts in the reply message, but I'm thinking that's a longer term goal. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
ken wrote: So, I have some thoughts in this direction, but I'm wondering: what do you want out of repl in terms of better MIME handling? All the text parts turned into UTF-8 and quoted would be a good start. I can then trim down in vi as normal. Yeah, to me that would make things 100% better. That's also on my to-do list. I think when I start thinking about better UTF-8 support then this will be an obvious benefit. I wouldn't mind some better intelligence in terms of picking out the good MIME parts in the reply message, but I'm thinking that's a longer term goal. right. for instance, if there's no text part (argh!), probably converting an html part to text and using that would be the next step. paul =- paul fox, p...@foxharp.boston.ma.us (arlington, ma, where it's 36.9 degrees) ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
On Tue, Jan 17, 2012 at 12:16, ra...@hep.wisc.edu wrote: So, I have some thoughts in this direction, but I'm wondering: what do you want out of repl in terms of better MIME handling? All the text parts turned into UTF-8 and quoted would be a good start. I can then trim down in vi as normal. Yeah, to me that would make things 100% better. That's also on my to-do list. I think when I start thinking about better UTF-8 support then this will be an obvious benefit. I wouldn't mind some better intelligence in terms of picking out the good MIME parts in the reply message, but I'm thinking that's a longer term goal. right. for instance, if there's no text part (argh!), probably converting an html part to text and using that would be the next step. +1 and also, when there's multipart/alternative choose text/plain over e.g. text/html. The problem with that approach is that sometimes the text part just says ``There is no text part, use an HTML capable mail reader''. I'm seeing more of them these days. In mh-e we use the content that is being displayed to the user when we reply. If no content is displayed we do not include the original post. Otherwise we let the user figure out which content is most useful and include that in the reply. Would there be some way to figure this out by caching the details of the last show command which indicates which part was displayed (text or html)? Thanks Jeff -- Jeffrey C. Honig j...@honig.net http://www.honig.net/jch GnuPG ID:14E29E13 http://www.honig.net/jch/key.shtml ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] repl and mime handling
The problem with that approach is that sometimes the text part just says ``There is no text part, use an HTML capable mail reader''. I'm seeing more of them these days. Yeah, I've seen those occasionally but I'm willing to put that problem off for a little while. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
