Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
On Mon, Feb 6, 2012 at 12:51 PM, Joel Uckelman wrote: If/when IMAP support is added in, mhpath will return imap URLs. If one is using the existing classic storage model, then local pathnames are returned. IMAP URLs aren't paths, at least not in the sense I take 'path' to have in 'mhpath'. Maybe we could add 'mhurl' to produce URLs (and file: URLs for local storage!) and leave mhpath producing paths? Tomato, tomahto. Path can have different meanings depending on your context, but if you want to be pedantic, maybe have a new program called mhloc to represent the location of an nmh resource. mhpath can then just be mhloc, but with a pre-configured setting that if the resource is to a local file, it provides a local pathname instead of a file: URL. Regardless, mhpath should not barf if the resource happens to be a non-local pathname. It will still return a URL for such resources. --ewh ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
In the IMAP case, you don't want to download the entire message just to satisfy an mhpath request. The value in IMAP is its ability to treat MIME sections as separate objects. By sucking down entire messages, all you've done is downgrade IMAP to POP. I understand where you're coming from, but I have to ask ... when people use mhpath to get the path of an MH message, what, exactly, are they trying to accomplish? Usually it's something along the lines of use some Unix text processing tool on this message. For that you need to download the whole thing. Obviously for something like scan you don't need the whole message (well, depending on your scan format, you might, or at least some of the beginning of the message). I think mhpath isn't part of the normal flow of MH, but if it is for people, I'm interested in how you use it (I use it, but only occasionally, and my usage is such that I'd need the whole message). So if you want to have mhpath return IMAP URLs, okay, I'm fine with that. But personally I think that's not as useful as having mhpath download the whole message. Adding this to nmh is doable, but it's not something you can design by committee. If we're going to implement a VFS abstraction layer, a couple of people need to step up and volunteer to design and implement a prototype. This needs working code to shake out architecture bugs, and to provide scaffolding to work out the best way to expose this to the user commands. Well, here's where I think we are regarding that: - Paul suggested what such an API should look like (specifically, he said man 3 db. He said he'd do some work on designing this API if people thought it would be a good idea. - My judge of the consensus of the list was that it was a good idea; okay, there weren't many responses. I (and maybe a few others) said we thought it was a good idea. No one said they thought it was a bad idea. - That's where things stopped. I've been busy, so I didn't want to bug Paul about it. I suspect Paul's been busy too, and didn't want to work on it until he had a chunk of time free. - My gut tells me that even thought there is a danger of the design by committee problem, we should discuss this on nmh-workers. I think there is enough good ideas here that it would be useful to solicit input from people. I could be persuaded otherwise. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
On 2012-02-06, at 6:28 AM, Ken Hornstein wrote: I understand where you're coming from, but I have to ask ... when people use mhpath to get the path of an MH message, what, exactly, are they trying to accomplish? Usually it's something along the lines of use some Unix text processing tool on this message. For that you need to download the whole thing. No, you need to download text parts. It would be pointless to download the 15 multi-MB TIFF attachments as well. We need to flesh out how MH will address per-message MIME parts and come up with a consistent syntax for expressing that across the commands. Once we have that, the mhpath solution should just fall out. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
No, you need to download text parts. It would be pointless to download the 15 multi-MB TIFF attachments as well. Huh, I guess I was thinking that you had to download the whole message body, but I guess I shouldn't go up against an IMAP expert like you :-) I took a closer look at RFC 3501 and I see how that works now. It occurs to me that we could crib the syntax for parts of MIME messages from RFC 3501 (look at section 6.4.5, the FETCH command). Well, maybe not directly, but it's a place to start. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
Ken Hornstein k...@pobox.com writes: In the IMAP case, you don't want to download the entire message just to satisfy an mhpath request. The value in IMAP is its ability to treat MIME sections as separate objects. By sucking down entire messages, all you've done is downgrade IMAP to POP. I understand where you're coming from, but I have to ask ... when people use mhpath to get the path of an MH message, what, exactly, are they trying to accomplish? Usually it's something along the lines of use some Unix text processing tool on this message. For that you need to download the whole thing. Not just download, but upload too, that is, modify messages. - My gut tells me that even thought there is a danger of the design by committee problem, we should discuss this on nmh-workers. I think there is enough good ideas here that it would be useful to solicit input from people. I could be persuaded otherwise. One of the design goals of mh, was transparency. I grant that we did not completely achieve that goal. I also grant that the Email world of 1977 was much, much, simpler than the world you are confronted with. Nonetheless, it would be good to keep that goal in mind. Norman Shapiro 798 Barron Avenue Palo Alto CA 94306-3109 (650) 565-8215 n...@dad.org ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
Not just download, but upload too, that is, modify messages. IMAP messages are immutable. You cannot change them after the fact. This us one of the many tricky issues with supporting an IMAP message store. E.g., it will be difficult to make anno behave correctly in all cases. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
On 2/6/2012 3:58 PM, Lyndon Nerenberg wrote: IMAP messages are immutable. You cannot change them after the fact. This us one of the many tricky issues with supporting an IMAP message store. E.g., it will be difficult to make anno behave correctly in all cases. if the underlying message store is immutable, i expect anno to do one of two things: 1. maintain state locally -- this means other IMAP users on the same or other hosts will not see anno's changes. 2. fail. overall i prefer #2. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
paul wrote: On 2/6/2012 3:58 PM, Lyndon Nerenberg wrote: IMAP messages are immutable. You cannot change them after the fact. This us one of the many tricky issues with supporting an IMAP message store. E.g., it will be difficult to make anno behave correctly in all cases. if the underlying message store is immutable, i expect anno to do one of two things: 1. maintain state locally -- this means other IMAP users on the same or other hosts will not see anno's changes. 2. fail. overall i prefer #2. right. this should be pretty similar to accessing a mail archive mounted read-only. (i use mhpath to edit and modify messages -- admittedly usually to do lists i've sent to myself. i wouldn't expect that to work from from imap.) paul =- paul fox, p...@foxharp.boston.ma.us (arlington, ma, where it's 39.2 degrees) ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
paul vixie writes: i think we're going to end up extending the message number to be a decimal not an integer, as in 53.6 to get MIME part 6 of message number 53. any place that accepts a message number will have to understand this If we do so, it will of course have to allow arbitrarily deep nesting, so we can address things like 53.6.4.1 (the 1st part of the 4th part of the 6th part of message #53). Tet ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
On Mon, 06 Feb 2012 07:11:57 PST, Lyndon Nerenberg said: No, you need to download text parts. It would be pointless to download the 15 multi-MB TIFF attachments as well. The fact it's text doesn't imply it needs downloading. I regularly get logfiles from our NetApp that run to 4M in size - they show up as a multipart/mixed with a 2M+ text/plain and 9 gzipped files. And if it's a crash report, the thing I want to look at *first* is in the third gzipped file, not the big text/plain. pgpU0Er8Ntiwy.pgp Description: PGP signature ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
It's not that hopeless. If the IMAP server supports the METADATA
extension (and most modern servers do), we can almost fully support
anno.
So ... looking at RFC 5464 ... for anno you would do something like:
SETMETADATA INBOX (/private/nmh/${imap-uid}-anno Replied: Mon, 06 Feb 2012)
As I understand it ... the limits on METADATA are per-mailbox. What are
some typical limits?
--Ken
___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
On 2012-02-06, at 9:09 AM, Ken Hornstein wrote:
SETMETADATA INBOX (/private/nmh/${imap-uid}-anno Replied: Mon, 06 Feb 2012)j
Exactly.
As I understand it ... the limits on METADATA are per-mailbox. What are
some typical limits?
I'm not sure. I will poll the IMAP protocol list and see what I can find out.
--lyndon
___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
Wrt to mhpath, I think it is getting over analyzed. All mhpath does is provide the *path* to an nmh resource, that's it. Issues like fetching or not fetching are out-of-scope. If/when IMAP support is added in, mhpath will return imap URLs. If one is using the existing classic storage model, then local pathnames are returned. So, for the following command: mhpath +inbox If using the classic storage model: /home/user/MHMail/inbox And if IMAP storage is used: imap://imap.example.com/inbox Any additional storage models would return URLs appropriate for that storage model. --ewh ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
On 2/6/2012 4:20 PM, Tethys wrote: If we do so, it will of course have to allow arbitrarily deep nesting, so we can address things like 53.6.4.1 (the 1st part of the 4th part of the 6th part of message #53). ouch! my eyes! ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
paul vixie writes: On 2/6/2012 4:20 PM, Tethys wrote: If we do so, it will of course have to allow arbitrarily deep nesting, so we can address things like 53.6.4.1 (the 1st part of the 4th part of the 6th part of message #53). ouch! my eyes! Yes, but representative of real world MIME structures. Tet ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
Thus spake Earl Hood: Wrt to mhpath, I think it is getting over analyzed. All mhpath does is provide the *path* to an nmh resource, that's it. Issues like fetching or not fetching are out-of-scope. If/when IMAP support is added in, mhpath will return imap URLs. If one is using the existing classic storage model, then local pathnames are returned. IMAP URLs aren't paths, at least not in the sense I take 'path' to have in 'mhpath'. Maybe we could add 'mhurl' to produce URLs (and file: URLs for local storage!) and leave mhpath producing paths? -- J. ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
On Mon, 06 Feb 2012 11:51:04 MST, Joel Uckelman said: in 'mhpath'. Maybe we could add 'mhurl' to produce URLs (and file: URLs for local storage!) and leave mhpath producing paths? m-hurl. What has been seen cannot be unseen. ;) pgpLvQ2xPjlNU.pgp Description: PGP signature ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
Re: [Nmh-workers] urls from mhpath, and message store abstraction layers.
i think it's itinerant upon those who think this should be done, to walk the walk. i also think that intermediate products such as example API's and proposals should be shared with the whole list. and lastly i think that silence will equal assent :-). Okay, I can see your point. I was going to say that I wouldn't mind being involved with the design process but you know what? If you and Lyndon are working on it, then I don't think I can really add value. Please let us know how you guys make out ... but I know how real life works, so I won't be biting my nails for something anytime soon. --Ken ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
[Nmh-workers] urls from mhpath, and message store abstraction layers.
On 2012-02-05, at 12:13 PM, Ken Hornstein wrote: % echo `mhpath 5 8 12` /tmp/.imapcache/88932 /tmp/.imapcache/92861 /tmp/.imapcache/1034785 Or whatever ... you get the idea. mhpath would fetch the messages in question and put them somewhere local and return those pathnames. Sure, you could give a flag to mhpath to make it return IMAP URLs, but if we did that then I think we should return URLs for all messages. In the IMAP case, you don't want to download the entire message just to satisfy an mhpath request. The value in IMAP is its ability to treat MIME sections as separate objects. By sucking down entire messages, all you've done is downgrade IMAP to POP. MH could be a very good disconnected mode IMAP client, but getting this right involves a *lot* of subtleties. But before chasing IMAP message stores (or any others), we have to address Paul's VFS-like abstraction layer for the message store interface. This has been done before (not in MH), and while it's conceptually straight-forward, there are a lot of fiddly things you must get right. I have designed and implemented a few of these message store virtual interfaces, including a very crude prototype in MH as part of a test to see how hard adding IMAP support would be. [Answer: hard ;-) But not insurmountable.] Adding this to nmh is doable, but it's not something you can design by committee. If we're going to implement a VFS abstraction layer, a couple of people need to step up and volunteer to design and implement a prototype. This needs working code to shake out architecture bugs, and to provide scaffolding to work out the best way to expose this to the user commands. --lyndon ___ Nmh-workers mailing list Nmh-workers@nongnu.org https://lists.nongnu.org/mailman/listinfo/nmh-workers
