Re: Has anyone looked at JMAP?
>Take the closed-source API client. How does it ‘make reasonable efforts >to prevent and discourage other API Clients from using your >credentials’? It's not shipping source, but does embedding it somewhere >inside an ELF executable count as reasonable? I disassemble machine >code a lot, so perhaps it's only reasonable if they make some effort to >disguise it? I can't argue that it's NOT bullshit. I suspect the writers of that are used to apps on phones or tablets, where it's much harder to get at the actual executables. Getting back to the ORIGINAL thread ... regarding JMAP, I have no objections if anyone wants to work on adding it to nmh. But AFAICT, the same security issues exist with it as they do with IMAP in terms of API keys. Just in terms of protocol coverage by email providers, IMAP is the obvious choice. --Ken
Re: Has anyone looked at JMAP?
Ralph Corderoy wrote: > Take the closed-source API client. How does it ‘make reasonable efforts > to prevent and discourage other API Clients from using your > credentials’? It's not shipping source, but does embedding it somewhere > inside an ELF executable count as reasonable? I disassemble machine > code a lot, so perhaps it's only reasonable if they make some effort to > disguise it? I agree. It's a bullshit security design. A secret that is installed on every phone that has some app, and every windows platform? Ridiculous. > Or, we ship a proprietary closed-source blob, or download it if it's not > present, and lo, we've set the bar as high as those closed-source > shippers. uhm, yeah. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works|IoT architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ signature.asc Description: PGP signature
Re: Has anyone looked at JMAP?
Hi Ken, > It does seem unfortunate that the official rules don't permit OSS > projects https://developers.google.com/terms#b_confidential_matters says b. Confidential Matters Developer credentials (such as passwords, keys, and client IDs) are intended to be used by you and identify your API Client. You will keep your credentials confidential and make reasonable efforts to prevent and discourage other API Clients from using your credentials. Developer credentials may not be embedded in open source projects. Take the closed-source API client. How does it ‘make reasonable efforts to prevent and discourage other API Clients from using your credentials’? It's not shipping source, but does embedding it somewhere inside an ELF executable count as reasonable? I disassemble machine code a lot, so perhaps it's only reasonable if they make some effort to disguise it? How is that different to an open-source project shipping the API key as two parts: an encryption key and the encrypted API key? It seems reasonable to me. It's probably not too hard to make it as awkward to get the plain-text key as it is to disassemble. Or, we ship a proprietary closed-source blob, or download it if it's not present, and lo, we've set the bar as high as those closed-source shippers. IANAL. The answers I got from a FSF lawyer about the implications of signing their copyright assignment many years ago suggest to me that those who have signed it probably don't interpret it as a lawyer does. :-) -- Cheers, Ralph.
Re: Has anyone looked at JMAP?
>Oh, it's fairly easy for a user to create their own custom API key for their >own instance of fetchmail. What's *not* permitted is for a *project* to ship a >tarball or whatever with a key usable by everybody who installs the package. Well, we've been doing this for years with nmh! And it sounds like KMail does the same. I'm not really sure HOW you're supposed to do it for OSS projects otherwise. And ... well ... to show how COMPLETELY dumb this is, the same authentication scheme (OAauth2) is used for Microsoft exchange. When testing out DavMail for Exchange email access, my company refused to allow DavMail as an authorized client. So I simply changed DavMail to use the client identifier of Microsoft Office ... which I found on docs.microsoft.com. >Which of course is actually pretty reasonable - imagine the flamestorm if >openssh shipped a public/private keypair that it installed on every >machine. If you cound provide pointers on how to create your own custom API key (there is confusion on how difficult it is), I'd be glad to add that documentation to nmh. --Ken
Re: Has anyone looked at JMAP?
On Mon, 31 Aug 2020 20:44:42 -0400, Ken Hornstein said: > >- Paragraph 4.b.1, which states that "You will keep your credentials > >> confidential and make reasonable efforts to prevent and discourage other > >> API Clients from using your credentials. Developer credentials may not be > >> embedded in open source projects." prohibits the use of OAuth credentials > >> in free software projects. As I wrote above (and earlier), Google > >> tolerates (at the moment) that this specific point of their TOS is > >> violated. But that doesn't mean that violating them is without legal > >> risk. > > Oof, fair enough. It does seem unfortunate that the official rules don't > permit OSS projects; I wish there was a way for a user to create their > own custom API key and they could just add that to their account. Honestly, > I am fine with doing what KMail did (since that's what we did before). Oh, it's fairly easy for a user to create their own custom API key for their own instance of fetchmail. What's *not* permitted is for a *project* to ship a tarball or whatever with a key usable by everybody who installs the package. Which of course is actually pretty reasonable - imagine the flamestorm if openssh shipped a public/private keypair that it installed on every machine. pgpEIP4Rkm01d.pgp Description: PGP signature
Re: Has anyone looked at JMAP?
On Mon, Aug 31, 2020 at 5:44 PM Ken Hornstein wrote: > >- Paragraph 4.b.1, which states that "You will keep your credentials > >> confidential and make reasonable efforts to prevent and discourage other > >> API Clients from using your credentials. Developer credentials may not > be > >> embedded in open source projects." prohibits the use of OAuth > credentials > >> in free software projects. As I wrote above (and earlier), Google > >> tolerates (at the moment) that this specific point of their TOS is > >> violated. But that doesn't mean that violating them is without legal > >> risk. > > Oof, fair enough. It does seem unfortunate that the official rules don't > permit OSS projects; I wish there was a way for a user to create their > own custom API key and they could just add that to their account. > Honestly, > I am fine with doing what KMail did (since that's what we did before). According to emacs-devel, there _is_ a way for users to create their own API key, although it's supposedly pretty involved, and includes an unbounded (in time) review step. There is apparently a python-based package that smooths the process, but it seems to be designed to intentionally avoid automation. Regardless, it isn't a good conceptual fit for GNU because it's pretty user-unfriendly, and it requires running a bunch of "unfree javascript", so they don't want to officially recommend it. FWIW, there are apparently some other clauses that make it clear that they don't want you to automate the API key process (and a suggesting that they'll invalidate keys and break the automated paths), as well as some language that prevents adapting the KMail code into a server for other programs. The core Gnus folks have roughly said "Google has made their intentions clear, so it's not worth _our_ effort to try to sidestep them", and there are in theory conversations underway to see if a KMail-style arrangement can be reached that doesn't trip up the lawyers. No telling how that will turn out, but it seems pretty clear to me (still from the back bleachers) that part of GMail wants to stop supporting clients like Gnus, even if other parts are open to an informal arrangement. GNU has a talent for invoke/provoking formality, though. ~Chad
Re: Has anyone looked at JMAP?
>- Paragraph 4.b.1, which states that "You will keep your credentials >> confidential and make reasonable efforts to prevent and discourage other >> API Clients from using your credentials. Developer credentials may not be >> embedded in open source projects." prohibits the use of OAuth credentials >> in free software projects. As I wrote above (and earlier), Google >> tolerates (at the moment) that this specific point of their TOS is >> violated. But that doesn't mean that violating them is without legal >> risk. Oof, fair enough. It does seem unfortunate that the official rules don't permit OSS projects; I wish there was a way for a user to create their own custom API key and they could just add that to their account. Honestly, I am fine with doing what KMail did (since that's what we did before). --Ken
Re: Has anyone looked at JMAP?
On Mon, Aug 31, 2020 at 3:46 PM Ken Hornstein wrote: > >AIUI, Google was planning on discontinuing LSA access late this year for > >gmail accounts (hosted GSuite accounts had a different timeline but the > >same goal). Instead, apps can apply for an app-specific "secret", but on > >terms that specifically disallow open-source code from shipping the > secret. > > Well ... we've been dealing with this as well. Our reading of the terms > is that the issue isn't with open source software at all, but more about > how you prove that you're "you" (we shipped a secret that worked until > very recently). I don't see anything that really disallows OSS, though! > Here's the part that came across emacs-devel: Google's terms of service for OAuth services are available at > https://developers.google.com/terms . Only a lawyer can tell you in brief > terms what the concrete requirements are. > ... - Paragraph 4.b.1, which states that "You will keep your credentials > confidential and make reasonable efforts to prevent and discourage other > API Clients from using your credentials. Developer credentials may not be > embedded in open source projects." prohibits the use of OAuth credentials > in free software projects. As I wrote above (and earlier), Google > tolerates (at the moment) that this specific point of their TOS is > violated. But that doesn't mean that violating them is without legal > risk. Hope that helps! ~Chad
Re: Has anyone looked at JMAP?
>I haven't gotten very far into the details, but the move to a stateless >protocol seemed like an improvement (at least potentially) over IMAP to me. Well, so ... here's my opinion on that. The "stateless" part really is talking about the network protocol. My reading of the JMAP spec is that you definitely need to remember things between requests. Like if you find out a mailbox or message id, you need to remember for later use. Yes, I see what you mean about you can do some interesting batching, where the output of one query can be fed into another query ... but I am not sure that is directly usable by nmh. Nmh really wants to work on "message numbers", so you need to map that to "message identifiers", be it IMAP or JMAP. The solutions for that are relatively obvious, but that does mean most of the time you won't be able to do a query without translating those results into MH message numbers and that prevents batching. But in terms of cutting down round trips ... you can batch up a lot of stuff in an IMAP request, and you're allowed to send multiple commands at once (each command is prefixed with a tag so you can distinguish the responses). And you can do intelligent things like only ask for the parts of the message you care about. So, I _think_ it's probably a wash in terms of how nmh would use it. And preliminary tests suggested that the round trips weren't too awful (see the archives). >As to GMail, there's some uncertainty about the scheduling, since a big >step was apparently delayed recently, but it has to do with "Less Secure >App" access: > > https://support.google.com/accounts/answer/6010255?hl=en Right, we already support that! >AIUI, Google was planning on discontinuing LSA access late this year for >gmail accounts (hosted GSuite accounts had a different timeline but the >same goal). Instead, apps can apply for an app-specific "secret", but on >terms that specifically disallow open-source code from shipping the secret. Well ... we've been dealing with this as well. Our reading of the terms is that the issue isn't with open source software at all, but more about how you prove that you're "you" (we shipped a secret that worked until very recently). I don't see anything that really disallows OSS, though! --Ken
Re: Has anyone looked at JMAP?
I haven't gotten very far into the details, but the move to a stateless protocol seemed like an improvement (at least potentially) over IMAP to me. If the nmh approach was to daemonize locally, then it doesn't matter, but that way seemed a bit fraught to me. The basis of jmap seems to be batching requests into higher level (a reasonable approximation to mh-level, I perhaps naively thought) email concepts, so as to remove the need for a stateful connection. As to GMail, there's some uncertainty about the scheduling, since a big step was apparently delayed recently, but it has to do with "Less Secure App" access: https://support.google.com/accounts/answer/6010255?hl=en AIUI, Google was planning on discontinuing LSA access late this year for gmail accounts (hosted GSuite accounts had a different timeline but the same goal). Instead, apps can apply for an app-specific "secret", but on terms that specifically disallow open-source code from shipping the secret. Emacs-devel was looking into it specifically because KMail supposedly had a way around the problem, which turns out to be "ignore the legal terms under a tacit agreement that Google won't overtly get mad", which is (as you might guess) less palatable to the GNU folks. The delay in the schedule has taken the pressure off for a bit, but the issue is still fundamentally unresolved. >From up here in the bleacher seats, it seems clear that the common webmail systems are moving strongly toward client lock-in in the name of security. Fastmail and JMAP looked like a potential alternative for the future. Yeah, they're not suggesting ditching IMAP yet, and clearly that would be a futile move. Similarly, they have a draft contacts and calendaring system using the same approach as JMAP, but they're waiting for use-cases to settle down (probably, for JMAP itself to settle down) before trying to solidify the specs. Thanks, ~Chad
Re: Has anyone looked at JMAP?
>I don't know if it will be easier to interface mh folders to or not. So, "solution in search of a problem" might have been a bit harsh. I understand where they are coming from. But ... fundamentally, here are the issues I see with nmh<->IMAP/JMAP interface: - Authentication - Being able to detect when the contents of a folder changes - Mapping unique server message identifiers to MH message numbers As far as I can tell, all of these issues are the same between IMAP and JMAP. That is to say, I believe they are fundamentally solvable, but it will require some effort. If people disagree with me, then fine; speak up! Like I said before, I am open to be proven wrong. --Ken
Re: Has anyone looked at JMAP?
Ken Hornstein wrote: >> I came across JMAP (jmap.io) this weekend, and have been looking at it in >> my spare time. It looks like it intends to solve most of the problems that >> I personally had with mixing mh-style and mobile-app email usage, and might >> even be a (speculation warning) useful bridge to the future for nmh. > I have looked at it. It honestly seems to me like a solution in search > of a problem, but fine (or maybe it's designed for people who cannot > imagine any network protocol that is not REST). But I cannot see how it > really solves any of the fundamental problems of mixing "mh-style" and > mobile-app email usage; as far as I can tell, those core issues still > exist. But I am open to being persuaded otherwise! So, if you want > to expand on your opinion, please do! It is being driven by a variety of web properties and frameworks. Not necessarily big ones. Think WordPress and phpBB and tikiWiki, not GCE. I know a few people involved, but i haven't been involved myself, but I think that you'll see it used to access to things like webboards, and the like in a cross-platform, NNTP-like way. I don't know if it will be easier to interface mh folders to or not. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works|IoT architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ signature.asc Description: PGP signature
Re: Has anyone looked at JMAP?
>I came across JMAP (jmap.io) this weekend, and have been looking at it in >my spare time. It looks like it intends to solve most of the problems that >I personally had with mixing mh-style and mobile-app email usage, and might >even be a (speculation warning) useful bridge to the future for nmh. I have looked at it. It honestly seems to me like a solution in search of a problem, but fine (or maybe it's designed for people who cannot imagine any network protocol that is not REST). But I cannot see how it really solves any of the fundamental problems of mixing "mh-style" and mobile-app email usage; as far as I can tell, those core issues still exist. But I am open to being persuaded otherwise! So, if you want to expand on your opinion, please do! >I >certainly remember a time when skipping over IMAP would have seemed crazy, >but the people on emacs-devel are currently wrestling with the implications >of at least Google and MS both moving towards shutting down pure IMAP >access (somewhat stalled by current conditions, but clearly stated intent). I am not aware of any effort by Google to shut down IMAP; could you provide a link? MS, yes, wants you to use their horrible Exchange protocol. If you're talking about how you have to enable IMAP for G Suite users ... well, fine. My limited, imperfect understanding is that "native" Gmail is a proprietary protocol, so JMAP isn't relevant. If you want a protocol that is relatively universal, I think right now IMAP is the best choice. Even the people behind JMAP provide an IMAP server on their systems. --Ken
Has anyone looked at JMAP?
If this is too speculative and/or disruptive for this list these days, please ignore it with my apologies, but... I came across JMAP (jmap.io) this weekend, and have been looking at it in my spare time. It looks like it intends to solve most of the problems that I personally had with mixing mh-style and mobile-app email usage, and might even be a (speculation warning) useful bridge to the future for nmh. I certainly remember a time when skipping over IMAP would have seemed crazy, but the people on emacs-devel are currently wrestling with the implications of at least Google and MS both moving towards shutting down pure IMAP access (somewhat stalled by current conditions, but clearly stated intent). If anyone here had a considered opinion, I'd like to hear it. Thanks, ~Chad
