Hi Robert, Given a couple of the recent questions on the list about Cyrus and JMAP support, I think it'd be good to have a quick guide in the docs for people wanting to get started with it.
We can then also keep it updated with the state of implementation(such as you've outlined below) which will let people know what functionality is and isn't yet available. Are you able to write up a brief set of instructions on getting the sample client running against a Cyrus installation and I can fold them into the docs? (We can discuss at the Cyrus meeting later today) Cheers, Nicola ----- Original message ----- From: "rost via Cyrus-devel" <cyrus-devel@lists.andrew.cmu.edu> To: cyrus-devel@lists.andrew.cmu.edu Subject: JMAP: Current status Date: Tue, 05 Jan 2016 18:38:32 +0100 Hi, as promised in our last Hangout, here is my writeup of the current state of JMAP in Cyrus. Since I’ll be off from Cyrus development for a couple of weeks, this double-serves as my TODO list. But first things first: all my Cyrus contributions outlined below are 100% sponsored by FastMail. What works already =============== Contacts ------------ Mostly. But see next section. Calendars -------------- Mostly. But see next section. Messages -------------- Are work in progress and my current focus. - getMessages: works mostly. Not supported: message threads, inReplyTo or proper html-to-text body conversion (see [http_jmap.c:2248](https://github.com/rsto/cyrus-imapd/blob/jmap/imap/http_jmap.c#L2248)). - setMessages: supports to create drafts, send mails. Does not support creation of messages in multiple mailboxes, or any mailbox moves. - getMailboxes: mostly, except threads. - setMailboxes: mostly - getMessageList: supports filters Cassandane tests ------------------------- There are currently 59 JMAP tests in Cassandane. What doesn’t work, yet ================== Attachments ----------------- I have worked on a proof-concept of supporting the JMAP File API (aka “blobs”) in Cyrus, but it’s far from complete and it didn’t seem promising enough to merge it into the main JMAP branch. As discussed with Bron, I rather would like to get back to the design stage. For that, I welcome any input! My current assumptions are that the majority of blobs will belong to exactly one JMAP object, and most of the blobs uploaded by an account will only be attached to one object. Also: while I assume most blobs belong to mails, some users may have (rarely changing) avatar blobs in most of their contacts. I assume calendar event attachments to be rarely used. Under these assumptions, I plan the blob service to keep a blob in its containing object as long as possible. E.g. if a JMAP client wants to download a mail attachment, the blob service should be able to use the blobId to locate the containing message in the mailboxes. It should not create a copy of the blob. That’s very similar to the current JMAP proxy implementation. Things get tricky though, once such a blob is used as an attachment for another object or even another entity (e.g. from message blob to avatar blob). In order to better understand what a JMAP client might want to do, I’ve summarised all use cases I could come up with in a [Google spreadsheet](https://docs.google.com/spreadsheets/d/15CvwT-aYw8ks3PbCS3Svm2sPfKWTbIB_JNBEBLMoGSc/edit?pref=2&pli=1#gid=0). Note that the prioritisation is solely mine and some of these use cases probably shouldn’t be supported at all. Also, in a sane blob service, one control flow should over multiple use cases. The JMAP blob ref-counting will require a tight grip on what happens to mails, calendars and contacts. Maybe the notify service could be used to support asynchronous ref-counting? Also: Since calendar managed attachments are almost a subset of the more generic JMAP blobs (except for deletions!), these two features might even be merged. The blob service might deserve its own httpd and I could imagine it being decoupled from the JMAP code. The blob HTTP service should also be configurable to only redirect to “safe” URLs for custom calendar or contact URL attachments. Multi-user accounts --------------------------- All the current JMAP code operates on the userid currently authenticated to Cyrus httpd. That is, the ‘accountId’ property in JMAP requests is not really supported. Authentication -------------------- All the JMAP methods in httpd currently require an account logged in via Basic Authentication. Remote mailboxes -------------------------- As discussed with Bron, we don’t take remote mailboxes into account, yet. Events --------- The JMAP event service hooks into notifications, so that’s almost done. What’s AFAIK missing is the service layer (Bron knows more). Messages --------------- Anything not mentioned in the “What works” section doesn’t works. Notably, that’s search snippets and threads. Phrase-Matching -------------------- The JMAP filters require phrase matching for text properties, but as a placeholder we currently only support case-insensitive substring search. What could work better ================== Lookup message by guid ---------------------------------- We use message guids as JMAP message ids. Currently, that requires O(n), where n is the number of records across all a users mailboxes. That really should become O(1) or O(lgN) Lookup mailbox by unique-id ------------------------------------------ We use mailbox unique-ids for JMAP mailbox ids. To look them up, I added already a stub in [mboxlist](https://github.com/rsto/cyrus-imapd/blob/jmap/imap/mboxlist.c#L598), but it’s O(n) (n is the number of a users mailboxes). Should be O(1) or O(lgN) Filters --------- The contacts and calendar code reuses the same JMAP filter code. We agreed to first implement naive search for these objects. Currently, we match every record in a a users calendar or contacts mailboxes against the JMAP filter. At least for calendar events, this could be sped up, e.g. it would be straight-forward to only pull out calendar events for a custom time range or even create the SQL statements on demand based on the filter contents. Also, for message filters, there is a very naive filter implementation. That’s just meant as a placeholder, and should be refactored to make use of Xapian. Error reporting -------------------- The JMAP spec requires all invalid properties of a request to be reported. But contacts fail at the first property error. Calendars and Messages try hard to report all erroneous properties. None of the JMAP error handlers report an error description. Code organization ------------------------- While code reuse is OK, contacts, calendars and messages still use slightly different idioms. Note to me: Possibly use jmap_req as context, and pass around sub-requests, e.g. jmap_setMessages_req embeds jmap_req. That could help unify control and data flow. <EOF> Cheers, Robert
