Re: My thoughts on JMAP implementation
I gonna start answering this one ... Le 14/12/2015 08:58, Matthieu Baechler a écrit : > And the last part ... > > On 13/12/2015 12:43, Tellier Benoit wrote: > > [...] > >> - I heard JMAP handles unique ID accross email copies. I guess you are >> compelled to support it. How do you plan to manage it across >> implementations ? > > That's a very interesting question. > > What we think would be a good design is : > > 1/ make Message a top-level entity, that means it should exist outside > any mailbox. From a datastore point of view, it means having an > identifier that is not bound to the mailbox (unlike uid). Let's call > that Message.id Yes, we agree. I have one question : what should be the type of message.id ? I think it can be easily MailboxId... > > 2/ Message <-> Mailbox relation is no longer a composition but an > aggregation, it means that Message lifecycle is not bound to Mailbox and > that Message.uid is now a relation id Ok > > 3/ Some backends may not be able to express this new design (we think > about filesystem backend) so we think backend have to expose their > capabilities and a backend consumer should require a mandatory > capabilities list. That way, we could go forward without being limitated > by backends not feature-rich enough. The compatibility should be checked > at server start and prevent incompatible components to start. I think this is a very good idea. I can already think of other "capabilities", like : - custom IMAP flag handling - custom IMAP flag indexing - quota processing - ACL handling - Thread ID handling ... It would be, in my opinion an answer to the multiplication of feature while supporting multiple backends. Such a feature is definitely needed. > > There's at least the Quota part the is not solved yet : how do you count > Messages duplicated in several Mailboxes ? We can count it once per mailbox, It would be correct but does not reflect your storage reality. We can attach the message blob to the ID (independently of all messages copy) thus we can pay the cost in terms of quota on the first append, and not in the face of copies. The problem with above behavior is if I append message to QuotaRoot A, then copy it to a mailbox belonging with QuotaRoot B. Delete it in quota root A. Then quotas for QuotaRoot A will still take the message into account, and not QuotaRoot B (what is not expected). Recovering from such a situation seems really hard to me. What do you think about it ? > > Maybe we missed some design issue, feel free to push this discussion > further. > - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: server/protocols vs protocols (was Re: My thoughts on JMAP implementation)
Le 14/12/2015 08:19, Matthieu Baechler a écrit : > > > On 13/12/2015 12:43, Tellier Benoit wrote: > > [...] > >> - First point, I believe /server/protocols is only here to wrap up the >> /protocols stuff. We of course should have the servlets in >> server/protocols. The responsibilities of such server/protocols are for >> me to identify commands and encode responses + directly talk to the >> client. /protocols is responsible of interacting with other James >> entities (such as repositories and mailbox project) in order to reply to >> the user commands. Such an organization allows not to be tied to jetty. >> What do you think of it? > > I'm ok with this idea, we took some shortcut while implementing this > feature but we can now make it fit the usual James split of components. > > Note that Jetty is already abstracted away. Servlet are not, but it > might be difficult to abstract when we'll need to interact with headers > in response. We'll see how to handle this. The need to hook on the transport layer is a common need seing the differents protocols implementations : - IMAP StartTls - I did the same for ManageSieve - IMAP IDLE - ... ( there might be others ) > >> This, I guess mean moving the RequestHandler, and most of the models in >> /protocols. I don't believe it would be harder than that, and allow >> protocols/jmap to become the first lib for JMAP in JAVA (kind of >> advertisement for James ? Maybe it can be helpful...) > > Not a lot of value right now but if people want to use it out of James > context, why not ? > +1 , I just wanted it to be discussed here. And to add to your point servlets are defined in javax. Maybe moving server/protocols/jmap to protocols/jmap when you are done with the implementation would be enough. - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: server/data/data-memory (was : Re: My thoughts on JMAP implementation)
In my JAMES-1618 PR I introduced an imMemory UsersRepository (as I wanted to test SieveRepository integration). Le 14/12/2015 08:45, Matthieu Baechler a écrit : > > > On 13/12/2015 12:43, Tellier Benoit wrote: > >> - Why not introducing /server/data/data-memory for your >> MemoryAccessTokenRepository ? I have upcoming commits to create the >> project, maybe moving it to data-memory would make sens. > > The only rational for not doing it yet is to avoid project > proliferation. We only have a single class in data-jmap for in-memory > implementation. > > If you create a server/data/data-memory project, we can put > MemoryAccessTokenRepository in it, that's cool. > I want to ask a really stupid question : "Is project proliferation a bad thing?" - What we pay is time to run tests and compile. Small projects that take ~ 2 s to compile do not seem that expensive. - If projects are well organized in a hierarchical structure, we are not lost, as we can redirect our selves to the right project. - We should pay attention to code duplication (more on that in my next mail ) - What is dangerous is project for backends nobody is able / volontary to maintain... My opinion. - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: james-server-util* location (was : Re: My thoughts on JMAP implementation)
Here we have an interesting topic ;-) If I remember well; backend commons was introduced to share common code across James project without introducing meaningless / cycle prone / multiples and fast cardinality growing dependencies. I happened to be on Cassandra between data-cassandra and mailbox/cassandra . Now I propose you something I encountered yesterday : - JSIEVE body extension matches message content in a similar way Message Search does. MessageSearcher (located in mailbox/store) is generic as it can be used to match text either in raw streams and mime parts. It seems logic to base JSIEVE body implementation on this tool. Well, the question is then "Where should I locate MessageSearcher ?" - In JSIEVE ? - In server/containers/utils ? - In mailbox/store ? In my opinion, it looks like the Cassandra problem... ( I did not answered this issue yet. I wanted to wait to launch this discussion, but hey, here we are... ) Note 1 : I think such problem might arise more frequently as our understanding of James existing code is growing, and might help to factorize code. Note 2 : backend-common name is not immutable... Le 14/12/2015 08:28, Matthieu Baechler a écrit : > > > On 13/12/2015 12:43, Tellier Benoit wrote: > > [...] > >> - james-server-util is amaizing, as james-server-util-java8 . I like >> the idea of using these tools broadly. But maybe these projects should >> be located in backends-common. rather than in server/container ? I find >> it makes more sens... > > backends-common was meant to contain backend related code (cassandra for > a start, but maybe hbase or any other backend can follow). > > IMO, general purpose util classes doesn't belong to backends-common. > > I agree server/container is not a great choice either, but that's where > james-server-util was located for 2+ years. > > If we take a look at server/container content : > > * cassandra-guice > * cli > * core > * filesystem-api > * jetty > * lifecycle-api > * mailbox-adapter > * spring > * util > * util-java8 > > IMO, cassandra-guice and spring match 'container' but others don't. > > I don't know how to sort this out, does someone have an idea about that ? > Most of these projects looks better here than anywhere else ( lifecycle-api, spring, and cassandra-guice). cli and mailbox-adapter should be kept together. It makes sens to me to have them either in containers, bu will also make sens in app. I would keep it like today. Doesn't jetty better belongs to protocols ? I think util and util-java8 better belongs in backend-commons. filesystem-api can be moved to data, but keeping it in containers does also makes sens. I don't know what to say about core... To sum up : ... /backeds-commons/util /backeds-commons/util-java8 ... /server/container/cassandra-guice /server/container/cli (/server/container/core) /server/container/filesystem-api /server/container/lifecycle-api /server/container/mailbox-adapter /server/container/spring ... /server/protocols/jetty What do you think about it ? - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: jmap-integration-testing (was : Re: My thoughts on JMAP implementation)
Le 14/12/2015 08:42, Matthieu Baechler a écrit : > > > On 13/12/2015 12:43, Tellier Benoit wrote: > > [...] > >>- jmap-integration-testing project is strange to me. >> >>1/ Its location. I think it has nothing to do in protocols, and >> would rather see it in a new folder. Something like >> /server/integration-tests/jmap/cassandra .(again server/protocols is, >> according to me, about decoding requests and encoding responses) > > jmap-integration-testing aims at testing jmap protocol. It makes sense > to have it next to jmap. > > About whether it should be next to server/protocols/jmap or > protocols/jmap : it's integration tests, we need a server running, so > server/protocols/jmap seems to be the best fit. Do you agree ? Yes > >>2/ I think you should make it easier for one to implement other >> backends, and separate Cassandra stuff from generalizable testing stuff. > > > To some degree, I disagree with the "everything must be generic" policy. > > On one hand, it's not a problem to put interfaces to be able to switch > implementation of various part of James and it's what we did in jmap (we > don't have any dependency on a specific backend). > > On the other hand, testing every configuration is a dead-end. Tests are > already too long to execute and cartesian product of pluggable parts is > really huge. :-) I agree, cartesian product is hard ... > > So our take is : just allow anybody to test the configuration it want > (after all, it's just a matter of splitting today > jmap-integration-testing in two parts) but don't pay upfront the cost of > having those configurations tested. +1 It makes even more sens as back-ends already a standard behavior normalized by both Mapper unit tests and IMAP MPT integration tests. (If I were you, I would even have loaded the memory mailbox to speed test up). > > Obviously it is a choice we can discuss further if anybody feels it's a > critical design decision. > - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-890) Blogging Protocols
[ https://issues.apache.org/jira/browse/JAMES-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15055894#comment-15055894 ] Pierre Smits commented on JAMES-890: There seems to be no interest within the community. At this moment in time it seems more an issue of the 'wish' type, than an improvement Better to close this, so that it doesn't clog up the list of outstanding bugs and improvements. > Blogging Protocols > -- > > Key: JAMES-890 > URL: https://issues.apache.org/jira/browse/JAMES-890 > Project: James Server > Issue Type: Wish >Affects Versions: Trunk >Reporter: Robert Burrell Donkin > > James supports a number of older messaging protocols (SMTP, NNTP, IMAP, > POP3). It lacks support for newer messaging protocols for example Atom. > I've opened this issue to help gather use cases -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
