Author: btellier Date: Sat Nov 28 13:12:02 2015 New Revision: 1716969 URL: http://svn.apache.org/viewvc?rev=1716969&view=rev Log: MAILBOX-211 Event system documentation
Added: james/project/trunk/server/src/site/xdoc/config-events.xml Modified: james/project/trunk/server/src/site/xdoc/config.xml Added: james/project/trunk/server/src/site/xdoc/config-events.xml URL: http://svn.apache.org/viewvc/james/project/trunk/server/src/site/xdoc/config-events.xml?rev=1716969&view=auto ============================================================================== --- james/project/trunk/server/src/site/xdoc/config-events.xml (added) +++ james/project/trunk/server/src/site/xdoc/config-events.xml Sat Nov 28 13:12:02 2015 @@ -0,0 +1,154 @@ +<?xml version="1.0"?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> +<document> + + <properties> + <title>Apache James Server 3 - Quota Configuration</title> + </properties> + + <body> + + <section name="Events System Configuration"> + + <p>Consult <a href="http://svn.apache.org/repos/asf/james/server/trunk/app/src/main/resources/events-template.xml">events-template.xml</a> in SVN to get some examples and hints.</p> + + <p>Use this configuration to define the type of Event System you want.</p> + + <p> + James relies on an event system. Each operations performed on the mailbox will trigger related events. Some software + components (MailboxListeners) can register themselves on this event system to be called when an event is fired. + </p> + + <p> + Here are typical use cases for Mailbox Listeners (non exhaustive list) : + <ul> + <li>Message search indexation, for instance in Lucene or ElasticSearch</li> + <li>Local cache invalidation (caching mailbox project)</li> + <li>Quota calculation</li> + <li>IMAP IDLE feature : live notification of actions performed on a mailbox, allowing publish subscribe on mailboxes events</li> + <li>Message Sequence Number consistence</li> + </ul> + The Mailbox Listeners can be classified in two categories : + <ul> + <li>Mailbox registered : The mailbox listener is only notified on events affecting this mailbox. IDLE is a good example of this.</li> + <li>Global Listeners : This event listener is triggered upon each events.</li> + </ul> + Note that Global Listeners can also be classified in two categories : + <ul> + <li>Those which needs to be triggered only once in your cluster. For instance ElasticSearch indexing is an example of this.</li> + <li>Those which needs to be triggered on each servers. For instance, each Lucene indexer needs to be triggered on each server + for the search feature to stay consistent.</li> + </ul> + </p> + + <p> + The default implementation is a synchronous in memory event system. The performance are really good, as their is no need to serialize + events, and no network overhead. However, this event system is limited to one computer and you might want a distributed systems. + </p> + + <p> + Other implementations, distributed environment friendly are available. + </p> + + <p> + The simplest one is broadcast based. Each James servers listen the same message queue, and each James server will be notified upon events. + Here are the pros and cons of this implementations : + <ul> + Pros: + <li>It supports every type of listener described above</li> + <li>It allows you to scale your James infrastructure without changing your middlewares. You just need a message queue</li> + Cons : + <li>Your scalability is limited as each servers is notified on all events</li> + <li>Network overhead on event transmissions</li> + <li>Event serialization and deserialization</li> + </ul> + To use this implementation, you need two other components (that will be discussed) : a publishing system and an event serializer + </p> + + <p> + The other mode is based on registrations. + Each server reads messages from a dedicated message queue, and other servers send messages addressed to this sever on this message queue. + Registrations are performed on an eternal data-store supporting document deletion after a fixed amount of time. + These registrations are periodically refreshed. This data-store is then triggered on event generation, that, if needed are serialized and + send to the given queues. + The pros and cons of this implementations are : + <ul> + Pros : + <li>Linear scalability</li> + <li>A server receives only events concerning him</li> + Cons : + <li>Possible event serialization costs</li> + <li>Registration and registration refresh costs</li> + <li>Need to find interested servers on event delivery</li> + <li>Network overhead on event transmissions</li> + </ul> + </p> + + <h2>Failure modes</h2> + + <ul> + Default implementation : + <li>The default implementation might not deliver some events on server stop.</li> + </ul> + + <ul> + Broadcast implementation : + <li>The broadcast implementation might not deliver some events on server stop.</li> + <li>The broadcast implementation is tight to limitation of the underlying publisher.</li> + </ul> + + <ul> + Registered implementation : + <li>The registered implementation might not deliver some events on server stop.</li> + <li>The registered implementation is tight to limitation of the underlying publisher, and underlying registration system.</li> + </ul> + + <h2>Publisher</h2> + + <p> + Available implementation is Kafka based. Kafka ensure at least one delivery. This means some messages might be + delivered two times. You need to compile and run James using Java 8 in order to use the Kafka messaging system. + </p> + + <h2>Event serializer</h2> + + <p>There are two types of event serialization systems : + <ul> + <li>Json : events are converted to JSON</li> + <li>Message Pack : a binary representation of JSON. 2 times smaller in average but two times longer to compute. It allows you to trade + bandwidth and data readability against CPU time.</li> + </ul> + </p> + + <h2>Registration systems</h2> + + <p> + Available implementation is based on Cassandra. It is used on an AP fashion, enforcing availability instead of consistency. Some + messages might get delivered to no more registered servers. This is just extra work. Worst, messages might not be delivered to + recently registered servers. But we make sure that we have the more up to date version of the registrations we can, and will not time out in + the face of network partitions, nor enforce some default behaviour. + </p> + + </section> + + </body> + +</document> + Modified: james/project/trunk/server/src/site/xdoc/config.xml URL: http://svn.apache.org/viewvc/james/project/trunk/server/src/site/xdoc/config.xml?rev=1716969&r1=1716968&r2=1716969&view=diff ============================================================================== --- james/project/trunk/server/src/site/xdoc/config.xml (original) +++ james/project/trunk/server/src/site/xdoc/config.xml Sat Nov 28 13:12:02 2015 @@ -70,6 +70,11 @@ <td></td> </tr> <tr> + <td><a href="http://svn.apache.org/repos/asf/james/server/trunk/app/src/main/resources/events-template.xml">events.xml</a></td> + <td><a href="config-events.html">Event system Configuration</a></td> + <td></td> + </tr> + <tr> <td><a href="http://svn.apache.org/repos/asf/james/server/trunk/app/src/main/resources/mailrepositorystore-template.xml">mailrepositorystore.xml</a></td> <td><a href="config-mailrepositorystore.html">Mail Repository Stores Configuration</a></td> <td></td> --------------------------------------------------------------------- To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org