[jira] Created: (IMAP-199) MaildirMessage should get populated lazy
MaildirMessage should get populated lazy Key: IMAP-199 URL: https://issues.apache.org/jira/browse/IMAP-199 Project: JAMES Imap Issue Type: Improvement Components: Maildir Mailbox Reporter: Norman Maurer Priority: Minor At the moment the MaildirMessage get generated by parsing the File as soon as it is created. I think we should lazy parse it when we need it. Like parsing the message to get the headers only when they are requested etc -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Proposed store api change (NEXTUID)
... and one more general point :Should the UID API oblige (or not?) each store to use the UIDNEXT from the memory ? (or Should the caching UIDNEXT mecanism be an optional utility, meaning that each store could re-implement it in a different way - cached, persistent or not ?). Tks, Eric On 17/08/2010 17:56, Eric Charles wrote: Hi Norman, I've read the http://www.rfc-editor.org/rfc/rfc3501.txt (section 2.3.1 Message Numbers) and http://www.rfc-editor.org/rfc/rfc2683.txt (section 3.4.3. UIDs and UIDVALIDITY) A first point is RFC talks about backend server not being able to store the UIDs. In this case, the UID are to be regenerated each time, with a different UIDVALIDITY, so a there are no risk to confuse mails. I had also to read twice the sentence and associated explanation It seems to be a common misunderstanding that the UIDVALIDITY and the UID, taken together, form a 64-bit identifier that uniquely identifies a message on a server . However, it is said at another place : The combination of mailbox name, UIDVALIDITY, and UID must refer to a single immutable message on that server forever. Finally, this may give as requirement that the store API should not prevent to implement a store that wouldn't be capable of storing UIDs (seems strange, but considered at numerous places in RFCs). I think the current store API already allows that ? On the UIDVALIDITY, it is now generated for example in JPA with a Math.abs(RANDOM.nextInt()). RFC states: A good UIDVALIDITY value to use in this case is a 32-bit representation of the creation date/time of the mailbox. It seems reasonable to provide utility methods such as existing randomUidValidity() to the store impl, each store having the freedom to use it or not. (was just wondering what is the difference between the imap-mailbox and imap-store projects - not always obvious at first sight to define the responsibilities of each). Coming to the UIDNEXT and as you pointed, I also understand that the returned UIDNEXT value has nothing to do with the UID that will be given to the next coming message. That value needs however to be equals or higher. I suppose the idea would be to have per mailbox a cache in memory. That cache would be used to return the UIDNEXT (that would be the current cache value), but also to assign the UID for coming mails (cache+1). I am wondering how we can ensure in case of abrupt shutdown that the last value of the cache be stored. If we can't ensure that, and this will be probably the case, there we an have a strategy to init the cache with a value recomputed from the all the stored UID (something like give me the highest value from all the UID of that mailbox). This would need an initial step when the cache is not initialized but should not be a penality for a JPA store, even for mailbox with many mails. Don't know for the other stores (jcr, maildir,...) ? Should we care on the cache time-to-live? If we don't care about that, we will have a growing memory, even if for each mailbox, we only need an Integer (that wouldn't represent much KB). But there is also the possibility to define a ttl of a few hours, with a scheduled cache manager that would cleanup things. Tks, Eric On 15/08/2010 17:25, Norman Maurer wrote: Hi there, After looking a bit over the store api again the last days I think there is some room for improvements. This improvements will break the api (again), so I think we should do it now and after that cut the 0.1 release. I will try to explain you why I think there should be some improvements made and whats my point of view. Please feel free to comment .. NEXTUID (IMAP-193): The NEXTUID generation / house-keeping is just a big performance killer. We really guaranteer to use the value of NEXTUID for the next message which will get saved. Thats not needed. We just need to guaranteer its equal or greater then the value returned by NEXTUID. So its prolly more performant to just hold the informations in memory and update it every x writes (or something like that). So the implementation could use an auto-increment field to generate the unique uid when storing the message or just an AtomicInteger for generation. Maybe again with a new abstract class called UIDKeeper ? Does this sound like something which make sense ? - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [IMAP] Proposed store api change (NEXTUID)
Eric, you are right about the UIDVALIDITY, the default shouldn't be a random number, but the current timestamp which would guarantee that it won't occur again. I also thought about checking whether the next uid would wrap the uid counter to the negative - which would mean we need to regenerate the uids and create a new UIDVALIDITY. However, it's just to improbable to have more then 9 quintillion mails arriving in a mailbox. UIDVALIDITY + UID as id or not simply means: the combination MUST never refer to any other mail - ever. If for some reason you have to regenerate the uids or do so by default, the UIDVALIDITY MUST change - making the previous uids invalid. The refer forever part is in a STRONGLY ENCOURAGES section. Caching NEXTUIDS etc sounds really complicated for a simple matter. I'll try to understand why this is such a performance blocker. It's just reading or writing a value, isn't it? Later... Regards Tim Am Dienstag, den 17.08.2010, 17:56 +0200 schrieb Eric Charles: Hi Norman, I've read the http://www.rfc-editor.org/rfc/rfc3501.txt (section 2.3.1 Message Numbers) and http://www.rfc-editor.org/rfc/rfc2683.txt (section 3.4.3. UIDs and UIDVALIDITY) A first point is RFC talks about backend server not being able to store the UIDs. In this case, the UID are to be regenerated each time, with a different UIDVALIDITY, so a there are no risk to confuse mails. I had also to read twice the sentence and associated explanation It seems to be a common misunderstanding that the UIDVALIDITY and the UID, taken together, form a 64-bit identifier that uniquely identifies a message on a server . However, it is said at another place : The combination of mailbox name, UIDVALIDITY, and UID must refer to a single immutable message on that server forever. Finally, this may give as requirement that the store API should not prevent to implement a store that wouldn't be capable of storing UIDs (seems strange, but considered at numerous places in RFCs). I think the current store API already allows that ? On the UIDVALIDITY, it is now generated for example in JPA with a Math.abs(RANDOM.nextInt()). RFC states: A good UIDVALIDITY value to use in this case is a 32-bit representation of the creation date/time of the mailbox. It seems reasonable to provide utility methods such as existing randomUidValidity() to the store impl, each store having the freedom to use it or not. (was just wondering what is the difference between the imap-mailbox and imap-store projects - not always obvious at first sight to define the responsibilities of each). Coming to the UIDNEXT and as you pointed, I also understand that the returned UIDNEXT value has nothing to do with the UID that will be given to the next coming message. That value needs however to be equals or higher. I suppose the idea would be to have per mailbox a cache in memory. That cache would be used to return the UIDNEXT (that would be the current cache value), but also to assign the UID for coming mails (cache+1). I am wondering how we can ensure in case of abrupt shutdown that the last value of the cache be stored. If we can't ensure that, and this will be probably the case, there we an have a strategy to init the cache with a value recomputed from the all the stored UID (something like give me the highest value from all the UID of that mailbox). This would need an initial step when the cache is not initialized but should not be a penality for a JPA store, even for mailbox with many mails. Don't know for the other stores (jcr, maildir,...) ? Should we care on the cache time-to-live? If we don't care about that, we will have a growing memory, even if for each mailbox, we only need an Integer (that wouldn't represent much KB). But there is also the possibility to define a ttl of a few hours, with a scheduled cache manager that would cleanup things. Tks, Eric On 15/08/2010 17:25, Norman Maurer wrote: Hi there, After looking a bit over the store api again the last days I think there is some room for improvements. This improvements will break the api (again), so I think we should do it now and after that cut the 0.1 release. I will try to explain you why I think there should be some improvements made and whats my point of view. Please feel free to comment .. NEXTUID (IMAP-193): The NEXTUID generation / house-keeping is just a big performance killer. We really guaranteer to use the value of NEXTUID for the next message which will get saved. Thats not needed. We just need to guaranteer its equal or greater then the value returned by NEXTUID. So its prolly more performant to just hold the informations in memory and update it every x writes (or something like that). So the implementation could use an auto-increment field to generate the unique uid when storing the message or just an AtomicInteger for generation. Maybe again with a new abstract
Re: [IMAP] Proposed store api change (NEXTUID)
Well, I think its not the generation of the UIDNEXT which is a performance problem. Its how it is used atm. We currently use it as uid for the next message which will get append to the mailbox. It would be more performant to use an auto_increment column in jpa for example. Other backends have other features which can match the uid generation stuff. Bye, Norman 2010/8/17 Tim-Christian Mundt d...@tim-erwin.de: Eric, you are right about the UIDVALIDITY, the default shouldn't be a random number, but the current timestamp which would guarantee that it won't occur again. I also thought about checking whether the next uid would wrap the uid counter to the negative - which would mean we need to regenerate the uids and create a new UIDVALIDITY. However, it's just to improbable to have more then 9 quintillion mails arriving in a mailbox. UIDVALIDITY + UID as id or not simply means: the combination MUST never refer to any other mail - ever. If for some reason you have to regenerate the uids or do so by default, the UIDVALIDITY MUST change - making the previous uids invalid. The refer forever part is in a STRONGLY ENCOURAGES section. Caching NEXTUIDS etc sounds really complicated for a simple matter. I'll try to understand why this is such a performance blocker. It's just reading or writing a value, isn't it? Later... Regards Tim Am Dienstag, den 17.08.2010, 17:56 +0200 schrieb Eric Charles: Hi Norman, I've read the http://www.rfc-editor.org/rfc/rfc3501.txt (section 2.3.1 Message Numbers) and http://www.rfc-editor.org/rfc/rfc2683.txt (section 3.4.3. UIDs and UIDVALIDITY) A first point is RFC talks about backend server not being able to store the UIDs. In this case, the UID are to be regenerated each time, with a different UIDVALIDITY, so a there are no risk to confuse mails. I had also to read twice the sentence and associated explanation It seems to be a common misunderstanding that the UIDVALIDITY and the UID, taken together, form a 64-bit identifier that uniquely identifies a message on a server . However, it is said at another place : The combination of mailbox name, UIDVALIDITY, and UID must refer to a single immutable message on that server forever. Finally, this may give as requirement that the store API should not prevent to implement a store that wouldn't be capable of storing UIDs (seems strange, but considered at numerous places in RFCs). I think the current store API already allows that ? On the UIDVALIDITY, it is now generated for example in JPA with a Math.abs(RANDOM.nextInt()). RFC states: A good UIDVALIDITY value to use in this case is a 32-bit representation of the creation date/time of the mailbox. It seems reasonable to provide utility methods such as existing randomUidValidity() to the store impl, each store having the freedom to use it or not. (was just wondering what is the difference between the imap-mailbox and imap-store projects - not always obvious at first sight to define the responsibilities of each). Coming to the UIDNEXT and as you pointed, I also understand that the returned UIDNEXT value has nothing to do with the UID that will be given to the next coming message. That value needs however to be equals or higher. I suppose the idea would be to have per mailbox a cache in memory. That cache would be used to return the UIDNEXT (that would be the current cache value), but also to assign the UID for coming mails (cache+1). I am wondering how we can ensure in case of abrupt shutdown that the last value of the cache be stored. If we can't ensure that, and this will be probably the case, there we an have a strategy to init the cache with a value recomputed from the all the stored UID (something like give me the highest value from all the UID of that mailbox). This would need an initial step when the cache is not initialized but should not be a penality for a JPA store, even for mailbox with many mails. Don't know for the other stores (jcr, maildir,...) ? Should we care on the cache time-to-live? If we don't care about that, we will have a growing memory, even if for each mailbox, we only need an Integer (that wouldn't represent much KB). But there is also the possibility to define a ttl of a few hours, with a scheduled cache manager that would cleanup things. Tks, Eric On 15/08/2010 17:25, Norman Maurer wrote: Hi there, After looking a bit over the store api again the last days I think there is some room for improvements. This improvements will break the api (again), so I think we should do it now and after that cut the 0.1 release. I will try to explain you why I think there should be some improvements made and whats my point of view. Please feel free to comment .. NEXTUID (IMAP-193): The NEXTUID generation / house-keeping is just a big performance killer. We really guaranteer to use the value of NEXTUID for the next message which will get saved. Thats not needed. We just need
[jira] Created: (IMAP-200) NameSpaceResponseEncoder need to append delimiter on response
NameSpaceResponseEncoder need to append delimiter on response - Key: IMAP-200 URL: https://issues.apache.org/jira/browse/IMAP-200 Project: JAMES Imap Issue Type: Bug Components: Protocol Reporter: Norman Maurer Assignee: Norman Maurer Fix For: 0.1 If the Namespace contains a char it need to append the delimiter to the namespace. See: http://www.ietf.org/rfc/rfc2342.txt Example 5.5: === A server that supports only the Personal Namespace, with a leading prefix of INBOX to personal mailboxes and a hierarchy delimiter of . C: A001 NAMESPACE S: * NAMESPACE ((INBOX. .)) NIL NIL S: A001 OK NAMESPACE command completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
svn commit: r986474 - /james/imap/trunk/message/src/main/java/org/apache/james/imap/encode/NamespaceResponseEncoder.java
Author: norman
Date: Tue Aug 17 20:22:26 2010
New Revision: 986474
URL: http://svn.apache.org/viewvc?rev=986474view=rev
Log:
Append delimiter to Namespace if Namespace length is bigger then 0 (IMAP-200)
Modified:
james/imap/trunk/message/src/main/java/org/apache/james/imap/encode/NamespaceResponseEncoder.java
Modified:
james/imap/trunk/message/src/main/java/org/apache/james/imap/encode/NamespaceResponseEncoder.java
URL:
http://svn.apache.org/viewvc/james/imap/trunk/message/src/main/java/org/apache/james/imap/encode/NamespaceResponseEncoder.java?rev=986474r1=986473r2=986474view=diff
==
---
james/imap/trunk/message/src/main/java/org/apache/james/imap/encode/NamespaceResponseEncoder.java
(original)
+++
james/imap/trunk/message/src/main/java/org/apache/james/imap/encode/NamespaceResponseEncoder.java
Tue Aug 17 20:22:26 2010
@@ -72,8 +72,14 @@ public class NamespaceResponseEncoder ex
private void encode(Namespace namespace, ImapResponseComposer composer)
throws IOException {
composer.openParen();
-composer.quote(namespace.getPrefix());
-composer.quote(Character.toString(namespace.getDelimiter()));
+String prefix = namespace.getPrefix();
+String delimiter = Character.toString(namespace.getDelimiter());
+
+if (prefix.length() 0) {
+prefix = prefix + delimiter;
+}
+composer.quote(prefix);
+composer.quote(delimiter);
composer.closeParen();
}
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
