... and one more general point :Should the UID API oblige (or not?)
each store to use the UIDNEXT from the memory ?
(or "Should the caching UIDNEXT mecanism be an optional utility, meaning
that each store could re-implement it in a different way - cached,
persistent or not ?").
Tks,
Eric
On 17/08/2010 17:56, Eric Charles wrote:
Hi Norman,
I've read the http://www.rfc-editor.org/rfc/rfc3501.txt (section 2.3.1
Message Numbers) and http://www.rfc-editor.org/rfc/rfc2683.txt
(section 3.4.3. UIDs and UIDVALIDITY)
A first point is RFC talks about backend server not being able to
store the UIDs. In this case, the UID are to be regenerated each time,
with a different UIDVALIDITY, so a there are no risk to confuse mails.
I had also to read twice the sentence and associated explanation "It
seems to be a common misunderstanding that "the UIDVALIDITY and the
UID, taken together, form a 64-bit identifier that uniquely identifies
a message on a server" ." However, it is said at another place : "The
combination of mailbox name, UIDVALIDITY, and UID must refer to a
single immutable message on that server forever".
Finally, this may give as requirement that the store API should not
prevent to implement a store that wouldn't be capable of storing UIDs
(seems strange, but considered at numerous places in RFCs).
I think the current store API already allows that ?
On the UIDVALIDITY, it is now generated for example in JPA with a
Math.abs(RANDOM.nextInt()). RFC states: "A good UIDVALIDITY value to
use in this case is a 32-bit representation of the creation date/time
of the mailbox".
It seems reasonable to provide utility methods such as existing
randomUidValidity() to the store impl, each store having the freedom
to use it or not.
(was just wondering what is the difference between the imap-mailbox
and imap-store projects - not always obvious at first sight to define
the responsibilities of each).
Coming to the UIDNEXT and as you pointed, I also understand that the
returned UIDNEXT value has nothing to do with the UID that will be
given to the next coming message. That value needs however to be
equals or higher.
I suppose the idea would be to have per mailbox a cache in memory.
That cache would be used to return the UIDNEXT (that would be the
current cache value), but also to assign the UID for coming mails
(cache+1).
I am wondering how we can ensure in case of abrupt shutdown that the
last value of the cache be stored.
If we can't ensure that, and this will be probably the case, there we
an have a strategy to init the cache with a value recomputed from the
all the stored UID (something like "give me the highest value from all
the UID of that mailbox).
This would need an initial step when the cache is not initialized but
should not be a penality for a JPA store, even for mailbox with many
mails. Don't know for the other stores (jcr, maildir,...) ?
Should we care on the cache time-to-live? If we don't care about that,
we will have a growing memory, even if for each mailbox, we only need
an Integer (that wouldn't represent much KB). But there is also the
possibility to define a ttl of a few hours, with a scheduled cache
manager that would cleanup things.
Tks,
Eric
On 15/08/2010 17:25, Norman Maurer wrote:
Hi there,
After looking a bit over the store api again the last days I think
there is some room for improvements. This improvements will break the
api (again), so I think we should do it now and after that cut the 0.1
release. I will try to explain you why I think there should be some
improvements made and whats my point of view. Please feel free to
comment ..
NEXTUID (IMAP-193):
The NEXTUID generation / house-keeping is just a big performance
killer. We really guaranteer to use the value of NEXTUID for the next
message which will get saved. Thats not needed. We just need to
guaranteer its equal or greater then the value returned by NEXTUID. So
its prolly more performant to just hold the informations in memory and
update it every x writes (or something like that). So the
implementation could use an auto-increment field to generate the
unique uid when storing the message or just an AtomicInteger for
generation. Maybe again with a new abstract class called UIDKeeper ?
Does this sound like something which make sense ?
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
