Thanks for this!
I am translating this in my head this way:
1. Mail Receiver receives a mail and queues it in ActiveMQ.
2. Mail Processor has multiple consumer threads watching the ActiveM queue.
3. If one mailbox write operation blocks for 5 minutes, the other consumer
threads of the Mail Processor will continue.
Is this roughly correct?
Is this relevant or is the spool outgoing: (below xml of
mailetcontainer-template.conf )
<!-- This block is responsible for processing messages on the spool. -->
<mailetcontainer enableJmx="true">
<!-- MailAddress used for PostMaster -->
<context>
<postmaster>postmaster@localhost</postmaster>
</context>
<!-- Number of spool threads -->
<spooler>
<threads>20</threads>
</spooler>
Thanks again for your time!
-tim
On Jun 6, 2012, at 8:43 AM, Eric Charles wrote:
> Hi Tim,
>
> 1. James doesn't block on connection (but don't forget the intermediate
> spooling via ActiveMQ).
>
> 2. To answer this question, you need to realize James reads the mails via a
> Camel context, getting them from ActiveMQ.
>
> Sorry if these response are too short, I was just wanting to take more time
> to provide you with a complete picture, but I understand you need to go
> forward.
>
> What you can do it put a break point in the append method, and send mails to
> James (running in your IDE). You will see the various threads and also that
> it is non-blocking.... exception made of the various lock with do via the
> JVMMailboxPathLocker, see for example this snippet from StoreMessageManager
>
> final Message<Id> message = createMessage(internalDate, size,
> bodyStartOctet, contentIn, flags, propertyBuilder);
> return locker.executeWithLock(mailboxSession, new
> StoreMailboxPath<Id>(getMailboxEntity()), new
> MailboxPathLocker.LockAwareExecution<Long>() {
>
> @Override
> public Long execute() throws MailboxException {
> MessageMetaData data = appendMessageToStore(message,
> mailboxSession);
>
> SortedMap<Long, MessageMetaData> uids = new TreeMap<Long,
> MessageMetaData>();
> uids.put(data.getUid(), data);
> dispatcher.added(mailboxSession, uids, getMailboxEntity());
> return data.getUid();
> }
> }, true);
>
>
> This being said, you will also need to take care of the various
> locking/concurrent access depending on the technology you use to store your
> mails (database, file, jcr,...).
>
> I don't have a Yes/No answer, it must be looked and analysed in context.
>
> Happy to further discuss,
>
> Eric
>
>
>
> On 06/06/2012 02:24 PM, Timothy Prepscius wrote:
>> Ok, I'm sorry to spam about this question, but I would really like a
>> response.
>> I am making assumptions, which may be incorrect.
>> Because of maven it is difficult for me to get a full source tree to read.
>>
>>
>> 1. Does james block on a single connection?
>> (I assume no, because this would be just be crazy)
>>
>> 2. Does james block on a transaction within a connection, meaning server X
>> talks to you on a single connection, transmits M1, M2, M3.
>> Does james receive M1, queue a transaction in a different thread, and
>> continue on to M2, -OR- does it receive M1, block on transaction processor,
>> and then continue onto M2?
>>
>> (I hope doesn't block, if so I should implement a thread pool for the
>> mailbox-write.)
>>
>>
>> Thanks,
>>
>> -tim
>>
>> On Jun 4, 2012, at 9:28 AM, Timothy Prepscius wrote:
>>
>>>
>>> I got rid of the NPE on java7 instead of switching to java6. So no
>>> problems there anymore.
>>> I will investigate how to do the jira in a minute.
>>>
>>>
>>> I have (perhaps) only one more question:
>>>
>>> Should I implement the Mailbox as a multi-threaded non blocking pool?
>>> (Does James block the entire server on a Mailbox write?)
>>>
>>> Or should I implement the Mailbox as a single-threaded blocking write
>>> mechanism?
>>> (Is there somewhere in James a thread pool handling requests from the smtp
>>> socket?)
>>>
>>>
>>> Thanks,
>>>
>>> -tim
>>>
>>> On Jun 4, 2012, at 4:49 AM, Eric Charles wrote:
>>>
>>>> Hi Timothy,
>>>>
>>>> 1.- I have committed your patch [1], thx for this.
>>>> 2.- Do you still have NPE with JDK6. If yes, can you post here the full
>>>> stacktrace?
>>>> 3.- Please open a JIRA [2] for your proposal for getFullContent() and
>>>> explain there in a few words why it would be more optimal.
>>>>
>>>> Thx, Eric
>>>>
>>>> [1] http://svn.apache.org/viewvc?rev=1345862&view=rev
>>>> [2] https://issues.apache.org/jira/browse/MAILBOX
>>>>
>>>>
>>>> On 06/04/2012 05:00 AM, Timothy Prepscius wrote:
>>>>>
>>>>> On Jun 3, 2012, at 10:59 PM, Timothy Prepscius wrote:
>>>>>
>>>>>> Sorry, hit the wrong button I guess.
>>>>>>
>>>>>> On Jun 3, 2012, at 10:40 PM, Eric Charles wrote:
>>>>>>
>>>>>>> Hi Timothy,
>>>>>>> Can you post to the mailing list so everyone can comment on this?
>>>>>>> Thx, Eric
>>>>>>>
>>>>>>> On 06/03/2012 07:23 PM, Timothy Prepscius wrote:
>>>>>>>> hey, could I also offer one code change, which will not conflict with
>>>>>>>> anything anywhere (I think) ?
>>>>>>>>
>>>>>>>> It optimizes the getFullContent for the SimpleMessage
>>>>>>>>
>>>>>>>> ---
>>>>>>>>
>>>>>>>> tprepscius$ svn diff
>>>>>>>> Index:
>>>>>>>> src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
>>>>>>>> ===================================================================
>>>>>>>> ---
>>>>>>>> src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
>>>>>>>> (revision 1345622)
>>>>>>>> +++
>>>>>>>> src/main/java/org/apache/james/mailbox/store/mail/model/impl/SimpleMessage.java
>>>>>>>> (working copy)
>>>>>>>> @@ -96,6 +96,10 @@
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> + @Override
>>>>>>>> + public InputStream getFullContent() throws IOException {
>>>>>>>> + return content.newStream(0, -1);
>>>>>>>> + }
>>>>>>>>
>>>>>>>> public Date getInternalDate() {
>>>>>>>> return internalDate;
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jun 3, 2012, at 1:14 PM, Timothy Prepscius wrote:
>>>>>>>>
>>>>>>>>> hmm, I just added httpclient-4.0.2.jar to the class path and got the
>>>>>>>>> same error.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> a second ago I unzipped the httpclient jar
>>>>>>>>> removed the entire META-INF/* dir tree
>>>>>>>>> and zipped it up again manually and now it works.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> hmmm
>>>>>>>>>
>>>>>>>>> not sure what that means :-)
>>>>>>>>>
>>>>>>>>> -tim
>>>>>>>>>
>>>>>>>>> On Jun 3, 2012, at 1:03 PM, Timothy Prepscius wrote:
>>>>>>>>>
>>>>>>>>>> vanilla is fine, no prob
>>>>>>>>>>
>>>>>>>>>> it is indeed 1.7
>>>>>>>>>>
>>>>>>>>>> tprepscius@blue:~/Projects/James/apache-james/bin$ java -version
>>>>>>>>>> java version "1.7.0_03"
>>>>>>>>>> OpenJDK Runtime Environment (IcedTea7 2.1.1pre)
>>>>>>>>>> (7~u3-2.1.1~pre1-1ubuntu3)
>>>>>>>>>> OpenJDK Client VM (build 22.0-b10, mixed mode, sharing)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Jun 3, 2012, at 12:47 PM, Eric Charles wrote:
>>>>>>>>>>
>>>>>>>>>>> Thx Timothy, I will commit your patch in trunk tomorrow.
>>>>>>>>>>>
>>>>>>>>>>> For NPE, do you run fine vanilla james? We had such NPE when
>>>>>>>>>>> running with jdk7, although it was running fine with JDK6.
>>>>>>>>>>> Thx, Eric
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 06/03/2012 03:28 PM, Timothy Prepscius wrote:
>>>>>>>>>>>> yah:
>>>>>>>>>>>>
>>>>>>>>>>>> <constructor-arg index="0" ref="memory-sessionMapperFactory"/>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hey- I have a question:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> So I used another mail package to test my ideas, called, JES-2,
>>>>>>>>>>>> and now I'm moving to James for production.
>>>>>>>>>>>>
>>>>>>>>>>>> I want to create a new james-mailbox.
>>>>>>>>>>>>
>>>>>>>>>>>> I struggled for many many hours doing this and hit the same road
>>>>>>>>>>>> block over and over.
>>>>>>>>>>>> Unfortunately it is a configuration problem, not a coding problem,
>>>>>>>>>>>> so it becomes an indeterminate time problem- which isn't fun ;-)
>>>>>>>>>>>>
>>>>>>>>>>>> Here is what I did:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. tar/untar clone the james-mailbox/memory-mailbox to
>>>>>>>>>>>> james-mailbox/custom-mailbox
>>>>>>>>>>>>
>>>>>>>>>>>> 2. rename the java packages and class names to
>>>>>>>>>>>> xxx.customer.CustomerBlahBlahBlah.
>>>>>>>>>>>> (things compile and have different names/packages than the memory)
>>>>>>>>>>>>
>>>>>>>>>>>> 3. cloned the spring-mailbox-memory.xml to
>>>>>>>>>>>> spring-mailbox-custom.xml, changed names inside to reflect names
>>>>>>>>>>>> of #2
>>>>>>>>>>>>
>>>>>>>>>>>> 4. modified the spring-mailbox.xml to include this new
>>>>>>>>>>>> spring-mailbox-custom.xml
>>>>>>>>>>>>
>>>>>>>>>>>> ---------
>>>>>>>>>>>>
>>>>>>>>>>>> 5. mvn'd the code, shut down the mail server, transferred the
>>>>>>>>>>>> jars for mailbox-custom and the modified mailbox-spring.
>>>>>>>>>>>>
>>>>>>>>>>>> (at other points I was transferring everything I could find, but
>>>>>>>>>>>> it made no difference)
>>>>>>>>>>>>
>>>>>>>>>>>> 6. modified the run.sh to include the jar in the class path.
>>>>>>>>>>>>
>>>>>>>>>>>> 7. ran sudo ./run.sh
>>>>>>>>>>>>
>>>>>>>>>>>> ----------
>>>>>>>>>>>>
>>>>>>>>>>>> Error is something like:
>>>>>>>>>>>>
>>>>>>>>>>>> a NullPointerException when parsing XML for spring-server.xml.
>>>>>>>>>>>> with a huge stack trace which doesn't yield too much interesting
>>>>>>>>>>>> (to my eyes) except that I know it is init-ing.
>>>>>>>>>>>>
>>>>>>>>>>>> I can't remember where that XML file is. I found it yesterday,
>>>>>>>>>>>> but now I've forgotten where it is.
>>>>>>>>>>>> Anyhow, when I found it I noticed that it had an import to the
>>>>>>>>>>>> spring-mailbox.xml…
>>>>>>>>>>>>
>>>>>>>>>>>> But I've unzipped those jars I transferred, and the
>>>>>>>>>>>> spring-mailbox.xml seems to be there.
>>>>>>>>>>>>
>>>>>>>>>>>> ----
>>>>>>>>>>>>
>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> Is the guy/girl who wrote the hbase on this list?
>>>>>>>>>>>> Do you have any notes on the process of integrating that mailbox
>>>>>>>>>>>> into james?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> (just noticed my writing is incredibly bad this morning, couldn't
>>>>>>>>>>>> sleep, apologize if I'm writing like a 2 year old.)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Jun 3, 2012, at 3:17 AM, Eric Charles wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Tim,
>>>>>>>>>>>>> Thx for the feedback.
>>>>>>>>>>>>> Any patch to correct this?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thx, Eric
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 06/03/2012 08:06 AM, Timothy Prepscius wrote:
>>>>>>>>>>>>>> http://svn.apache.org/repos/asf/james/mailbox/trunk/spring/src/main/resources/META-INF/org/apache/james/spring-mailbox-memory.xml
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <constructor-arg index="0"
>>>>>>>>>>>>>> ref="maildir-sessionMapperFactory"/>
>>>>>>>>>>>>>> might be wrong.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -tim
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>>>>>>>>
>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>>>>>>>>>>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>>>>>>>>>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>>>
>>>>
>>>> --
>>>> eric | http://about.echarles.net | @echarles
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>
>
> --
> eric | http://about.echarles.net | @echarles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
