svn commit: r928611 - /james/server/trunk/spring-deployment/src/main/config/james/log4j.properties
Author: norman Date: Mon Mar 29 06:52:15 2010 New Revision: 928611 URL: http://svn.apache.org/viewvc?rev=928611view=rev Log: Add some logging for jackrabbit Modified: james/server/trunk/spring-deployment/src/main/config/james/log4j.properties Modified: james/server/trunk/spring-deployment/src/main/config/james/log4j.properties URL: http://svn.apache.org/viewvc/james/server/trunk/spring-deployment/src/main/config/james/log4j.properties?rev=928611r1=928610r2=928611view=diff == --- james/server/trunk/spring-deployment/src/main/config/james/log4j.properties (original) +++ james/server/trunk/spring-deployment/src/main/config/james/log4j.properties Mon Mar 29 06:52:15 2010 @@ -129,6 +129,9 @@ log4j.appender.DOMAINLIST.layout.Convers +# logger for jackrabbit +log4j.logger.org.apache.jackrabbit=ERROR, CONS, FILE + # logger for activemq log4j.logger.org.apache.xbean.spring=INFO, CONS, FILE log4j.logger.org.apache.activemq=INFO, CONS, FILE - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
svn commit: r928612 - /james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/StatCmdHandler.java
Author: norman
Date: Mon Mar 29 06:52:44 2010
New Revision: 928612
URL: http://svn.apache.org/viewvc?rev=928612view=rev
Log:
Fix stat command, which was failing on empty inbox. This effected thunderbird
sometimes.. (Thx Eric for the pointer)
Modified:
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/StatCmdHandler.java
Modified:
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/StatCmdHandler.java
URL:
http://svn.apache.org/viewvc/james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/StatCmdHandler.java?rev=928612r1=928611r2=928612view=diff
==
---
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/StatCmdHandler.java
(original)
+++
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/StatCmdHandler.java
Mon Mar 29 06:52:44 2010
@@ -58,19 +58,21 @@ public class StatCmdHandler implements C
try {
ListLong uidList = (ListLong)
session.getState().get(POP3Session.UID_LIST);
ListLong deletedUidList = (ListLong)
session.getState().get(POP3Session.DELETED_UID_LIST);
-
-MailboxSession mailboxSession = (MailboxSession)
session.getState().get(POP3Session.MAILBOX_SESSION);
- IteratorMessageResult results =
session.getUserMailbox().getMessages(MessageRange.range(uidList.get(0),
uidList.get(uidList.size() -1)), new FetchGroupImpl(FetchGroup.MINIMAL),
mailboxSession);
-
long size = 0;
int count = 0;
- ListMessageResult validResults = new
ArrayListMessageResult();
-while (results.hasNext()) {
- MessageResult result = results.next();
-if (deletedUidList.contains(result.getUid()) == false) {
-size += result.getSize();
-count++;
-validResults.add(result);
+if (uidList.isEmpty() == false) {
+MailboxSession mailboxSession = (MailboxSession)
session.getState().get(POP3Session.MAILBOX_SESSION);
+IteratorMessageResult results =
session.getUserMailbox().getMessages(MessageRange.range(uidList.get(0),
uidList.get(uidList.size() -1)), new FetchGroupImpl(FetchGroup.MINIMAL),
mailboxSession);
+
+
+ListMessageResult validResults = new
ArrayListMessageResult();
+while (results.hasNext()) {
+MessageResult result = results.next();
+if (deletedUidList.contains(result.getUid()) == false)
{
+size += result.getSize();
+count++;
+validResults.add(result);
+}
}
}
StringBuilder responseBuffer =
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [PROPOSAL] Replace MailRepository with Mailbox / MailboxManager for POP3Server ...
Hi Eric, I was able to reproduce your problem and just checked in the fix. Please svn up and try again.. Thx for your feedback. Bye, Norman 2010/3/29 Norman Maurer nor...@apache.org: Hi Eric, I just tried to acces an empty inbox via thunderbird (pop3) here. It worked without any probs. Im using windows 7 and thunderbird 3.0.3 Bye, Norman 2010/3/28 Norman Maurer nor...@apache.org: -- Forwarded message -- From: Norman Maurer nor...@apache.org Date: 2010/3/28 Subject: Re: [PROPOSAL] Replace MailRepository with Mailbox / MailboxManager for POP3Server ... To: Eric Charles eric.char...@u-mangate.com Hi Eric, could you maybe use wireshark or something similar to log what thunderbird tries todo exactly.. ? Thx, Norman 2010/3/28 Eric Charles eric.char...@u-mangate.com: Hi Norman, I just made the following tests: - Fresh checkout from trunk and mvn package (some new dependencies were downloaded such as jsieve, mailet-sdk,...) = same problem on a local deployment with thunderbird : it hangs when mailbox is empty. - I after defined my production account on evolution (http://www.gnome.org/projects/evolution) : I didn't have the problem anymore, but really strange, evolution downloaded some messages from yesterday. Those messages had been deleted from my inbox via thunderbird, but was retrieved via evolution... - I tested with evolution the local deployment I just made : I can read the inbox even if empty. The problem seems to reside on the thunderbird/james conversation when inbox is empty. Tks, Eric On 03/27/2010 10:08 PM, Norman Maurer wrote: Hi Eric, I just tested pop3 with imail and no messages in inbox and everything just works. Will try to test it with thunderbird later.. Bye, Norman 2010/3/27 Norman Maurernor...@apache.org: Hi Eric, I will try to find out whats wrong with the pop3 inbox stuff. If you find out anything else please feel free to just report it.. Thx, Norman 2010/3/27 Eric Charleseric.char...@u-mangate.com: Hi Norman, Many tks for your quick answer. I had defined in my Thunderbird 3.0.3 client (on linux) the pop3 account as the default one. When launching thunderbird, the connection was succefully done Host contacted, sending login information, saying after 1 second Determining which messages to index (never saw that message before...), hanging there a few minutes, and finally saying connection to server ... timed out. After that, Imap account behaved also such a way... I also tried with latest Thundrebird 3.0.3: same result... I have now defined the imap account as the default one, and it's working fine even without any mail in the inbox. However, for the pop3 account, I still have the same behaviour. Anyway, I have a temporary workaround right now. I will try to debug it but it may take some time to jump into the new architecture (I hacked james 3 years ago, but now we have some new great stuff such as jpa, activemq, jackrabbit,...). Regarding the OOM, I saw some days ago the JAMES-981 regarding the ConsumerTemplate. However, the problem was still present even after you commited JAMES-981 into trunk, so I don't think the problem was coming from that. I tried to analyse the 2GB heap dump with eclipse memory analyser, but it failed (too much swapping, my RAM is only 1GB). Currently, the process memory with the new trunk version I deployed today seems stable. I will keep posted if I still have the issue, or if finally, the memory remains stable. Tks a lot and keep up the excellent work, Eric On 01/-10/-28163 08:59 PM, Norman Maurer wrote: Hi Eric, thx for the feedback . Does this happens with pop3 or imap ? Or with both? What client you use? About the OOM Exception, chances are good that this is fixed now. I did something bad with ConsumerTemplate in RemoteDelivery nailet, which could lead to such problems. Bye Norman 2010/3/27, Eric Charleseric.char...@u-mangate.com: Great Norman, I just upgraded to the current trunk to benefit from : - JAMES-983: common POP3/IMAP MailRepository. - JAMES-984: activemq 5.3.1 hoping it will resolve some OutOfMemory exceptions I had (*) I deployed and now (configured with enableVirtualHosting=true and JDBCDomainList), I've got the following: - When my mailbox contains at least one mail, No problem: I can read it without any problem. - When my mailbox is empty, my mail client simply waits and finally give me a timeout (closing my client and launching it again with mails in the inbox, everything is fine again). Can anybody else reproduce it? Is there additional specific configuration I have to apply ? Tks a lot, Eric (*) Just for information, about the OutOfMemory I had with activemq 3.5.0: - Testing different configurations from http://activemq.apache.org/javalangoutofmemory.html didn't help. - Stressing a local deployment with apache-sever-postage, server was holding the
svn commit: r928614 - /james/server/trunk/spring-deployment/src/main/config/james/lib/
Author: norman Date: Mon Mar 29 07:02:04 2010 New Revision: 928614 URL: http://svn.apache.org/viewvc?rev=928614view=rev Log: Thats not true for current trunk Removed: james/server/trunk/spring-deployment/src/main/config/james/lib/ - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
JAMES Server Nightly Build Report
An automated nightly build of JAMES has been posted to http://people.apache.org/builds/james/nightly/ Any unit test errors from the build should be reported below: BUILD SUCCESSFUL Total time: 4 minutes 38 seconds Done. Packages Will Be Uploaded To The Nightly Repository. Please review http://people.apache.org/builds/james/nightly/BUILD.log. - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] Created: (IMAP-125) Add support for UIDPLUS
Add support for UIDPLUS --- Key: IMAP-125 URL: https://issues.apache.org/jira/browse/IMAP-125 Project: JAMES Imap Issue Type: New Feature Reporter: Norman Maurer Add support for UIDPLUS extension. See http://www.rfc-archive.org/getrfc.php?rfc=2359 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
svn commit: r928673 - /james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/DKIMVerifier.java
Author: bago
Date: Mon Mar 29 10:02:12 2010
New Revision: 928673
URL: http://svn.apache.org/viewvc?rev=928673view=rev
Log:
Catch IllegalStateExcetpion and rethrow as PermFailException on Invalid public
key.
Modified:
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/DKIMVerifier.java
Modified:
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/DKIMVerifier.java
URL:
http://svn.apache.org/viewvc/james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/DKIMVerifier.java?rev=928673r1=928672r2=928673view=diff
==
---
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/DKIMVerifier.java
(original)
+++
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/DKIMVerifier.java
Mon Mar 29 10:02:12 2010
@@ -116,30 +116,33 @@ public class DKIMVerifier extends DKIMCo
* @throws PermFailException when the keys are not applicable
*/
public static void apply(PublicKeyRecord pkr, SignatureRecord sign) throws
PermFailException {
-if (!pkr.getGranularityPattern().matcher(sign.getIdentityLocalPart())
-.matches()) {
-throw new PermFailException(inapplicable key identity local=
-+ sign.getIdentityLocalPart() + Pattern:
-+ pkr.getGranularityPattern().pattern());
-}
-
-if (!pkr.isHashMethodSupported(sign.getHashMethod())) {
-throw new PermFailException(inappropriate hash for a=
-+ sign.getHashKeyType() + / + sign.getHashMethod());
-}
-if (!pkr.isKeyTypeSupported(sign.getHashKeyType())) {
-throw new PermFailException(inappropriate key type for a=
-+ sign.getHashKeyType() + / + sign.getHashMethod());
-}
-
-if (pkr.isDenySubdomains()) {
-if (!sign.getIdentity().toString().toLowerCase().endsWith(
-(@ + sign.getDToken()).toLowerCase())) {
-throw new PermFailException(
-AUID in subdomain of SDID is not allowed by the
public key record.);
+try {
+if
(!pkr.getGranularityPattern().matcher(sign.getIdentityLocalPart())
+.matches()) {
+throw new PermFailException(inapplicable key identity local=
++ sign.getIdentityLocalPart() + Pattern:
++ pkr.getGranularityPattern().pattern());
}
+
+if (!pkr.isHashMethodSupported(sign.getHashMethod())) {
+throw new PermFailException(inappropriate hash for a=
++ sign.getHashKeyType() + / + sign.getHashMethod());
+}
+if (!pkr.isKeyTypeSupported(sign.getHashKeyType())) {
+throw new PermFailException(inappropriate key type for a=
++ sign.getHashKeyType() + / + sign.getHashMethod());
+}
+
+if (pkr.isDenySubdomains()) {
+if (!sign.getIdentity().toString().toLowerCase().endsWith(
+(@ + sign.getDToken()).toLowerCase())) {
+throw new PermFailException(
+AUID in subdomain of SDID is not allowed by the
public key record.);
+}
+}
+} catch (IllegalStateException e) {
+throw new PermFailException(Invalid public key: +e.getMessage());
}
-
}
/**
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
svn commit: r928674 - /james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/impl/BodyHasherImpl.java
Author: bago
Date: Mon Mar 29 10:03:19 2010
New Revision: 928674
URL: http://svn.apache.org/viewvc?rev=928674view=rev
Log:
Make sure to validate the signature templates and throw PermFailException if an
invalid template is provided (otherwise it was throwing NPE later)
Modified:
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/impl/BodyHasherImpl.java
Modified:
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/impl/BodyHasherImpl.java
URL:
http://svn.apache.org/viewvc/james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/impl/BodyHasherImpl.java?rev=928674r1=928673r2=928674view=diff
==
---
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/impl/BodyHasherImpl.java
(original)
+++
james/jdkim/trunk/main/src/main/java/org/apache/james/jdkim/impl/BodyHasherImpl.java
Mon Mar 29 10:03:19 2010
@@ -47,6 +47,12 @@ public class BodyHasherImpl implements B
throw new PermFailException(Unsupported algorythm:
+ sign.getHashAlgo(), e);
}
+
+try {
+sign.validate();
+} catch (IllegalStateException e) {
+throw new PermFailException(Invalid signature template, e);
+}
int limit = sign.getBodyHashLimit();
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] Created: (JAMES-985) Store the MimeMessage in an extra datastore and just load it on demand
Store the MimeMessage in an extra datastore and just load it on demand -- Key: JAMES-985 URL: https://issues.apache.org/jira/browse/JAMES-985 Project: JAMES Server Issue Type: Sub-task Reporter: Norman Maurer JMS not really good in terms of performance when big messages are send around. We should store the MimeMessage of the Mail object in a extra storage and just load it when necessary. We could use claim check EIP for this: http://camel.apache.org/claim-check.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
svn commit: r928767 - /james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/TopCmdHandler.java
Author: norman
Date: Mon Mar 29 13:26:55 2010
New Revision: 928767
URL: http://svn.apache.org/viewvc?rev=928767view=rev
Log:
Write a CRLF after each header on TOP command
Modified:
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/TopCmdHandler.java
Modified:
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/TopCmdHandler.java
URL:
http://svn.apache.org/viewvc/james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/TopCmdHandler.java?rev=928767r1=928766r2=928767view=diff
==
---
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/TopCmdHandler.java
(original)
+++
james/server/trunk/pop3server/src/main/java/org/apache/james/pop3server/core/TopCmdHandler.java
Mon Mar 29 13:26:55 2010
@@ -110,6 +110,10 @@ public class TopCmdHandler extends RetrC
IteratorHeader headers = result.headers();
while (headers.hasNext()) {
headers.next().writeTo(outChannel);
+
+ // we need to write out the CRLF after each
header
+out.write(\r\n.getBytes());
+
}
// headers and body are seperated by a CRLF
out.write(\r\n.getBytes());
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
Re: [PROPOSAL] Replace MailRepository with Mailbox / MailboxManager for POP3Server ...
Hi Norman, With wireshark, thunderbird/james hangs at POP3 STAT command. I saw you patched the StatCmdHandler. It now works even without any message in my mailbox (after update/package/deploy). Great! However: - I received a few exceptions complaining about the transaction (can not initiate a new one while still an active) while retrieving messages and the spool was doing some work. - I saw yesterday some strange behaviour when I made tests with different clients : deleted messages were still fetched by new clients. I will further investigate to post with more consolidated info. Many tks, Eric On 03/29/2010 08:53 AM, Norman Maurer wrote: Hi Eric, I was able to reproduce your problem and just checked in the fix. Please svn up and try again.. Thx for your feedback. Bye, Norman 2010/3/29 Norman Maurernor...@apache.org: Hi Eric, I just tried to acces an empty inbox via thunderbird (pop3) here. It worked without any probs. Im using windows 7 and thunderbird 3.0.3 Bye, Norman 2010/3/28 Norman Maurernor...@apache.org: -- Forwarded message -- From: Norman Maurernor...@apache.org Date: 2010/3/28 Subject: Re: [PROPOSAL] Replace MailRepository with Mailbox / MailboxManager for POP3Server ... To: Eric Charleseric.char...@u-mangate.com Hi Eric, could you maybe use wireshark or something similar to log what thunderbird tries todo exactly.. ? Thx, Norman 2010/3/28 Eric Charleseric.char...@u-mangate.com: Hi Norman, I just made the following tests: - Fresh checkout from trunk and mvn package (some new dependencies were downloaded such as jsieve, mailet-sdk,...) = same problem on a local deployment with thunderbird : it hangs when mailbox is empty. - I after defined my production account on evolution (http://www.gnome.org/projects/evolution) : I didn't have the problem anymore, but really strange, evolution downloaded some messages from yesterday. Those messages had been deleted from my inbox via thunderbird, but was retrieved via evolution... - I tested with evolution the local deployment I just made : I can read the inbox even if empty. The problem seems to reside on the thunderbird/james conversation when inbox is empty. Tks, Eric On 03/27/2010 10:08 PM, Norman Maurer wrote: Hi Eric, I just tested pop3 with imail and no messages in inbox and everything just works. Will try to test it with thunderbird later.. Bye, Norman 2010/3/27 Norman Maurernor...@apache.org: Hi Eric, I will try to find out whats wrong with the pop3 inbox stuff. If you find out anything else please feel free to just report it.. Thx, Norman 2010/3/27 Eric Charleseric.char...@u-mangate.com: Hi Norman, Many tks for your quick answer. I had defined in my Thunderbird 3.0.3 client (on linux) the pop3 account as the default one. When launching thunderbird, the connection was succefully done Host contacted, sending login information, saying after 1 second Determining which messages to index (never saw that message before...), hanging there a few minutes, and finally saying connection to server ... timed out. After that, Imap account behaved also such a way... I also tried with latest Thundrebird 3.0.3: same result... I have now defined the imap account as the default one, and it's working fine even without any mail in the inbox. However, for the pop3 account, I still have the same behaviour. Anyway, I - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
OutOfMemory exception
Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... Yesterday, I had a more explicit stacktrace (see STACK TRACE 2 in annex) talking about Too many open files in the MimeMessageInputStreamSource constructor when trying file = File.createTempFile(key, .m64). Normally, the reference to the file should disappear when invoking the dispose method (doing a file.delete();). But, I've got many old files in my /tmp (one more file each minute), so I suppose that for a reason or another, those files are not deleted. This could cause the Too many open files problem, leaving the JMS Producers in memory with any Consumers... ? Finally, I don't know if the OOM comes from the Too many open files or still from somewhere else ? Please note that I define the address-error processors as Null (don't want to keep those mails on disk) and that I have activated the SpamAssassin mailet. But even without those extra configs, I've got the OOM. Any idea, Eric STACK TRACE 1 INFO 16:23:01,385 | james.smtpserver | Successfully spooled mail Mail1269872581383-7178 from johnsmith...@u-mangate.org on 120.56.194.53 for [johnsmith...@u-mangate.org] INFO 16:23:01,386 | james.smtpserver | Successfully spooled mail Mail1269872581383-7179 from catchthism...@u-mangate.net on 120.56.194.53 for [catchthism...@u-mangate.net] INFO 16:23:20,887 | james.smtpserver | Successfully spooled mail Mail1269872600878-7183 from 3edd9016.4070...@u-mangate.be on 120.56.194.53 for [3edd9016.4070...@u-mangate.be] INFO 16:23:22,007 | james.smtpserver | Successfully spooled mail Mail1269872602002-7185 from 3c3cd8ce0b.4020...@u-mangate.org on 120.56.194.53 for [3c3cd8ce0b.4020...@u-mangate.org] INFO 16:23:23,336 | james.smtpserver | Successfully spooled mail Mail1269872603328-7186 from elizabethsorensen...@bbr-vt.de on 95.169.149.179 for [angelo.bid...@u-mangate.net] INFO 16:23:34,899 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 1115 INFO 16:23:37,254 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1240 ms, Index update took 0 ms INFO 16:23:37,255 | james.smtpserver | Successfully spooled mail Mail1269872616013-7188 from 3ee83341.3030...@u-mangate.be on 120.56.194.53 for [3ee83341.3030...@u-mangate.be] INFO 16:23:42,140 | james.smtpserver | Successfully spooled mail Mail1269872622133-7190 from m...@u-mangate.net on 120.56.194.53 for [m...@u-mangate.net] INFO 16:23:56,193 | james.smtpserver | Successfully spooled mail Mail1269872633794-7195 from shauna.escoba...@superiorsleep.co.nz on 121.147.241.76 for [angelo.bid...@u-mangate.net] INFO 16:23:56,195 | james.smtpserver | Successfully spooled mail Mail1269872633798-7196 from 3ee822b7.10...@u-mangate.be on 120.56.194.53 for [3ee822b7.10...@u-mangate.be] INFO 16:23:59,740 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 3543 ms, Index update took 0 ms INFO 16:23:59,743 | james.smtpserver | Successfully spooled mail Mail1269872636196-7198 from ric.char...@u-mangate.org on 120.56.194.53 for [ric.char...@u-mangate.org] INFO 16:24:01,066 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1322 ms, Index update took 0 ms INFO 16:24:01,066 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1321 ms, Index update took 0 ms INFO 16:24:01,069 | james.smtpserver | Successfully spooled mail Mail1269872639742-7199 from stephane.liu...@u-mangate.net on 120.56.194.53 for [stephane.liu...@u-mangate.net] INFO 16:24:10,625 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 7100 INFO 16:24:10,626 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 2458 ms, Index update took 7099 ms INFO 16:24:19,753 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 5770 INFO 16:24:19,753 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 3358 ms, Index update took 5769 ms INFO 16:24:28,832 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 4584 INFO 16:24:53,501 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 22422 INFO 16:24:53,502 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 0 ms, Index update took
Re: OutOfMemory exception
Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charles eric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... Yesterday, I had a more explicit stacktrace (see STACK TRACE 2 in annex) talking about Too many open files in the MimeMessageInputStreamSource constructor when trying file = File.createTempFile(key, .m64). Normally, the reference to the file should disappear when invoking the dispose method (doing a file.delete();). But, I've got many old files in my /tmp (one more file each minute), so I suppose that for a reason or another, those files are not deleted. This could cause the Too many open files problem, leaving the JMS Producers in memory with any Consumers... ? Finally, I don't know if the OOM comes from the Too many open files or still from somewhere else ? Please note that I define the address-error processors as Null (don't want to keep those mails on disk) and that I have activated the SpamAssassin mailet. But even without those extra configs, I've got the OOM. Any idea, Eric STACK TRACE 1 INFO 16:23:01,385 | james.smtpserver | Successfully spooled mail Mail1269872581383-7178 from johnsmith...@u-mangate.org on 120.56.194.53 for [johnsmith...@u-mangate.org] INFO 16:23:01,386 | james.smtpserver | Successfully spooled mail Mail1269872581383-7179 from catchthism...@u-mangate.net on 120.56.194.53 for [catchthism...@u-mangate.net] INFO 16:23:20,887 | james.smtpserver | Successfully spooled mail Mail1269872600878-7183 from 3edd9016.4070...@u-mangate.be on 120.56.194.53 for [3edd9016.4070...@u-mangate.be] INFO 16:23:22,007 | james.smtpserver | Successfully spooled mail Mail1269872602002-7185 from 3c3cd8ce0b.4020...@u-mangate.org on 120.56.194.53 for [3c3cd8ce0b.4020...@u-mangate.org] INFO 16:23:23,336 | james.smtpserver | Successfully spooled mail Mail1269872603328-7186 from elizabethsorensen...@bbr-vt.de on 95.169.149.179 for [angelo.bid...@u-mangate.net] INFO 16:23:34,899 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 1115 INFO 16:23:37,254 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1240 ms, Index update took 0 ms INFO 16:23:37,255 | james.smtpserver | Successfully spooled mail Mail1269872616013-7188 from 3ee83341.3030...@u-mangate.be on 120.56.194.53 for [3ee83341.3030...@u-mangate.be] INFO 16:23:42,140 | james.smtpserver | Successfully spooled mail Mail1269872622133-7190 from m...@u-mangate.net on 120.56.194.53 for [m...@u-mangate.net] INFO 16:23:56,193 | james.smtpserver | Successfully spooled mail Mail1269872633794-7195 from shauna.escoba...@superiorsleep.co.nz on 121.147.241.76 for [angelo.bid...@u-mangate.net] INFO 16:23:56,195 | james.smtpserver | Successfully spooled mail Mail1269872633798-7196 from 3ee822b7.10...@u-mangate.be on 120.56.194.53 for [3ee822b7.10...@u-mangate.be] INFO 16:23:59,740 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 3543 ms, Index update took 0 ms INFO 16:23:59,743 | james.smtpserver | Successfully spooled mail Mail1269872636196-7198 from ric.char...@u-mangate.org on 120.56.194.53 for [ric.char...@u-mangate.org] INFO 16:24:01,066 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1322 ms, Index update took 0 ms INFO 16:24:01,066 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1321 ms, Index update took 0 ms INFO 16:24:01,069 | james.smtpserver | Successfully spooled mail Mail1269872639742-7199 from stephane.liu...@u-mangate.net on 120.56.194.53 for [stephane.liu...@u-mangate.net] INFO 16:24:10,625 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 7100 INFO 16:24:10,626 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 2458 ms, Index update took 7099 ms INFO 16:24:19,753 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 5770 INFO 16:24:19,753 |
Re: [PROPOSAL] Replace MailRepository with Mailbox / MailboxManager for POP3Server ...
Hi Eric, just post the infos when you have them, and I will look what I can do.. Bye, Norman 2010/3/29 Eric Charles eric.char...@u-mangate.com: Hi Norman, With wireshark, thunderbird/james hangs at POP3 STAT command. I saw you patched the StatCmdHandler. It now works even without any message in my mailbox (after update/package/deploy). Great! However: - I received a few exceptions complaining about the transaction (can not initiate a new one while still an active) while retrieving messages and the spool was doing some work. - I saw yesterday some strange behaviour when I made tests with different clients : deleted messages were still fetched by new clients. I will further investigate to post with more consolidated info. Many tks, Eric On 03/29/2010 08:53 AM, Norman Maurer wrote: Hi Eric, I was able to reproduce your problem and just checked in the fix. Please svn up and try again.. Thx for your feedback. Bye, Norman 2010/3/29 Norman Maurernor...@apache.org: Hi Eric, I just tried to acces an empty inbox via thunderbird (pop3) here. It worked without any probs. Im using windows 7 and thunderbird 3.0.3 Bye, Norman 2010/3/28 Norman Maurernor...@apache.org: -- Forwarded message -- From: Norman Maurernor...@apache.org Date: 2010/3/28 Subject: Re: [PROPOSAL] Replace MailRepository with Mailbox / MailboxManager for POP3Server ... To: Eric Charleseric.char...@u-mangate.com Hi Eric, could you maybe use wireshark or something similar to log what thunderbird tries todo exactly.. ? Thx, Norman 2010/3/28 Eric Charleseric.char...@u-mangate.com: Hi Norman, I just made the following tests: - Fresh checkout from trunk and mvn package (some new dependencies were downloaded such as jsieve, mailet-sdk,...) = same problem on a local deployment with thunderbird : it hangs when mailbox is empty. - I after defined my production account on evolution (http://www.gnome.org/projects/evolution) : I didn't have the problem anymore, but really strange, evolution downloaded some messages from yesterday. Those messages had been deleted from my inbox via thunderbird, but was retrieved via evolution... - I tested with evolution the local deployment I just made : I can read the inbox even if empty. The problem seems to reside on the thunderbird/james conversation when inbox is empty. Tks, Eric On 03/27/2010 10:08 PM, Norman Maurer wrote: Hi Eric, I just tested pop3 with imail and no messages in inbox and everything just works. Will try to test it with thunderbird later.. Bye, Norman 2010/3/27 Norman Maurernor...@apache.org: Hi Eric, I will try to find out whats wrong with the pop3 inbox stuff. If you find out anything else please feel free to just report it.. Thx, Norman 2010/3/27 Eric Charleseric.char...@u-mangate.com: Hi Norman, Many tks for your quick answer. I had defined in my Thunderbird 3.0.3 client (on linux) the pop3 account as the default one. When launching thunderbird, the connection was succefully done Host contacted, sending login information, saying after 1 second Determining which messages to index (never saw that message before...), hanging there a few minutes, and finally saying connection to server ... timed out. After that, Imap account behaved also such a way... I also tried with latest Thundrebird 3.0.3: same result... I have now defined the imap account as the default one, and it's working fine even without any mail in the inbox. However, for the pop3 account, I still have the same behaviour. Anyway, I - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
svn commit: r928850 - /james/server/trunk/spoolmanager/src/main/java/org/apache/james/AbstractMailServer.java
Author: norman
Date: Mon Mar 29 17:59:39 2010
New Revision: 928850
URL: http://svn.apache.org/viewvc?rev=928850view=rev
Log:
Dispose source mail after copy it. This is prolly the cause of OOM reported by
Eric. Thx again for the report
Modified:
james/server/trunk/spoolmanager/src/main/java/org/apache/james/AbstractMailServer.java
Modified:
james/server/trunk/spoolmanager/src/main/java/org/apache/james/AbstractMailServer.java
URL:
http://svn.apache.org/viewvc/james/server/trunk/spoolmanager/src/main/java/org/apache/james/AbstractMailServer.java?rev=928850r1=928849r2=928850view=diff
==
---
james/server/trunk/spoolmanager/src/main/java/org/apache/james/AbstractMailServer.java
(original)
+++
james/server/trunk/spoolmanager/src/main/java/org/apache/james/AbstractMailServer.java
Mon Mar 29 17:59:39 2010
@@ -52,6 +52,7 @@ import org.apache.james.core.MailHeaders
import org.apache.james.core.MailImpl;
import org.apache.james.impl.jamesuser.JamesUsersRepository;
import org.apache.james.lifecycle.Configurable;
+import org.apache.james.lifecycle.LifecycleUtil;
import org.apache.james.lifecycle.LogEnabled;
import org.apache.james.services.MailServer;
import org.apache.james.transport.camel.InMemoryMail;
@@ -298,7 +299,10 @@ public abstract class AbstractMailServer
*/
public void sendMail(Mail mail) throws MessagingException {
try {
-producerTemplate.sendBody(getToUri(mail), ExchangePattern.InOnly,
new InMemoryMail(mail));
+ Mail newMail = new InMemoryMail(mail);
+producerTemplate.sendBody(getToUri(mail), ExchangePattern.InOnly,
newMail);
+
+LifecycleUtil.dispose(mail);
} catch (Exception e) {
logger.error(Error storing message: + e.getMessage(),e);
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
Re: OutOfMemory exception
Hi Eric, I found the cause for the not deleted temporary files. Hopefully this is the cause of the OOM. Could try to svn up and run again ? Thx, Norman 2010/3/29 Norman Maurer norman.mau...@googlemail.com: Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charles eric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... Yesterday, I had a more explicit stacktrace (see STACK TRACE 2 in annex) talking about Too many open files in the MimeMessageInputStreamSource constructor when trying file = File.createTempFile(key, .m64). Normally, the reference to the file should disappear when invoking the dispose method (doing a file.delete();). But, I've got many old files in my /tmp (one more file each minute), so I suppose that for a reason or another, those files are not deleted. This could cause the Too many open files problem, leaving the JMS Producers in memory with any Consumers... ? Finally, I don't know if the OOM comes from the Too many open files or still from somewhere else ? Please note that I define the address-error processors as Null (don't want to keep those mails on disk) and that I have activated the SpamAssassin mailet. But even without those extra configs, I've got the OOM. Any idea, Eric STACK TRACE 1 INFO 16:23:01,385 | james.smtpserver | Successfully spooled mail Mail1269872581383-7178 from johnsmith...@u-mangate.org on 120.56.194.53 for [johnsmith...@u-mangate.org] INFO 16:23:01,386 | james.smtpserver | Successfully spooled mail Mail1269872581383-7179 from catchthism...@u-mangate.net on 120.56.194.53 for [catchthism...@u-mangate.net] INFO 16:23:20,887 | james.smtpserver | Successfully spooled mail Mail1269872600878-7183 from 3edd9016.4070...@u-mangate.be on 120.56.194.53 for [3edd9016.4070...@u-mangate.be] INFO 16:23:22,007 | james.smtpserver | Successfully spooled mail Mail1269872602002-7185 from 3c3cd8ce0b.4020...@u-mangate.org on 120.56.194.53 for [3c3cd8ce0b.4020...@u-mangate.org] INFO 16:23:23,336 | james.smtpserver | Successfully spooled mail Mail1269872603328-7186 from elizabethsorensen...@bbr-vt.de on 95.169.149.179 for [angelo.bid...@u-mangate.net] INFO 16:23:34,899 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 1115 INFO 16:23:37,254 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1240 ms, Index update took 0 ms INFO 16:23:37,255 | james.smtpserver | Successfully spooled mail Mail1269872616013-7188 from 3ee83341.3030...@u-mangate.be on 120.56.194.53 for [3ee83341.3030...@u-mangate.be] INFO 16:23:42,140 | james.smtpserver | Successfully spooled mail Mail1269872622133-7190 from m...@u-mangate.net on 120.56.194.53 for [m...@u-mangate.net] INFO 16:23:56,193 | james.smtpserver | Successfully spooled mail Mail1269872633794-7195 from shauna.escoba...@superiorsleep.co.nz on 121.147.241.76 for [angelo.bid...@u-mangate.net] INFO 16:23:56,195 | james.smtpserver | Successfully spooled mail Mail1269872633798-7196 from 3ee822b7.10...@u-mangate.be on 120.56.194.53 for [3ee822b7.10...@u-mangate.be] INFO 16:23:59,740 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 3543 ms, Index update took 0 ms INFO 16:23:59,743 | james.smtpserver | Successfully spooled mail Mail1269872636196-7198 from ric.char...@u-mangate.org on 120.56.194.53 for [ric.char...@u-mangate.org] INFO 16:24:01,066 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1322 ms, Index update took 0 ms INFO 16:24:01,066 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1321 ms, Index update took 0 ms INFO 16:24:01,069 | james.smtpserver | Successfully spooled mail Mail1269872639742-7199 from stephane.liu...@u-mangate.net on 120.56.194.53 for [stephane.liu...@u-mangate.net] INFO 16:24:10,625 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 7100 INFO 16:24:10,626 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB
Re: OutOfMemory exception
Norman, Done. Now, we have to wait... I saw some created *.m64 that were removed. But there are other ones that remains in /tmp. I sometimes run a jmap (java memory map) to produce a heap dump and analyse it. At the beginning, everything seems ok, and often, when I come back, the OOM has already occured. I changed the -Xmx512m to -Xmx256m to have a quicker exception (if any, let's hope not). The *.m64 are a clue and we should ensure that they are removed in all circumstances. This may be the easy part, as we have some visible clues (the files). We should also ensure after that there are no other causes to the leaks. Tks, Eric On 03/29/2010 08:00 PM, Norman Maurer wrote: Hi Eric, I found the cause for the not deleted temporary files. Hopefully this is the cause of the OOM. Could try to svn up and run again ? Thx, Norman 2010/3/29 Norman Maurernorman.mau...@googlemail.com: Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charleseric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... Yesterday, I had a more explicit stacktrace (see STACK TRACE 2 in annex) talking about Too many open files in the MimeMessageInputStreamSource constructor when trying file = File.createTempFile(key, .m64). Normally, the reference to the file should disappear when invoking the dispose method (doing a file.delete();). But, I've got many old files in my /tmp (one more file each minute), so I suppose that for a reason or another, those files are not deleted. This could cause the Too many open files problem, leaving the JMS Producers in memory with any Consumers... ? Finally, I don't know if the OOM comes from the Too many open files or still from somewhere else ? Please note that I define the address-error processors as Null (don't want to keep those mails on disk) and that I have activated the SpamAssassin mailet. But even without those extra configs, I've got the OOM. Any idea, Eric STACK TRACE 1 INFO 16:23:01,385 | james.smtpserver | Successfully spooled mail Mail1269872581383-7178 from johnsmith...@u-mangate.org on 120.56.194.53 for [johnsmith...@u-mangate.org] INFO 16:23:01,386 | james.smtpserver | Successfully spooled mail Mail1269872581383-7179 from catchthism...@u-mangate.net on 120.56.194.53 for [catchthism...@u-mangate.net] INFO 16:23:20,887 | james.smtpserver | Successfully spooled mail Mail1269872600878-7183 from 3edd9016.4070...@u-mangate.be on 120.56.194.53 for [3edd9016.4070...@u-mangate.be] INFO 16:23:22,007 | james.smtpserver | Successfully spooled mail Mail1269872602002-7185 from 3c3cd8ce0b.4020...@u-mangate.org on 120.56.194.53 for [3c3cd8ce0b.4020...@u-mangate.org] INFO 16:23:23,336 | james.smtpserver | Successfully spooled mail Mail1269872603328-7186 from elizabethsorensen...@bbr-vt.de on 95.169.149.179 for [angelo.bid...@u-mangate.net] INFO 16:23:34,899 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 1115 INFO 16:23:37,254 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1240 ms, Index update took 0 ms INFO 16:23:37,255 | james.smtpserver | Successfully spooled mail Mail1269872616013-7188 from 3ee83341.3030...@u-mangate.be on 120.56.194.53 for [3ee83341.3030...@u-mangate.be] INFO 16:23:42,140 | james.smtpserver | Successfully spooled mail Mail1269872622133-7190 from m...@u-mangate.net on 120.56.194.53 for [m...@u-mangate.net] INFO 16:23:56,193 | james.smtpserver | Successfully spooled mail Mail1269872633794-7195 from shauna.escoba...@superiorsleep.co.nz on 121.147.241.76 for [angelo.bid...@u-mangate.net] INFO 16:23:56,195 | james.smtpserver | Successfully spooled mail Mail1269872633798-7196 from 3ee822b7.10...@u-mangate.be on 120.56.194.53 for [3ee822b7.10...@u-mangate.be] INFO 16:23:59,740 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 3543 ms, Index update took 0 ms INFO 16:23:59,743 | james.smtpserver | Successfully spooled mail Mail1269872636196-7198 from ric.char...@u-mangate.org on 120.56.194.53 for [ric.char...@u-mangate.org] INFO 16:24:01,066 |
Re: OutOfMemory exception
Hi Eric, sure.. we have to find the OOM cause. What would be interesting, could you check somehow if the .m64 files are files which are related to successfully delivered mail ? Thx, Norman 2010/3/29 Eric Charles eric.char...@u-mangate.com: Norman, Done. Now, we have to wait... I saw some created *.m64 that were removed. But there are other ones that remains in /tmp. I sometimes run a jmap (java memory map) to produce a heap dump and analyse it. At the beginning, everything seems ok, and often, when I come back, the OOM has already occured. I changed the -Xmx512m to -Xmx256m to have a quicker exception (if any, let's hope not). The *.m64 are a clue and we should ensure that they are removed in all circumstances. This may be the easy part, as we have some visible clues (the files). We should also ensure after that there are no other causes to the leaks. Tks, Eric On 03/29/2010 08:00 PM, Norman Maurer wrote: Hi Eric, I found the cause for the not deleted temporary files. Hopefully this is the cause of the OOM. Could try to svn up and run again ? Thx, Norman 2010/3/29 Norman Maurernorman.mau...@googlemail.com: Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charleseric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... Yesterday, I had a more explicit stacktrace (see STACK TRACE 2 in annex) talking about Too many open files in the MimeMessageInputStreamSource constructor when trying file = File.createTempFile(key, .m64). Normally, the reference to the file should disappear when invoking the dispose method (doing a file.delete();). But, I've got many old files in my /tmp (one more file each minute), so I suppose that for a reason or another, those files are not deleted. This could cause the Too many open files problem, leaving the JMS Producers in memory with any Consumers... ? Finally, I don't know if the OOM comes from the Too many open files or still from somewhere else ? Please note that I define the address-error processors as Null (don't want to keep those mails on disk) and that I have activated the SpamAssassin mailet. But even without those extra configs, I've got the OOM. Any idea, Eric STACK TRACE 1 INFO 16:23:01,385 | james.smtpserver | Successfully spooled mail Mail1269872581383-7178 from johnsmith...@u-mangate.org on 120.56.194.53 for [johnsmith...@u-mangate.org] INFO 16:23:01,386 | james.smtpserver | Successfully spooled mail Mail1269872581383-7179 from catchthism...@u-mangate.net on 120.56.194.53 for [catchthism...@u-mangate.net] INFO 16:23:20,887 | james.smtpserver | Successfully spooled mail Mail1269872600878-7183 from 3edd9016.4070...@u-mangate.be on 120.56.194.53 for [3edd9016.4070...@u-mangate.be] INFO 16:23:22,007 | james.smtpserver | Successfully spooled mail Mail1269872602002-7185 from 3c3cd8ce0b.4020...@u-mangate.org on 120.56.194.53 for [3c3cd8ce0b.4020...@u-mangate.org] INFO 16:23:23,336 | james.smtpserver | Successfully spooled mail Mail1269872603328-7186 from elizabethsorensen...@bbr-vt.de on 95.169.149.179 for [angelo.bid...@u-mangate.net] INFO 16:23:34,899 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: cleanup took 1115 INFO 16:23:37,254 | org.apache.activemq.store.kahadb.MessageDatabase | Slow KahaDB access: Journal append took: 1240 ms, Index update took 0 ms INFO 16:23:37,255 | james.smtpserver | Successfully spooled mail Mail1269872616013-7188 from 3ee83341.3030...@u-mangate.be on 120.56.194.53 for [3ee83341.3030...@u-mangate.be] INFO 16:23:42,140 | james.smtpserver | Successfully spooled mail Mail1269872622133-7190 from m...@u-mangate.net on 120.56.194.53 for [m...@u-mangate.net] INFO 16:23:56,193 | james.smtpserver | Successfully spooled mail Mail1269872633794-7195 from shauna.escoba...@superiorsleep.co.nz on 121.147.241.76 for [angelo.bid...@u-mangate.net] INFO 16:23:56,195 | james.smtpserver | Successfully spooled mail Mail1269872633798-7196 from 3ee822b7.10...@u-mangate.be on 120.56.194.53 for [3ee822b7.10...@u-mangate.be] INFO 16:23:59,740 |
svn commit: r928874 - in /james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel: AbstractProcessorRouteBuilder.java DisposeProcessor.java
Author: norman
Date: Mon Mar 29 19:14:15 2010
New Revision: 928874
URL: http://svn.apache.org/viewvc?rev=928874view=rev
Log:
Get sure we dispose Mail on stop()
Added:
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/DisposeProcessor.java
Modified:
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/AbstractProcessorRouteBuilder.java
Modified:
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/AbstractProcessorRouteBuilder.java
URL:
http://svn.apache.org/viewvc/james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/AbstractProcessorRouteBuilder.java?rev=928874r1=928873r2=928874view=diff
==
---
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/AbstractProcessorRouteBuilder.java
(original)
+++
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/AbstractProcessorRouteBuilder.java
Mon Mar 29 19:14:15 2010
@@ -87,7 +87,7 @@ public abstract class AbstractProcessorR
public void configure() throws Exception {
Processor terminatingMailetProcessor = new MailetProcessor(new
TerminatingMailet(), logger);
-
+Processor disposeProcessor = new DisposeProcessor();
ListHierarchicalConfiguration processorConfs =
config.configurationsAt(processor);
for (int i = 0; i processorConfs.size(); i++) {
@@ -213,8 +213,8 @@ public abstract class AbstractProcessorR
.choice()
-// if the mailstate is GHOST whe should just stop
here.
-.when(new MailStateEquals(Mail.GHOST)).stop()
+// if the mailstate is GHOST whe should just
dispose and stop here.
+.when(new
MailStateEquals(Mail.GHOST)).process(disposeProcessor).stop()
// check if the state of the mail is the same as
the
// current processor. If not just route it to the
right endpoint via recipientList and stop processing.
Added:
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/DisposeProcessor.java
URL:
http://svn.apache.org/viewvc/james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/DisposeProcessor.java?rev=928874view=auto
==
---
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/DisposeProcessor.java
(added)
+++
james/server/trunk/spoolmanager/src/main/java/org/apache/james/transport/camel/DisposeProcessor.java
Mon Mar 29 19:14:15 2010
@@ -0,0 +1,37 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the*
+ * License); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the*
+ * specific language governing permissions and limitations *
+ * under the License. *
+ /
+
+package org.apache.james.transport.camel;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.Processor;
+import org.apache.james.lifecycle.LifecycleUtil;
+
+/**
+ * Processor which dispose body object if needed
+ *
+ *
+ */
+public class DisposeProcessor implements Processor{
+
+ public void process(Exchange arg0) throws Exception {
+ LifecycleUtil.dispose(arg0.getIn().getBody());
+ }
+
+}
-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
Re: OutOfMemory exception
Hi Norman, The .m64 are all to unkown user but to well known domain (so for unkn...@known.com). They are from various size (with and without attachment). They are well formed (I can open the downloaded file with thunderbird) However, when I sent a mail to unkn...@known.com, I don't see it in the /tmp Tks, Eric On 03/29/2010 08:43 PM, Norman Maurer wrote: Hi Eric, sure.. we have to find the OOM cause. What would be interesting, could you check somehow if the .m64 files are files which are related to successfully delivered mail ? Thx, Norman 2010/3/29 Eric Charleseric.char...@u-mangate.com: Norman, Done. Now, we have to wait... I saw some created *.m64 that were removed. But there are other ones that remains in /tmp. I sometimes run a jmap (java memory map) to produce a heap dump and analyse it. At the beginning, everything seems ok, and often, when I come back, the OOM has already occured. I changed the -Xmx512m to -Xmx256m to have a quicker exception (if any, let's hope not). The *.m64 are a clue and we should ensure that they are removed in all circumstances. This may be the easy part, as we have some visible clues (the files). We should also ensure after that there are no other causes to the leaks. Tks, Eric On 03/29/2010 08:00 PM, Norman Maurer wrote: Hi Eric, I found the cause for the not deleted temporary files. Hopefully this is the cause of the OOM. Could try to svn up and run again ? Thx, Norman 2010/3/29 Norman Maurernorman.mau...@googlemail.com: Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charleseric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: OutOfMemory exception
Hi Norman, I just deployed your new commit with the new camel DisposeProcess. Good news : I don't see anymore the m64 file in tmp (well I see 1 file, on the second after, it is no more there, so the dispose works as it should). I keep you posted with our eventual future OOM. Tks, Eric On 03/29/2010 09:18 PM, Eric Charles wrote: Hi Norman, The .m64 are all to unkown user but to well known domain (so for unkn...@known.com). They are from various size (with and without attachment). They are well formed (I can open the downloaded file with thunderbird) However, when I sent a mail to unkn...@known.com, I don't see it in the /tmp Tks, Eric On 03/29/2010 08:43 PM, Norman Maurer wrote: Hi Eric, sure.. we have to find the OOM cause. What would be interesting, could you check somehow if the .m64 files are files which are related to successfully delivered mail ? Thx, Norman 2010/3/29 Eric Charleseric.char...@u-mangate.com: Norman, Done. Now, we have to wait... I saw some created *.m64 that were removed. But there are other ones that remains in /tmp. I sometimes run a jmap (java memory map) to produce a heap dump and analyse it. At the beginning, everything seems ok, and often, when I come back, the OOM has already occured. I changed the -Xmx512m to -Xmx256m to have a quicker exception (if any, let's hope not). The *.m64 are a clue and we should ensure that they are removed in all circumstances. This may be the easy part, as we have some visible clues (the files). We should also ensure after that there are no other causes to the leaks. Tks, Eric On 03/29/2010 08:00 PM, Norman Maurer wrote: Hi Eric, I found the cause for the not deleted temporary files. Hopefully this is the cause of the OOM. Could try to svn up and run again ? Thx, Norman 2010/3/29 Norman Maurernorman.mau...@googlemail.com: Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charleseric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: OutOfMemory exception
Oops, no, the files are still there (only unkn...@known.com). Eric On 03/29/2010 10:16 PM, Eric Charles wrote: Hi Norman, I just deployed your new commit with the new camel DisposeProcess. Good news : I don't see anymore the m64 file in tmp (well I see 1 file, on the second after, it is no more there, so the dispose works as it should). I keep you posted with our eventual future OOM. Tks, Eric On 03/29/2010 09:18 PM, Eric Charles wrote: Hi Norman, The .m64 are all to unkown user but to well known domain (so for unkn...@known.com). They are from various size (with and without attachment). They are well formed (I can open the downloaded file with thunderbird) However, when I sent a mail to unkn...@known.com, I don't see it in the /tmp Tks, Eric On 03/29/2010 08:43 PM, Norman Maurer wrote: Hi Eric, sure.. we have to find the OOM cause. What would be interesting, could you check somehow if the .m64 files are files which are related to successfully delivered mail ? Thx, Norman 2010/3/29 Eric Charleseric.char...@u-mangate.com: Norman, Done. Now, we have to wait... I saw some created *.m64 that were removed. But there are other ones that remains in /tmp. I sometimes run a jmap (java memory map) to produce a heap dump and analyse it. At the beginning, everything seems ok, and often, when I come back, the OOM has already occured. I changed the -Xmx512m to -Xmx256m to have a quicker exception (if any, let's hope not). The *.m64 are a clue and we should ensure that they are removed in all circumstances. This may be the easy part, as we have some visible clues (the files). We should also ensure after that there are no other causes to the leaks. Tks, Eric On 03/29/2010 08:00 PM, Norman Maurer wrote: Hi Eric, I found the cause for the not deleted temporary files. Hopefully this is the cause of the OOM. Could try to svn up and run again ? Thx, Norman 2010/3/29 Norman Maurernorman.mau...@googlemail.com: Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charleseric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: OutOfMemory exception
Hi Eric, you said all the files are related to address-errors , could you show me your address error processor config? Does the Problem still exist when you enable the ValidRcptHandler in the smtpserver.xml file? Thx Norman 2010/3/30, Eric Charles eric.char...@u-mangate.com: Oops, no, the files are still there (only unkn...@known.com). Eric On 03/29/2010 10:16 PM, Eric Charles wrote: Hi Norman, I just deployed your new commit with the new camel DisposeProcess. Good news : I don't see anymore the m64 file in tmp (well I see 1 file, on the second after, it is no more there, so the dispose works as it should). I keep you posted with our eventual future OOM. Tks, Eric On 03/29/2010 09:18 PM, Eric Charles wrote: Hi Norman, The .m64 are all to unkown user but to well known domain (so for unkn...@known.com). They are from various size (with and without attachment). They are well formed (I can open the downloaded file with thunderbird) However, when I sent a mail to unkn...@known.com, I don't see it in the /tmp Tks, Eric On 03/29/2010 08:43 PM, Norman Maurer wrote: Hi Eric, sure.. we have to find the OOM cause. What would be interesting, could you check somehow if the .m64 files are files which are related to successfully delivered mail ? Thx, Norman 2010/3/29 Eric Charleseric.char...@u-mangate.com: Norman, Done. Now, we have to wait... I saw some created *.m64 that were removed. But there are other ones that remains in /tmp. I sometimes run a jmap (java memory map) to produce a heap dump and analyse it. At the beginning, everything seems ok, and often, when I come back, the OOM has already occured. I changed the -Xmx512m to -Xmx256m to have a quicker exception (if any, let's hope not). The *.m64 are a clue and we should ensure that they are removed in all circumstances. This may be the easy part, as we have some visible clues (the files). We should also ensure after that there are no other causes to the leaks. Tks, Eric On 03/29/2010 08:00 PM, Norman Maurer wrote: Hi Eric, I found the cause for the not deleted temporary files. Hopefully this is the cause of the OOM. Could try to svn up and run again ? Thx, Norman 2010/3/29 Norman Maurernorman.mau...@googlemail.com: Hi Eric, thx for the report. I see exact the same problem today here.. (The OOM). Didn't notice the files in the tmp folder, but I think thats a good pointer. I will try to debug the problem later or tomorrow. But I suspect you are right about the tmp files and jms producers.. Did you run a kill -3 pid to see what threads are active etc ? Thx, Norman Ps: Patches are welcome :) 2010/3/29 Eric Charleseric.char...@u-mangate.com: Hi, The last two weeks, I deployed various trunk snapshots. After james running less than 1 day, I always ran into a OutOfMemory Exception. I analysed the logs (for example STACK TRACE 1 in annex), and always found Camel complaining when trying to deliver on JMS queue. I tried to adapt the ActiveMQ configuration based on http://activemq.apache.org/javalangoutofmemory.html but nothing helped. I also analyzed various heap dump :sometimes, 65% was used by org.apache.mina.transport.socket.nio.NioSocketSession, sometimes by a activemq class,... - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
