Benoit Tellier created JAMES-3891: ------------------------------------- Summary: ActiveMQ: restarting james can block mail queue Key: JAMES-3891 URL: https://issues.apache.org/jira/browse/JAMES-3891 Project: James Server Issue Type: Improvement Components: Queue Affects Versions: master, 3.7.3 Reporter: Benoit Tellier Fix For: 3.8.0, 3.7.4
Given a mail queue currently containing emails and being consumed, When I restart James, Then after the restart the email consumptions sometime do not resume. The following log can be seen: {code:java} james1 | java.io.FileNotFoundException: /root/var/store/activemq/blob-transfer/10/ID_james1.local-33677-1677150190846-5_1_1_1_1037 (No such file or directory) james1 | at java.base/java.io.FileInputStream.open0(Native Method) james1 | at java.base/java.io.FileInputStream.open(Unknown Source) james1 | at java.base/java.io.FileInputStream.<init>(Unknown Source) james1 | at org.apache.james.queue.activemq.FileSystemBlobStrategy.getInputStream(FileSystemBlobStrategy.java:101) james1 | at org.apache.activemq.blob.BlobDownloader.getInputStream(BlobDownloader.java:38) james1 | at org.apache.activemq.command.ActiveMQBlobMessage.getInputStream(ActiveMQBlobMessage.java:132) james1 | at org.apache.james.queue.activemq.MimeMessageBlobMessageSource.getInputStream(MimeMessageBlobMessageSource.java:45) james1 | at org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:307) james1 | at org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:296) james1 | at org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:292) james1 | at org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:276) james1 | at org.apache.james.server.core.MimeMessageWrapper.<init>(MimeMessageWrapper.java:158) james1 | ... 19 common frames omitted james1 | Wrapped by: javax.mail.MessagingException: IOException while copying message james1 | at org.apache.james.server.core.MimeMessageWrapper.<init>(MimeMessageWrapper.java:165) james1 | at org.apache.james.server.core.MailImpl.setMessage(MailImpl.java:505) james1 | at com.github.fge.lambdas.consumers.ConsumerChainer.lambda$sneakyThrow$9(ConsumerChainer.java:73) james1 | at java.base/java.util.Optional.ifPresent(Unknown Source) james1 | at org.apache.james.server.core.MailImpl$Builder.build(MailImpl.java:279) james1 | at org.apache.james.queue.jms.JMSCacheableMailQueue.createMail(JMSCacheableMailQueue.java:391) james1 | at org.apache.james.queue.activemq.ActiveMQCacheableMailQueue.createMailQueueItem(ActiveMQCacheableMailQueue.java:230) james1 | at org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:249) james1 | ... 12 common frames omitted james1 | Wrapped by: org.apache.james.queue.api.MailQueue$MailQueueException: Unable to dequeue next message james1 | at org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:260) james1 | at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44) james1 | at reactor.core.publisher.FluxRepeatPredicate$RepeatPredicateSubscriber.resubscribe(FluxRepeatPredicate.java:120) james1 | at reactor.core.publisher.MonoRepeatPredicate.subscribeOrReturn(MonoRepeatPredicate.java:48) james1 | at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:55) james1 | at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194) james1 | at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) james1 | at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) james1 | at java.base/java.util.concurrent.FutureTask.run(Unknown Source) james1 | at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) james1 | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) james1 | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) james1 | at java.base/java.lang.Thread.run(Unknown Source) james1 | Wrapped by: reactor.core.Exceptions$ErrorCallbackNotImplemented: org.apache.james.queue.api.MailQueue$MailQueueException: Unable to dequeue next message; james1 | nested exception is: james1 | javax.mail.MessagingException: IOException while copying message; james1 | nested exception is: james1 | java.io.FileNotFoundException: /root/var/store/activemq/blob-transfer/10/ID_james1.local-33677-1677150190846-5_1_1_1_1037 (No such file or directory) {code} The issue is that blobs are handle via file so their deletion is not linked to the JMS trasaction: the JMS transaction can be rolled back for a deleted blob, thus leading to messages without blob. Steps to reproduce: - Send 5000 mails to james of 500 bytes each {code:java} smtp-source -A -C100 -r 1 -l 500 -m 5000 -s 20 -d -c -f a...@other.com -M other.com -t bob@localhost 172.24.0.4:25 {code} - Once all mails are queued, restart james several time during processing - Increasing the count of spooler workers helps. As a fix I propose to discard emails not backed by a blob. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org