chibenwa commented on code in PR #3194: URL: https://github.com/apache/james-project/pull/3194#discussion_r4070261282
########## 0076-artemis-mailqueue-migration.md: ########## @@ -0,0 +1,65 @@ +# 76. Migration from ActiveMQ Classic to ActiveMQ Artemis for Embedded Mail Queue Review Comment: Remove this file please ########## src/adr/0076-artemis-mailqueue-migration.md: ########## @@ -0,0 +1,65 @@ +# 76. Migration from ActiveMQ Classic to ActiveMQ Artemis for Embedded Mail Queue + +Date: 2026-09-22 + +## Status + +Accepted & implemented. + +## Context + +Apache James provides a mail spool queue mechanism implemented via JMS (`server/queue/queue-activemq`). +Previously, the embedded broker relied on Apache ActiveMQ "Classic" (5.x/6.x) with KahaDB persistence adapter and ActiveMQ-specific `BlobMessage` / `FileSystemBlobTransferPolicy`. + +Under real-world and high workloads, ActiveMQ Classic suffered from severe design and performance limitations: + +1. **Throughput and Latency Bottleneck:** + - Out-of-the-box ActiveMQ Classic experiences high latency (~100 ms) and limited throughput (~25-400 msgs/s depending on storage sync and KahaDB locking). + - Java profiling revealed that up to 95% of total request processing time inside James was spent waiting on ActiveMQ / KahaDB queue commits. + +2. **Head-of-Line Blocking with Delayed Mails (JAMES-4192):** + - In James Remote Delivery, retries are scheduled with delays (e.g., 30–60 minutes) upon encountering temporary exceptions (such as greylisting / SMTP 421). + - Dequeueing uses JMS message selectors (`JAMES_NEXT_DELIVERY <= currentTimeMillis() OR FORCE_DELIVERY = true`). + - In ActiveMQ Classic, messages are read into memory in batches governed by `maxPageSize` (default: 200). If ≥200 delayed messages reside at the head of the queue, the selector rejects them, but ActiveMQ Classic **stops evaluating further pages**. + - As a result, the entire outgoing delivery stalls for the duration of the delay window, completely starving non-delayed, ready-to-deliver messages. + +3. **Proprietary Blob Messages:** + - Out-of-band blob messages (`BlobMessage`, `FileSystemBlobTransferPolicy`) in ActiveMQ Classic are non-portable, lack clean garbage-collection guarantees, and complicate embedded deployments. + +Apache ActiveMQ Artemis is the modern, next-generation message broker from the ActiveMQ project, designed from the ground up for asynchronous non-blocking I/O, low latency, native journal persistence, and Jakarta Messaging 3.x compliance. + +## Decision + +Migrate the embedded message queue broker in Apache James from ActiveMQ Classic to **Apache ActiveMQ Artemis**: + +1. **Embedded Broker Architecture (`EmbeddedActiveMQ.java`):** + - Use `org.apache.activemq.artemis.core.server.embedded.EmbeddedActiveMQ` running in-VM (`vm://0`). + - Use Artemis native high-performance journal storage for bindings, journal, paging, and large messages (`setPersistenceEnabled(true)`). + - Configure optimal embedded JMS client connection factory with `setConsumerWindowSize(0)` and `setBlockOnAcknowledge(true)`. + +2. **Resolution of Delayed Delivery and Head-of-Line Blocking (JAMES-4192):** + - ActiveMQ Artemis natively supports the Jakarta Messaging delivery delay specification via an internal dedicated scheduler (`ScheduledDeliveryHandler`). + - Delayed and scheduled messages are held out-of-band by the scheduler and do not occupy active queue paging buffers (`maxPageSize`), completely eliminating the consumer starvation and head-of-line blocking defect seen in ActiveMQ Classic. + - Delayed messages remain tracked on the destination (accessible via `scheduledCount`), preventing inconsistencies between queue reporting and delivery state. Review Comment: Could we add a test regarding this claim ? ########## ARTEMIS_MIGRATION_EN.md: ########## @@ -0,0 +1,156 @@ +# Migration Guide: Apache James ActiveMQ (Classic) to Apache ActiveMQ Artemis Review Comment: remove this file please ########## ARTEMIS_MIGRATION.md: ########## @@ -0,0 +1,157 @@ +# Миграция Apache James с ActiveMQ (Classic) на Apache ActiveMQ Artemis + +В данном документе подробно описаны архитектурные и кодовые изменения, выполненные в проекте James (`C:\soft\james_src\james-project-fast`) для замены встроенного брокера сообщений **Apache ActiveMQ (Classic 6.x)** на **Apache ActiveMQ Artemis (2.56.0)**. Review Comment: remove this file please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
