Hello all,
As I had already been writing here, I did encounter significant issues
during a recent deployment [1]
[1] https://www.mail-archive.com/server-dev@james.apache.org/msg73848.html
This did lead to [2] implementing backpressure for IMAP FETCH. Which had
been mitigating the issue.
[2] https://issues.apache.org/jira/projects/JAMES/issues/JAMES-3997
But not really well-enough. As the count of users/mails increases I
ended up with some new OutOfMemory exception related to IMAP usage from
this weekend.
I thus did take the time to write a test regarding backpressure [3] (not
reading the socket and instrumenting the mailbox layer to see what is
actually pulled) and started playing with some related Netty settings [4].
[3] https://github.com/apache/james-project/pull/2128
[4] https://github.com/apache/james-project/pull/2129
However high/low level write buffer watermarks seems ineffective: it
takes dozens of several MB messages to be written for the back-pressure
to quick-in. And the default values (32KB/64KB) are very low compared to
a problematic message size. Netty expertise is more than welcome here!
Another problem is that as of today message content is loaded as a byte
array by the mailbox layer. For a request like IMAP FETCH (BODY[]) this
is ineffective and we could rather be streaming it straight from the
object store (even applying backpressure from within a single message
write). Yet this would require a major refactoring of mailbox / imap
code. And also a bullet proof lifecycle management for connections/
temporary files.
Thoughts?
Benoit
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
