[jira] [Commented] (JAMES-3191) Key registration is slow
[ https://issues.apache.org/jira/browse/JAMES-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123521#comment-17123521 ] Benoit Tellier commented on JAMES-3191: --- https://github.com/linagora/james-project/pull/3408 solved the performance issue du to a rabbitmq overuse due to an infinite processing loop. https://github.com/linagora/james-project/pull/3390 contributed reactive registration to the event bus > Key registration is slow > > > Key: JAMES-3191 > URL: https://issues.apache.org/jira/browse/JAMES-3191 > Project: James Server > Issue Type: Improvement > Components: eventbus, IMAPServer >Affects Versions: master >Reporter: Benoit Tellier >Priority: Major > Labels: perf > > We noticed that on *master* SELECT IMAP command is significatively slow > (several minutes) on top of the distributed profile. (See attached > instrumentation average time and percentiles) > A quick performance review links this to listener registration (by key). > A code review leads to: > - The low channel number (3) maybe this count can be safely raised to a > higher number? Like 10? Maybe even configurable? > - EventBus::Register operation is handling IO but is synchronous. The > reactor scheduler backing it up is unspecifed (thus is likely the parrallel > one). We should let the caller specify the scheduler he whishes to run. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3191) Key registration is slow
[ https://issues.apache.org/jira/browse/JAMES-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115825#comment-17115825 ] Benoit Tellier commented on JAMES-3191: --- > Did you run a profiler or use glowroot APM to have some insights on where the > time is spent exactly ? Well, I used the glowroot APM but observation in rabbitMQ admin api shows messages are not consumed fast enough and queues are growing, and Grafana boards also support this analysis. I posted all relevant details in a (private) Linagora operational ticket. > I have serious doubt that raising channel number from 3 to 10 will change > latency from several minutes to some milliseconds > 1. How did you discovered it was related to the channel number? That's a blind proposition that of course need to be fact checked. (I have no real better idea to be honnest) > Why do we need RabbitMQ for a SELECT? Easy. Because: - 1. You need propagating IMAP unsolicitated notifications, that happens upon changes triggered by another protocol session - 2. Maintain the IMAP MSN <-> UID mapping So SelectedMailboxImpl is a mailbox that can be registered to get updates for that given mailbox. > Maybe it is not the channel problem, but the lock time, in this case, we > might do more than just sending a message over RabbitMQ, and we should > prepare the message, then request the channel. That could be related to the change removing the ".block() call" in ReactorRabbitMQChannelPool::borrow(). 4d0da67 `JAMES-2774 Avoid nested block in ReactorRabbitMQChannelPool` is IMO a suspect... > Key registration is slow > > > Key: JAMES-3191 > URL: https://issues.apache.org/jira/browse/JAMES-3191 > Project: James Server > Issue Type: Improvement > Components: eventbus, IMAPServer >Affects Versions: master >Reporter: Benoit Tellier >Priority: Major > Labels: perf > > We noticed that on *master* SELECT IMAP command is significatively slow > (several minutes) on top of the distributed profile. (See attached > instrumentation average time and percentiles) > A quick performance review links this to listener registration (by key). > A code review leads to: > - The low channel number (3) maybe this count can be safely raised to a > higher number? Like 10? Maybe even configurable? > - EventBus::Register operation is handling IO but is synchronous. The > reactor scheduler backing it up is unspecifed (thus is likely the parrallel > one). We should let the caller specify the scheduler he whishes to run. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3191) Key registration is slow
[ https://issues.apache.org/jira/browse/JAMES-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115814#comment-17115814 ] Gautier DI FOLCO commented on JAMES-3191: - I have a few questions: # How did you discovered it was related to the channel number? # Why do we need RabbitMQ for a SELECT? Maybe it is not the channel problem, but the lock time, in this case, we might do more than just sending a message over RabbitMQ, and we should prepare the message, then request the channel. > Key registration is slow > > > Key: JAMES-3191 > URL: https://issues.apache.org/jira/browse/JAMES-3191 > Project: James Server > Issue Type: Improvement > Components: eventbus, IMAPServer >Affects Versions: master >Reporter: Benoit Tellier >Priority: Major > Labels: perf > > We noticed that on *master* SELECT IMAP command is significatively slow > (several minutes) on top of the distributed profile. (See attached > instrumentation average time and percentiles) > A quick performance review links this to listener registration (by key). > A code review leads to: > - The low channel number (3) maybe this count can be safely raised to a > higher number? Like 10? Maybe even configurable? > - EventBus::Register operation is handling IO but is synchronous. The > reactor scheduler backing it up is unspecifed (thus is likely the parrallel > one). We should let the caller specify the scheduler he whishes to run. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Commented] (JAMES-3191) Key registration is slow
[ https://issues.apache.org/jira/browse/JAMES-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17111845#comment-17111845 ] Matthieu Baechler commented on JAMES-3191: -- Did you run a profiler or use glowroot APM to have some insights on where the time is spent exactly ? I have serious doubt that raising channel number from 3 to 10 will change latency from several minutes to some milliseconds > Key registration is slow > > > Key: JAMES-3191 > URL: https://issues.apache.org/jira/browse/JAMES-3191 > Project: James Server > Issue Type: Improvement > Components: eventbus, IMAPServer >Affects Versions: master >Reporter: Benoit Tellier >Priority: Major > Labels: perf > > We noticed that on *master* SELECT IMAP command is significatively slow > (several minutes) on top of the distributed profile. (See attached > instrumentation average time and percentiles) > A quick performance review links this to listener registration (by key). > A code review leads to: > - The low channel number (3) maybe this count can be safely raised to a > higher number? Like 10? Maybe even configurable? > - EventBus::Register operation is handling IO but is synchronous. The > reactor scheduler backing it up is unspecifed (thus is likely the parrallel > one). We should let the caller specify the scheduler he whishes to run. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
