I wasn't planning to do that much messing with mordred. Eventually it just
goes away.
--- Noel
-----Original Message-----
From: Steve Short [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, February 11, 2003 13:28
To: James Users List
Subject: RE: RE : Really slow spooling again... :(
If you're messing with the Mordred stuff, do you want to consider a
simple statement cache so that the PreparedStatements aren't recompiled
all the time? It may not make a big difference considering the amount of
IO involved - but you never know.
Steve
> -----Original Message-----
> From: Noel J. Bergman [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, February 11, 2003 8:35 AM
> To: James Users List
> Subject: RE: RE : Really slow spooling again... :(
>
>
> Diego,
>
> At this time, there are no plans to replace mordred in James
> v2.1. Those are some of the things that will be part of James v3.
>
> I spent quite a bit of time with mordred yesterday. One
> thing that I found was that it really helps to add
> autoreconnect=true to the MySQL URL. I had a spool with over
> 200,000 items in it. Running James with no SMTP connections,
> so only the spool is using JDBC, I could not run for long
> before getting exceptions. With the auto-reconnect support,
> I was able to process the entire spool at a rate of roughly
> 1667 messages per minute without changing mordred at all.
>
> I would like to see spool processing much faster than that,
> but where I am at in my testing is this: if I don't rate
> limit postal many of the messages accummulate in the queue.
> If I rate limit postal to 600 messages per minute, I can run
> unimpeded for hours with 5% free CPU. Since normally I have
> 0% free CPU during a load test, I am figuring that for this
> system, 600 MPM is about the limit. That is with the spooler
> performing work. With the spooler discarding messages at the
> top of root, I can go about 1600 MPM, and wit the spooler
> disabled, I seem to recall about 3000 MPM. So there appears
> to be considerable room for improvement in the spooler.
>
> I was planning to make some tweaks to mordred, and perhaps
> there are some changes we could make to the table structure,
> but right now the biggest improvement in reliability for me
> over the stock config.xml was to add the autoreconnect=true
> parameter to the MySQL URL.
>
> Ultimately, I suspect that we'll have to revisit the spooling
> mechanism, and possibly do some rate limiting on the incoming
> connection based upon the spooler.
>
> --- Noel
>
> -----Original Message-----
> From: Diego Castillo [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, February 11, 2003 4:30
> To: 'James Users List'
> Cc: Diego Castillo, INEXBEE
> Subject: RE : Really slow spooling again... :(
>
>
> Hi Noel,
>
> I have seen your mail on Mordred and some issues with JDBC
> spool, which you seem to have corrected in latest build.
>
> I am experiencing myself some problems on heavy load
> conditions (more than 5 mails per second). MySQL gets tired
> and starts responding slowly. This slows James down, so the
> spool queue grows and MySQL gets even slower. At some point
> it takes really long to take a mail out of the spool (>60s),
> and Mordred fires a DB connection timeout. Then it gets what
> looks to me like a deadlock and James hangs (see stack trace
> below). I need to restart James and OPTIMIZE TABLE spool,
> following Danny's advice.
>
> Is this the issue you have corrected in 2.1.1a7? I am using
> 2.1. I am looking forward searchable datasources via JNDI,
> which Serge seems to have added to HEAD. Will this be part of
> the 2.1.1 final release?
>
> Regards,
>
>
> Diego
>
>
> *** Stack trace ***
>
> database-connections.maildb: java.lang.Throwable
> at org.apache.james.util.mordred.PoolConnEntry
> .lock(PoolConnEntry.java:84)
> at org.apache.james.util.mordred.JdbcDataSource
> .getConnection(JdbcDataSource.java:127)
> at org.apache.james.mailrepository.JDBCSpoolRepository
> .loadPendingMessages(JDBCSpoolRepository.java:228)
> at org.apache.james.mailrepository.JDBCSpoolRepository
> .getNextPendingMessage(JDBCSpoolRepository.java:203)
> at org.apache.james.mailrepository.JDBCSpoolRepository
> .accept(JDBCSpoolRepository.java:137)
> at org.apache.james.transport.mailets.RemoteDelivery
> .run(RemoteDelivery.java:577)
> at java.lang.Thread.run(Thread.java:536)
>
> -----Message d'origine-----
> De : Noel J. Bergman [mailto:[EMAIL PROTECTED]]
> Envoye : mardi 11 fevrier 2003 01:21
> A : James Users List
> Objet : RE: Really slow spooling again... :(
>
> Kenny,
>
> Try adding some additional threads to RemoteDelivery, but
> unless you also configured RemoteDelivery's output spool for
> JDBC, do *not* do it unless you are using v2.1.1a7 or later,
> due to a synchronization error that I found and fixed.
>
> Yes, the record at openrbl.org doesn't look good for you.
> Are you a colo, or do you need a host?
>
> --- Noel
>
> -----Original Message-----
> From: Kenny Smith [mailto:[EMAIL PROTECTED]]
> Sent: Monday, February 10, 2003 18:31
> To: James Users List
> Subject: Re: Really slow spooling again... :(
>
>
> Hi Noel et al,
>
> It appears that connections from my server are getting denied
> by hotmail and yahoo (my ISP just got labeled as a spammer...
> I'm trying to find a new ISP), so I'm getting a lot of
> timeouts and a lot of retries. I can only assume that it's
> just going through the spool really slowly because it takes a
> long time to timeout. :/
>
> Me need new ISP. *sigh*
>
> Kenny
>
> Noel J. Bergman wrote:
>
> > Kenny,
> >
> > I have run tests most nights lately, each about 1 million messages.
> They
> > run through the spooler until they encounter a matcher that discards
> them
> > all. The tests average about 1400 messages per minute on
> the 400 mhz
> > Celeron test bed.
> >
> > The test basically exercises the heck out of the SMTP
> handler and the
> > spooler. The spooler is configured in mysql as:
> >
> >
> >
> >
> >
> > The data-source is the standard entry for MySQL, except that the
> > database is "test" on that machine.
> >
> > What database (type and version) are you using? One thing that I
> found a
> > while ago was that on very rare occassions, I could go into
> MySQL, do
> a
> > manual SQL query, and notice an excessive amount of time
> (let's say 1
> > second instead of 0.006). Restarting MySQL (and James because of
> > mordred) cleared
> > that up.
> >
> > The fact that they are in transport is insufficient. What
> you need to
> > do is see where in the processor they are. Look for entries of the
> > kind:
> >
> > Checking with
> > Servicing by
> >
> > Those should tell us specifically where in the transport the message
> has
> > gotten, and the timestamps will tell us how long it is
> taking to move
> > through the spool.
> >
> > --- Noel
> >
> > -----Original Message-----
> > From: Kenny Smith [mailto:[EMAIL PROTECTED]]
> > Sent: Monday, February 10, 2003 13:10
> > To: [EMAIL PROTECTED]
> > Subject: Really slow spooling again... :(
> >
> > Hi all,
> >
> > I'm having all of my mail backup in the spool tables (using
> JDBC) with
> a
> > message_state of 'transport'. I've turned up my logging to
> DEBUG, but
> > I'm not seeing anything in the logs that looks relevant to slow
> > performance. I sent 230 email 2 hours ago and there are still 190 in
> the
> > spool/transport waiting to get sent.
> >
> > I saw the recent conversation about indexes on the spool
> table causing
> > slow downs after a while, so I stopped James, dumped the table,
> removed
> > the indicies, put the table back in and started James up. Nothing
> > appears to have changed.
> >
> > I'm running James 2.1 with jdk1.4 on Solaris.
> >
> > Any help is appreciated.
> >
> > Kenny Smith
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
