Re: JPAStressTest with derby, h2 and mysql

[Eric Charles]

Hi Norman,

Similar exceptions with properties.put("openjpa.LockTimeout", "30000");
I think the environment simply can not follow the load (more than one 
mail sent each ms).
Bootleneck can be anywhere (connection creation, database lock, 
openjpa,...).
This is why I put a sleep between each mail sending.
Real servers will better hold the pressure.

So I think I'm limited by my environment : it does not allow me to have 
a really heavy load to validate that IMAP-137 is resolved.
Well, it is probably, as before you commits, I directly had the 
duplicate key exception.

How much mails can you hold on your env with h2 inmemory ?

Tks,

Eric


On 05/01/2010 09:08 AM, Norman Maurer wrote:
Hi Eric,

thx for your tests. The order of the uid is caused by the threads,
thats nothing to worry about.

About the timeout, can you try to add this to the test case:

properties.put("openjpa.LockTimeout", "30000");

And keep in mind that we don't use any connection pooling in the test
case so the performance should be better when use it in deployment.

Bye,
Norman


2010/5/1 Eric Charles<eric.char...@u-mangate.com>:
   
Hi,

I just tested the JPAStressTest (for IMAP-137 JPA fails to persist
MailboxMembership Entity on heavy load) Norman committed yesterday.

I made some tests towards derby (memory and embedded) , h2 (memory and
embedded) and mysql with following configs (and adding the needed jdbc
drivers in pom.xml):

        // Derby Memory
        properties.put("openjpa.ConnectionDriverName",
org.apache.derby.jdbc.EmbeddedDriver.class.getName());
        properties.put("openjpa.ConnectionURL",
"jdbc:derby:memory:derbyimap;create=ue");

        // Derby Embedded
//        properties.put("openjpa.ConnectionDriverName",
org.apache.derby.jdbc.EmbeddedDriver.class.getName());
//        properties.put("openjpa.ConnectionURL",
"jdbc:derby:derbyimap;create=ue;user=root;password=root");

        // H2 Memory
//        properties.put("openjpa.ConnectionDriverName",
org.h2.Driver.class.getName());
//        properties.put("openjpa.ConnectionURL",
"jdbc:h2:mem:h2imap;DB_CLOSE_DELAY=");

        // H2 Embedded
//        properties.put("openjpa.ConnectionDriverName",
org.h2.Driver.class.getName());
//        properties.put("openjpa.ConnectionURL",
"jdbc:h2:~/h2/h2imap;USER=ot;PASSWORD=root");

        //Mysql
//        properties.put("openjpa.ConnectionDriverName",
com.mysql.jdbc.Driver.class.getName());
//        properties.put("openjpa.ConnectionURL",
"jdbc:mysql://localhost:3306/msqlimap?user=ot&password=root&createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=utf-8");


I first changed the maximum number of messages from 1.000 to 100.000 and saw
on my linux dev PC (james server, stresstest and databases all running on
same dev PC):

Derby: after many mails sending:
java.io.IOException: Too many open files
...
Exception in thread "pool-1-thread-1044" java.lang.NoClassDefFoundError:
org/apache/james/imap/api/display/HumanReadableText
...

H2: after 400 mails:
java.io.IOException: Too many open files
...
Caused by:<openjpa-1.2.2-r422266:898935 nonfatal general error>
org.apache.openjpa.persistence.PersistenceException: Timeout trying to lock
table MEMBERSHIP [50200-79] {prepstmnt 406576
INSERT INTO Membership (mailboxId, uid, answered, deleted, draft,
        flagged, internalDate, recent, seen, size, MESSAGE_ID)
    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
[params=ong) 1, (long) 49, (int) 0, (int) 0, (int) 0, (int) 0, (Timestamp)
2010-05-01 08:37:10.293, (int) 0, (int) 0, (int) 25, (long) 4151]}
[codeP200, state=HYT00]
...

MySql : after +/- 1000 (always around 1002 and 1007) mails :
Exception in thread "pool-1-thread-84"<openjpa-1.2.2-r422266:898935
nonfatal user error>  org.apache.openjpa.persistence.NoResultException: The
query on candidate type "class
org.apache.james.imap.jpa.mail.model.JPAMailbox" with filter "SELECT mailbox
FROM Mailbox mailbox WHERE mailbox.mailboxId =idParam" was configured to
have a unique result, but no instance matched the query.
....
org.apache.james.imap.mailbox.MailboxNotFoundException: Mailbox
'#mail.INBOX' not found.
...

3 different databases, 3 different behaviour/exceptions.

Running 3 components (james server, stresstest, database) on my
"not-so-strong-PC" may be an issue.

To make the tests successful, I added a Thread.sleep(5) (15 ms for embedded
derby and h2) between each mail sending.


I suppose we can say that in a real environment, the load will never be that
heavy, and that real server will perform better.


I also saw that the uid order is not respected:

Append message with uidb58
Append message with uidb59
Append message with uidb56
Append message with uidb57

This could come from a print mismatch, and not from uid being generated in a
bad order.

Tks in advance for your feedbacks, if any,


Eric


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org


     
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

   


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org