I checked 2 thread dumps you provided so I can only comment what I saw there. I did not see any deadlock. Only problem I saw was in main thread where it was waiting for connection.
1. "main" prio=10 tid=0x0000000040feb000 nid=0x4fb0 in Object.wait() [0x 00007fefd3812000] 2. java.lang.Thread.State: WAITING (on object monitor) 3. at java.lang.Object.$$YJP$$wait(Native Method) 4. at java.lang.Object.wait(Object.java) 5. at java.lang.Object.wait(Object.java:485) 6. at org.apache.commons.pool.impl.GenericObjectPool.borrowObject (GenericObjectPool.java:1104) 7. - locked <0x00000000d9b90c40> (a org.apache.commons.pool.impl. GenericObjectPool$Latch) 8. at org.apache.commons.dbcp.PoolingDataSource.getConnection (PoolingDataSource.java:106) 9. at org.apache.commons.dbcp.BasicDataSource.getConnection (BasicDataSource.java:1044) 10. at org.apache.jackrabbit.core.util.db.ConnectionHelper. getConnection(ConnectionHelper.java:439) You should be able to get thread dump any time unless JVM process is completely frozen IMO. Do you use jstack to get thread dump? Marek ---------- Původní zpráva ---------- Od: Daniel Hobi <[email protected]> Datum: 8. 3. 2013 Předmět: Re: JVM hangs during startup (indexing) (2) "Hi Marek Thanks for your suggestions! Unfortunately, changing the JVM gc options so that the JVM will no longer hang just worked ONCE. Therefore I was not able to get a "nice" stack trace anymore :-( Connection Pool: We had set the max connection pool size to 5. As you suggested I set it to 15. Same effect. Even 50 or 100 did not help. Interestingly, postgres shows just 3 open connection for jackrabbit. Is this a suspicious behavior or does it simply not need more connections and is therefore not spawning new connections? Enabling debug, setting login and socket timeout on the postgres-jdbc-driver did not help either. I even changed the pool implementation from dbcp to c3p0 in jackrabbit-core- 2.6.0 to be "sure" it is not a pool bug. Further we are not able to reproduce the hang with: - Slower machine, postgres 8.3 - Very fast machine, postgres 8.4 (only hanged 1 of 5 times so far) - Faster machine, mysql This issue is slowly driving me nuts ;-) Thanks! Daniel Am 07.03.2013 um 16:24 schrieb [email protected]: > Also randomly we had problem with not released connections in JBoss > connection pool and Postgres. Now long time no occurrence. > In any case you can check number/list of open connections in psql . In our > case it was > 97 open connection (by default 100 open connections in postgres and 3 are > reserved for admin). It happenned even if we had much smaller > max connection pool size. As it happened randomly we did not find any > solution. We tried to change connection pool size and it does not > happen any more. Still no idea what was cause of this. In any case using > psql you can find out easily if it is also your problem. > > Query is for 9.x: SELECT * FROM pg_stat_activity; > Marek > > -- > Marek Slama > [email protected] > > > > ---------- Původní zpráva ---------- > Od: [email protected] > Datum: 7. 3. 2013 > Předmět: Re: JVM hangs during startup (indexing) (2) > > "Quick look: It hangs in connection pool - connection pool if it cannot > return free connection to caller it waits forever by default. > How big is your connection pool? (Maximum size? It would be good to patch > connection pool to see who/how many connections is > used and why connections are used not release. In our config we use > connection max pool size 15 and it is enough.) So first > fast fix would be to increase max connection pool size but if there is > problem with not released connection it will not help for long. > > Good news is it is not problem with memory or gc (at least IMO). > > Marek > > -- > Marek Slama > [email protected] > > > > ---------- Původní zpráva ---------- > Od: Daniel Hobi <[email protected]> > Datum: 7. 3. 2013 > Předmět: JVM hangs during startup (indexing) (2) > > "Hi there > > > > Some updates on our issue started here: > > http://mail-archives.apache.org/mod_mbox/jackrabbit-users/201301.mbox/%3C 201 > 30115135117.186960%40gmx.net%3E > > > > We're still facing the JVM freeze (possibly during minor gcs) even if... > > ...we updated Jackrabbit from 2.2.13 to 2.6.0. > > ...we updated the Sun JVM to the latest version (1.6.39) and even tried > OpenJDK. > > ...we increased the JVM heap size (up to 1.5gb). > > ...we upgraded PostgreSQL to the latest version (8.4.x / 9.2.x) available. > > > > But there is good news too: > > - I've just played around with the JVM's garbage collection > options. Even if the index process still hangs the JVM will no longer block. > I was able to get an interesting thread dump: > <http://pastebin.com/uSrXpej8> http://pastebin.com/uSrXpej8 > > - With the addition JVM Param -XX:MaxNewSize=10m we can make the > hang occur faster (less than 100 minor gcs) > > > > I would be very grateful if someone could have a look at this. We're running > (slowly but surely) out of ideas :-( > > > > Thanks! > > Daniel"""
