[Resin-interest] An interesting memory leak
Here is an interesting one, If I package my application up as a JAR and put it in WEB-INF/lib then I get memory leaks in the perm gen space as none of the Class objects are garbage collected. This is easily re-producable every time. If however I put the application's .class files inside WEB-INF/class and don't bother with the JAR then it will work fine, no memory leaks, runs forever. Any ideas on what I'm doing wrong or if there is some subtle issue with classloaders here that I don't understand ? ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] java.sql.SQLException: ORA-01001: invalid cursor
On our production Resin 2.1.17 periodicaly we get the following error: [2006-12-07 15:31:56.630] close-idle 153:jdbc/data [active:4, total:10] [2006-12-07 15:31:56.659] close 153:jdbc/data [2006-12-07 15:31:56.659] 153:close() [2006-12-07 15:31:56.660] 153:exn-close(java.sql.SQLException: ORA-01001: invalid cursor ) [2006-12-07 15:31:56.661] java.sql.SQLException: ORA-01001: invalid cursor at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134) at oracle.jdbc.ttc7.TTIoer.processError(TTIoer.java:289) at oracle.jdbc.ttc7.Oclose.receive(Oclose.java:122) at oracle.jdbc.ttc7.TTC7Protocol.close(TTC7Protocol.java:723) at oracle.jdbc.driver.OracleStatement.close(OracleStatement.java :720) at oracle.jdbc.driver.OracleConnection.close_statements( OracleConnection.java:2387) at oracle.jdbc.driver.OracleConnection.close(OracleConnection.java :1480) at com.caucho.sql.SpyConnection.close(SpyConnection.java:557) at com.caucho.sql.XAConnectionAdapter.close(XAConnectionAdapter.java :453) at com.caucho.sql.PoolItem.close(PoolItem.java:650) at com.caucho.sql.DBPool.handleAlarm(DBPool.java:1734) at com.caucho.util.Alarm$AlarmThread.evaluateAlarm(Alarm.java:319) at com.caucho.util.Alarm$AlarmThread.run(Alarm.java:279) is this connection finally become closed or not? --- Sergey Plehov ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] EJB and Resin 3.1
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi there, is it possible to create EJBs packaged in a JAR file and have them available without creating a WAR file and a servlet to access them? Are there any examples on this? I see in the server logfile that my EJBs are compiled, but I get a NamingException when trying to access the from JNDI (from local scope, e.g. packaged in an EAR file). Thanks in advance Markus Wolf - -- NMMN - New Media Markets Networks http://www.nmmn.com/ Langbehnstrasse 6 22761 Hamburg Tel. 040 284 118 - 720 Fax 040 284 118 - 999 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFeXJiDBHISU1oEKERAh2zAJ4r9hI3RAuidG77BgqvysiYJmnSgwCfShVR DXumVHlx44Iy2qmkoWvw+Ns= =slki -END PGP SIGNATURE- ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] Another weird OpenSSL/Resin Error.
Environment: Windows 2003 Server, Resin Pro 3.0.21, OpenSSL 0.9.8b, Java 1.4.2_12 Saw this SSL-related error in our logs this morning just after a spontaneous restart of resing caused by the JVM (jvm.dll): [06:35:48.766] Loaded Socket JNI library. [06:35:48.781] http listening to *:80 [06:35:49.234] https listening to *:443 [06:35:49.234] hmux listening to 127.0.0.1:6802 [06:35:49.250] Resin started in 7000ms [06:35:50.797] resin-file: init [06:37:16.967] web-app root directory should not be the same as resin.home [06:37:16.967] /d:/resin [06:37:16.998] WebApp[] starting [06:37:20.420] java.lang.StackOverflowError [06:37:24.998] java.lang.StackOverflowError [06:37:35.764] java.lang.StackOverflowError [06:37:38.685] java.io.IOException: 4668:error:140C5042:SSL routines:SSL_UNDEFINED_FUNCTION:called a function you should not call:.\ssl\ssl_lib.c:2071: [06:37:38.685] [06:37:38.685] at com.caucho.vfs.JniStream.writeNative(Native Method) [06:37:38.685] at com.caucho.vfs.JniStream.write(JniStream.java:138) [06:37:38.685] at com.caucho.vfs.WriteStream.flushBuffer (WriteStream.java:389) [06:37:38.685] at com.caucho.server.connection.AbstractHttpResponse.finish (AbstractHttpResponse.java:1974) [06:37:38.685] at com.caucho.server.connection.AbstractHttpResponse.close (AbstractHttpResponse.java:260) [06:37:38.685] at com.caucho.server.webapp.WebAppFilterChain.doFilter (WebAppFilterChain.java:190) [06:37:38.685] at com.caucho.server.dispatch.ServletInvocation.service (ServletInvocation.java:229) [06:37:38.685] at com.caucho.server.http.HttpRequest.handleRequest (HttpRequest.java:274) [06:37:38.685] at com.caucho.server.port.TcpConnection.run (TcpConnection.java:511) [06:37:38.685] at com.caucho.util.ThreadPool.runTasks (ThreadPool.java:516) [06:37:38.685] at com.caucho.util.ThreadPool.run(ThreadPool.java:442) [06:37:38.685] at java.lang.Thread.run(Thread.java:534) Rob ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] An interesting memory leak
On Dec 8, 2006, at 00:10 , sksamuel wrote: Here is an interesting one, If I package my application up as a JAR and put it in WEB-INF/lib then I get memory leaks in the perm gen space as none of the Class objects are garbage collected. This is easily re-producable every time. If however I put the application's .class files inside WEB-INF/class and don't bother with the JAR then it will work fine, no memory leaks, runs forever. Any ideas on what I'm doing wrong or if there is some subtle issue with classloaders here that I don't understand ? If what you describe is really happening, it's gotta be nominated for the Bizzaro Bug of the Month or something. :-) You're sure the perm space is being exhausted? Are you using -XX: +PrintGCDetails or something similar to see the results of gc's on the perm space? Also, very important, what environment are you running? Versions of OS, Resin, Java? Rob ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] EJB and Resin 3.1
Do you mean simply having a web application as a place where you deploy your EJBs that are then accessed from other web applications? Or do you mean simply not packaging your web application as an .ear/.war file and still have your EJBs deployed? For the latter case, I don't use .war/.ear files for deployment and I don't remember this being a problem when I used to use EJB with Resin. For the first... the only problem I could think of would be if your application was not set up to start automatically when the server starts, as it could be waiting for the first web access to start the web context. I'm not sure if an EJB call would have the same effect, I don't think so. In any case, one of the first hurdles when deploying EJB is actually finding them in the JNDI tree, so it might be a problem of simply not looking under a wrong name. I used to have a test.jsp page that simply crawled the JNDI tree, it was useful to debug such kind of problems as JNDI does not answer you you almost got it!, so an extra 'jdb/', 'ejb' in your lookup name could be simply your problem. I don't have it here at the moment, but it is quite simple to create such a page, simply to crawl the JNDI tree. Good luck! D. -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi there, is it possible to create EJBs packaged in a JAR file and have them available without creating a WAR file and a servlet to access them? Are there any examples on this? I see in the server logfile that my EJBs are compiled, but I get a NamingException when trying to access the from JNDI (from local scope, e.g. packaged in an EAR file). Thanks in advance Markus Wolf - -- NMMN - New Media Markets Networks http://www.nmmn.com/ Langbehnstrasse 6 22761 Hamburg Tel. 040 284 118 - 720 Fax 040 284 118 - 999 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFeXJiDBHISU1oEKERAh2zAJ4r9hI3RAuidG77BgqvysiYJmnSgwCfShVR DXumVHlx44Iy2qmkoWvw+Ns= =slki -END PGP SIGNATURE- ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] com.caucho.vfs.JniStream.readNative flakiness? [Was: Another weird OpenSSL/Resin Error.]
Environment: Windows 2003 Server, Resin Pro 3.0.21, OpenSSL 0.9.8b, Java 1.4.2_12 More information on what caused the JVM/Resin to spontaneously reboot. I get these fairly often (multiple times per month). They're not always the same exact thing. Often the stack trace references one of the SSL DLL's, but other times, like this one, it references the JVM DLL. In any event, since the Java frames section references some Caucho JNI calls, I'm sending this in. Scott, let me know if you want an official bug filed. I'm attaching two files to this email since the emailer likes to reformat stuff into unreadable chunks. One is the weird SSL_UNDEFINED_FUNCTION error I have never seen before and reported a few minutes. The other is the top portion of the JVM crash log where the jvm.dll was the source of the crash and Caucho JNI routines are being called. Btw, whenever the crash log mentions an SLL DLL as the source of the crash, the Java frames section is always something along these lines: Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j com.caucho.vfs.JniStream.readNative(J[BII)I+0 j com.caucho.vfs.JniStream.read([BII)I+43 j com.caucho.vfs.ReadStream.readBuffer()Z+53 j com.caucho.vfs.ReadStream.waitForRead()Z+12 j com.caucho.server.port.TcpConnection.run()V+181 j com.caucho.util.ThreadPool.runTasks()V+187 j com.caucho.util.ThreadPool.run()V+85 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub Which is similar to the attached JVM DLL crash. What's up with com.caucho.vfs.JniStream.readNative ? Rob SSL_UNDEFINED_FUNCTION.log Description: Binary data jvmDLLCrash.log Description: Binary data ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] An interesting memory leak
On Dec 8, 2006, at 8:37 AM, Rob Lockstone wrote: On Dec 8, 2006, at 00:10 , sksamuel wrote: Here is an interesting one, If I package my application up as a JAR and put it in WEB-INF/lib then I get memory leaks in the perm gen space as none of the Class objects are garbage collected. This is easily re-producable every time. If however I put the application's .class files inside WEB-INF/class and don't bother with the JAR then it will work fine, no memory leaks, runs forever. Any ideas on what I'm doing wrong or if there is some subtle issue with classloaders here that I don't understand ? If what you describe is really happening, it's gotta be nominated for the Bizzaro Bug of the Month or something. :-) You're sure the perm space is being exhausted? Are you using -XX: +PrintGCDetails or something similar to see the results of gc's on the perm space? Also, very important, what environment are you running? Versions of OS, Resin, Java? This sounds like the JDK/introspection/debug perm-gen GC issue (which is really weird). It's a JDK bug, which appears to be fixed in the most recent JDKs. -- Scott Rob ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] resin-interest Digest, Vol 6, Issue 8
If what you describe is really happening, it's gotta be nominated for the Bizzaro Bug of the Month or something. :-) You're sure the perm space is being exhausted? Are you using -XX: +PrintGCDetails or something similar to see the results of gc's on the perm space? Yes of course. and -XX:+TraceClassUnloading so I can see that they do get unloaded when deployed in the classes dir. I'll send you my jar see if it happens on yours too ? ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] com.caucho.vfs.JniStream.readNative flakiness? [Was: Another weird OpenSSL/Resin Error.]
On Dec 8, 2006, at 8:56 AM, Rob Lockstone wrote: Environment: Windows 2003 Server, Resin Pro 3.0.21, OpenSSL 0.9.8b, Java 1.4.2_12 More information on what caused the JVM/Resin to spontaneously reboot. I get these fairly often (multiple times per month). They're not always the same exact thing. Often the stack trace references one of the SSL DLL's, but other times, like this one, it references the JVM DLL. In any event, since the Java frames section references some Caucho JNI calls, I'm sending this in. Scott, let me know if you want an official bug filed. That would be great. I'm attaching two files to this email since the emailer likes to reformat stuff into unreadable chunks. One is the weird SSL_UNDEFINED_FUNCTION error I have never seen before and reported a few minutes. The other is the top portion of the JVM crash log where the jvm.dll was the source of the crash and Caucho JNI routines are being called. Btw, whenever the crash log mentions an SLL DLL as the source of the crash, the Java frames section is always something along these lines: Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j com.caucho.vfs.JniStream.readNative(J[BII)I+0 j com.caucho.vfs.JniStream.read([BII)I+43 j com.caucho.vfs.ReadStream.readBuffer()Z+53 j com.caucho.vfs.ReadStream.waitForRead()Z+12 j com.caucho.server.port.TcpConnection.run()V+181 j com.caucho.util.ThreadPool.runTasks()V+187 j com.caucho.util.ThreadPool.run()V+85 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub Which is similar to the attached JVM DLL crash. What's up with com.caucho.vfs.JniStream.readNative ? readNative itself should be solid. We've run it for days under very heavy load (CPU load around 40 on multi-cpu systems) with no issues. But there has been less time stressing openssl, so there might be something odd there. -- Scott Rob SSL_UNDEFINED_FUNCTION.log jvmDLLCrash.log ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] com.caucho.vfs.JniStream.readNative flakiness? [Was: Another weird OpenSSL/Resin Error.]
On Dec 8, 2006, at 09:10 , Scott Ferguson wrote: On Dec 8, 2006, at 8:56 AM, Rob Lockstone wrote: Environment: Windows 2003 Server, Resin Pro 3.0.21, OpenSSL 0.9.8b, Java 1.4.2_12 More information on what caused the JVM/Resin to spontaneously reboot. I get these fairly often (multiple times per month). They're not always the same exact thing. Often the stack trace references one of the SSL DLL's, but other times, like this one, it references the JVM DLL. In any event, since the Java frames section references some Caucho JNI calls, I'm sending this in. Scott, let me know if you want an official bug filed. That would be great. Ok, I reported it. Looks like Mantis took it twice. The first time, it complained that the file I was trying to upload wasn't valid. So I created a new report. But once I submitted that one, it took me to the list of issues and I saw that it had created two entries for the same bug. The numbers are 1499 and 1500. Feel free to remove one of them as a dup. Rob ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Error using custom log formatter
On Dec 7, 2006, at 6:05 PM, jason rutherglen wrote:
Where org.apache.solr.cluster.ClusterLogFormatter extends
java.util.logging.Formatter. Get this error:
[17:49:24.453] com.caucho.config.LineConfigException: WEB-INF/
web.xml:16: java.l
ang.IllegalAccessException: Class
com.caucho.config.BeanTypeStrategy can not acc
ess a member of class java.util.logging.Formatter with modifiers
protected
I've added this as a bug report. It definitely looks like an
introspection issue, but I'm not sure why the introspection would not
be allowed.
-- Scott
--
package org.apache.solr.cluster;
import java.util.logging.*;
import java.io.*;
import java.text.*;
import java.util.Date;
import java.sql.SQLException;
import java.util.concurrent.locks.*;
import org.apache.commons.lang.exception.*;
import org.apache.commons.dbutils.DbUtils;
/**
* Prints out full stack traces of all nested exceptions
*
* @author jasonr
*/
public class ClusterLogFormatter extends java.util.logging.Formatter {
Date dat = new Date();
private final static String format = {0,date} {0,time};
private MessageFormat formatter;
private ReentrantLock lock = new ReentrantLock();
private Object args[] = new Object[1];
public static void main(String[] args) {
LogManager logManager = LogManager.getLogManager();
//logManager.
}
// Line separator string. This is the value of the line.separator
// property at the moment that the SimpleFormatter was created.
private String lineSeparator = (String)
java.security.AccessController.doPrivileged(
new sun.security.action.GetPropertyAction
(line.separator));
public ClusterLogFormatter() {
super();
}
public String format(LogRecord record) {
lock.lock();
try {
StringBuffer sb = new StringBuffer();
// Minimize memory allocations here.
dat.setTime(record.getMillis());
args[0] = dat;
StringBuffer text = new StringBuffer();
if (formatter == null) {
formatter = new MessageFormat(format);
}
formatter.format(args, text, null);
sb.append(text);
sb.append( );
if (record.getSourceClassName() != null) {
sb.append(record.getSourceClassName());
} else {
sb.append(record.getLoggerName());
}
if (record.getSourceMethodName() != null) {
sb.append( );
sb.append(record.getSourceMethodName());
}
sb.append(lineSeparator);
String message = formatMessage(record);
sb.append(record.getLevel().getLocalizedName());
sb.append(: );
sb.append(message);
sb.append(lineSeparator);
if (record.getThrown() != null) {
Throwable throwable = record.getThrown();
if (throwable instanceof SQLException) {
SQLException sqlException = (SQLException)throwable;
StringWriter stringWriter = new StringWriter();
DbUtils.printStackTrace(sqlException, new PrintWriter
(stringWriter));
sb.append(stringWriter.toString());
} else {
String string = ExceptionUtils.getFullStackTrace(throwable);
sb.append(string);
}
/**
* try {
* StringWriter sw = new StringWriter();
* PrintWriter pw = new PrintWriter(sw);
* record.getThrown().printStackTrace(pw);
* pw.close();
* sb.append(sw.toString());
* } catch (Exception ex) {
* }
**/
}
return sb.toString();
} finally {
lock.unlock();
}
}
}
___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
___
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] EJB and Resin 3.1
If you were using the snapshot version 3.1.s061203, you might ran into a Resin bug. I found my EJB3 stopped working on this version and noticed Resin sample stateless EJB not working either. By the way, Markus, you can find sample code under: resin.home/webapps/resin-doc/tutorial/ejb-stateless/WEB-INF/classes/example/... -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Markus Wolf Sent: Friday, December 08, 2006 8:48 AM To: General Discussion for the Resin application server Subject: Re: [Resin-interest] EJB and Resin 3.1 -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Do you mean simply having a web application as a place where you deploy your EJBs that are then accessed from other web applications? Or do you mean simply not packaging your web application as an .ear/.war file and still have your EJBs deployed? I've meant to create an EAR package containting WARs and EJB-JARs based on annotations and EJB3. But instead of using the Dependency Injection I want to do a JNDI lookup, because we want to access the EJBs from Quercus. If we don't specify a name for a SessionBean in the annotation what is the default JNDI name Resin deploys them at? If I find some time I'll send a demo application package next week. Thanks Markus - -- NMMN - New Media Markets Networks http://www.nmmn.com/ Langbehnstrasse 6 22761 Hamburg Tel. 040 284 118 - 720 Fax 040 284 118 - 999 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFeZclDBHISU1oEKERAq1iAJ9ONsfVpqgn+S3Id6079GmEdVWw+ACfdQFm LAIoBM2HA/uk949rbuXYXIA= =zBtw -END PGP SIGNATURE- ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] com.caucho.vfs.JniStream.readNative flakiness? [Was: Another weird OpenSSL/Resin Error.]
On Dec 8, 2006, at 10:34 AM, Rob Lockstone wrote: Fwiw, I just restarted a server and got three of these SSL-related crashes in a row. It's just really, really bad when you try to restart a server and add it to a pool of active servers and it starts taking requests. Just awful. So it sounds like there might be a startup synchronization issue. Your previous case shows something similar. Again, I'd like to request that, at least for Windows, Caucho includes its own we trust this build of OpenSSL DLL's for use with Resin. The constant crashing is painful to watch. And since I'm the resin guy, even though I'm not a sysadmin, I get blamed when everything goes to pot. It's not fun. :-( We can add the OpenSSL version we tested with. That's a good idea. We can't distribute the dll itself. I'll see if I can figure out how to reproduce it. It may be a bit tricky to track down, though. Java's thread dumps are so much nicer. -- Scott Rob SSLdllCrash.log On Dec 8, 2006, at 09:10 , Scott Ferguson wrote: On Dec 8, 2006, at 8:56 AM, Rob Lockstone wrote: Environment: Windows 2003 Server, Resin Pro 3.0.21, OpenSSL 0.9.8b, Java 1.4.2_12 More information on what caused the JVM/Resin to spontaneously reboot. I get these fairly often (multiple times per month). They're not always the same exact thing. Often the stack trace references one of the SSL DLL's, but other times, like this one, it references the JVM DLL. In any event, since the Java frames section references some Caucho JNI calls, I'm sending this in. Scott, let me know if you want an official bug filed. That would be great. I'm attaching two files to this email since the emailer likes to reformat stuff into unreadable chunks. One is the weird SSL_UNDEFINED_FUNCTION error I have never seen before and reported a few minutes. The other is the top portion of the JVM crash log where the jvm.dll was the source of the crash and Caucho JNI routines are being called. Btw, whenever the crash log mentions an SLL DLL as the source of the crash, the Java frames section is always something along these lines: Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j com.caucho.vfs.JniStream.readNative(J[BII)I+0 j com.caucho.vfs.JniStream.read([BII)I+43 j com.caucho.vfs.ReadStream.readBuffer()Z+53 j com.caucho.vfs.ReadStream.waitForRead()Z+12 j com.caucho.server.port.TcpConnection.run()V+181 j com.caucho.util.ThreadPool.runTasks()V+187 j com.caucho.util.ThreadPool.run()V+85 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub Which is similar to the attached JVM DLL crash. What's up with com.caucho.vfs.JniStream.readNative ? readNative itself should be solid. We've run it for days under very heavy load (CPU load around 40 on multi-cpu systems) with no issues. But there has been less time stressing openssl, so there might be something odd there. -- Scott Rob SSL_UNDEFINED_FUNCTION.log jvmDLLCrash.log ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] com.caucho.vfs.JniStream.readNative flakiness? [Was: Another weird OpenSSL/Resin Error.]
On Dec 8, 2006, at 6:00 PM, Rob Lockstone wrote: We can add the OpenSSL version we tested with. That's a good idea. We can't distribute the dll itself. Damn. :-( Well, can you distribute the source AND your particular build flags, etc? That should allow us to build the dll with the same source/process and hopefully obtain the exact same dll's. I've been using the shining light build: http://www.slproweb.com/ products/Win32OpenSSL.html. If you're rebuilding openssl, you need to make sure you've set the threaded flag (although looking at their docs, it looks like it's set automatically in windows.) It used to be that openssl's default build would compile in non-multithreaded on unix. The build info for resinssl should be part of the pro distribution in modules/c/resinssl if that's helpful tracking this down. -- Scott Rob I'll see if I can figure out how to reproduce it. It may be a bit tricky to track down, though. Java's thread dumps are so much nicer. -- Scott Rob SSLdllCrash.log On Dec 8, 2006, at 09:10 , Scott Ferguson wrote: On Dec 8, 2006, at 8:56 AM, Rob Lockstone wrote: Environment: Windows 2003 Server, Resin Pro 3.0.21, OpenSSL 0.9.8b, Java 1.4.2_12 More information on what caused the JVM/Resin to spontaneously reboot. I get these fairly often (multiple times per month). They're not always the same exact thing. Often the stack trace references one of the SSL DLL's, but other times, like this one, it references the JVM DLL. In any event, since the Java frames section references some Caucho JNI calls, I'm sending this in. Scott, let me know if you want an official bug filed. That would be great. I'm attaching two files to this email since the emailer likes to reformat stuff into unreadable chunks. One is the weird SSL_UNDEFINED_FUNCTION error I have never seen before and reported a few minutes. The other is the top portion of the JVM crash log where the jvm.dll was the source of the crash and Caucho JNI routines are being called. Btw, whenever the crash log mentions an SLL DLL as the source of the crash, the Java frames section is always something along these lines: Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j com.caucho.vfs.JniStream.readNative(J[BII)I+0 j com.caucho.vfs.JniStream.read([BII)I+43 j com.caucho.vfs.ReadStream.readBuffer()Z+53 j com.caucho.vfs.ReadStream.waitForRead()Z+12 j com.caucho.server.port.TcpConnection.run()V+181 j com.caucho.util.ThreadPool.runTasks()V+187 j com.caucho.util.ThreadPool.run()V+85 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub Which is similar to the attached JVM DLL crash. What's up with com.caucho.vfs.JniStream.readNative ? readNative itself should be solid. We've run it for days under very heavy load (CPU load around 40 on multi-cpu systems) with no issues. But there has been less time stressing openssl, so there might be something odd there. -- Scott Rob SSL_UNDEFINED_FUNCTION.log jvmDLLCrash.log ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] com.caucho.vfs.JniStream.readNative flakiness? [Was: Another weird OpenSSL/Resin Error.]
On Dec 8, 2006, at 18:43 , Scott Ferguson wrote: On Dec 8, 2006, at 6:00 PM, Rob Lockstone wrote: We can add the OpenSSL version we tested with. That's a good idea. We can't distribute the dll itself. Damn. :-( Well, can you distribute the source AND your particular build flags, etc? That should allow us to build the dll with the same source/process and hopefully obtain the exact same dll's. I've been using the shining light build: http://www.slproweb.com/ products/Win32OpenSSL.html. Yes, that's exactly the one I'm using. Except the 'b' version. I tried using 'd' with 3.0.21 and it blew up, wouldn't even load the dll's at startup. Have you used 'd' with .21? If so, which dll's and where do you place them? Currently, I place libeay32.dll, libssl32.dll, and ssleay32.dll (the latter two appear to be identical, at least in size) into the resin home directory. Rob If you're rebuilding openssl, you need to make sure you've set the threaded flag (although looking at their docs, it looks like it's set automatically in windows.) It used to be that openssl's default build would compile in non-multithreaded on unix. The build info for resinssl should be part of the pro distribution in modules/c/resinssl if that's helpful tracking this down. -- Scott Rob I'll see if I can figure out how to reproduce it. It may be a bit tricky to track down, though. Java's thread dumps are so much nicer. -- Scott Rob SSLdllCrash.log On Dec 8, 2006, at 09:10 , Scott Ferguson wrote: On Dec 8, 2006, at 8:56 AM, Rob Lockstone wrote: Environment: Windows 2003 Server, Resin Pro 3.0.21, OpenSSL 0.9.8b, Java 1.4.2_12 More information on what caused the JVM/Resin to spontaneously reboot. I get these fairly often (multiple times per month). They're not always the same exact thing. Often the stack trace references one of the SSL DLL's, but other times, like this one, it references the JVM DLL. In any event, since the Java frames section references some Caucho JNI calls, I'm sending this in. Scott, let me know if you want an official bug filed. That would be great. I'm attaching two files to this email since the emailer likes to reformat stuff into unreadable chunks. One is the weird SSL_UNDEFINED_FUNCTION error I have never seen before and reported a few minutes. The other is the top portion of the JVM crash log where the jvm.dll was the source of the crash and Caucho JNI routines are being called. Btw, whenever the crash log mentions an SLL DLL as the source of the crash, the Java frames section is always something along these lines: Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j com.caucho.vfs.JniStream.readNative(J[BII)I+0 j com.caucho.vfs.JniStream.read([BII)I+43 j com.caucho.vfs.ReadStream.readBuffer()Z+53 j com.caucho.vfs.ReadStream.waitForRead()Z+12 j com.caucho.server.port.TcpConnection.run()V+181 j com.caucho.util.ThreadPool.runTasks()V+187 j com.caucho.util.ThreadPool.run()V+85 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub Which is similar to the attached JVM DLL crash. What's up with com.caucho.vfs.JniStream.readNative ? readNative itself should be solid. We've run it for days under very heavy load (CPU load around 40 on multi-cpu systems) with no issues. But there has been less time stressing openssl, so there might be something odd there. -- Scott Rob SSL_UNDEFINED_FUNCTION.log jvmDLLCrash.log ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
