Re: [zones-discuss] Weird Solaris 8 container problem (fwd)
Hey Rich, Looks like it is crashing in the jvm (in JIT code). There are also some jni libraries loaded and being called into by other threads. It would help to know what library is mapped in. I think support should be contacted. My first guess is that they are hitting either an old bug in the jvm, or whatever jni library they are using. This could be an s8c issue, but I'd have somebody look ath the java first. -Steve L. On Tue, Jun 23, 2009 at 09:35:06AM -0700, Rich Teer wrote: > Hi Steve, > > Here are the answers to your questions, as provided by my customer. > I hope the attached pstack and mdb sessions get through unscathed! > > Cheers, > > -- > Rich Teer, SCSA, SCNA, SCSECA > > URLs: http://www.rite-group.com/rich > http://www.linkedin.com/in/richteer > > -- Forwarded message -- > Date: Thu, 18 Jun 2009 12:55:23 -0700 > > Thank you Rich, > > I am collecting the requested information. Can you help me please answer > these questions better? Here are preliminary versions of the answers > > > > Do you mean you've tried both 1.2.2 and 1.5? Is the failure identical > > with > > both jvms? > > Answer: The server would not start up with Java 1.5. The Solaris > "native" version of 1.2.2 crashes on the first (?) execution of the Java > code. > > > > The ::stack should be correct. Have you looked at the core from the > > global zone? It shouldn't matter, but it can't hurt. Also use pstack > > on the core. > > Answer: I am attaching the results of pstack and MDB session to this > email > I did not look at the core from the global zone. I am not sure how to do > that. > > > > > Is it in hotspot (dynamcially generated) code? If so the function > names > > will > > just be hex. Is it dying the jvm itself, or in jni code (java > bindings to > > native code provided by Vantive)? > > I don't know how to answer this question. > The server uses libjvm.so rather than creates a JVM session (process) > using fork() or exec(). > > > > Is the application threaded? > > Yes, it is threaded. > > > Thank you, > > Vlad > > P.S. Currently playing with MDB with a little success. I don't know > assembler > > > > > > -Original Message- > > From: Rich Teer [mailto:rich.t...@rite-group.com] > > Sent: Thursday, June 18, 2009 10:14 AM > > To: Vladimir Ryzhov; Andy Woodward > > Subject: Re: [zones-discuss] Weird Solaris 8 container problem (fwd) > > > > Hi guys, > > > > Here's a response I got on the Zones mailing list about the weird > > Vantive crashes. Could you please reply to me with the answers to > > Steve's questions, and I'll forward them to the list. > > > > -- > > Rich Teer, SCSA, SCNA, SCSECA > > > > URLs: http://www.rite-group.com/rich > > http://www.linkedin.com/in/richteer > > > > -- Forwarded message -- > > Date: Wed, 17 Jun 2009 17:59:04 -0700 > > From: Steve Lawrence > > To: Rich Teer > > Cc: Zones discuss > > Subject: Re: [zones-discuss] Weird Solaris 8 container problem > > > > On Wed, Jun 17, 2009 at 02:56:59PM -0700, Rich Teer wrote: > > > Hi all, > > > > > > IHAC who is trying to run one of their applications in a Solaris 8 > > > branded zone. The global OS is Solaris 10 5/09 and we're using > > > Solaris 8 containers version 1.0.1, on a Sun Fire 280R server with > > > a 750 MHz CPU and 6 GB of RAM. > > > > > > Although apparently quite tempremental, their app runs acceptably > > > when run on Solaris 8 natively (i.e., S8 on bare metal rather than > > > in a branded zone), but crashes very frequently when run in the > > > branded container. Of course, the applicatiopn source code is > > > unavailable... :-( > > > > > > The application, called Vantive, talks to an Oracle 8.1.6 database, > > > and is written in Java. The Vantive app ships with version 1.2.2 > > > of the Java runtime, and we've tried version 1.5. > > > > Do you mean you've tried both 1.2.2 and 1.5? Is the failure identical > > with > > both jvms? > > > > > > > > Annoyingly, when we try trussing the errant process, it doesn't > crash! > > > When a crash does happen, a core dump usually occurs, but hasn't > been > > > too helpful. The crashes do seem to be happening from within the > JVM, > > > if the ::stack output from mdb is to be believed. > > > > The ::stack should be correct. Have you looked at the core from the > > global zone? It shouldn't matter, but it can't hurt. Also use pstack > > on the core. > > > > Is it in hotspot (dynamcially generated) code? If so the function > names > > will > > just be hex. Is it dying the jvm itself, or in jni code (java > bindings to > > native code provided by Vantive)? > > > > > > > > Does this ring any bells? Is there anything we can do to help debug > > this? > > > Note that the branded zone seems to work just fine apart from this > one > > > (rather major) issue. > > > > No bells. The only Java problem I've seen was in 1.1.8, and it does > not > > exist on java 1.2+. Best to contact
zones-discuss@opensolaris.org
I note that I got myself into a boatload of trouble by building zones and removing /usr without removing those others. LiveUpdate in particular crashed and burned. Once you remove /usr, you need to go all the way and make a whole-root zone. (this is not documented, that I can see; the docs warn against removing /lib, /platform, /sbin and /bin but don't mention /usr) ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Weird Solaris 8 container problem (fwd)
Hi Steve, Here are the answers to your questions, as provided by my customer. I hope the attached pstack and mdb sessions get through unscathed! Cheers, -- Rich Teer, SCSA, SCNA, SCSECA URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer -- Forwarded message -- Date: Thu, 18 Jun 2009 12:55:23 -0700 Thank you Rich, I am collecting the requested information. Can you help me please answer these questions better? Here are preliminary versions of the answers > Do you mean you've tried both 1.2.2 and 1.5? Is the failure identical > with > both jvms? Answer: The server would not start up with Java 1.5. The Solaris "native" version of 1.2.2 crashes on the first (?) execution of the Java code. > The ::stack should be correct. Have you looked at the core from the > global zone? It shouldn't matter, but it can't hurt. Also use pstack > on the core. Answer: I am attaching the results of pstack and MDB session to this email I did not look at the core from the global zone. I am not sure how to do that. > > Is it in hotspot (dynamcially generated) code? If so the function names > will > just be hex. Is it dying the jvm itself, or in jni code (java bindings to > native code provided by Vantive)? I don't know how to answer this question. The server uses libjvm.so rather than creates a JVM session (process) using fork() or exec(). > Is the application threaded? Yes, it is threaded. Thank you, Vlad P.S. Currently playing with MDB with a little success. I don't know assembler > -Original Message- > From: Rich Teer [mailto:rich.t...@rite-group.com] > Sent: Thursday, June 18, 2009 10:14 AM > To: Vladimir Ryzhov; Andy Woodward > Subject: Re: [zones-discuss] Weird Solaris 8 container problem (fwd) > > Hi guys, > > Here's a response I got on the Zones mailing list about the weird > Vantive crashes. Could you please reply to me with the answers to > Steve's questions, and I'll forward them to the list. > > -- > Rich Teer, SCSA, SCNA, SCSECA > > URLs: http://www.rite-group.com/rich > http://www.linkedin.com/in/richteer > > -- Forwarded message -- > Date: Wed, 17 Jun 2009 17:59:04 -0700 > From: Steve Lawrence > To: Rich Teer > Cc: Zones discuss > Subject: Re: [zones-discuss] Weird Solaris 8 container problem > > On Wed, Jun 17, 2009 at 02:56:59PM -0700, Rich Teer wrote: > > Hi all, > > > > IHAC who is trying to run one of their applications in a Solaris 8 > > branded zone. The global OS is Solaris 10 5/09 and we're using > > Solaris 8 containers version 1.0.1, on a Sun Fire 280R server with > > a 750 MHz CPU and 6 GB of RAM. > > > > Although apparently quite tempremental, their app runs acceptably > > when run on Solaris 8 natively (i.e., S8 on bare metal rather than > > in a branded zone), but crashes very frequently when run in the > > branded container. Of course, the applicatiopn source code is > > unavailable... :-( > > > > The application, called Vantive, talks to an Oracle 8.1.6 database, > > and is written in Java. The Vantive app ships with version 1.2.2 > > of the Java runtime, and we've tried version 1.5. > > Do you mean you've tried both 1.2.2 and 1.5? Is the failure identical > with > both jvms? > > > > > Annoyingly, when we try trussing the errant process, it doesn't crash! > > When a crash does happen, a core dump usually occurs, but hasn't been > > too helpful. The crashes do seem to be happening from within the JVM, > > if the ::stack output from mdb is to be believed. > > The ::stack should be correct. Have you looked at the core from the > global zone? It shouldn't matter, but it can't hurt. Also use pstack > on the core. > > Is it in hotspot (dynamcially generated) code? If so the function names > will > just be hex. Is it dying the jvm itself, or in jni code (java bindings to > native code provided by Vantive)? > > > > > Does this ring any bells? Is there anything we can do to help debug > this? > > Note that the branded zone seems to work just fine apart from this one > > (rather major) issue. > > No bells. The only Java problem I've seen was in 1.1.8, and it does not > exist on java 1.2+. Best to contact support to debug. Could be a jvm (or > other bug) that they were lucky enough to never hit on their native s8 > system. Given the truss thing, it is likely a race/timing issue, they > they > may have never hit on slower systems. Is the application threaded? > > -Steve L. > > > > > > Any thoughts greatfully received! > > > > -- > > Rich Teer, SCSA, SCNA, SCSECA > > > > URLs: http://www.rite-group.com/rich > > http://www.linkedin.com/in/richteer > > ___ > > zones-discuss mailing list > > zones-discuss@opensolaris.org pstack.core Description: pstack.core mdb-session Description: mdb-session ___ zones-discuss mailing list zones-discuss@opensolaris.org
