Re: JAXBContext leaks memory
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Mark,
I know you posted this a while ago, but I have some comments:
On 5/4/2010 9:10 AM, Mark Shifman wrote:
> public T getNextElement(String theElement, String elementAfter,
> Class clazz) {
> String elname = "";
> T h = null;
> try {
> while(reader.hasNext()){
> if(reader.peek().isStartElement()){
>elname =
> reader.peek().asStartElement().getName().getLocalPart();
>if(elname.equals(theElement)){
> h= u.unmarshal(reader, clazz).getValue();
> return h;
>}
> } else if(reader.peek().isEndElement()){
> elname =
> reader.peek().asEndElement().getName().getLocalPart();
> if(elname.equals(elementAfter)){
> return h;
> }
> }
>
> reader.nextEvent();
> }
> } catch (XMLStreamException e) {
> throw new RuntimeException(e);
> } catch (JAXBException e) {
> throw new RuntimeException(e);
> }
> return h;
> }
You could put the "return h" above the exception handlers to make it
clear that the try block must run to completion in order to return a
value. That's just a matter of taste, though.
> It also has a close method to clean up after I have gotten all the elements.
> public void close(){
> try {
> reader.close();
> } catch (XMLStreamException e) {
> //quietly
Don't do this: at least log the exception to stdout, or you may miss
something important at some point.
> }
> IOUtils.closeQuietly(jxb_in);
> u=null;
> }
You might want to put the call to IOUtils.closeQuietly into a finally
block: a RuntimeException could skip the call to your "close" method.
Whenever there's a close method available, you should call it.
Whenever you call it, you should probably call it in a finally blocks.
>>> private InputStream jxb_in;
>>>
>>> public static JAXBMascot getInstance(InputStream in) {
>>> JAXBMascot m = new JAXBMascot();
>>> try {
>>> m.setJxb_in(in);
>>>
>>> m.setReader(XMLInputFactory.newInstance().createXMLEventReader(in));
>>> } catch (Exception e) {
>>> log.fatal("error getting JAXBMascot instance");
>>> IOUtils.closeQuietly(in);
>>> throw new RuntimeException(e);
>>> }
Should you close even if no Exception has been thrown? The above does
not cover Error conditions, for example. Perhaps you wanted to catch
Throwable instead?
- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkvxpQMACgkQ9CaO5/Lv0PCyOgCfabMgxsAJzNplBsspeQnn83i2
KSwAn36FoEeFGJIQppwTpD5Ju338qIVX
=HAkp
-END PGP SIGNATURE-
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: JAXBContext leaks memory
2010/5/3 Mark Shifman : > Using jmap -histo pid, I can watch > com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in number > (...) > > The JAXBContext instance is created with a singleton that is an enum (using > Josh Block's pattern): > The cause might be somewhere else. JAXBContextMascot.INSTANCE is a singleton, which means (roughly speaking) that it is present in the memory while its containing Class is alive, and the Class is alive while the ClassLoader is alive. I wonder how enums are implemented. WebappClassLoader#clearReferencesStaticFinal() clears static fields of classes, but it skips fields which name contains '$'. E.g. internal $VALUES field in the enum class that holds array of all values will be skipped. Thus, it looks like an indication of the leak, but not the leak itself. (From your later emails it looks that you figured that already). There are known cases when a leak was not detectable with profilers. E.g. JavaDoc in JreMemoryLeakPreventionListener.java source code for the xmlParsingProtection field mentions one. IIRC, there was some sun bugreport for that, but I do not have a reference. I hope that that is not your case. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: JAXBContext leaks memory
On 17/05/2010 18:36, Mark Shifman wrote:
> Hi Pid, et al:
>
> Things are curiouser and curiouser. I decided to deploy the struts blank app
> after starting tomcat with startup.sh.
> Well there is no memory leak after undeploy and redeploy. If I start tomcat
> with jsvc, I get a memory leak.
> This is the minimal init script I used to start the tomcat daemon.
>
> snip..
> # Source function library.
> . /etc/init.d/functions
>
> # Adapt the following lines to your configuration
> JAVA_HOME=/usr/java/jdk1.6.0_18
> CATALINA_HOME=/home/vir6.0.26/apache-tomcat-6.0.26
> DAEMON_HOME=$CATALINA_HOME/bin
> TOMCAT_USER=mas
>
> #CATALINA_OPTS="-Djava.awt.headless=true "
> #LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_HOME/conf/logging.properties"
> #JAVA_OPTS="-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
> CLASSPATH=\
> $JAVA_HOME/lib/tools.jar:\
> $CATALINA_HOME/bin/commons-daemon.jar:\
> $CATALINA_HOME/bin/bootstrap.jar
>
> start(){
> $DAEMON_HOME/jsvc \
> -jvm server \
> -user $TOMCAT_USER \
> -home $JAVA_HOME \
> -Dcatalina.base=$CATALINA_HOME \
> -Dcatalina.home=$CATALINA_HOME \
> -Djava.io.tmpdir=$CATALINA_HOME/temp \
> -outfile $CATALINA_HOME/logs/catalina.out \
> -errfile '&1' \
> -cp $CLASSPATH \
> org.apache.catalina.startup.Bootstrap
> echo "starting: " $CATALINA_HOME
> echo "tomcat user: " $TOMCAT_USER
> #
> # To get a verbose JVM
> #-verbose \
> # To get a debug of jsvc.
> #-debug \
> }
>
> stop(){
> #killproc jsvc
> #PID=`cat /var/run/jsvc.pid`
> #kill $PID
>
> $DAEMON_HOME/jsvc \
> -stop \
> -pidfile /var/run/jsvc.pid \
> org.apache.catalina.startup.Bootstrap
>
> echo "stopping tomcat: " $CATALINA_HOME
> echo "tomcat user: " $TOMCAT_USER
>
> }
>
>
> case "$1" in
> start)
> #
> # Start Tomcat
> #
> start
> ;;
>
> status)
> status jsvc
> ;;
>
> stop)
> #
> # Stop Tomcat
> #
> stop
> ;;
>
> restart)
> stop
> start
> ;;
>
> *)
> echo "Usage tomcat6.0.26 start/stop/restart/status"
> exit 1;;
> esac
>
> When I load the heap dump and do a classloaderexplorerquery using MAT I get
> this:
>
> Class Name |
> Defined Classes | No. of Instances
> --
> org.apache.catalina.loader.WebappClassLoader @ 0x94977290|
> 260 | 186
> |- parent org.apache.catalina.loader.StandardClassLoader @ 0x94861d10|
> 935 |3,750
> |- org.apache.commons.beanutils.converters.ConverterFacade |
> | 54
> |- org.apache.commons.logging.impl.Jdk14Logger |
> | 38
> |- org.apache.commons.beanutils.converters.ArrayConverter|
> | 27
> |- org.apache.commons.beanutils.WeakFastHashMap |
> |4
> |- org.apache.commons.beanutils.converters.BooleanConverter |
> |4
> |- org.apache.commons.beanutils.converters.ByteConverter |
> |4
> |- org.apache.commons.beanutils.converters.CharacterConverter|
> |4
> |- org.apache.commons.beanutils.converters.DoubleConverter |
> |4
> |- org.apache.commons.beanutils.converters.FloatConverter|
> |4
> |- org.apache.commons.beanutils.converters.IntegerConverter |
> |4
> |- org.apache.commons.beanutils.converters.LongConverter |
> |4
> |- org.apache.commons.beanutils.converters.ShortConverter|
> |4
> |- org.apache.commons.beanutils.converters.DateConverter |
> |3
> |- org.apache.commons.beanutils.converters.BigDecimalConverter |
> |2
> |- org.apache.commons.beanutils.converters.BigIntegerConverter |
> |2
> |- org.apache.commons.beanutils.converters.ClassConverter|
> |2
> |- org.apache.commons.beanutils.converters.FileConverter |
> |2
> |- org.apache.commons.beanutils.converters.SqlDateConverter |
> |2
> |- org.apache.commons.beanutils.converters.SqlTimeConverter |
> |2
> |- org.apache.commons.beanutils.converters.SqlTimestampConverter |
> |2
> |- org.apache.commons.beanutils.converters.StringConverter |
Re: JAXBContext leaks memory
Hi Pid, et al:
Things are curiouser and curiouser. I decided to deploy the struts blank app
after starting tomcat with startup.sh.
Well there is no memory leak after undeploy and redeploy. If I start tomcat
with jsvc, I get a memory leak.
This is the minimal init script I used to start the tomcat daemon.
snip..
# Source function library.
. /etc/init.d/functions
# Adapt the following lines to your configuration
JAVA_HOME=/usr/java/jdk1.6.0_18
CATALINA_HOME=/home/vir6.0.26/apache-tomcat-6.0.26
DAEMON_HOME=$CATALINA_HOME/bin
TOMCAT_USER=mas
#CATALINA_OPTS="-Djava.awt.headless=true "
#LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_HOME/conf/logging.properties"
#JAVA_OPTS="-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
CLASSPATH=\
$JAVA_HOME/lib/tools.jar:\
$CATALINA_HOME/bin/commons-daemon.jar:\
$CATALINA_HOME/bin/bootstrap.jar
start(){
$DAEMON_HOME/jsvc \
-jvm server \
-user $TOMCAT_USER \
-home $JAVA_HOME \
-Dcatalina.base=$CATALINA_HOME \
-Dcatalina.home=$CATALINA_HOME \
-Djava.io.tmpdir=$CATALINA_HOME/temp \
-outfile $CATALINA_HOME/logs/catalina.out \
-errfile '&1' \
-cp $CLASSPATH \
org.apache.catalina.startup.Bootstrap
echo "starting: " $CATALINA_HOME
echo "tomcat user: " $TOMCAT_USER
#
# To get a verbose JVM
#-verbose \
# To get a debug of jsvc.
#-debug \
}
stop(){
#killproc jsvc
#PID=`cat /var/run/jsvc.pid`
#kill $PID
$DAEMON_HOME/jsvc \
-stop \
-pidfile /var/run/jsvc.pid \
org.apache.catalina.startup.Bootstrap
echo "stopping tomcat: " $CATALINA_HOME
echo "tomcat user: " $TOMCAT_USER
}
case "$1" in
start)
#
# Start Tomcat
#
start
;;
status)
status jsvc
;;
stop)
#
# Stop Tomcat
#
stop
;;
restart)
stop
start
;;
*)
echo "Usage tomcat6.0.26 start/stop/restart/status"
exit 1;;
esac
When I load the heap dump and do a classloaderexplorerquery using MAT I get
this:
Class Name | Defined
Classes | No. of Instances
--
org.apache.catalina.loader.WebappClassLoader @ 0x94977290|
260 | 186
|- parent org.apache.catalina.loader.StandardClassLoader @ 0x94861d10|
935 |3,750
|- org.apache.commons.beanutils.converters.ConverterFacade |
| 54
|- org.apache.commons.logging.impl.Jdk14Logger |
| 38
|- org.apache.commons.beanutils.converters.ArrayConverter|
| 27
|- org.apache.commons.beanutils.WeakFastHashMap |
|4
|- org.apache.commons.beanutils.converters.BooleanConverter |
|4
|- org.apache.commons.beanutils.converters.ByteConverter |
|4
|- org.apache.commons.beanutils.converters.CharacterConverter|
|4
|- org.apache.commons.beanutils.converters.DoubleConverter |
|4
|- org.apache.commons.beanutils.converters.FloatConverter|
|4
|- org.apache.commons.beanutils.converters.IntegerConverter |
|4
|- org.apache.commons.beanutils.converters.LongConverter |
|4
|- org.apache.commons.beanutils.converters.ShortConverter|
|4
|- org.apache.commons.beanutils.converters.DateConverter |
|3
|- org.apache.commons.beanutils.converters.BigDecimalConverter |
|2
|- org.apache.commons.beanutils.converters.BigIntegerConverter |
|2
|- org.apache.commons.beanutils.converters.ClassConverter|
|2
|- org.apache.commons.beanutils.converters.FileConverter |
|2
|- org.apache.commons.beanutils.converters.SqlDateConverter |
|2
|- org.apache.commons.beanutils.converters.SqlTimeConverter |
|2
|- org.apache.commons.beanutils.converters.SqlTimestampConverter |
|2
|- org.apache.commons.beanutils.converters.StringConverter |
|2
|- org.apache.commons.beanutils.converters.URLConverter |
|2
|- org.apache.commons.beanutils.BeanUtilsBean|
|1
|- org.apache.commons.beanutils.BeanUtilsBean$1 |
Re: JAXBContext leaks memory
On 14/05/2010 19:21, Mark Shifman wrote:
> After playing around I don't think the leak is from JAXBContext. My web app
> is running under struts 1.3.10. I tested
> the blank web apps that comes with struts 1.3.10 and it showed a memory leak
> on undeploying and redeploying. (so did the
> blank web app for the current struts 2). Using Eclipse MAT the retained
> WebappClassLoader showed a bunch of commons.beanutils
> classes but I am not sure how to follow it any farther.
>
> So either there is something funky with commons.beanutils 1.8.0 (also tried
> the newer version)(ie something tricky with reflection or commons-logging)
> or I have some how done something really stupid with my tomcat/jvm
> configuration that goofs up garbage collection.
Interesting. I had more of a nose around in JAXBContext and the
peripheral objects and found some things I'm slightly curious about but
nothing that looks leaky. I'm using JAXB myself and haven't seen much
of a side-effect.
I'd be happy to have a look at Struts if you'd like to detail some of
what you've found. Maybe we'll find something that can be contributed
to the Tomcat JreMemoryLeakPrevention.
p
> mas
>
> On 05/03/2010 12:15 PM, Mark Shifman wrote:
>> I have a web app running under tomcat-6.0.26 with
>> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>>
>> Using jmap -histo pid, I can watch
>> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in number
>> after running my unmarshal action, followed by undeploy and redeploy. Find
>> Leaks in the manager also finds leaks.
>>
>> The JAXBContext instance is created with a singleton that is an enum (using
>> Josh Block's pattern):
>>
>> public enum JAXBContextMascot {
>> INSTANCE("com.matrixscience.xmlns.schema.mascot_search_results_2" );
>> private JAXBContext ctx;
>> JAXBContextMascot(String contextPath) {
>> try {
>> ctx =JAXBContext.newInstance(clazz);
>> } catch (JAXBException e) {
>> throw new RuntimeException(e);
>> }
>> }
>> public Unmarshaller createUnmarshaller(){
>> try {
>> return ctx.createUnmarshaller();
>> } catch (JAXBException e) {
>> throw new RuntimeException(e);
>> }
>> }
>> }
>>
>> Am I doing something wrong which is causing the memory leak?
>> JAXBContext.newInstance() can take a ClassLoader argument. Is there some
>> ClassLoader I should be using that will get around this?
>>
>> Any help would be appreciated.
>> mas
>>
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
signature.asc
Description: OpenPGP digital signature
Re: JAXBContext leaks memory
After playing around I don't think the leak is from JAXBContext. My web app is
running under struts 1.3.10. I tested
the blank web apps that comes with struts 1.3.10 and it showed a memory leak on
undeploying and redeploying. (so did the
blank web app for the current struts 2). Using Eclipse MAT the retained
WebappClassLoader showed a bunch of commons.beanutils
classes but I am not sure how to follow it any farther.
So either there is something funky with commons.beanutils 1.8.0 (also tried the
newer version)(ie something tricky with reflection or commons-logging)
or I have some how done something really stupid with my tomcat/jvm
configuration that goofs up garbage collection.
mas
On 05/03/2010 12:15 PM, Mark Shifman wrote:
> I have a web app running under tomcat-6.0.26 with
> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>
> Using jmap -histo pid, I can watch
> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in number
> after running my unmarshal action, followed by undeploy and redeploy. Find
> Leaks in the manager also finds leaks.
>
> The JAXBContext instance is created with a singleton that is an enum (using
> Josh Block's pattern):
>
> public enum JAXBContextMascot {
> INSTANCE("com.matrixscience.xmlns.schema.mascot_search_results_2" );
> private JAXBContext ctx;
> JAXBContextMascot(String contextPath) {
> try {
> ctx =JAXBContext.newInstance(clazz);
> } catch (JAXBException e) {
> throw new RuntimeException(e);
> }
> }
> public Unmarshaller createUnmarshaller(){
> try {
> return ctx.createUnmarshaller();
> } catch (JAXBException e) {
> throw new RuntimeException(e);
> }
> }
> }
>
> Am I doing something wrong which is causing the memory leak?
> JAXBContext.newInstance() can take a ClassLoader argument. Is there some
> ClassLoader I should be using that will get around this?
>
> Any help would be appreciated.
> mas
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: JAXBContext leaks memory
jvisualvm is an excellent tool, and so is MAT
(http://www.eclipse.org/mat/downloads.php). If you can figure out
which instances are increasing (as Pid indicated) its very easy to
check the path of those objects to the garbage collector using one of
these tools. From there its usually easy to see what the cause of the
leak is and correct it. I'm not familiar with jconsole, but I have
used the mat/jvisualvm approach several times before.
On Wed, May 5, 2010 at 1:36 AM, Mark Shifman wrote:
> Thanks.
> I'll try Jconsole.
> mas
>
> On 05/04/2010 12:28 PM, Pid wrote:
>> On 04/05/2010 14:10, Mark Shifman wrote:
>>>
>>> On 05/03/2010 02:53 PM, Pid wrote:
On 03/05/2010 18:30, Mark Shifman wrote:
>
> On 05/03/2010 12:48 PM, Pid wrote:
>> On 03/05/2010 17:15, Mark Shifman wrote:
>>> I have a web app running under tomcat-6.0.26 with
>>> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>>>
>>> Using jmap -histo pid, I can watch
>>> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in
>>> number
>>> after running my unmarshal action, followed by undeploy and redeploy.
>>> Find Leaks in the manager also finds leaks.
Do you see log messages referring to potential leaks in the catalina.out
log (assuming you're on a unix variant)?
If so, can you post them please?
>>>
>>> There are no messages in catalia.out concerning leaks (I am using Linux
>>> 2.6.18-92.1.22.el5)
>>> I also got rid of timeBetweenEvictionRunsMillis in datasource since it
>>> causes a leaky TimerThread).
>>>
What does the manager 'find leaks' command report exactly?
>>> ...
>>> leak (use a profiler to confirm):
>>> /yp_results
>>
>> There are some useful commands in the JDK which may help track down
>> exactly which class is causing the problem.
>>
>> jmap -histo
>>
>> (and other jmap subcommands)
>>
>> If you take a snapshot periodically, esp after reload cycles you may be
>> able to see which classes are increasing in number.
>>
>>
>> If you can get a VisualVM working, or connect a JConsole to the remote
>> VM you may be able to poke around and see which classes aren't being
>> garbage collected.
>>
>>
>>> My webapp is named yp_results.
>> After a few undeploy/redeploy cycles does the number of
>> WebappClassLoader's also increase?
>
> Yes it increases 1 for each undeploy/redeploy cycle.
> snip... <
>> Maybe.
>>
>>> JAXBContext.newInstance() can take a ClassLoader argument. Is there
>>> some ClassLoader I should be using that will get around this?
OK, so I've looked at JAXBContext (and JAXBContextImpl) and it doesn't
(after quick read through) look like it's storing the classloader
argument anywhere during the newInstance call, which is the usual source
of leaks.
>>
>>
>> Where is the jar with the above code, in a webapp?
> The code above in in the war for the web app in a class in
> WEB-INF/classes/org/blablabla
>
> It is called via a class that looks like this:
>
> public class JAXBMascot {
> protected static Log log = LogFactory.getLog(JAXBMascot.class);
> private XMLEventReader reader;
> private Unmarshaller u =
> JAXBContextMascot.INSTANCE.createUnmarshaller();
You're setting the XMLEventReader, Unmarshaller & InputStream as
instance field values, rather than completing the parsing in the
getInstance() method?
>>> I have really big xmls to unmarshall so I am using streaming them in and
>>> unmarshalling the elements I want
>>> and then insert into my database. I need the reader to see where I am and
>>> then the umarshaller
>>
>> I'll have a look at the code below a bit later, am pushed for time right
>> now.
>>
>>
>> p
>>
>>
>>
>>> I didn't show the all the methods of JAXBMascot but here is workhorse:
>>> public T getNextElement(String theElement, String elementAfter,
>>> Class clazz) {
>>> String elname = "";
>>> T h = null;
>>> try {
>>> while(reader.hasNext()){
>>> if(reader.peek().isStartElement()){
>>> elname =
>>> reader.peek().asStartElement().getName().getLocalPart();
>>> if(elname.equals(theElement)){
>>> h= u.unmarshal(reader,
>>> clazz).getValue();
>>> return h;
>>> }
>>> } else if(reader.peek().isEndElement()){
>>> elname =
>>> reader.peek().asEndElement().getName().getLocalPart();
>>> if(elname.equals(elementAfter)){
>>> return h;
>>> }
>>> }
>>>
>>> reader.nextEvent();
>>>
Re: JAXBContext leaks memory
Thanks.
I'll try Jconsole.
mas
On 05/04/2010 12:28 PM, Pid wrote:
> On 04/05/2010 14:10, Mark Shifman wrote:
>>
>> On 05/03/2010 02:53 PM, Pid wrote:
>>> On 03/05/2010 18:30, Mark Shifman wrote:
On 05/03/2010 12:48 PM, Pid wrote:
> On 03/05/2010 17:15, Mark Shifman wrote:
>> I have a web app running under tomcat-6.0.26 with
>> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>>
>> Using jmap -histo pid, I can watch
>> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in
>> number
>> after running my unmarshal action, followed by undeploy and redeploy.
>> Find Leaks in the manager also finds leaks.
>>>
>>> Do you see log messages referring to potential leaks in the catalina.out
>>> log (assuming you're on a unix variant)?
>>>
>>> If so, can you post them please?
>>
>> There are no messages in catalia.out concerning leaks (I am using Linux
>> 2.6.18-92.1.22.el5)
>> I also got rid of timeBetweenEvictionRunsMillis in datasource since it
>> causes a leaky TimerThread).
>>
>>>
>>> What does the manager 'find leaks' command report exactly?
>> ...
>> leak (use a profiler to confirm):
>> /yp_results
>
> There are some useful commands in the JDK which may help track down
> exactly which class is causing the problem.
>
> jmap -histo
>
> (and other jmap subcommands)
>
> If you take a snapshot periodically, esp after reload cycles you may be
> able to see which classes are increasing in number.
>
>
> If you can get a VisualVM working, or connect a JConsole to the remote
> VM you may be able to poke around and see which classes aren't being
> garbage collected.
>
>
>> My webapp is named yp_results.
>>>
> After a few undeploy/redeploy cycles does the number of
> WebappClassLoader's also increase?
Yes it increases 1 for each undeploy/redeploy cycle.
>>>
snip... <
>>>
> Maybe.
>
>> JAXBContext.newInstance() can take a ClassLoader argument. Is there
>> some ClassLoader I should be using that will get around this?
>>>
>>> OK, so I've looked at JAXBContext (and JAXBContextImpl) and it doesn't
>>> (after quick read through) look like it's storing the classloader
>>> argument anywhere during the newInstance call, which is the usual source
>>> of leaks.
>
>
> Where is the jar with the above code, in a webapp?
The code above in in the war for the web app in a class in
WEB-INF/classes/org/blablabla
It is called via a class that looks like this:
public class JAXBMascot {
protected static Log log = LogFactory.getLog(JAXBMascot.class);
private XMLEventReader reader;
private Unmarshaller u =
JAXBContextMascot.INSTANCE.createUnmarshaller();
>>>
>>> You're setting the XMLEventReader, Unmarshaller & InputStream as
>>> instance field values, rather than completing the parsing in the
>>> getInstance() method?
>> I have really big xmls to unmarshall so I am using streaming them in and
>> unmarshalling the elements I want
>> and then insert into my database. I need the reader to see where I am and
>> then the umarshaller
>
> I'll have a look at the code below a bit later, am pushed for time right
> now.
>
>
> p
>
>
>
>> I didn't show the all the methods of JAXBMascot but here is workhorse:
>> public T getNextElement(String theElement, String elementAfter,
>> Class clazz) {
>> String elname = "";
>> T h = null;
>> try {
>> while(reader.hasNext()){
>>if(reader.peek().isStartElement()){
>> elname =
>> reader.peek().asStartElement().getName().getLocalPart();
>> if(elname.equals(theElement)){
>> h= u.unmarshal(reader, clazz).getValue();
>> return h;
>> }
>>} else if(reader.peek().isEndElement()){
>> elname =
>> reader.peek().asEndElement().getName().getLocalPart();
>> if(elname.equals(elementAfter)){
>> return h;
>> }
>> }
>>
>>reader.nextEvent();
>> }
>> } catch (XMLStreamException e) {
>> throw new RuntimeException(e);
>> } catch (JAXBException e) {
>> throw new RuntimeException(e);
>> }
>> return h;
>> }
>>
>> It also has a close method to clean up after I have gotten all the elements.
>> public void close(){
>> try {
>> reader.close();
>> } catch (XMLStreamException e) {
>> //quietly
>> }
>> IOUtils.closeQuietly(jxb_in);
>> u=null;
>>
Re: JAXBContext leaks memory
On 04/05/2010 14:10, Mark Shifman wrote:
>
> On 05/03/2010 02:53 PM, Pid wrote:
>> On 03/05/2010 18:30, Mark Shifman wrote:
>>>
>>> On 05/03/2010 12:48 PM, Pid wrote:
On 03/05/2010 17:15, Mark Shifman wrote:
> I have a web app running under tomcat-6.0.26 with
> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>
> Using jmap -histo pid, I can watch
> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in
> number
> after running my unmarshal action, followed by undeploy and redeploy.
> Find Leaks in the manager also finds leaks.
>>
>> Do you see log messages referring to potential leaks in the catalina.out
>> log (assuming you're on a unix variant)?
>>
>> If so, can you post them please?
>
> There are no messages in catalia.out concerning leaks (I am using Linux
> 2.6.18-92.1.22.el5)
> I also got rid of timeBetweenEvictionRunsMillis in datasource since it causes
> a leaky TimerThread).
>
>>
>> What does the manager 'find leaks' command report exactly?
> ...
> leak (use a profiler to confirm):
> /yp_results
There are some useful commands in the JDK which may help track down
exactly which class is causing the problem.
jmap -histo
(and other jmap subcommands)
If you take a snapshot periodically, esp after reload cycles you may be
able to see which classes are increasing in number.
If you can get a VisualVM working, or connect a JConsole to the remote
VM you may be able to poke around and see which classes aren't being
garbage collected.
> My webapp is named yp_results.
>>
After a few undeploy/redeploy cycles does the number of
WebappClassLoader's also increase?
>>>
>>> Yes it increases 1 for each undeploy/redeploy cycle.
>>
>>> snip... <
>>
Maybe.
> JAXBContext.newInstance() can take a ClassLoader argument. Is there some
> ClassLoader I should be using that will get around this?
>>
>> OK, so I've looked at JAXBContext (and JAXBContextImpl) and it doesn't
>> (after quick read through) look like it's storing the classloader
>> argument anywhere during the newInstance call, which is the usual source
>> of leaks.
Where is the jar with the above code, in a webapp?
>>> The code above in in the war for the web app in a class in
>>> WEB-INF/classes/org/blablabla
>>>
>>> It is called via a class that looks like this:
>>>
>>> public class JAXBMascot {
>>> protected static Log log = LogFactory.getLog(JAXBMascot.class);
>>> private XMLEventReader reader;
>>> private Unmarshaller u =
>>> JAXBContextMascot.INSTANCE.createUnmarshaller();
>>
>> You're setting the XMLEventReader, Unmarshaller & InputStream as
>> instance field values, rather than completing the parsing in the
>> getInstance() method?
> I have really big xmls to unmarshall so I am using streaming them in and
> unmarshalling the elements I want
> and then insert into my database. I need the reader to see where I am and
> then the umarshaller
I'll have a look at the code below a bit later, am pushed for time right
now.
p
> I didn't show the all the methods of JAXBMascot but here is workhorse:
> public T getNextElement(String theElement, String elementAfter,
> Class clazz) {
> String elname = "";
> T h = null;
> try {
> while(reader.hasNext()){
> if(reader.peek().isStartElement()){
>elname =
> reader.peek().asStartElement().getName().getLocalPart();
>if(elname.equals(theElement)){
> h= u.unmarshal(reader, clazz).getValue();
> return h;
>}
> } else if(reader.peek().isEndElement()){
> elname =
> reader.peek().asEndElement().getName().getLocalPart();
> if(elname.equals(elementAfter)){
> return h;
> }
> }
>
> reader.nextEvent();
> }
> } catch (XMLStreamException e) {
> throw new RuntimeException(e);
> } catch (JAXBException e) {
> throw new RuntimeException(e);
> }
> return h;
> }
>
> It also has a close method to clean up after I have gotten all the elements.
> public void close(){
> try {
> reader.close();
> } catch (XMLStreamException e) {
> //quietly
> }
> IOUtils.closeQuietly(jxb_in);
> u=null;
> }
> I don't think I am leaving any stuff hanging around but memory leaks are very
> sneaky.
> mas
>>
>> This looks a bit odd to me, but I don't know what the rest of the
>> instance does.
Re: JAXBContext leaks memory
On 05/03/2010 02:53 PM, Pid wrote:
> On 03/05/2010 18:30, Mark Shifman wrote:
>>
>> On 05/03/2010 12:48 PM, Pid wrote:
>>> On 03/05/2010 17:15, Mark Shifman wrote:
I have a web app running under tomcat-6.0.26 with
JreMemoryLeakPreventionListener, java jdk1.6.0_18.
Using jmap -histo pid, I can watch
com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in
number
after running my unmarshal action, followed by undeploy and redeploy.
Find Leaks in the manager also finds leaks.
>
> Do you see log messages referring to potential leaks in the catalina.out
> log (assuming you're on a unix variant)?
>
> If so, can you post them please?
There are no messages in catalia.out concerning leaks (I am using Linux
2.6.18-92.1.22.el5)
I also got rid of timeBetweenEvictionRunsMillis in datasource since it causes a
leaky TimerThread).
>
> What does the manager 'find leaks' command report exactly?
...
leak (use a profiler to confirm):
/yp_results
My webapp is named yp_results.
>
>>> After a few undeploy/redeploy cycles does the number of
>>> WebappClassLoader's also increase?
>>
>> Yes it increases 1 for each undeploy/redeploy cycle.
>
>> snip... <
>
>>> Maybe.
>>>
JAXBContext.newInstance() can take a ClassLoader argument. Is there some
ClassLoader I should be using that will get around this?
>
> OK, so I've looked at JAXBContext (and JAXBContextImpl) and it doesn't
> (after quick read through) look like it's storing the classloader
> argument anywhere during the newInstance call, which is the usual source
> of leaks.
>
>>> Where is the jar with the above code, in a webapp?
>> The code above in in the war for the web app in a class in
>> WEB-INF/classes/org/blablabla
>>
>> It is called via a class that looks like this:
>>
>> public class JAXBMascot {
>> protected static Log log = LogFactory.getLog(JAXBMascot.class);
>> private XMLEventReader reader;
>> private Unmarshaller u =
>> JAXBContextMascot.INSTANCE.createUnmarshaller();
>
> You're setting the XMLEventReader, Unmarshaller & InputStream as
> instance field values, rather than completing the parsing in the
> getInstance() method?
I have really big xmls to unmarshall so I am using streaming them in and
unmarshalling the elements I want
and then insert into my database. I need the reader to see where I am and then
the umarshaller
I didn't show the all the methods of JAXBMascot but here is workhorse:
public T getNextElement(String theElement, String elementAfter,
Class clazz) {
String elname = "";
T h = null;
try {
while(reader.hasNext()){
if(reader.peek().isStartElement()){
elname =
reader.peek().asStartElement().getName().getLocalPart();
if(elname.equals(theElement)){
h= u.unmarshal(reader, clazz).getValue();
return h;
}
} else if(reader.peek().isEndElement()){
elname =
reader.peek().asEndElement().getName().getLocalPart();
if(elname.equals(elementAfter)){
return h;
}
}
reader.nextEvent();
}
} catch (XMLStreamException e) {
throw new RuntimeException(e);
} catch (JAXBException e) {
throw new RuntimeException(e);
}
return h;
}
It also has a close method to clean up after I have gotten all the elements.
public void close(){
try {
reader.close();
} catch (XMLStreamException e) {
//quietly
}
IOUtils.closeQuietly(jxb_in);
u=null;
}
I don't think I am leaving any stuff hanging around but memory leaks are very
sneaky.
mas
>
> This looks a bit odd to me, but I don't know what the rest of the
> instance does...
>
>
> p
>
>> private InputStream jxb_in;
>>
>> public static JAXBMascot getInstance(InputStream in) {
>> JAXBMascot m = new JAXBMascot();
>> try {
>> m.setJxb_in(in);
>>
>> m.setReader(XMLInputFactory.newInstance().createXMLEventReader(in));
>> } catch (Exception e) {
>> log.fatal("error getting JAXBMascot instance");
>> IOUtils.closeQuietly(in);
>> throw new RuntimeException(e);
>> }
>>
>> return m;
>> }
>>
>> }
>>
>> This is also in the webapp in WEB-INF/classe
Re: JAXBContext leaks memory
On 03/05/2010 18:30, Mark Shifman wrote:
>
> On 05/03/2010 12:48 PM, Pid wrote:
>> On 03/05/2010 17:15, Mark Shifman wrote:
>>> I have a web app running under tomcat-6.0.26 with
>>> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>>>
>>> Using jmap -histo pid, I can watch
>>> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in number
>>> after running my unmarshal action, followed by undeploy and redeploy. Find
>>> Leaks in the manager also finds leaks.
Do you see log messages referring to potential leaks in the catalina.out
log (assuming you're on a unix variant)?
If so, can you post them please?
What does the manager 'find leaks' command report exactly?
>> After a few undeploy/redeploy cycles does the number of
>> WebappClassLoader's also increase?
>
> Yes it increases 1 for each undeploy/redeploy cycle.
> snip... <
>> Maybe.
>>
>>> JAXBContext.newInstance() can take a ClassLoader argument. Is there some
>>> ClassLoader I should be using that will get around this?
OK, so I've looked at JAXBContext (and JAXBContextImpl) and it doesn't
(after quick read through) look like it's storing the classloader
argument anywhere during the newInstance call, which is the usual source
of leaks.
>> Where is the jar with the above code, in a webapp?
> The code above in in the war for the web app in a class in
> WEB-INF/classes/org/blablabla
>
> It is called via a class that looks like this:
>
> public class JAXBMascot {
> protected static Log log = LogFactory.getLog(JAXBMascot.class);
> private XMLEventReader reader;
> private Unmarshaller u =
> JAXBContextMascot.INSTANCE.createUnmarshaller();
You're setting the XMLEventReader, Unmarshaller & InputStream as
instance field values, rather than completing the parsing in the
getInstance() method?
This looks a bit odd to me, but I don't know what the rest of the
instance does...
p
> private InputStream jxb_in;
>
> public static JAXBMascot getInstance(InputStream in) {
> JAXBMascot m = new JAXBMascot();
> try {
> m.setJxb_in(in);
>
> m.setReader(XMLInputFactory.newInstance().createXMLEventReader(in));
> } catch (Exception e) {
> log.fatal("error getting JAXBMascot instance");
> IOUtils.closeQuietly(in);
> throw new RuntimeException(e);
> }
>
> return m;
> }
>
> }
>
> This is also in the webapp in WEB-INF/classes/org/blablabla
signature.asc
Description: OpenPGP digital signature
Re: JAXBContext leaks memory
On 05/03/2010 02:18 PM, Caldarale, Charles R wrote: >> From: Mark Shifman [mailto:mark.shif...@yale.edu] >> Subject: Re: JAXBContext leaks memory >> >> This is also in the webapp in WEB-INF/classes/org/blablabla > > When you say "also", does that mean the class is in more than one place? > (That would be a Bad Thing.) no it lives in one place only. > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you received > this in error, please contact the sender and delete the e-mail and its > attachments from all computers. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: JAXBContext leaks memory
> From: Mark Shifman [mailto:mark.shif...@yale.edu] > Subject: Re: JAXBContext leaks memory > > This is also in the webapp in WEB-INF/classes/org/blablabla When you say "also", does that mean the class is in more than one place? (That would be a Bad Thing.) - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: JAXBContext leaks memory
On 05/03/2010 12:48 PM, Pid wrote:
> On 03/05/2010 17:15, Mark Shifman wrote:
>> I have a web app running under tomcat-6.0.26 with
>> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>>
>> Using jmap -histo pid, I can watch
>> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in number
>> after running my unmarshal action, followed by undeploy and redeploy. Find
>> Leaks in the manager also finds leaks.
>
> After a few undeploy/redeploy cycles does the number of
> WebappClassLoader's also increase?
Yes it increases 1 for each undeploy/redeploy cycle.
>
>> The JAXBContext instance is created with a singleton that is an enum (using
>> Josh Block's pattern):
>>
>> public enum JAXBContextMascot {
>> INSTANCE("com.matrixscience.xmlns.schema.mascot_search_results_2" );
>> private JAXBContext ctx;
>> JAXBContextMascot(String contextPath) {
>> try {
>> ctx =JAXBContext.newInstance(clazz);
>
> Where does the 'clazz' variable come from?
My bad I edited the email argg. should of course be contextPath.
>
>> } catch (JAXBException e) {
>> throw new RuntimeException(e);
>> }
>> }
>> public Unmarshaller createUnmarshaller(){
>> try {
>> return ctx.createUnmarshaller();
>> } catch (JAXBException e) {
>> throw new RuntimeException(e);
>> }
>> }
>> }
>>
>> Am I doing something wrong which is causing the memory leak?
>
> Maybe.
>
>> JAXBContext.newInstance() can take a ClassLoader argument. Is there some
>> ClassLoader I should be using that will get around this?
>
> Where is the jar with the above code, in a webapp?
The code above in in the war for the web app in a class in
WEB-INF/classes/org/blablabla
It is called via a class that looks like this:
public class JAXBMascot {
protected static Log log = LogFactory.getLog(JAXBMascot.class);
private XMLEventReader reader;
private Unmarshaller u =
JAXBContextMascot.INSTANCE.createUnmarshaller();
private InputStream jxb_in;
public static JAXBMascot getInstance(InputStream in) {
JAXBMascot m = new JAXBMascot();
try {
m.setJxb_in(in);
m.setReader(XMLInputFactory.newInstance().createXMLEventReader(in));
} catch (Exception e) {
log.fatal("error getting JAXBMascot instance");
IOUtils.closeQuietly(in);
throw new RuntimeException(e);
}
return m;
}
}
This is also in the webapp in WEB-INF/classes/org/blablabla
thanks
mas
>
>
> p
>
>> Any help would be appreciated.
>> mas
>>
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: JAXBContext leaks memory
On 03/05/2010 17:15, Mark Shifman wrote:
> I have a web app running under tomcat-6.0.26 with
> JreMemoryLeakPreventionListener, java jdk1.6.0_18.
>
> Using jmap -histo pid, I can watch
> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in number
> after running my unmarshal action, followed by undeploy and redeploy. Find
> Leaks in the manager also finds leaks.
After a few undeploy/redeploy cycles does the number of
WebappClassLoader's also increase?
> The JAXBContext instance is created with a singleton that is an enum (using
> Josh Block's pattern):
>
> public enum JAXBContextMascot {
> INSTANCE("com.matrixscience.xmlns.schema.mascot_search_results_2" );
> private JAXBContext ctx;
> JAXBContextMascot(String contextPath) {
> try {
> ctx =JAXBContext.newInstance(clazz);
Where does the 'clazz' variable come from?
> } catch (JAXBException e) {
> throw new RuntimeException(e);
> }
> }
> public Unmarshaller createUnmarshaller(){
> try {
> return ctx.createUnmarshaller();
> } catch (JAXBException e) {
> throw new RuntimeException(e);
> }
> }
> }
>
> Am I doing something wrong which is causing the memory leak?
Maybe.
> JAXBContext.newInstance() can take a ClassLoader argument. Is there some
> ClassLoader I should be using that will get around this?
Where is the jar with the above code, in a webapp?
p
> Any help would be appreciated.
> mas
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
signature.asc
Description: OpenPGP digital signature
JAXBContext leaks memory
I have a web app running under tomcat-6.0.26 with
JreMemoryLeakPreventionListener, java jdk1.6.0_18.
Using jmap -histo pid, I can watch
com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl, etc increase in number
after running my unmarshal action, followed by undeploy and redeploy. Find
Leaks in the manager also finds leaks.
The JAXBContext instance is created with a singleton that is an enum (using
Josh Block's pattern):
public enum JAXBContextMascot {
INSTANCE("com.matrixscience.xmlns.schema.mascot_search_results_2" );
private JAXBContext ctx;
JAXBContextMascot(String contextPath) {
try {
ctx =JAXBContext.newInstance(clazz);
} catch (JAXBException e) {
throw new RuntimeException(e);
}
}
public Unmarshaller createUnmarshaller(){
try {
return ctx.createUnmarshaller();
} catch (JAXBException e) {
throw new RuntimeException(e);
}
}
}
Am I doing something wrong which is causing the memory leak?
JAXBContext.newInstance() can take a ClassLoader argument. Is there some
ClassLoader I should be using that will get around this?
Any help would be appreciated.
mas
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
