"OutOfMemoryException: PermGen space" error with Ant?
Hello, I was using dbunit's Ant integration and have a for loop that exports a bunch of dataset into many small xml files (less than 5-50kb each). After repeating task for ~50 times, I got an OutOfMemoryException: PermGen space. (DbUnit v2.4.8 w/ Ant 1.8.2) C:\test\build.xml:364: java.lang.OutOfMemoryError: PermGen space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at org.apache.tools.ant.AntClassLoader.defineClassFromData(AntClassLoader.java:1124) at org.apache.tools.ant.AntClassLoader.getClassFromStream(AntClassLoader.java:1295) at org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1351) at org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1311) at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1064) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at org.apache.tools.ant.AntClassLoader.defineClassFromData(AntClassLoader.java:1124) at org.apache.tools.ant.AntClassLoader.getClassFromStream(AntClassLoader.java:1295) at org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1351) at org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1311) at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1064) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at org.apache.tools.ant.AntClassLoader.defineClassFromData(AntClassLoader.java:1124) at org.apache.tools.ant.AntClassLoader.getClassFromStream(AntClassLoader.java:1295) at org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1351) at org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1311) at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1064) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at org.apache.tools.ant.AntClassLoader.defineClassFromData(AntClassLoader.java:1124) at org.apache.tools.ant.AntClassLoader.getClassFromStream(AntClassLoader.java:1295) Although I read that using -Xmx256m will solve the problem, I am curious why this problem occurred in the first place? I am able to import a single 50MB xml file without specifying -Xmx option, so in theory, exporting 100 50kb xml files should NOT cause any problems. Also, the problem can be simulated by repeating the following ~100 times in a single Ant task. ... ... ... ... ... When I look at task manager, the Java process (running Ant) seems to consume more and more memory, once it reaches above 130MB, the exception is thrown. Is there some sort optimization that could be done in the code to alleviate this problem? I also tried the latest Ant 1.8.4, with similar errors: java.lang.OutOfMemoryError: PermGen space at java.lang.Throwable.getStackTraceElement(Native Method) at java.lang.Throwable.getOurStackTrace(Throwable.java:591) at java.lang.Throwable.printStackTrace(Throwable.java:462) at java.lang.Throwable.printStackTrace(Throwable.java:451) at org.apache.tools.ant.Main.runBuild(Main.java:838) at org.apache.tools.ant.Main.startAnt(Main.java:217) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109) PermGen space java.lang.OutOfMemoryError: PermGen space at java.lang.Runtime.exit(Runtime.java:90) at java.lang.System.exit(System.java:904) at org.apache.tools.ant.Main.exit(Main.java:245) at org.apache.tools.ant.Main.startAnt(Main.java:235) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109) Thank you! Charles - To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
Re: Fwd: extends & buildlist on 2.3.0-rc1 ... it gets worse
Hi Mitch,
I've just replied on the ivy-user mailing list.
I think this issue should get fixed before the 2.3.0 final release.
Could you at least create a JIRA issue for this bug. Attaching a test case
would be really helpfull.
(attaching a patch that fixes the problem would be even more helpfull :-) )
I don't have time right now to look at it though.
I hope to have a bit more time in july/august...
Maarten
From: Mitch Gitman
To: Ant Developers List
Sent: Friday, June 22, 2012 7:53 PM
Subject: Fwd: extends & buildlist on 2.3.0-rc1 ... it gets worse
I'm forwarding this thread I started on the ivy-user list.
Can someone advise me how to handle this? Should I file a JIRA issue? Is
someone aware if there's a JIRA issue already for this problem? For that
matter, has anyone else noticed Ivy buildlist and extends not being able to
coexist?
Once we've established that there's a JIRA issue with a reproducible use
case, what's the best way to ensure it gets addressed? Would someone like
Maarten want to tackle this, or should I? My fear is that, if I tackle it,
the fix is not going to find its way into the code base.
Well, my bigger fear is that Ivy 2.3.0 is going to be released without this
problem being addressed.
-- Forwarded message --
From: Mitch Gitman
Date: Fri, Jun 22, 2012 at 10:21 AM
Subject: Re: extends & buildlist on 2.3.0-rc1 ... it gets worse
To: ivy-u...@ant.apache.org
OK, I stripped away all use of the extends feature, and sure enough,
buildlist produced a normal, expected, non-randomized project build order.
This indicates that something that had been working in Ivy 2.2.0 is no
longer working in Ivy 2.3.0-rc1.
Considering that:
A. The buildlist task is a prerequisite for being productive with Ivy.*
B. The extends feature is a prerequisite for being productive with Ivy.*
The apparent fact that these two features are now mutually exclusive with
Ivy 2.3.0-rc1 (where they weren't with Ivy 2.2.0) represents a serious bug.
I think my next step is to forward this thread to the ant-dev list and ask
how to proceed.
* For those who wish to say, "Mitch, we've been able to get by just fine
without (buildlist|extends)," I'm perfectly happy to have that discussion,
but my primary concern now is facilitating a fix.
On Thu, Jun 21, 2012 at 10:12 PM, Mitch Gitman wrote:
> One other data point. As long as my undesired simplification wasn't
> helping things, I went back to both a bootstrap-parent and a master-parent.
> I also tried setting haltOnError="false" on ivy:buildlist. With that, the
> task successfully got through everything. However, it continued to create a
> seriously trashed build order.
>
> Just as an experiment, I might try converting all the ivy.xml files to not
> use the extends feature and then see what happens when running buildlist.
> My hypothesis is, this will work. Not that this is a desired state of
> affairs, but at least it isolates the problem to the interaction between
> buildlist and extends.
>
>
> On Thu, Jun 21, 2012 at 9:23 PM, Mitch Gitman wrote:
>
>> Over a week ago, I'd sent out a message to this list, "buildlist task
>> chokes on absolute path to parent Ivy module." I'd found that the extends
>> feature worked fine with an absolute path to the parent ivy.xml if I was
>> building any single Ivy module, but the buildlist task would fail to find
>> the absolute path.
>>
>>
>>
>> I'd noted that I'd been using Ivy 2.2.0. Now that I've upgraded to Ivy
>> 2.3.0-rc1, my problems have only gotten worse. The first problem I
>> encountered had nothing to do with absolute paths to parents. I'm still
>> using relative paths to parents. Instead, it arose from my using two
>> different parent Ivy modules: bootstrap-parent and master-parent. Some
>> projects extended the former; others the latter. For example:
>>
>>
>> > revision="${version}"
>>
>> location="../bootstrap-parent/ivy.xml" />
>>
>>
>>
>>
>>
>> But then when I pointed the ivy:buildlist Ant task at a project stack
>> that included a mix of both parents and their children, I saw this error:
>>
>> impossible to parse ivy file for …/foo-client/homeowner/build.xml:
>> ivyfile=.../foo-client/homeowner/ivy.xml
>> exception=java.text.ParseException: Problem occurred while parsing ivy
>> file: inconsistent module descriptor file found in
>> 'file:/.../master-parent/ivy.xml': bad module name:
>> expected='bootstrap-parent' found='master-parent'; in
>> file:/.../foo-client/homeowner/ivy.xml
>>
>>
>>
>> What's happening is, the homeowner module extends bootstrap-parent, but
>> somehow the relative path to master-parent/ivy.xml is supplanting the
>> relative path to bootstrap-parent/ivy.xml. It appears buildlist doesn't
>> know how to deal with more than one parent, even though there's no
>> interaction between the two parents.
>>
>>
>>
>> After this, even though I didn't really want to, I thought, "Why not make
>> things simple for buildlist and use ju
Fwd: extends & buildlist on 2.3.0-rc1 ... it gets worse
I'm forwarding this thread I started on the ivy-user list.
Can someone advise me how to handle this? Should I file a JIRA issue? Is
someone aware if there's a JIRA issue already for this problem? For that
matter, has anyone else noticed Ivy buildlist and extends not being able to
coexist?
Once we've established that there's a JIRA issue with a reproducible use
case, what's the best way to ensure it gets addressed? Would someone like
Maarten want to tackle this, or should I? My fear is that, if I tackle it,
the fix is not going to find its way into the code base.
Well, my bigger fear is that Ivy 2.3.0 is going to be released without this
problem being addressed.
-- Forwarded message --
From: Mitch Gitman
Date: Fri, Jun 22, 2012 at 10:21 AM
Subject: Re: extends & buildlist on 2.3.0-rc1 ... it gets worse
To: ivy-u...@ant.apache.org
OK, I stripped away all use of the extends feature, and sure enough,
buildlist produced a normal, expected, non-randomized project build order.
This indicates that something that had been working in Ivy 2.2.0 is no
longer working in Ivy 2.3.0-rc1.
Considering that:
A. The buildlist task is a prerequisite for being productive with Ivy.*
B. The extends feature is a prerequisite for being productive with Ivy.*
The apparent fact that these two features are now mutually exclusive with
Ivy 2.3.0-rc1 (where they weren't with Ivy 2.2.0) represents a serious bug.
I think my next step is to forward this thread to the ant-dev list and ask
how to proceed.
* For those who wish to say, "Mitch, we've been able to get by just fine
without (buildlist|extends)," I'm perfectly happy to have that discussion,
but my primary concern now is facilitating a fix.
On Thu, Jun 21, 2012 at 10:12 PM, Mitch Gitman wrote:
> One other data point. As long as my undesired simplification wasn't
> helping things, I went back to both a bootstrap-parent and a master-parent.
> I also tried setting haltOnError="false" on ivy:buildlist. With that, the
> task successfully got through everything. However, it continued to create a
> seriously trashed build order.
>
> Just as an experiment, I might try converting all the ivy.xml files to not
> use the extends feature and then see what happens when running buildlist.
> My hypothesis is, this will work. Not that this is a desired state of
> affairs, but at least it isolates the problem to the interaction between
> buildlist and extends.
>
>
> On Thu, Jun 21, 2012 at 9:23 PM, Mitch Gitman wrote:
>
>> Over a week ago, I'd sent out a message to this list, "buildlist task
>> chokes on absolute path to parent Ivy module." I'd found that the extends
>> feature worked fine with an absolute path to the parent ivy.xml if I was
>> building any single Ivy module, but the buildlist task would fail to find
>> the absolute path.
>>
>>
>>
>> I'd noted that I'd been using Ivy 2.2.0. Now that I've upgraded to Ivy
>> 2.3.0-rc1, my problems have only gotten worse. The first problem I
>> encountered had nothing to do with absolute paths to parents. I'm still
>> using relative paths to parents. Instead, it arose from my using two
>> different parent Ivy modules: bootstrap-parent and master-parent. Some
>> projects extended the former; others the latter. For example:
>>
>>
>> > revision="${version}"
>>
>> location="../bootstrap-parent/ivy.xml" />
>>
>>
>>
>>
>>
>> But then when I pointed the ivy:buildlist Ant task at a project stack
>> that included a mix of both parents and their children, I saw this error:
>>
>> impossible to parse ivy file for …/foo-client/homeowner/build.xml:
>> ivyfile=.../foo-client/homeowner/ivy.xml
>> exception=java.text.ParseException: Problem occurred while parsing ivy
>> file: inconsistent module descriptor file found in
>> 'file:/.../master-parent/ivy.xml': bad module name:
>> expected='bootstrap-parent' found='master-parent'; in
>> file:/.../foo-client/homeowner/ivy.xml
>>
>>
>>
>> What's happening is, the homeowner module extends bootstrap-parent, but
>> somehow the relative path to master-parent/ivy.xml is supplanting the
>> relative path to bootstrap-parent/ivy.xml. It appears buildlist doesn't
>> know how to deal with more than one parent, even though there's no
>> interaction between the two parents.
>>
>>
>>
>> After this, even though I didn't really want to, I thought, "Why not make
>> things simple for buildlist and use just one parent, master-parent?"
>>
>>
>>
>> Here's where things really got wild. With Ivy 2.2.0, buildlist worked
>> just fine, provided I gave it relative paths to the different parents. Now,
>> even with a relative path and even with a single parent, buildlist on
>> 2.3.0-rc1 goes nuts. I go so far as to introduce a buildlist Ivy conf to
>> force project A to sort before project B. How does buildlist on 2.3.0-rc1
>> interpret this? It puts B before A. (And no, I can guarantee I have no
>> circular dependencies.)
>>
>>
>>
>> Can someone say, are there any integration tests that test the
>> interac
