Re: GWT 2.5 compiler is running out of memory

[Antonio Petrelli]

Il giorno venerdì 1 febbraio 2013 18:02:11 UTC+1, dhoffer ha scritto:

> With that set I was able to remove ...but just wondering 
> what  means when using threads.  Does that control the 
> number of threads, have no effect?
>

You have to set the "gwt.jjs.maxThreads" property, in my case:
-Dgwt.jjs.maxThreads=8 
The number of threads is calculated as the minimum between the number of 
local workers and the above property value.

HTH
Antonio

-- 
You received this message because you are subscribed to the Google Groups 
"Google Web Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To post to this group, send email to google-web-toolkit@googlegroups.com.
Visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: GWT 2.5 compiler is running out of memory

[Thomas Broyer]

On Friday, February 1, 2013 6:02:11 PM UTC+1, dhoffer wrote:
>
> Yes 
> using 
> -Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev.ThreadedPermutationWorkerFactory
>  
> fixed the problem!  Everything compiles fine now.
>
> With that set I was able to remove ...but just wondering 
> what  means when using threads.  Does that control the 
> number of threads, have no effect?
>

localWorkers controls the number of "workers" (defaults to 1 in the 
Compiler, but gwt-maven-plugin sets it to 
Runtime#availableProcessors()
 by 
default); the workers are created by a factory. The default factory 
(ExternalPermutationWOrkerFactory) spawns external processes (looking at 
the code, you can override the JVM opts the the subprocesses by setting the 
gwt.jjs.javaArgs system property, and even the java command –to use a 
different JVM for instance– using the gwt.jjs.javaCommand system prop).
Google probably uses a custom factory the distribute the load to a cluster, 
in this case the localWorkers is not used (if the factory returns false 
from its isLocal() method).

-- 
You received this message because you are subscribed to the Google Groups 
"Google Web Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To post to this group, send email to google-web-toolkit@googlegroups.com.
Visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: GWT 2.5 compiler is running out of memory

[David Hoffer]

Yes
using 
-Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev.ThreadedPermutationWorkerFactory
fixed the problem!  Everything compiles fine now.

With that set I was able to remove ...but just wondering
what 
means when using threads.  Does that control the number of threads, have no
effect?

Thanks!
-Dave


On Fri, Feb 1, 2013 at 9:18 AM, Thomas Broyer  wrote:

> By default, GWT spawns new processes for the "workers", you can switch to
> using threads by
> passing 
> -Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev.ThreadedPermutationWorkerFactory
> in .
> Either that or try an even smaller value for localWorkers (or use a
> smaller memory opts)
>
>
> On Friday, February 1, 2013 5:10:52 PM UTC+1, dhoffer wrote:
>
>> I've got 8 cores so I set  to 4...that makes a big
>> difference in how much RAM is uses, with that set it never spikes up near
>> 8GB.  However it still fails...with this error message:
>>
>> [INFO]Compiling 12 permutations
>> [INFO]   Compiling permutation 1...
>> [INFO]   Process output
>> [INFO]  Error occurred during initialization of VM
>> [INFO]   Process output
>> [INFO]  Error occurred during initialization of VM
>> [INFO]  Could not reserve enough space for object heap
>> [INFO]  Could not reserve enough space for object heap
>> [INFO]   Process output
>> [INFO]  Error occurred during initialization of VM
>> [INFO]  Could not reserve enough space for object heap
>> [INFO]   Compiling permutation 4...
>>
>>
>> I do have some local changes...it's possible the local changes have some
>> errors in thembut I need the compiler to tell me what they are if
>> that's the case.  Instead it fails to compile, not sure what to do to
>> resolve this.  Note that if I didn't kill the build at this point...it
>> would continue on with the next permutation...and not report this error at
>> the end.
>>
>> -Dave
>>
>>
>>
>> On Fri, Feb 1, 2013 at 6:17 AM, David Hoffer  wrote:
>>
>>> The last error was this:
>>>
>>> [INFO]  #
>>> [INFO]  # There is insufficient memory for the Java Runtime
>>> Environment to continue.
>>> [INFO]  # Native memory allocation (malloc) failed to allocate
>>> 32744 bytes for ChunkPool::allocate
>>> [INFO]  #
>>> [INFO]  # There is insufficient memory for the Java Runtime
>>> Environment to continue.
>>> [INFO]  # Native memory allocation (malloc) failed to allocate
>>> 450768 bytes for Chunk::new
>>>
>>> Not sure if it matters but we did recently upgrade to Java 1.7 build 11
>>> (the previous builds had a major security flaw and we had to upgrade
>>> everything) but that happened a couple days before I started seeing this I
>>> believe.
>>>
>>> I'll try the  suggestion...
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Feb 1, 2013 at 3:03 AM, Thomas Broyer  wrote:
>>>
 Which kind of OutOfMemory are you seeing?
 Also, try to set  to some value lower than the number of
 cores/cpus of your system: if you don't set localWorkers, the
 gwt-maven-plugin will use the number of cores/cpus by default, and
 obviously parallelizing work increases the memory needs.


 On Thursday, January 31, 2013 11:32:10 PM UTC+1, dhoffer wrote:
>
> I've got a rather large GWT build using Maven and the gwt-maven-plugin
> and I'm getting out of memory errors building now.  I'm not sure what's
> changed, it used to take considerable memory but now its taking so much I
> can't build.  I'm running Windows 7 64 bit with 8GB RAM and using Java 7 
> 64
> bit to compile, here are some of my settings:
>
> MAVEN_OPTS=-Xms128m -Xmx1024m -XX:MaxPermSize=256m
>
> Then in the gwt-maven-plugin configuration I
> have -Xmx2048m -Xmx2048m -XX:MaxPermSize=512m extraJvm**Args>
>
> I've tried several combinations of values here with no luck...with
> these values...
> - When I start the maven command line build the system has about 3GB
> of memory used
> - When it it starts the GWT part of the build its at about 4GB used.
> - The I have lots of rebind operations going on generating
> code...during this time it goes to about 5GB used.
> - Then when it hits the permutations step this is where it breaks
> down...it very quickly goes to about 7.5 GB used.  Then I get out of 
> memory
> errors and the build fails.
>
> So there seems to be a couple of issues, first the GWT compiler
> doesn't seem to work within the -Xmx2048m value, I used to use -Xmx1024m
> and it does about the same thing...it uses a lot more memory than
> that...and then fails.  In this case it used 3.5 GB before it failed.  And
> secondly, I can't give GWT more memory because my system doesn't have more
> to give it...this was with almost all other programs closed...and it still
> failed.
>
> Does anyone have an idea why my build might be using so much memory
> and how to lim

Re: GWT 2.5 compiler is running out of memory

[Thomas Broyer]

By default, GWT spawns new processes for the "workers", you can switch to 
using threads by 
passing 
-Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev.ThreadedPermutationWorkerFactory
 
in .
Either that or try an even smaller value for localWorkers (or use a smaller 
memory opts)

On Friday, February 1, 2013 5:10:52 PM UTC+1, dhoffer wrote:
>
> I've got 8 cores so I set  to 4...that makes a big 
> difference in how much RAM is uses, with that set it never spikes up near 
> 8GB.  However it still fails...with this error message:
>
> [INFO]Compiling 12 permutations
> [INFO]   Compiling permutation 1...
> [INFO]   Process output
> [INFO]  Error occurred during initialization of VM
> [INFO]   Process output
> [INFO]  Error occurred during initialization of VM
> [INFO]  Could not reserve enough space for object heap
> [INFO]  Could not reserve enough space for object heap
> [INFO]   Process output
> [INFO]  Error occurred during initialization of VM
> [INFO]  Could not reserve enough space for object heap
> [INFO]   Compiling permutation 4...
>
>
> I do have some local changes...it's possible the local changes have some 
> errors in thembut I need the compiler to tell me what they are if 
> that's the case.  Instead it fails to compile, not sure what to do to 
> resolve this.  Note that if I didn't kill the build at this point...it 
> would continue on with the next permutation...and not report this error at 
> the end.
>
> -Dave
>
>
>
> On Fri, Feb 1, 2013 at 6:17 AM, David Hoffer 
> > wrote:
>
>> The last error was this:
>>
>> [INFO]  #
>> [INFO]  # There is insufficient memory for the Java Runtime 
>> Environment to continue.
>> [INFO]  # Native memory allocation (malloc) failed to allocate 
>> 32744 bytes for ChunkPool::allocate
>> [INFO]  #
>> [INFO]  # There is insufficient memory for the Java Runtime 
>> Environment to continue.
>> [INFO]  # Native memory allocation (malloc) failed to allocate 
>> 450768 bytes for Chunk::new
>>
>> Not sure if it matters but we did recently upgrade to Java 1.7 build 11 
>> (the previous builds had a major security flaw and we had to upgrade 
>> everything) but that happened a couple days before I started seeing this I 
>> believe.
>>
>> I'll try the  suggestion...
>>
>>
>>
>>
>>
>> On Fri, Feb 1, 2013 at 3:03 AM, Thomas Broyer 
>> > wrote:
>>
>>> Which kind of OutOfMemory are you seeing?
>>> Also, try to set  to some value lower than the number of 
>>> cores/cpus of your system: if you don't set localWorkers, the 
>>> gwt-maven-plugin will use the number of cores/cpus by default, and 
>>> obviously parallelizing work increases the memory needs.
>>>
>>>
>>> On Thursday, January 31, 2013 11:32:10 PM UTC+1, dhoffer wrote:

 I've got a rather large GWT build using Maven and the gwt-maven-plugin 
 and I'm getting out of memory errors building now.  I'm not sure what's 
 changed, it used to take considerable memory but now its taking so much I 
 can't build.  I'm running Windows 7 64 bit with 8GB RAM and using Java 7 
 64 
 bit to compile, here are some of my settings:

 MAVEN_OPTS=-Xms128m -Xmx1024m -XX:MaxPermSize=256m

 Then in the gwt-maven-plugin configuration I 
 have -Xmx2048m -Xmx2048m -XX:MaxPermSize=512m>>> extraJvmArgs>

 I've tried several combinations of values here with no luck...with 
 these values...
 - When I start the maven command line build the system has about 3GB of 
 memory used
 - When it it starts the GWT part of the build its at about 4GB used.
 - The I have lots of rebind operations going on generating 
 code...during this time it goes to about 5GB used.
 - Then when it hits the permutations step this is where it breaks 
 down...it very quickly goes to about 7.5 GB used.  Then I get out of 
 memory 
 errors and the build fails.

 So there seems to be a couple of issues, first the GWT compiler doesn't 
 seem to work within the -Xmx2048m value, I used to use -Xmx1024m and it 
 does about the same thing...it uses a lot more memory than that...and then 
 fails.  In this case it used 3.5 GB before it failed.  And secondly, I 
 can't give GWT more memory because my system doesn't have more to give 
 it...this was with almost all other programs closed...and it still failed.

 Does anyone have an idea why my build might be using so much memory and 
 how to limit this?

 Btw, at the start of the permutation compile step sometimes it reports 
 6 permutations and sometimes 12...seems to have the same problem in both 
 cases but I'm not sure why it sometimes has 12 and other times 6.

 -Dave



  -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Google Web Toolkit" group.
>>> To unsubscribe from this group and stop receiving e

Re: GWT 2.5 compiler is running out of memory

[David Hoffer]

I've got 8 cores so I set  to 4...that makes a big difference
in how much RAM is uses, with that set it never spikes up near 8GB.
 However it still fails...with this error message:

[INFO]Compiling 12 permutations
[INFO]   Compiling permutation 1...
[INFO]   Process output
[INFO]  Error occurred during initialization of VM
[INFO]   Process output
[INFO]  Error occurred during initialization of VM
[INFO]  Could not reserve enough space for object heap
[INFO]  Could not reserve enough space for object heap
[INFO]   Process output
[INFO]  Error occurred during initialization of VM
[INFO]  Could not reserve enough space for object heap
[INFO]   Compiling permutation 4...


I do have some local changes...it's possible the local changes have some
errors in thembut I need the compiler to tell me what they are if
that's the case.  Instead it fails to compile, not sure what to do to
resolve this.  Note that if I didn't kill the build at this point...it
would continue on with the next permutation...and not report this error at
the end.

-Dave



On Fri, Feb 1, 2013 at 6:17 AM, David Hoffer  wrote:

> The last error was this:
>
> [INFO]  #
> [INFO]  # There is insufficient memory for the Java Runtime
> Environment to continue.
> [INFO]  # Native memory allocation (malloc) failed to allocate
> 32744 bytes for ChunkPool::allocate
> [INFO]  #
> [INFO]  # There is insufficient memory for the Java Runtime
> Environment to continue.
> [INFO]  # Native memory allocation (malloc) failed to allocate
> 450768 bytes for Chunk::new
>
> Not sure if it matters but we did recently upgrade to Java 1.7 build 11
> (the previous builds had a major security flaw and we had to upgrade
> everything) but that happened a couple days before I started seeing this I
> believe.
>
> I'll try the  suggestion...
>
>
>
>
>
> On Fri, Feb 1, 2013 at 3:03 AM, Thomas Broyer  wrote:
>
>> Which kind of OutOfMemory are you seeing?
>> Also, try to set  to some value lower than the number of
>> cores/cpus of your system: if you don't set localWorkers, the
>> gwt-maven-plugin will use the number of cores/cpus by default, and
>> obviously parallelizing work increases the memory needs.
>>
>>
>> On Thursday, January 31, 2013 11:32:10 PM UTC+1, dhoffer wrote:
>>>
>>> I've got a rather large GWT build using Maven and the gwt-maven-plugin
>>> and I'm getting out of memory errors building now.  I'm not sure what's
>>> changed, it used to take considerable memory but now its taking so much I
>>> can't build.  I'm running Windows 7 64 bit with 8GB RAM and using Java 7 64
>>> bit to compile, here are some of my settings:
>>>
>>> MAVEN_OPTS=-Xms128m -Xmx1024m -XX:MaxPermSize=256m
>>>
>>> Then in the gwt-maven-plugin configuration I
>>> have -Xmx2048m -Xmx2048m -XX:MaxPermSize=512m>> extraJvmArgs>
>>>
>>> I've tried several combinations of values here with no luck...with these
>>> values...
>>> - When I start the maven command line build the system has about 3GB of
>>> memory used
>>> - When it it starts the GWT part of the build its at about 4GB used.
>>> - The I have lots of rebind operations going on generating code...during
>>> this time it goes to about 5GB used.
>>> - Then when it hits the permutations step this is where it breaks
>>> down...it very quickly goes to about 7.5 GB used.  Then I get out of memory
>>> errors and the build fails.
>>>
>>> So there seems to be a couple of issues, first the GWT compiler doesn't
>>> seem to work within the -Xmx2048m value, I used to use -Xmx1024m and it
>>> does about the same thing...it uses a lot more memory than that...and then
>>> fails.  In this case it used 3.5 GB before it failed.  And secondly, I
>>> can't give GWT more memory because my system doesn't have more to give
>>> it...this was with almost all other programs closed...and it still failed.
>>>
>>> Does anyone have an idea why my build might be using so much memory and
>>> how to limit this?
>>>
>>> Btw, at the start of the permutation compile step sometimes it reports 6
>>> permutations and sometimes 12...seems to have the same problem in both
>>> cases but I'm not sure why it sometimes has 12 and other times 6.
>>>
>>> -Dave
>>>
>>>
>>>
>>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Google Web Toolkit" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to google-web-toolkit+unsubscr...@googlegroups.com.
>> To post to this group, send email to google-web-toolkit@googlegroups.com.
>> Visit this group at
>> http://groups.google.com/group/google-web-toolkit?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google Web Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsub

Re: GWT 2.5 compiler is running out of memory

[David Hoffer]

The last error was this:

[INFO]  #
[INFO]  # There is insufficient memory for the Java Runtime
Environment to continue.
[INFO]  # Native memory allocation (malloc) failed to allocate
32744 bytes for ChunkPool::allocate
[INFO]  #
[INFO]  # There is insufficient memory for the Java Runtime
Environment to continue.
[INFO]  # Native memory allocation (malloc) failed to allocate
450768 bytes for Chunk::new

Not sure if it matters but we did recently upgrade to Java 1.7 build 11
(the previous builds had a major security flaw and we had to upgrade
everything) but that happened a couple days before I started seeing this I
believe.

I'll try the  suggestion...





On Fri, Feb 1, 2013 at 3:03 AM, Thomas Broyer  wrote:

> Which kind of OutOfMemory are you seeing?
> Also, try to set  to some value lower than the number of
> cores/cpus of your system: if you don't set localWorkers, the
> gwt-maven-plugin will use the number of cores/cpus by default, and
> obviously parallelizing work increases the memory needs.
>
>
> On Thursday, January 31, 2013 11:32:10 PM UTC+1, dhoffer wrote:
>>
>> I've got a rather large GWT build using Maven and the gwt-maven-plugin
>> and I'm getting out of memory errors building now.  I'm not sure what's
>> changed, it used to take considerable memory but now its taking so much I
>> can't build.  I'm running Windows 7 64 bit with 8GB RAM and using Java 7 64
>> bit to compile, here are some of my settings:
>>
>> MAVEN_OPTS=-Xms128m -Xmx1024m -XX:MaxPermSize=256m
>>
>> Then in the gwt-maven-plugin configuration I have -Xmx2048m
>> -Xmx2048m -XX:MaxPermSize=512m
>>
>> I've tried several combinations of values here with no luck...with these
>> values...
>> - When I start the maven command line build the system has about 3GB of
>> memory used
>> - When it it starts the GWT part of the build its at about 4GB used.
>> - The I have lots of rebind operations going on generating code...during
>> this time it goes to about 5GB used.
>> - Then when it hits the permutations step this is where it breaks
>> down...it very quickly goes to about 7.5 GB used.  Then I get out of memory
>> errors and the build fails.
>>
>> So there seems to be a couple of issues, first the GWT compiler doesn't
>> seem to work within the -Xmx2048m value, I used to use -Xmx1024m and it
>> does about the same thing...it uses a lot more memory than that...and then
>> fails.  In this case it used 3.5 GB before it failed.  And secondly, I
>> can't give GWT more memory because my system doesn't have more to give
>> it...this was with almost all other programs closed...and it still failed.
>>
>> Does anyone have an idea why my build might be using so much memory and
>> how to limit this?
>>
>> Btw, at the start of the permutation compile step sometimes it reports 6
>> permutations and sometimes 12...seems to have the same problem in both
>> cases but I'm not sure why it sometimes has 12 and other times 6.
>>
>> -Dave
>>
>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "Google Web Toolkit" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-web-toolkit+unsubscr...@googlegroups.com.
> To post to this group, send email to google-web-toolkit@googlegroups.com.
> Visit this group at
> http://groups.google.com/group/google-web-toolkit?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google Web Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To post to this group, send email to google-web-toolkit@googlegroups.com.
Visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: GWT 2.5 compiler is running out of memory

[Thomas Broyer]

Which kind of OutOfMemory are you seeing?
Also, try to set  to some value lower than the number of 
cores/cpus of your system: if you don't set localWorkers, the 
gwt-maven-plugin will use the number of cores/cpus by default, and 
obviously parallelizing work increases the memory needs.

On Thursday, January 31, 2013 11:32:10 PM UTC+1, dhoffer wrote:
>
> I've got a rather large GWT build using Maven and the gwt-maven-plugin and 
> I'm getting out of memory errors building now.  I'm not sure what's 
> changed, it used to take considerable memory but now its taking so much I 
> can't build.  I'm running Windows 7 64 bit with 8GB RAM and using Java 7 64 
> bit to compile, here are some of my settings:
>
> MAVEN_OPTS=-Xms128m -Xmx1024m -XX:MaxPermSize=256m
>
> Then in the gwt-maven-plugin configuration I have -Xmx2048m 
> -Xmx2048m -XX:MaxPermSize=512m
>
> I've tried several combinations of values here with no luck...with these 
> values...
> - When I start the maven command line build the system has about 3GB of 
> memory used
> - When it it starts the GWT part of the build its at about 4GB used.
> - The I have lots of rebind operations going on generating code...during 
> this time it goes to about 5GB used.
> - Then when it hits the permutations step this is where it breaks 
> down...it very quickly goes to about 7.5 GB used.  Then I get out of memory 
> errors and the build fails.
>
> So there seems to be a couple of issues, first the GWT compiler doesn't 
> seem to work within the -Xmx2048m value, I used to use -Xmx1024m and it 
> does about the same thing...it uses a lot more memory than that...and then 
> fails.  In this case it used 3.5 GB before it failed.  And secondly, I 
> can't give GWT more memory because my system doesn't have more to give 
> it...this was with almost all other programs closed...and it still failed.
>
> Does anyone have an idea why my build might be using so much memory and 
> how to limit this?
>
> Btw, at the start of the permutation compile step sometimes it reports 6 
> permutations and sometimes 12...seems to have the same problem in both 
> cases but I'm not sure why it sometimes has 12 and other times 6.
>
> -Dave
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google Web Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-web-toolkit+unsubscr...@googlegroups.com.
To post to this group, send email to google-web-toolkit@googlegroups.com.
Visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.