Re: [appengine-java] Re: Java Heap Space Error, Correct Increasing Method?
You need to supply --jvm_flag as a command-line argument to dev_appserver.sh
(aka KickStart), not to java.exe. That should work the same way for the ant,
dev_appserver/ task. (You can see this in config/user/ant-macros.xml).
On Mon, Jun 28, 2010 at 3:51 PM, Mike Dillon mikedillo...@gmail.com wrote:
Another thing to consider. I'm not using eclipse, and I'm building and
running the server with 'ant runserver'. I just realized that ant
isn't calling
dev_appserver.sh so I haven't been executing those args at all. When I
use dev_appserver.sh to run my server it chokes on --jvm_flag but will
run with just the -Xmx, however that has no affect.
- Mike
On Jun 28, 2:47 pm, Mike Dillon mikedillo...@gmail.com wrote:
Thanks,
Yes that is exactly whats going on, the linux box has 4gb of ram, my
mac has 2gb...
And yes I tried the --jvm_flag=-Xmx512m on my mac with no success.
- Mike
On Jun 28, 2:42 pm, Don Schwarz schwa...@google.com wrote:
Have you tried --jvm_flag=-Xmxwhateverm on your Mac? It should work
there
too.
As for the Linux box, the default heap size is different depending on
whether you have a server-class machine or not, so it is possible
that you
are seeing a much higher default heap size just by virtue of using a
machine
with more RAM or more processors.
The -Xmx setting you added to dev_appserver.sh has no effect. Your
code is
executed in a subprocess, which is why --jvm_flag must be used to pass
Java
args through to this process.
On Mon, Jun 28, 2010 at 6:38 PM, Mike Dillon mikedillo...@gmail.com
wrote:
Thanks Don,
I forgot to mention that I'm using a mac with the latest snow leopard
update. It seems that these heap space
commands are incorrect for my machine, because we just tested it on a
linux box and it works fine, even
without the --jvm_flag prefix.
Any ideas?
- Mike
On Jun 28, 1:18 pm, Don Schwarz schwa...@google.com wrote:
I believe you want the following flag:
--jvm_flag=-Xmx512m
On Sun, Jun 27, 2010 at 3:13 PM, Mike Dillon
mikedillo...@gmail.com
wrote:
Hello all,
Im working on a project that reads GTFS archives as part of its
functionality. When we are importing
a particular set of data that has ~60,000 entries my dev server
locks
up around the 27,000 entry. The
error is the java heap space error. I would like to know if
anyone has
successfully upped their heap
space. I searched, and found general instructions but after
implementing them I'm still failing around
the same entry. This is leading me to believe that my command
line
args are not taking affect. Here
is what I have so far:
appengine-sdk-java/bin/dev_appserver.sh:
#!/bin/bash
# Launches the development AppServer
[ -z ${DEBUG} ] || set -x # trace if $DEBUG env. var. is
non-zero
SDK_BIN=`dirname $0 | sed -e s#^\\([^/]\\)#${PWD}/\\1#` # sed
makes
absolute
SDK_LIB=$SDK_BIN/../lib
SDK_CONFIG=$SDK_BIN/../config/sdk
java -Xms1024m -Xmx1024m -ea -cp
$SDK_LIB/appengine-tools-api.jar \
com.google.appengine.tools.KickStart \
com.google.appengine.tools.development.DevAppServerMain $*
Is this correct?
Also the way we get our data is through a custom import function
that
chunks the data into groups of 20
and then processes them and stores them into the datastore. We
are
only using one instance of the
persistance manager as per the recommendations to use a singleton
class. Each batch hits a servlet on
our app and then does its work. Should we close the persistence
manager after every batch is completed?
I'm wondering if this would be a memory leak.
I believe our documentation recommends storing the
PersistenceManagerFactory
as a singleton, but using a new PersistenceManager each time
(remembering
to
close it before discarding).
However, the local implementation of the datastore does keep
everything
in
memory (and simply flushes to disk periodically), so you will run
into
size
issues at some point.
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%2B
unsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this
[appengine-java] Re: Java Heap Space Error, Correct Increasing Method?
Solved!
Okay here is a quick format for how to increase your java heap space:
If you are running your dev server by running ./dev_appserver.sh war
dir then do this:
#!/bin/bash
# Launches the development AppServer
[ -z ${DEBUG} ] || set -x # trace if $DEBUG env. var. is non-zero
SDK_BIN=`dirname $0 | sed -e s#^\\([^/]\\)#${PWD}/\\1#` # sed makes
absolute
SDK_LIB=$SDK_BIN/../lib
SDK_CONFIG=$SDK_BIN/../config/sdk
java -ea -cp $SDK_LIB/appengine-tools-api.jar \
com.google.appengine.tools.KickStart \
com.google.appengine.tools.development.DevAppServerMain --jvm_flag=-
Xmxmem size here $*
*** Notice where the --jvm_flag is !
If you are using ant runserver then you need to edit appengine-sdk-
java/config/user/ang-macros.xml :
Find this, and add an arg value:
macrodef name=dev_appserver description=Runs the App Engine
Development App Server
attribute name=war description=The exploded war directory
containing the application/
attribute name=port default=8080 description=The port the
server starts on/
attribute name=address default=localhost description=The
interface the server binds to/
element name=options optional=true description=Additional
options for dev_appserver/
element name=args optional=true description=Additional
arguments for the java task/
sequential
java classname=com.google.appengine.tools.KickStart
classpath=${appengine.tools.classpath}
fork=true failonerror=true
arg
value=com.google.appengine.tools.development.DevAppServerMain/
arg value=--po...@{port}/
arg value=--addre...@{address}/
arg value=--jvm_flag=-Xmxmem size here/
options/
arg value=@{war}/
args/
/java
/sequential
/macrodef
Thanks for the help Don.
- Mike
On Jun 28, 3:51 pm, Mike Dillon mikedillo...@gmail.com wrote:
Another thing to consider. I'm not using eclipse, and I'm building and
running the server with 'ant runserver'. I just realized that ant
isn't calling
dev_appserver.sh so I haven't been executing those args at all. When I
use dev_appserver.sh to run my server it chokes on --jvm_flag but will
run with just the -Xmx, however that has no affect.
- Mike
On Jun 28, 2:47 pm, Mike Dillon mikedillo...@gmail.com wrote:
Thanks,
Yes that is exactly whats going on, the linux box has 4gb of ram, my
mac has 2gb...
And yes I tried the --jvm_flag=-Xmx512m on my mac with no success.
- Mike
On Jun 28, 2:42 pm, Don Schwarz schwa...@google.com wrote:
Have you tried --jvm_flag=-Xmxwhateverm on your Mac? It should work
there
too.
As for the Linux box, the default heap size is different depending on
whether you have a server-class machine or not, so it is possible that
you
are seeing a much higher default heap size just by virtue of using a
machine
with more RAM or more processors.
The -Xmx setting you added to dev_appserver.sh has no effect. Your code
is
executed in a subprocess, which is why --jvm_flag must be used to pass
Java
args through to this process.
On Mon, Jun 28, 2010 at 6:38 PM, Mike Dillon mikedillo...@gmail.com
wrote:
Thanks Don,
I forgot to mention that I'm using a mac with the latest snow leopard
update. It seems that these heap space
commands are incorrect for my machine, because we just tested it on a
linux box and it works fine, even
without the --jvm_flag prefix.
Any ideas?
- Mike
On Jun 28, 1:18 pm, Don Schwarz schwa...@google.com wrote:
I believe you want the following flag:
--jvm_flag=-Xmx512m
On Sun, Jun 27, 2010 at 3:13 PM, Mike Dillon mikedillo...@gmail.com
wrote:
Hello all,
Im working on a project that reads GTFS archives as part of its
functionality. When we are importing
a particular set of data that has ~60,000 entries my dev server
locks
up around the 27,000 entry. The
error is the java heap space error. I would like to know if anyone
has
successfully upped their heap
space. I searched, and found general instructions but after
implementing them I'm still failing around
the same entry. This is leading me to believe that my command line
args are not taking affect. Here
is what I have so far:
appengine-sdk-java/bin/dev_appserver.sh:
#!/bin/bash
# Launches the development AppServer
[ -z ${DEBUG} ] || set -x # trace if $DEBUG env. var. is non-zero
SDK_BIN=`dirname $0 | sed -e s#^\\([^/]\\)#${PWD}/\\1#` # sed
makes
absolute
SDK_LIB=$SDK_BIN/../lib
SDK_CONFIG=$SDK_BIN/../config/sdk
java -Xms1024m -Xmx1024m -ea -cp
$SDK_LIB/appengine-tools-api.jar \
com.google.appengine.tools.KickStart \
com.google.appengine.tools.development.DevAppServerMain $*
Is this correct?
Also the way we get our data is through a custom import function
that
chunks the data
[appengine-java] Re: Java Heap Space Error, Correct Increasing Method?
Thanks Toby, I had figured this out before you replied but didn't
refresh the page before I posted my solution. I appreciate the help!
On Jun 28, 3:59 pm, Toby Reyelts to...@google.com wrote:
You need to supply --jvm_flag as a command-line argument to dev_appserver.sh
(aka KickStart), not to java.exe. That should work the same way for the ant,
dev_appserver/ task. (You can see this in config/user/ant-macros.xml).
On Mon, Jun 28, 2010 at 3:51 PM, Mike Dillon mikedillo...@gmail.com wrote:
Another thing to consider. I'm not using eclipse, and I'm building and
running the server with 'ant runserver'. I just realized that ant
isn't calling
dev_appserver.sh so I haven't been executing those args at all. When I
use dev_appserver.sh to run my server it chokes on --jvm_flag but will
run with just the -Xmx, however that has no affect.
- Mike
On Jun 28, 2:47 pm, Mike Dillon mikedillo...@gmail.com wrote:
Thanks,
Yes that is exactly whats going on, the linux box has 4gb of ram, my
mac has 2gb...
And yes I tried the --jvm_flag=-Xmx512m on my mac with no success.
- Mike
On Jun 28, 2:42 pm, Don Schwarz schwa...@google.com wrote:
Have you tried --jvm_flag=-Xmxwhateverm on your Mac? It should work
there
too.
As for the Linux box, the default heap size is different depending on
whether you have a server-class machine or not, so it is possible
that you
are seeing a much higher default heap size just by virtue of using a
machine
with more RAM or more processors.
The -Xmx setting you added to dev_appserver.sh has no effect. Your
code is
executed in a subprocess, which is why --jvm_flag must be used to pass
Java
args through to this process.
On Mon, Jun 28, 2010 at 6:38 PM, Mike Dillon mikedillo...@gmail.com
wrote:
Thanks Don,
I forgot to mention that I'm using a mac with the latest snow leopard
update. It seems that these heap space
commands are incorrect for my machine, because we just tested it on a
linux box and it works fine, even
without the --jvm_flag prefix.
Any ideas?
- Mike
On Jun 28, 1:18 pm, Don Schwarz schwa...@google.com wrote:
I believe you want the following flag:
--jvm_flag=-Xmx512m
On Sun, Jun 27, 2010 at 3:13 PM, Mike Dillon
mikedillo...@gmail.com
wrote:
Hello all,
Im working on a project that reads GTFS archives as part of its
functionality. When we are importing
a particular set of data that has ~60,000 entries my dev server
locks
up around the 27,000 entry. The
error is the java heap space error. I would like to know if
anyone has
successfully upped their heap
space. I searched, and found general instructions but after
implementing them I'm still failing around
the same entry. This is leading me to believe that my command
line
args are not taking affect. Here
is what I have so far:
appengine-sdk-java/bin/dev_appserver.sh:
#!/bin/bash
# Launches the development AppServer
[ -z ${DEBUG} ] || set -x # trace if $DEBUG env. var. is
non-zero
SDK_BIN=`dirname $0 | sed -e s#^\\([^/]\\)#${PWD}/\\1#` # sed
makes
absolute
SDK_LIB=$SDK_BIN/../lib
SDK_CONFIG=$SDK_BIN/../config/sdk
java -Xms1024m -Xmx1024m -ea -cp
$SDK_LIB/appengine-tools-api.jar \
com.google.appengine.tools.KickStart \
com.google.appengine.tools.development.DevAppServerMain $*
Is this correct?
Also the way we get our data is through a custom import function
that
chunks the data into groups of 20
and then processes them and stores them into the datastore. We
are
only using one instance of the
persistance manager as per the recommendations to use a singleton
class. Each batch hits a servlet on
our app and then does its work. Should we close the persistence
manager after every batch is completed?
I'm wondering if this would be a memory leak.
I believe our documentation recommends storing the
PersistenceManagerFactory
as a singleton, but using a new PersistenceManager each time
(remembering
to
close it before discarding).
However, the local implementation of the datastore does keep
everything
in
memory (and simply flushes to disk periodically), so you will run
into
size
issues at some point.
--
You received this message because you are subscribed to the Google
Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B
unsubscr...@googlegroups.comgoogle-appengine-java%2B
unsubscr...@googlegroups.com
.
For more options, visit this
