Damn, this is a really good catch. I think it should be rather simple to wait with committing the transaction until the full response is generated.
I'll look into it. In general, and as has been mentioned several times before, we obviously need to look at how we could stream the batch operations, rather than keeping it all in memory.. /Jake On Fri, Sep 2, 2011 at 5:39 AM, jadell <josh.ad...@gmail.com> wrote: > I'm sending a batch REST request, with 15000 operations (10000 create > nodes, > and 5000 create relationships between those nodes.) In webadmin, I can see > all the nodes and relationships have been created, and I can browse them > using the data browser. So the all my entities were successfully created. > But the HTTP response is a "500 Java heap space" error. It looks from the > stack trace in the log like the failure occurred when mapping the results > into the HTTP response. This could be a pretty nasty surprise for someone > who thinks their batch failed when all the entities were actually created. > > Is there anyway to not commit the transaction on the server until the > response is generated and sent successfully? > > Here is the stack trace from the log: > > > > Sep 1, 2011 11:30:01 PM com.sun.jersey.spi.container.ContainerResponse > mapMappableContainerException > SEVERE: The exception contained within MappableContainerException could not > be mapped to a response, re-throwing to the HTTP container > java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:2882) > at > > java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) > at > java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) > at java.lang.StringBuffer.append(StringBuffer.java:219) > at org.apache.commons.lang.StringUtils.join(StringUtils.java:3088) > at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133) > at > > org.neo4j.server.rest.repr.BatchOperationResults.toJSON(BatchOperationResults.java:99) > at > > org.neo4j.server.rest.web.BatchOperationService.performBatchOperations(BatchOperationService.java:88) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:184) > at > > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) > at > > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:276) > at > > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:83) > at > > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:133) > at > > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:71) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1171) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1103) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1053) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1043) > at > > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:406) > at > > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:477) > at > > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:662) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) > at > > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > Sep 1, 2011 11:30:01 PM org.mortbay.log.Slf4jLog warn > SEVERE: /db/data/batch > java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:2882) > at > > java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) > at > java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) > at java.lang.StringBuffer.append(StringBuffer.java:219) > at org.apache.commons.lang.StringUtils.join(StringUtils.java:3088) > at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133) > at > > org.neo4j.server.rest.repr.BatchOperationResults.toJSON(BatchOperationResults.java:99) > at > > org.neo4j.server.rest.web.BatchOperationService.performBatchOperations(BatchOperationService.java:88) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:184) > at > > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) > at > > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:276) > at > > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:83) > at > > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:133) > at > > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:71) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1171) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1103) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1053) > at > > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1043) > at > > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:406) > at > > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:477) > at > > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:662) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) > at > > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) > > > > -- Josh Adell > > -- > View this message in context: > http://neo4j-community-discussions.438527.n3.nabble.com/REST-batch-completes-with-error-code-tp3303247p3303247.html > Sent from the Neo4j Community Discussions mailing list archive at > Nabble.com. > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
