Hi, It depends on how you do the import. Could you get a heap histogram for a large import (jmap -histo:live <pid>), and could you tell us how you do the import exactly?
If you use "create table ... as select" then it can't run out of memory (as far as I know), because no undo log is needed. If you execute many insert or update statements, then the undo log is used, and at least a few bytes are needed per record (the actual data is stored to disk at some point, but the position of the data is not). So in theory it could run out of memory, even thought the transaction would need to be huge. By the way, with the new MVStore storage (not production ready yet), there should no longer be a limit on the size of a transaction, as no undo log is needed. Regards, Thomas On Tue, Sep 10, 2013 at 10:51 AM, Ryan How <r...@exemail.com.au> wrote: > Howcome it is getting OOM on import? It shouldn't do that should it? I've > imported several GB of data before, I did get OOM, but increasing heap size > to around 1GB worked for me. I didn't need to go to crazy sizes... > > > On 10/09/2013 4:32 PM, Noel Grandin wrote: > >> >> On 2013-09-06 20:15, Brian Craft wrote: >> >>> I need to load about 1G of data into an existing db, while maintaining >>> data coherence. Wrapping the inserts in one transaction results in >>> out-of-memory problems in the jvm. I increased the max heap size to 8g w/o >>> improvement. I can split it into a bunch of smaller commits, which works >>> fine, but then on error I need a bunch of application code to delete the >>> transactions which succeeded. The deletes will need their own transactions, >>> which could also fail. >>> >>> Is there any better way to do this? >>> >>> >> Not really. >> One strategy would be to copy the DB, since with H2 it's just a single >> file, and then run your import process. >> If it fails, just replace the modified DB with the backup. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "H2 Database" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > h2-database+unsubscribe@**googlegroups.com<h2-database%2bunsubscr...@googlegroups.com> > . > To post to this group, send email to h2-database@googlegroups.com. > Visit this group at > http://groups.google.com/**group/h2-database<http://groups.google.com/group/h2-database> > . > For more options, visit > https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> > . > -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/groups/opt_out.
