Each insert is a separate "call" to a separate "insert" tag that I defined in my dbutils. Each insert is in a separate transaction.
So, for instance, here's my "insert" tag: <define:tag name="insert"> <sql:transaction dataSource="${ds}"> <j:catch var="ex"> <sql:update sql="insert into ${table} VALUES (${values})" var="upd"/> </j:catch> <j:if test="${ex != null}"> FAILED INSERT. <j:expr value="${ex}"/> <ant:fail message="Failed table insert" /> </j:if> </sql:transaction> </define:tag> I essentially call this about 18000 times with different parameters. Watching the task manager, the memory usage slowly increases as it inserts rows. > -----Original Message----- > From: Paul Libbrecht [mailto:[EMAIL PROTECTED] > Sent: Monday, July 09, 2007 4:13 PM > To: Jakarta Commons Users List > Subject: Re: [jelly] How to make a large Jelly-SQL script > more memory efficient? > > David, > > I do not see any other way than programmatically separating > the "lines" of the SQL query. Is this doable ? Like, if it's > a backup, it probably has a chance that each query is a line, or ? > > If yes, then it should be pretty easy to use a > LineNumberReader and feed each line as an SQL query... or do > I mistake ? > > Alternatively, we could tweak the sql taglib to actually read > the SQL script and not load it as a string but this would > mean to decompose the lines in an appropriate way. I do not > know how that could be done. If you know of a generic way, > let's open a jira issue and work on that, it'd be easy. > > thanks > > paul > > > Le 10 juil. 07 à 01:04, Karr, David a écrit : > > > I have a large Jelly-SQL script (about 3.8 mb) that just does SQL > > calls to insert rows into a database. Each row it inserts > is pretty > > small, but it inserts a lot of rows (relatively). It currently > > inserts about 18000 rows. What I'm finding is that the > script won't > > even run on Windows (2 gig process limit). It takes too > much memory. > > A previous version of the script only inserted about 11000 > rows, and > > it was able to run on Windows. The vast majority of the script is > > generated by a Java app that processes a spreadsheet. > > > > While the script is running, I watch it in Task Manager, > and I see the > > memory very slowly increasing. It runs for quite a while. > It finally > > runs out of memory in the JVM and fails. > > > > Is there some strategy for building a script like this so > it is more > > memory-efficient? > > > > The script currently has a top-level "j:jelly" element, imports a > > utility package (some dbutils), and then imports the > generated portion > > of the script. > > > > I can temporarily work around this by building multiple top-level > > scripts that call separate pieces of the big script, but that's > > annoying. I'm also trying to get this set up on a Unix > system, to get > > a larger process size. > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]