I did stuff like this with BBEdit (on a Mac). But opening the file took about 10 minutes ;-) Good luck!
-will On 13.12.2010, at 11:09, Ernst Bunders wrote: > > > 2010/12/13 Will Scheidegger <[email protected]> > Hm... very interested in a solution too as we see this happening > - on imports > - on exports > - on deletion > of massive nodes. > > For starters if you don't have all article in a single-level structure (i.e. > if you have folders and subfolders) you can try to open the XML in a good > text editor and split it up into several chunks which can then be partially > imported. > > Well, we don't have them in a single level structure, and what you propose is > exactly what I'm doing just now. Currently I am trying Jedit, which gave me > good results in the past, but the same problem seems to occur here. Perhaps > there are streaming xml editors? I don't know. > > Is there an editor in specific that you would like to recommend? > > regards, > > Ernst > > > -will > > > On 13.12.2010, at 10:44, Ernst Bunders wrote: > >> Hello >> >> We are in the final stages of (re)creating one of our largest sites in >> magnolia (http://geschiedenis.vpro.nl). This site has a massive archive of >> articles and dossiers which we all diligently imported in the new site. >> >> So, the final step is to export the data from our test environment and re >> import it on the acceptance server. This is where trouble hits. I have a >> file website.geschiedenis.xml that is quite large. it's size is 292Mb, and >> cat website.geschiedenis.xml | grep "<sv:node" | wc -l >> yields: 289648. Quite a lot of nodes. >> >> The problem is that we can not import this file anymore. With max heap space >> settings of 2Gb we still get OutOfMemory exceptions. >> >> Apparently this whole xml tree is being constructed in memory, in a >> not-so-efficient way (lots of overhead if you can't fit 292 Mb of data in a >> 2Gb tree) >> >> We do not have a lot of knowledge about this subject yet. We don't know if >> and how Jackrabbit can be tuned to deal with different scenario's. So any >> help would be appreciated. This is clearly a problem we need to get out of >> the way, also because it impedes our possibilities for backup and restore. >> >> For now I am going to try to cut the file up into different smaller ones, >> see how that will go. >> >> regards, >> >> Ernst >> >> -- >> Ernst Bunders >> Ontwikkelaar VPRO > > > > ---------------------------------------------------------------- > For list details see > http://www.magnolia-cms.com/home/community/mailing-lists.html > To unsubscribe, E-mail to: <[email protected]> > ---------------------------------------------------------------- > > > > -- > Ernst Bunders > Ontwikkelaar VPRO ---------------------------------------------------------------- For list details see http://www.magnolia-cms.com/home/community/mailing-lists.html To unsubscribe, E-mail to: <[email protected]> ----------------------------------------------------------------
