[SQL] large xml database

2010-10-30 Thread Viktor Bojović
Hi, i have very big XML documment which is larger than 50GB and want to import it into databse, and transform it to relational schema. When splitting this documment to smaller independent xml documments i get ~11.1mil XML documents. I have spent lots of time trying to get fastest way to transform

Re: [SQL] large xml database

2010-10-30 Thread Andreas Joseph Krogh
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 10/30/2010 11:49 PM, Viktor Bojović wrote: Hi, i have very big XML documment which is larger than 50GB and want to import it into databse, and transform it to relational schema. When splitting this documment to smaller independent xml

Re: [SQL] large xml database

2010-10-30 Thread Rob Sargent
Andreas Joseph Krogh wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 10/30/2010 11:49 PM, Viktor Bojović wrote: Hi, i have very big XML documment which is larger than 50GB and want to import it into databse, and transform it to relational schema. When splitting this documment to

Re: [SQL] large xml database

2010-10-30 Thread James Cloos
VB == Viktor Bojović viktor.bojo...@gmail.com writes: VB i have very big XML documment which is larger than 50GB and want to VB import it into databse, and transform it to relational schema. Were I doing such a conversion, I'd use perl to convert the xml into something which COPY can grok. Any

Re: [SQL] large xml database

2010-10-30 Thread Lutz Steinborn
On Sat, 30 Oct 2010 23:49:29 +0200 Viktor Bojović viktor.bojo...@gmail.com wrote: many tries have failed because 8GB of ram and 10gb of swap were not enough. also sometimes i get that more than 2^32 operations were performed, and functions stopped to work. we have a similar problem and we