Re: [SQL] large xml database

2010-10-31 Thread Rob Sargent
Skipping much of the included thread, urgently. btw. you have mentioned "This I believe would parse nicely into a tidy but non-trivial schema directly", does it mean that postgre has a support for restoring the database schema from xml files? -- --- Viktor

Re: [SQL] large xml database

2010-10-31 Thread Viktor Bojović
On Sun, Oct 31, 2010 at 10:26 PM, Rob Sargent wrote: > Viktor Bojović wrote: > >> >> >> On Sun, Oct 31, 2010 at 9:42 PM, Rob Sargent > robjsarg...@gmail.com>> wrote: >> >> >> >> >>Viktor Bojovic' wrote: >> >> >> >>On Sun, Oct 31, 2010 at 2:26 AM, James Cloos >>mailto:cl...@j

Re: [SQL] large xml database

2010-10-31 Thread Rob Sargent
Viktor Bojović wrote: On Sun, Oct 31, 2010 at 9:42 PM, Rob Sargent > wrote: Viktor Bojovic' wrote: On Sun, Oct 31, 2010 at 2:26 AM, James Cloos mailto:cl...@jhcloos.com> >> wr

Re: [SQL] large xml database

2010-10-31 Thread Viktor Bojović
On Sun, Oct 31, 2010 at 9:42 PM, Rob Sargent wrote: > > > > Viktor Bojovic' wrote: > >> >> >> On Sun, Oct 31, 2010 at 2:26 AM, James Cloos > cl...@jhcloos.com>> wrote: >> >>> "VB" == Viktor Bojovic' > >>> writes: >> >>VB> i have very big XML documm

Re: [SQL] large xml database

2010-10-31 Thread Rob Sargent
Viktor Bojovic' wrote: On Sun, Oct 31, 2010 at 2:26 AM, James Cloos > wrote: > "VB" == Viktor Bojovic' mailto:viktor.bojo...@gmail.com>> writes: VB> i have very big XML documment which is larger than 50GB and want to VB> import it into databse

Re: [SQL] large xml database

2010-10-31 Thread Viktor Bojović
On Sun, Oct 31, 2010 at 2:26 AM, James Cloos wrote: > > "VB" == Viktor Bojović writes: > > VB> i have very big XML documment which is larger than 50GB and want to > VB> import it into databse, and transform it to relational schema. > > Were I doing such a conversion, I'd use perl to convert

Re: [SQL] large xml database

2010-10-31 Thread Viktor Bojović
On Sun, Oct 31, 2010 at 7:08 AM, Lutz Steinborn wrote: > On Sat, 30 Oct 2010 23:49:29 +0200 > Viktor Bojović wrote: > > > > > many tries have failed because 8GB of ram and 10gb of swap were not > enough. > > also sometimes i get that more than 2^32 operations were performed, and > > functions sto

Re: [SQL] large xml database

2010-10-30 Thread Lutz Steinborn
On Sat, 30 Oct 2010 23:49:29 +0200 Viktor Bojović wrote: > > many tries have failed because 8GB of ram and 10gb of swap were not enough. > also sometimes i get that more than 2^32 operations were performed, and > functions stopped to work. > we have a similar problem and we use the Amara xml To

Re: [SQL] large xml database

2010-10-30 Thread James Cloos
> "VB" == Viktor Bojović writes: VB> i have very big XML documment which is larger than 50GB and want to VB> import it into databse, and transform it to relational schema. Were I doing such a conversion, I'd use perl to convert the xml into something which COPY can grok. Any other language,

Re: [SQL] large xml database

2010-10-30 Thread Rob Sargent
Andreas Joseph Krogh wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 10/30/2010 11:49 PM, Viktor Bojović wrote: Hi, i have very big XML documment which is larger than 50GB and want to import it into databse, and transform it to relational schema. When splitting this documment to

Re: [SQL] large xml database

2010-10-30 Thread Andreas Joseph Krogh
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 10/30/2010 11:49 PM, Viktor Bojović wrote: > Hi, > i have very big XML documment which is larger than 50GB and want to import > it into databse, and transform it to relational schema. > When splitting this documment to smaller independent xml docu

[SQL] large xml database

2010-10-30 Thread Viktor Bojović
Hi, i have very big XML documment which is larger than 50GB and want to import it into databse, and transform it to relational schema. When splitting this documment to smaller independent xml documments i get ~11.1mil XML documents. I have spent lots of time trying to get fastest way to transform a