"Nan Jiang" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > >From: Peter Rabbitson <[EMAIL PROTECTED]> > >To: beginners@perl.org > >Subject: Re: Errors on processing 2GB XML file by using XML:Simple > >Date: Fri, 13 May 2005 09:46:09 -0500 > > > > > I keep receiving virtual memory error and program runtime error while > >using > > > XML:simple to process a 2GB XML file on Windows 2000. I have changed > > > virtual memory to 4 GB but still no use, so I wonder if anyone could > >offer > > > some help? Or is there any better module to process a 2GB XML file? > > > > > > >XML::Twig is your answer. Very clear documentation and examples. Enjoy > > > >-- > Peter, > > Thanks for the reply. I know my algorithm is not good (I have about 3 - 4 > foreach in loops) but I have to do it in that way as the XML file is > somewhat a more plain stucture rather than a tree-like structure. I'll try > XML:Twig anyway. >
Then use SAX. XML::SAX::Machines. You pick a driver and a writer from CPAN, and in between the two in the pipleine you refer to the package name of your filter or handler, which usually should be a subclass of XML::SAX::Base. You have to do your own state maintanance in the callbacks, but it is a very lightweight solution. Ive handled 10's of gigabytes of XML with this method; splitting files, archiving files, sticking records in a RDBMS, etc. Works great, and most importantly, fast. Todd W. -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] <http://learn.perl.org/> <http://learn.perl.org/first-response>
