I suppose sequential reads is one approach I may need to consider, especially if I develop this as a tool for our end-users (doing the XML import).
I was able to process the large file - It took some jumping through hoops, but I got the file imported. 20,669 records. I set ulimit -d to "unlimited" , but in order to do so, I had to login as root, set ulimit, then go into UV. Then the program completed. This is on a back up UV server with no other users. - Bob Worley Coburn Supply Co -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Joshua Gallant Sent: Thursday, January 12, 2012 10:30 AM To: U2 Users List Subject: Re: [U2] Extracting XML attributes When working with large XML files in the past I've always run into issues like this but I use a combination of xmapopen, xmapreadnext, and xmapclose. I couldn't ever come up with a great way to use the UV tools for parsing a large file at once so I open as a sequential file and parse the file until I have a full record. I then process that one record with the xml functions. There might be 15 other ways but this has always served its purpose. - Josh On 1/12/12 11:10 AM, "Bobby Worley" <bwor...@coburns.com> wrote: Does anybody have an experience reading in very large XML files into Universe on AIX using PrepareXML() function? We received a 176Mb XML file, and I'm running out of memory. I've maxed out ulimit -d and it still blows up with this error: ERROR MESSAGE A DOM error occured during parsing. -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bobby Worley Sent: Wednesday, November 16, 2011 7:34 AM To: U2 Users List Subject: Re: [U2] Extracting XML attributes Turns out it is a memory issue. My XML file is 16Mb. Setting ulimit -d 750000 resolved the issue. -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Bobby Worley Sent: Friday, November 11, 2011 10:42 AM To: U2 Users List Subject: Re: [U2] Extracting XML attributes My latest challenge: >PREPARE.XML FEED.XML MYXML Prepare the XMLDOM failed. XMLParser error message: A DOM error occured during parsing. UNIVERSE RELLEVEL 001 X 002 11.1.0 003 PICK 004 PICK.FORMAT 005 11.1.0 Aix Version 5.3.0.0 FEED.XML is 16mb. It prepares just fine on UV 10.1.17. Unfortunately it wont list on UV 10.1 because UV 10.1 is not aware of namespaces. This is making my pull my hair out... I don't need this on a Friday. Bob Worley Coburn Supply _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
