I'm involved with a project that is trying to parse xml files of size 20
Megs or greater. We are using xerces-c 1.5.0 xml parser on Windows 2000.
Validation is turned on. We are consistently running out of memory (Windows
2000 reports that we are running out of virtual memory and then programs
promptly start to crash as they can't allocate more) around the 3rd or 4th
file. As a test, I decided to run the parser repeatedly with one 3kb file
instead. At first, the parser went very quickly, parsing the same file
approximately 1000 times in 1 minute. After letting it run for several
hours (10+), it now takes the parser over 3 minutes to parse the file.
Also, using the task manager, I noticed that memory usage peaks at 190,000 K
while parsing the file over the 3 minute period.
My assumptions:
When parsing over a 3k file, memory should increase to some small finite
level and then go back down when parsing is finished (but not to previous
level due to overhead). This should happen repeatedly. I didn't expect to
see 190,000K and the slow down.
XML Description:
Very simplistic. Root node contains multiple nodes that are PCData. It
also contains a few nodes that have children. The children are PCData as
well.
This leads me to the following questions.
1) How large of a file can xerces handle?
2) Is this the kind of performance I should expect after continuous
parsing?
3) Is there anything that I can do differently? Is there anything that I
should be calling that I'm not.
Here is the code that I'm using.
void main(int argc, char* argv[])
{
XMLPlatformUtils::Initialize();
for(;;)
{
DOMParser parser;
parser.setDoValidation(true);
parser.setIncludeIgnorableWhitespace(false);
parser.setErrorHandler(new XmlErrorReporter());
parser.parse("test.xml");
DOM_Document doc = parser.getDocument();
DOM_Element root = doc.getDocumentElement();
}
XMLPlatformUtils::Terminate();
return 0;
}
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
