[EMAIL PROTECTED] wrote:
> Paul,
>
> This is interesting. Unfortunately, I have no control over the XML
> output. The file is from Goldmine. However, you have given me an
> idea...
>
> Is it possible to read an XML document in compressed format?
sure. you can e.g. use gzip.open to create a file object that
decompresses on the way in.
file = gzip.open("data.xml.gz")
for event, elem in ET.iterparse(file):
if elem.tag == "item":
elem.clear()
I tried compressing my 1 GB example, but all 1000-byte records in that
file are identical, so I got a 500x compression, which is a bit higher
than you can reasonably expect ;-) however, with that example, I get a
stable parsing time of 26 seconds, so it looks as if gzip can produce
data about as fast as a preloaded disk cache...
</F>
--
http://mail.python.org/mailman/listinfo/python-list
