On 2011-09-08 21:54, Steven Schveighoffer wrote:
On Thu, 08 Sep 2011 15:38:43 -0400, Jacob Carlborg <d...@me.com> wrote:
On 2011-09-08 15:22, Steven Schveighoffer wrote:
On Thu, 08 Sep 2011 09:16:40 -0400, Jacob Carlborg <d...@me.com> wrote:
The Tango XML parser doesn't read from a file, it takes the input as a
string. The parser isn't affected by I/O at all.
So you have to read the entire file before sending it to the parser?
Isn't that a bit limited? What if I have a 50MB file, I have to read it
into a continuous memory block first?
-Steve
I'm just telling how Tango currently works, not how the XML module in
Phobos should work. But I guess it might be somewhat limited. 50MB
isn't that big to read into memory?
Um... yeah, it is :) I have 1 GB of memory, my system starts thrashing
with an app that consumes 750MB. So that's like 13 xml files read?
Especially if I want to use DOM, I have to keep them around...
50MB is far from 750MB :), but I see your point.
Not to mention that the GC has to allocate a contiguous space for it. So
even if I have 100MB of garbage space, maybe none of it is usable, I
still have to allocate a new block. I'm just surprised there isn't at
least an option for a stream-based xml parser in Tango.
One thing this does though, I always assumed it was Tango's I/O that
accounts for its xml superiority. I wonder, does anyone count reading
the file in any of the benchmarks?
As far as I know it's because of two reasons: it doesn't allocate any
memory (uses slices) and all methods are final.
I have no idea about the benchmarks.
I still think we can come close without having to pre-read an entire file.
I hope so as well.
I think it would be nice to be able to do both. If you read the whole
file before sending it to the parser you would know it doesn't perform
any I/O operations.
I totally agree. I think there's ways to abstract the functionality for
both memory-based and device-based i/o into one interface (part of the
reason for the revamp).
-Steve
A ranged based API as Jonathan and others have said.
--
/Jacob Carlborg
