Re: High performance XML parser

2011-02-11 Thread Michael Rynn
On Mon, 07 Feb 2011 10:37:46 -0500, Robert Jacques wrote: On Mon, 07 Feb 2011 07:40:30 -0500, Steven Schveighoffer schvei...@yahoo.com wrote: On Fri, 04 Feb 2011 17:36:50 -0500, Tomek Sowiński j...@ask.me wrote: Steven Schveighoffer napisał: Here is how I would approach it (without doing

Re: High performance XML parser

2011-02-09 Thread Steven Schveighoffer
On Tue, 08 Feb 2011 19:16:37 -0500, Tomek Sowiński j...@ask.me wrote: Steven Schveighoffer napisał: The design I'm thinking is that the node iterator will own a buffer. One consequence is that the fields of the current node will point to the buffer akin to foreach(line; File.byLine), so

Re: High performance XML parser

2011-02-09 Thread Tomek Sowiński
Steven Schveighoffer napisał: OK, so you mean a buffer other than the I/O buffer. This means double buffering data. I was thinking of a solution that allows simply using the I/O buffer for parsing. I think this is one of the keys to Tango's xml performance. I'd be glad to hear

Re: High performance XML parser

2011-02-08 Thread Piotr Szturmaj
Tomek Sowiński wrote: I am now intensely accumulating information on how to go about creating a high-performance parser as it quickly became clear that my old one won't deliver. And if anything is clear is that memory is the key. One way is the slicing approach mentioned on this NG, notably

Re: High performance XML parser

2011-02-08 Thread Tomek Sowiński
Steven Schveighoffer napisał: The design I'm thinking is that the node iterator will own a buffer. One consequence is that the fields of the current node will point to the buffer akin to foreach(line; File.byLine), so in order to lift the input the user will have to dup (or process

Re: High performance XML parser

2011-02-08 Thread spir
On 02/09/2011 01:16 AM, Tomek Sowiński wrote: Steven Schveighoffer napisał: The design I'm thinking is that the node iterator will own a buffer. One consequence is that the fields of the current node will point to the buffer akin to foreach(line; File.byLine), so in order to lift the input the

Re: High performance XML parser

2011-02-07 Thread Steven Schveighoffer
On Fri, 04 Feb 2011 17:03:08 -0500, Simen kjaeraas simen.kja...@gmail.com wrote: Steven Schveighoffer schvei...@yahoo.com wrote: Here is how I would approach it (without doing any research). First, we need a buffered I/O system where you can easily access and manipulate the buffer. I

Re: High performance XML parser

2011-02-07 Thread Steven Schveighoffer
On Fri, 04 Feb 2011 17:36:50 -0500, Tomek Sowiński j...@ask.me wrote: Steven Schveighoffer napisał: Here is how I would approach it (without doing any research). First, we need a buffered I/O system where you can easily access and manipulate the buffer. I have proposed one a few months ago

Re: High performance XML parser

2011-02-07 Thread Robert Jacques
On Mon, 07 Feb 2011 07:40:30 -0500, Steven Schveighoffer schvei...@yahoo.com wrote: On Fri, 04 Feb 2011 17:36:50 -0500, Tomek Sowiński j...@ask.me wrote: Steven Schveighoffer napisał: Here is how I would approach it (without doing any research). First, we need a buffered I/O system where

Re: High performance XML parser

2011-02-07 Thread Kagamin
Tomek Sowiński Wrote: One way is the slicing approach mentioned on this NG, notably used by RapidXML. I already contacted Marcin (the author) to ensure that using solutions inspired by his lib is OK with him; it is. But I don't think I'll go this way. One reason is, surprisingly,

Re: High performance XML parser

2011-02-07 Thread Roman Ivanov
On 2/4/2011 4:47 PM, Tomek Sowiński wrote: Michel Fortin napisał: I agree it's important, especially when receiving XML over the network, but I also think it's important to be able to be able to support slicing. Imagine all the memory you could save by just making slices of a

Re: High performance XML parser

2011-02-06 Thread Jacob Carlborg
On 2011-02-04 22:02, Tomek Sowiński wrote: I am now intensely accumulating information on how to go about creating a high-performance parser as it quickly became clear that my old one won't deliver. And if anything is clear is that memory is the key. One way is the slicing approach mentioned

High performance XML parser

2011-02-04 Thread Tomek Sowiński
I am now intensely accumulating information on how to go about creating a high-performance parser as it quickly became clear that my old one won't deliver. And if anything is clear is that memory is the key. One way is the slicing approach mentioned on this NG, notably used by RapidXML. I

Re: High performance XML parser

2011-02-04 Thread Steven Schveighoffer
On Fri, 04 Feb 2011 16:02:39 -0500, Tomek Sowiński j...@ask.me wrote: I am now intensely accumulating information on how to go about creating a high-performance parser as it quickly became clear that my old one won't deliver. And if anything is clear is that memory is the key. One way is

Re: High performance XML parser

2011-02-04 Thread Michel Fortin
On 2011-02-04 16:02:39 -0500, Tomek Sowiński j...@ask.me said: I am now intensely accumulating information on how to go about creating a high-performance parser as it quickly became clear that my old one won't deliver. And if anything is clear is that memory is the key. One way is the

Re: High performance XML parser

2011-02-04 Thread Tomek Sowiński
Michel Fortin napisał: I agree it's important, especially when receiving XML over the network, but I also think it's important to be able to be able to support slicing. Imagine all the memory you could save by just making slices of a memory-mapped file. The difficulty is to support both

Re: High performance XML parser

2011-02-04 Thread Simen kjaeraas
Steven Schveighoffer schvei...@yahoo.com wrote: Here is how I would approach it (without doing any research). First, we need a buffered I/O system where you can easily access and manipulate the buffer. I have proposed one a few months ago in this NG. Second, I'd implement the XML lib as a

Re: High performance XML parser

2011-02-04 Thread Tomek Sowiński
Steven Schveighoffer napisał: Here is how I would approach it (without doing any research). First, we need a buffered I/O system where you can easily access and manipulate the buffer. I have proposed one a few months ago in this NG. Second, I'd implement the XML lib as a range where

Re: High performance XML parser

2011-02-04 Thread Denis Koroskin
On Sat, 05 Feb 2011 00:02:39 +0300, Tomek Sowiński j...@ask.me wrote: I am now intensely accumulating information on how to go about creating a high-performance parser as it quickly became clear that my old one won't deliver. And if anything is clear is that memory is the key. One way is

Re: High performance XML parser

2011-02-04 Thread Tomek Sowiński
Steven Schveighoffer napisał: Here is how I would approach it (without doing any research). First, we need a buffered I/O system where you can easily access and manipulate the buffer. I have proposed one a few months ago in this NG. Second, I'd implement the XML lib as a range

Re: High performance XML parser

2011-02-04 Thread Michel Fortin
On 2011-02-04 16:47:26 -0500, Tomek Sowiński j...@ask.me said: Michel Fortin napisał: I agree it's important, especially when receiving XML over the network, but I also think it's important to be able to be able to support slicing. Imagine all the memory you could save by just making slices