Re: Replacing std.xml

2013-09-03 Thread Andrei Alexandrescu
On 9/2/13 7:40 PM, Lionello Lunesu wrote: Having been the lead programmer on the Microsoft XML team for three years, I can easily say that the most popular XML API [on MS stack] is the XmlReader and XLinq in .NET. (This has nothing to do with LINQ, by the way.) I'd be willing to help make D

Re: Replacing std.xml

2013-09-03 Thread ilya-stromberg
On Thursday, 29 August 2013 at 16:14:28 UTC, Michel Fortin wrote: I wrote something like that a while ago. It only accepted arrays as input because of the lack of a buffered range concept that'd allow lookahead and efficient slicing from any kind of range, but that could be retrofitted in.

Re: Replacing std.xml

2013-09-03 Thread Michel Fortin
On 2013-09-03 16:11:37 +, ilya-stromberg ilya-stromberg-2...@yandex.ru said: On Thursday, 29 August 2013 at 16:14:28 UTC, Michel Fortin wrote: I wrote something like that a while ago. It only accepted arrays as input because of the lack of a buffered range concept that'd allow

Re: Replacing std.xml

2013-09-02 Thread Richard Webb
On 31/08/2013 16:43, ilya-stromberg wrote: It's the fastest Xml parser in the world, so may be you can find it useful: dotnot.org/blog/archives/2008/03/10/xml-benchmarks-parsequerymutateserialize/ dotnot.org/blog/archives/2008/03/12/why-is-dtango-so-fast-at-parsing-xml/ Has anyone done any

Re: Replacing std.xml

2013-09-02 Thread qznc
On Saturday, 31 August 2013 at 18:53:42 UTC, Michel Fortin wrote: On 2013-08-31 15:43:00 +, ilya-stromberg ilya-stromberg-2...@yandex.ru said: On Thursday, 29 August 2013 at 07:53:46 UTC, Tobias Pankrath wrote: There is http://dsource.org/projects/xmlp, which at some point has been

Re: Replacing std.xml

2013-09-02 Thread Michel Fortin
On 2013-09-02 13:34:18 +, qznc q...@web.de said: On Saturday, 31 August 2013 at 18:53:42 UTC, Michel Fortin wrote: For instance, Tango's SaxParser is based on its PullParser. This design requires the use a dynamic array to maintain a stack of opened elements. While not a huge performance

Re: Replacing std.xml

2013-09-02 Thread Lionello Lunesu
On 8/29/13 15:25, w0rp wrote: Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. First, and most importantly, what do we except out of a D XML library? I'd really

Re: Replacing std.xml

2013-09-02 Thread Peter Williams
On 03/09/13 12:40, Lionello Lunesu wrote: On 8/29/13 15:25, w0rp wrote: Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. First, and most importantly, what do

Re: Replacing std.xml

2013-09-01 Thread ilya-stromberg
On Saturday, 31 August 2013 at 18:03:10 UTC, Jacob Carlborg wrote: Unfortunately the Tango XML package will never end up in Phobos due to licensing issues. Yes, but we can always learn source code and put attention to the design solutions.

Re: Replacing std.xml

2013-09-01 Thread Jonathan M Davis
On Sunday, September 01, 2013 10:02:50 ilya-stromberg wrote: On Saturday, 31 August 2013 at 18:03:10 UTC, Jacob Carlborg wrote: Unfortunately the Tango XML package will never end up in Phobos due to licensing issues. Yes, but we can always learn source code and put attention to the design

Re: Replacing std.xml

2013-08-31 Thread ilya-stromberg
On Thursday, 29 August 2013 at 07:53:46 UTC, Tobias Pankrath wrote: There is http://dsource.org/projects/xmlp, which at some point has been proposed for std.xml2. But that stalled for some time now. Also, we have Tango Xml: https://github.com/SiegeLord/Tango-D2/tree/d2port/tango/text/xml

Re: Replacing std.xml

2013-08-31 Thread Jacob Carlborg
On 2013-08-31 17:43, ilya-stromberg wrote: Also, we have Tango Xml: https://github.com/SiegeLord/Tango-D2/tree/d2port/tango/text/xml It's the fastest Xml parser in the world, so may be you can find it useful: dotnot.org/blog/archives/2008/03/10/xml-benchmarks-parsequerymutateserialize/

Re: Replacing std.xml

2013-08-31 Thread Michel Fortin
On 2013-08-31 15:43:00 +, ilya-stromberg ilya-stromberg-2...@yandex.ru said: On Thursday, 29 August 2013 at 07:53:46 UTC, Tobias Pankrath wrote: There is http://dsource.org/projects/xmlp, which at some point has been proposed for std.xml2. But that stalled for some time now. Also, we

Re: Replacing std.xml

2013-08-31 Thread Jacob Carlborg
On 2013-08-31 20:53, Michel Fortin wrote: [^1]: IMHO, PullParser isn't a really good term for something that does not conform to the requirements of a parser in the XML spec. Tokenizer is a better term. I guess Pull is the key here. That it is the client's responsibility to fetch the next

Re: Replacing std.xml

2013-08-31 Thread Walter Bright
On 8/29/2013 12:25 AM, w0rp wrote: Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. First, and most importantly, what do we except out of a D XML library? I'd

Re: Replacing std.xml

2013-08-31 Thread deadalnix
On Thursday, 29 August 2013 at 18:58:57 UTC, H. S. Teoh wrote: On Thu, Aug 29, 2013 at 01:38:23PM -0400, Jonathan M Davis wrote: [...] Well, as I said, I couldn't remember exactly what the XML standard said about encodings, but if it can contain non-ASCII characters, then my first inclination

Replacing std.xml

2013-08-29 Thread w0rp
Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. First, and most importantly, what do we except out of a D XML library? I'd really like to have a discussion

Re: Replacing std.xml

2013-08-29 Thread Jonathan M Davis
On Thursday, August 29, 2013 09:25:35 w0rp wrote: Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. Someone needs to step forward, write it, and get it through

Re: Replacing std.xml

2013-08-29 Thread Johannes Pfau
Am Thu, 29 Aug 2013 09:25:35 +0200 schrieb w0rp devw...@gmail.com: Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. First, and most importantly, what do

Re: Replacing std.xml

2013-08-29 Thread Tobias Pankrath
On Thursday, 29 August 2013 at 07:25:36 UTC, w0rp wrote: Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. First, and most importantly, what do we except out

Re: Replacing std.xml

2013-08-29 Thread Robert Schadek
On 08/29/2013 09:51 AM, Johannes Pfau wrote: I most points here also apply to std.xml: http://wiki.dlang.org/Wish_list/std.json Those are not strict requirements though, I just summarized what I remembered from old discussions. I think, this even extends to access to all semi- and

Re: Replacing std.xml

2013-08-29 Thread Jacob Carlborg
On 2013-08-29 09:47, Jonathan M Davis wrote: Personally, I would have just said use ranges of dchar and be done with it without worrying about character encodings at all, but I don't remember what all the XML standard does with encodings. Won't that have the same problem as we talked about in

Re: Replacing std.xml

2013-08-29 Thread Jacob Carlborg
On 2013-08-29 10:15, Robert Schadek wrote: I think, this even extends to access to all semi- and structured-data. Think csv, sql nosql, you name it. Something which deserves a name like Uniform Access. I don't want to care if data is laid out differently. I want to define my struct or class

Re: Replacing std.xml

2013-08-29 Thread Jonathan M Davis
On Thursday, August 29, 2013 11:08:18 Jacob Carlborg wrote: On 2013-08-29 09:47, Jonathan M Davis wrote: Personally, I would have just said use ranges of dchar and be done with it without worrying about character encodings at all, but I don't remember what all the XML standard does with

Re: Replacing std.xml

2013-08-29 Thread Joakim
On Thursday, 29 August 2013 at 07:47:35 UTC, Jonathan M Davis wrote: There are several D XML libraries floating around, but no one has taken the time to get any of the prepared for the Phobos review queue, and I suspect that very few of them are range-based like the Phobos XML solution needs

Re: Replacing std.xml

2013-08-29 Thread Robert Schadek
On 08/29/2013 11:09 AM, Jacob Carlborg wrote: So you want serialization :). Which we currently are reviewing. Unfortunately there might be too many changes needed to get it in Phobos this time. well, sort of, but also with partial serialization (think sql update), more transparent interface

Re: Replacing std.xml

2013-08-29 Thread maarten van damme
and imagine someone forced to use xml who reads this answer from the community :p std.xml is a must, no doubt. 2013/8/29 Joakim joa...@airpost.net On Thursday, 29 August 2013 at 07:47:35 UTC, Jonathan M Davis wrote: There are several D XML libraries floating around, but no one has taken

Re: Replacing std.xml

2013-08-29 Thread Chris
On Thursday, 29 August 2013 at 09:24:31 UTC, Joakim wrote: I think it's great that there's no std.xml, as it implies that nobody using D would use a dumb tech like XML. Let's keep it that way. :) No way around XML. A must have, as has been said in this thread. But what would you suggest as

Re: Replacing std.xml

2013-08-29 Thread Jacob Carlborg
On 2013-08-29 11:23, Jonathan M Davis wrote: IIRC, everything in XML is ASCII anyway, with stuff like HTML codes to indicate Unicode characters. And if that's the case, avoiding unnecessary decoding is trivial when operating on strings. What! I hardly believe that. That might be the case for

Re: Replacing std.xml

2013-08-29 Thread Chris
On Thursday, 29 August 2013 at 13:20:40 UTC, Jacob Carlborg wrote: On 2013-08-29 11:23, Jonathan M Davis wrote: IIRC, everything in XML is ASCII anyway, with stuff like HTML codes to indicate Unicode characters. And if that's the case, avoiding unnecessary decoding is trivial when operating

Re: Replacing std.xml

2013-08-29 Thread H. S. Teoh
On Thu, Aug 29, 2013 at 01:14:19PM +0200, Chris wrote: On Thursday, 29 August 2013 at 09:24:31 UTC, Joakim wrote: I think it's great that there's no std.xml, as it implies that nobody using D would use a dumb tech like XML. Let's keep it that way. :) No way around XML. A must have, as has

Re: Replacing std.xml

2013-08-29 Thread Andrei Alexandrescu
On 8/29/13 12:25 AM, w0rp wrote: Hello everybody. I've been wondering, what are the current plans to replace std.xml? I'd like to help with the effort to get a final XML library in phobos. So, I have a few questions. First, and most importantly, what do we except out of a D XML library? I'd

Re: Replacing std.xml

2013-08-29 Thread Joakim
On Thursday, 29 August 2013 at 11:14:21 UTC, Chris wrote: On Thursday, 29 August 2013 at 09:24:31 UTC, Joakim wrote: I think it's great that there's no std.xml, as it implies that nobody using D would use a dumb tech like XML. Let's keep it that way. :) No way around XML. A must have, as

Re: Replacing std.xml

2013-08-29 Thread Chris
On Thursday, 29 August 2013 at 15:43:36 UTC, H. S. Teoh wrote: While I do agree that in the current state of affairs, XML support is a must, I also think that XML is just way overengineered, IMNSHO. It has adds too much overhead and therefore requires compression to be efficient, and it is

Re: Replacing std.xml

2013-08-29 Thread Michel Fortin
On 2013-08-29 07:47:17 +, Jonathan M Davis jmdavisp...@gmx.com said: On Thursday, August 29, 2013 09:25:35 w0rp wrote: The general idea in my mind is something SAX-like, with something a little DOM-like. What I personally think would be best is to have multiple parsers. First you have

Re: Replacing std.xml

2013-08-29 Thread Jason den Dulk
On Thursday, 29 August 2013 at 15:43:36 UTC, H. S. Teoh wrote: JSON is a nicer, simpler alternative, though there may be limitations with it that I don't know about. The main disavantage of JSON vs XML is lack of validation. Whenever I write code that works with JSON (or any data format), I

Re: Replacing std.xml

2013-08-29 Thread Jonathan M Davis
On Thursday, August 29, 2013 15:20:39 Jacob Carlborg wrote: On 2013-08-29 11:23, Jonathan M Davis wrote: IIRC, everything in XML is ASCII anyway, with stuff like HTML codes to indicate Unicode characters. And if that's the case, avoiding unnecessary decoding is trivial when operating on

Re: Replacing std.xml

2013-08-29 Thread Jonathan M Davis
On Thursday, August 29, 2013 12:14:28 Michel Fortin wrote: On 2013-08-29 07:47:17 +, Jonathan M Davis jmdavisp...@gmx.com said: On Thursday, August 29, 2013 09:25:35 w0rp wrote: The general idea in my mind is something SAX-like, with something a little DOM-like. What I personally

Re: Replacing std.xml

2013-08-29 Thread Brad Anderson
On Thursday, 29 August 2013 at 17:38:43 UTC, Jonathan M Davis wrote: Well, as I said, I couldn't remember exactly what the XML standard said about encodings, but if it can contain non-ASCII characters, then my first inclination is to say that it has to be UTF-8, UTF-16, or UTF-32 based on

Re: Replacing std.xml

2013-08-29 Thread w0rp
On Thursday, 29 August 2013 at 09:24:31 UTC, Joakim wrote: I think it's great that there's no std.xml, as it implies that nobody using D would use a dumb tech like XML. Let's keep it that way. :) JSON is better than XML in every way I can think of. Easier to map to data structures in

Re: Replacing std.xml

2013-08-29 Thread H. S. Teoh
On Thu, Aug 29, 2013 at 01:38:23PM -0400, Jonathan M Davis wrote: [...] Well, as I said, I couldn't remember exactly what the XML standard said about encodings, but if it can contain non-ASCII characters, then my first inclination is to say that it has to be UTF-8, UTF-16, or UTF-32 based on

Re: Replacing std.xml

2013-08-29 Thread Jacob Carlborg
On 2013-08-29 16:07, Chris wrote: And while we're at it, what about YAML? It's a subset of JSON which means the new json.d module will handle it, I suppose. YAML is a super set of JSON, not the other way around. But yes, I would like to have YAML support as well. -- /Jacob Carlborg

Re: Replacing std.xml

2013-08-29 Thread Jacob Carlborg
On 2013-08-29 19:38, Jonathan M Davis wrote: However, because all of the XML special symbols should be ASCII, you should still be able to avoid decoding characters for the most part. It's only when you have to actually look at the content that Unicode would potentially matter. So, the

Re: Replacing std.xml

2013-08-29 Thread Jacob Carlborg
On 2013-08-29 20:57, H. S. Teoh wrote: XML files can have *any* valid encoding, including nastiness like windows-1252 and relics like iso-8859-1. Actually, does the encoding really matters (as long as it's compatible with ASCII). Just use a range of ubytes, the parser will only be looking

Re: Replacing std.xml

2013-08-29 Thread Brad Anderson
On Thursday, 29 August 2013 at 18:58:57 UTC, H. S. Teoh wrote: No kidding! I was trying to write a program that navigates a website automatically using std.net.curl, and I'm running into all sorts of silly roadblocks, including std.encoding not supporting iso-8859-* encodings. It doesn't

Re: Replacing std.xml

2013-08-29 Thread Jonathan M Davis
On Thursday, August 29, 2013 21:28:09 Jacob Carlborg wrote: On 2013-08-29 19:38, Jonathan M Davis wrote: However, because all of the XML special symbols should be ASCII, you should still be able to avoid decoding characters for the most part. It's only when you have to actually look at

Re: Replacing std.xml

2013-08-29 Thread Brad Anderson
On Thursday, 29 August 2013 at 19:40:08 UTC, Brad Anderson wrote: That's a really great point. All of these modules that can't know the types and structure in advance should probably all use the same techniques for handling the situation. Perhaps a new module to unify all this stuff is in

Re: Replacing std.xml

2013-08-29 Thread Brad Anderson
On Thursday, 29 August 2013 at 08:15:39 UTC, Robert Schadek wrote: On 08/29/2013 09:51 AM, Johannes Pfau wrote: I most points here also apply to std.xml: http://wiki.dlang.org/Wish_list/std.json Those are not strict requirements though, I just summarized what I remembered from old

Re: Replacing std.xml

2013-08-29 Thread Sean Kelly
On Aug 29, 2013, at 11:57 AM, H. S. Teoh hst...@quickfur.ath.cx wrote: One way is to write the core code of std.xml in such a way that it handles all data as ubyte[] (or ushort[]/uint[] for 16-bit/32-bit encodings) so that it's encoding-independent. Then on top of this core, write some

Re: Replacing std.xml

2013-08-29 Thread Brad Anderson
On Thursday, 29 August 2013 at 20:08:10 UTC, Sean Kelly wrote: As long autoconversion is optional. When parsing XML or JSON or whatever, I generally only care about specific strings, and sometimes don't want anything decoded at all. Having decoding done automatically before the event fires

Re: Replacing std.xml

2013-08-29 Thread H. S. Teoh
On Thu, Aug 29, 2013 at 12:41:16PM -0700, Sean Kelly wrote: On Aug 29, 2013, at 11:57 AM, H. S. Teoh hst...@quickfur.ath.cx wrote: One way is to write the core code of std.xml in such a way that it handles all data as ubyte[] (or ushort[]/uint[] for 16-bit/32-bit encodings) so that it's

Re: Replacing std.xml

2013-08-29 Thread Jonathan M Davis
On Thursday, August 29, 2013 14:27:22 H. S. Teoh wrote: Right, that's why I said the core of std.xml should handle everything as bytes, only specially treating the ASCII values of , , , and other metacharacters. The tagname and tag body should just be a range over segments of the input. That

Re: Replacing std.xml

2013-08-29 Thread Chris
On Thursday, 29 August 2013 at 19:26:21 UTC, Jacob Carlborg wrote: On 2013-08-29 16:07, Chris wrote: And while we're at it, what about YAML? It's a subset of JSON which means the new json.d module will handle it, I suppose. YAML is a super set of JSON, not the other way around. But yes, I

Re: Replacing std.xml

2013-08-29 Thread Kiith-Sa
On Thursday, 29 August 2013 at 22:56:36 UTC, Chris wrote: On Thursday, 29 August 2013 at 19:26:21 UTC, Jacob Carlborg wrote: On 2013-08-29 16:07, Chris wrote: And while we're at it, what about YAML? It's a subset of JSON which means the new json.d module will handle it, I suppose. YAML is

Re: Replacing std.xml

2013-08-29 Thread Michel Fortin
On 2013-08-29 17:38:23 +, Jonathan M Davis jmdavisp...@gmx.com said: Well, as I said, I couldn't remember exactly what the XML standard said about encodings, but if it can contain non-ASCII characters, then my first inclination is to say that it has to be UTF-8, UTF-16, or UTF-32 based on