On Friday, November 19, 2010 15:17:35 Bruno Medeiros wrote: > On 19/11/2010 22:02, Jonathan M Davis wrote: > > On Friday, November 19, 2010 13:53:12 Bruno Medeiros wrote: > >> On 19/11/2010 21:27, Jonathan M Davis wrote: > >> > >> And by providing a lexer and a parser outside the standard library, > >> wouldn't it make it just as easy for those tools to be written? What's > >> the advantage of being in the standard library? I see only > >> disadvantages: to begin with it potentially increases the time that > >> Walter or other Phobos contributors may have to spend on it, even if > >> it's just reviewing patches or making sure the code works. > > > > If nothing, else, it makes it easier to keep in line with dmd itself. > > Since the dmd front end is LGPL, it's not possible to have a Boost port > > of it (like the Phobos version will be) without Walter's consent. And > > I'd be surprised if he did that for a third party library (though he > > seems to be pretty open on a lot of that kind of stuff). Not to mention, > > Walter and the core developers are _exactly_ the kind of people that you > > want working on a lexer or parser of the language itself, because > > they're the ones who work on it. > > > > - Jonathan M Davis > > Eh? That license argument doesn't make sense: if the lexer and parser > were to be based on DMD itself, then putting it in the standard library > is equivalent (in licensing terms) to licensing the lexer and parser > parts of DMD in Boost. More correctly, what I mean by equivalent, is > that there no reason why Walter would allow one thing and not the > other... (because on both cases he would have to issue that license)
It's very different to have D implementation of something - which is based on a C++ version but definitely different in some respects - be under Boost and generally available, and having the C++ implementation be under Boost - particularly when the C++ version covers far more than just a lexer and parser. Someone _could_ port the D code back to C++ and have that portion useable under Boost, but that's a lot more work than just taking the C++ code and using it, and it's only the portions of the compiler which were ported to D to which could be re-used that way. And since the Boost code could be used in a commercial product while the LGPL is more restricted, it could make a definite difference. I'm not a licensing expert, and I'm not an expert on what Walter does and doesn't want done with his code, but he put the compiler front end under the LGPL, not Boost, and he's given his permission to have the lexer alone ported to D and put under the Boost license in the standard library, which is very different from putting the entire front end under Boost. I expect that the parser will follow eventually, but even if it does, that's still not the entire front end. So, there is a difference in licenses does have a real impact. And no one can take the LGPL C++ code and port it to D - for the standard library or otherwise - without Walter's permission, because its his copyright on the code. As for the usefulness of a D lexer and parser, I've already had several programs or functions which I've wanted to write which would require it, and the lack has made them infeasible. For instance, I was considering posting a version of my datetime code without the unit tests in it, so that it would be easier to read the actual code (given the large number of unit tests), but I found that to accurately do that, you need a lexer for D, so I gave up on it for the time being. Having a function which stripped out unnecessary whitespace (and especially newlines) for string mixins would be great (particularly since line numbers get messed up with multi-line string mixins), but that would require CTFE-able D lexer to work correctly (though you might be able to hack together something which would mostly work), which we don't have. The D lexer won't be CTFE-able initially (though hopefully it will be once the CTFE capabilites of dmd improve), so you still won't be able to do that once the lexer is done, but it is a case where a lexer would be useful. Ideally, Phobos would be huge in a manner similar to how C# or Java's libraries are huge. It will take time to get there, and we'll need more developers, but I don't think that it really makes sense to not put things in the standard library because it might take more dev time - particularly when a D lexer is the sort of thing that likely won't need much changing once it's done, since it would only need to be changed when the language changed or when a bug with it was found (which would likely equate to a bug in the compiler anyway), so ultimately, the developer cost is likely fairly low. Additionally, Walter thinks that the development costs will be lower to have it in the standard library with an implementation similar to dmd's rather than having it separate. And it's his call. So, it's going to get done. There are several people around here who lament the lack of D parser in Phobos at least periodically, and I think that it will be good to have an appropriate lexer and parser for D in Phobos. Having other 3rd party stuff - like antlr - is great too, but that's no reason not to put it in the standard library. I think that we're just going to have to agree to disagree on this one. - Jonathan M Davis
