DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
Major changes since 0.9.16: - The registry is now officially located at http://code.dlang.org See also the current discussion thread in the D forum: http://forum.dlang.org/thread/l0o0jq$gvp$1...@digitalmars.com - Build types (debug, release, unittest etc.) can now be customized - Added support for multiple package repositories - Build flags are now handled in an abstract way to facilitate better compatibility of packages with different compilers (using a buildOptions field instead of dflags in most cases) - Configuration resolution on complex dependency graphs is much more robust now (i.e. configurations and subConfigurations fields of all used packages affect the finally chosen configurations as appropriate) - Robustness improvements for the VisualD project generator (avoids redundant flags and link dependencies) There are currently two active contributors, but a lot of work has piled up, mostly in the form of enhancement requests, so any additional help is valuable. GitHub projects: https://github.com/rejectedsoftware/dub https://github.com/rejectedsoftware/dub-registry Full change log: https://github.com/rejectedsoftware/dub/blob/master/CHANGELOG.md Download: http://code.dlang.org/download
Re: DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
On Thursday, 12 September 2013 at 10:34:56 UTC, Sönke Ludwig wrote: Major changes since 0.9.16: - The registry is now officially located at http://code.dlang.org See also the current discussion thread in the D forum: http://forum.dlang.org/thread/l0o0jq$gvp$1...@digitalmars.com - Build types (debug, release, unittest etc.) can now be customized - Added support for multiple package repositories - Build flags are now handled in an abstract way to facilitate better compatibility of packages with different compilers (using a buildOptions field instead of dflags in most cases) - Configuration resolution on complex dependency graphs is much more robust now (i.e. configurations and subConfigurations fields of all used packages affect the finally chosen configurations as appropriate) - Robustness improvements for the VisualD project generator (avoids redundant flags and link dependencies) There are currently two active contributors, but a lot of work has piled up, mostly in the form of enhancement requests, so any additional help is valuable. GitHub projects: https://github.com/rejectedsoftware/dub https://github.com/rejectedsoftware/dub-registry Full change log: https://github.com/rejectedsoftware/dub/blob/master/CHANGELOG.md Download: http://code.dlang.org/download Something is wrong with dub-0.9.18-setup.exe. CRC check fails.
Re: DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
On Thursday, 12 September 2013 at 10:34:56 UTC, Sönke Ludwig wrote: Major changes since 0.9.16: - The registry is now officially located at http://code.dlang.org See also the current discussion thread in the D forum: http://forum.dlang.org/thread/l0o0jq$gvp$1...@digitalmars.com - Build types (debug, release, unittest etc.) can now be customized - Added support for multiple package repositories - Build flags are now handled in an abstract way to facilitate better compatibility of packages with different compilers (using a buildOptions field instead of dflags in most cases) - Configuration resolution on complex dependency graphs is much more robust now (i.e. configurations and subConfigurations fields of all used packages affect the finally chosen configurations as appropriate) - Robustness improvements for the VisualD project generator (avoids redundant flags and link dependencies) There are currently two active contributors, but a lot of work has piled up, mostly in the form of enhancement requests, so any additional help is valuable. GitHub projects: https://github.com/rejectedsoftware/dub https://github.com/rejectedsoftware/dub-registry Full change log: https://github.com/rejectedsoftware/dub/blob/master/CHANGELOG.md Download: http://code.dlang.org/download Currently any attempt to access the precompiled binaries, eg: http://code.dlang.org/files/dub-0.9.18-linux-x86_64.tar.gz returns 500 - Internal Server Error. Internal error information: object.Exception@../../../opt/vibe/source/vibe/inet/message.d(352): Enforcement failed ./dub-registry(pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong)+0x6b) [0x5e31ab] ./dub-registry(void vibe.inet.message.InetHeaderMap.remove(immutable(char)[])+0x182) [0x744dfa] ./dub-registry(void vibe.http.fileserver.sendFile(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse, vibe.inet.path.Path, vibe.http.fileserver.HTTPFileServerSettings)+0x47c) [0x664844] ./dub-registry(void delegate(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse) vibe.http.fileserver.serveStaticFiles(vibe.inet.path.Path, vibe.http.fileserver.HTTPFileServerSettings).void callback(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse)+0x2ce) [0x66416e] ./dub-registry(void vibe.http.router.URLRouter.handleRequest(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse)+0x179) [0x6637d1] ./dub-registry(bool vibe.http.server.handleRequest(vibe.core.stream.Stream, immutable(char)[], vibe.http.server.HTTPServerListener, ref vibe.http.server.HTTPServerSettings, ref bool)+0x1545) [0x6592a5] ./dub-registry(void vibe.http.server.handleHTTPConnection(vibe.core.net.TCPConnection, vibe.http.server.HTTPServerListener)+0x143) [0x657c9b] ./dub-registry(void vibe.http.server.listenHTTPPlain(vibe.http.server.HTTPServerSettings, void delegate(vibe.http.server.HTTPServerRequest, vibe.http.server.HTTPServerResponse)).void doListen(vibe.http.server.HTTPServerSettings, vibe.http.server.HTTPServerListener, immutable(char)[]).void __lambda45(vibe.core.net.TCPConnection)+0x2c) [0x65424c] ./dub-registry(extern (C) nothrow void vibe.core.drivers.libevent2_tcp.onConnect(int, short, void*).void ClientTask.execute()+0x2d6) [0x67119e] ./dub-registry(void vibe.core.core.CoreTask.run()+0xf2) [0x67ef82] ./dub-registry(void core.thread.Fiber.run()+0x2a) [0x788fd2] ./dub-registry(fiber_entryPoint+0x61) [0x788edd] [(nil)]
Re: DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
Am 12.09.2013 18:16, schrieb Brad Anderson: Something is wrong with dub-0.9.18-setup.exe. CRC check fails. Thanks, fixed now. There are some weird issues with GZIP compression on the server so I disabled it for now. (I need to rewrite the compression code using zlib directly anyway to avoid excessive GC allocations)
Updates to D graph library
Hello all, Today I pushed a number of major (and breaking) changes to the master repository of the D graph library. I've provided a brief summary on my blog, which also describes how to revise any programs to work with the new code: http://braingam.es/2013/09/d-graph-library-updates/ I hope that these changes don't unduly inconvenience anyone currently using the library. My own take was that it's worth it in terms of moving to a generic design and offering some new higher-performance graph data structures, which I'll be describing in more detail some time soon. Feedback on the new code is welcome. I would also like to offer thanks to all the nice people on D.learn who contributed useful ideas that helped with this update :-) Thanks best wishes, -- Joe
Re: [OT] My C++ talk at GoingNative 2013
On Tuesday, 10 September 2013 at 13:08:29 UTC, Iain Buclaw wrote: All class methods are virtual by default in D, unless declared 'final'. There was an intense discussion a while back which ended in (I think) a decision by Walter to switch to final-by-default, but there has so far been no practical follow-up.
Re: DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
On Thursday, 12 September 2013 at 18:01:43 UTC, Craig Dillabaugh wrote: Currently any attempt to access the precompiled binaries, eg: http://code.dlang.org/files/dub-0.9.18-linux-x86_64.tar.gz returns 500 - Internal Server Error. Internal error information: object.Exception@../../../opt/vibe/source/vibe/inet/message.d(352): Enforcement failed ./dub-registry(pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong)+0x6b) [0x5e31ab] ./dub-registry(void vibe.inet.message.InetHeaderMap.remove(immutable(char)[])+0x182) [0x744dfa] ./dub-registry(void vibe.http.fileserver.sendFile(vibe.http.server.HTTPServerRequest, ... Try a different browser. I had this issue when using curl / wget, but it worked with Firefox.
Re: DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
On Thursday, 12 September 2013 at 21:06:22 UTC, Kapps wrote: On Thursday, 12 September 2013 at 18:01:43 UTC, Craig Dillabaugh wrote: Currently any attempt to access the precompiled binaries, eg: http://code.dlang.org/files/dub-0.9.18-linux-x86_64.tar.gz returns 500 - Internal Server Error. Internal error information: object.Exception@../../../opt/vibe/source/vibe/inet/message.d(352): Enforcement failed ./dub-registry(pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong)+0x6b) [0x5e31ab] ./dub-registry(void vibe.inet.message.InetHeaderMap.remove(immutable(char)[])+0x182) [0x744dfa] ./dub-registry(void vibe.http.fileserver.sendFile(vibe.http.server.HTTPServerRequest, ... Try a different browser. I had this issue when using curl / wget, but it worked with Firefox. Failed with Firefox, Opera, and Chromium. Maybe it is something with my PC, will try again from another PC.
Re: [OT] My C++ talk at GoingNative 2013
On Sep 12, 2013 9:16 PM, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote: On Tuesday, 10 September 2013 at 13:08:29 UTC, Iain Buclaw wrote: All class methods are virtual by default in D, unless declared 'final'. There was an intense discussion a while back which ended in (I think) a decision by Walter to switch to final-by-default, but there has so far been no practical follow-up. Not sure how long ago that was, but dconf everyone agreed to disagree and left it at we're not changing it. Regards -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: [OT] My C++ talk at GoingNative 2013
On Sep 12, 2013, at 2:46 PM, Iain Buclaw ibuc...@ubuntu.com wrote: Not sure how long ago that was, but dconf everyone agreed to disagree and left it at we're not changing it. Here's a portion of the discussion where Walter seemed to change his mind: http://forum.dlang.org/thread/yzsqwejxqlnzryhrk...@forum.dlang.org?page=26
Re: [OT] My C++ talk at GoingNative 2013
On Sep 12, 2013, at 2:46 PM, Iain Buclaw ibuc...@ubuntu.com wrote: On Sep 12, 2013 9:16 PM, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote: On Tuesday, 10 September 2013 at 13:08:29 UTC, Iain Buclaw wrote: All class methods are virtual by default in D, unless declared 'final'. There was an intense discussion a while back which ended in (I think) a decision by Walter to switch to final-by-default, but there has so far been no practical follow-up. Not sure how long ago that was, but dconf everyone agreed to disagree and left it at we're not changing it. I believe this decision happened after dconf as the result of a rather long discussion in digitalmars.D. Someone referenced something written by a C# creator.
Re: DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
On Thursday, 12 September 2013 at 21:30:38 UTC, Craig Dillabaugh wrote: On Thursday, 12 September 2013 at 21:06:22 UTC, Kapps wrote: On Thursday, 12 September 2013 at 18:01:43 UTC, Craig Dillabaugh wrote: Currently any attempt to access the precompiled binaries, eg: http://code.dlang.org/files/dub-0.9.18-linux-x86_64.tar.gz returns 500 - Internal Server Error. Internal error information: object.Exception@../../../opt/vibe/source/vibe/inet/message.d(352): Enforcement failed ./dub-registry(pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong)+0x6b) [0x5e31ab] ./dub-registry(void vibe.inet.message.InetHeaderMap.remove(immutable(char)[])+0x182) [0x744dfa] ./dub-registry(void vibe.http.fileserver.sendFile(vibe.http.server.HTTPServerRequest, ... Try a different browser. I had this issue when using curl / wget, but it worked with Firefox. Failed with Firefox, Opera, and Chromium. Maybe it is something with my PC, will try again from another PC. Same problem with Opera/Chrome from my home PC (Windows 7). My previous attempts were from my Linux machine at the office where I tried Firefox/Opera/Chromium. I am the only one experiencing this? Craig
Re: DUB 0.9.17/0.9.18 have been released + now on code.dlang.org
On Friday, 13 September 2013 at 00:56:27 UTC, Craig Dillabaugh wrote: On Thursday, 12 September 2013 at 21:30:38 UTC, Craig Dillabaugh wrote: On Thursday, 12 September 2013 at 21:06:22 UTC, Kapps wrote: On Thursday, 12 September 2013 at 18:01:43 UTC, Craig Dillabaugh wrote: Currently any attempt to access the precompiled binaries, eg: http://code.dlang.org/files/dub-0.9.18-linux-x86_64.tar.gz returns 500 - Internal Server Error. Internal error information: object.Exception@../../../opt/vibe/source/vibe/inet/message.d(352): Enforcement failed ./dub-registry(pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong)+0x6b) [0x5e31ab] ./dub-registry(void vibe.inet.message.InetHeaderMap.remove(immutable(char)[])+0x182) [0x744dfa] ./dub-registry(void vibe.http.fileserver.sendFile(vibe.http.server.HTTPServerRequest, ... Try a different browser. I had this issue when using curl / wget, but it worked with Firefox. Failed with Firefox, Opera, and Chromium. Maybe it is something with my PC, will try again from another PC. Same problem with Opera/Chrome from my home PC (Windows 7). My previous attempts were from my Linux machine at the office where I tried Firefox/Opera/Chromium. I am the only one experiencing this? Craig I'm getting this also on a windows PC from a Chrome browser.
Re: finding errors with templates without instantiating them
On 9/11/2013 10:14 PM, deadalnix wrote: On Thursday, 12 September 2013 at 05:04:29 UTC, Walter Bright wrote: On 9/11/2013 3:15 PM, Andrei Alexandrescu wrote: I'm not against it, but I find it a very small improvement. Its user base is exactly people who want to write and deploy a template without ever testing it at all, ever. I have a hard time catering to that hypothetical crowd. I'm against it because it makes for a rather bastardized semantic pass - expensive to specify, write and debug, and as you point out of highly suspect merit. Isn't that more or less required already for IFTI ? IFTI requires that the signature be analyzed, not the body.
Re: dub: should we make it the de jure package manager for D?
On Wed, 11 Sep 2013 15:50:52 -0700 H. S. Teoh hst...@quickfur.ath.cx wrote: On Wed, Sep 11, 2013 at 05:11:38PM -0400, Nick Sabalausky wrote: Seriously, this is just like writing assembly code in 1975. Screw up a single opcode, and the computer just blindly barges onward interpreting random bytes as instructions, wreaking havoc to your entire system while you stare at the screen believing that it's still computing what you think you told it to. Who knew that JS was an underhanded way of getting people to write assembly code by hand again? :-P Exactly. I really do believe that's a very accurate way to put it. I don't like the prevailing notion that such dynamic languages have strong typing. Yes, they *technically* have strong typing, but most benefits are thrown right out the window. So *in effect* it's mostly the same as not having strong typing at all - either way you're still getting tons of seemingly random, unpredictable, barely-tracable, and *potentially dangerous* behavior any time the slightest thing goes wrong. Just because a language has addressed dangerous low-level behaviors doesn't mean it's addressed dangerous high-level behaviors. All they've really accomplished is make the same old problems and catastrophes operate on a higher level than before (but more slowly and with greater power drain). So like you said: Reliability-wise, they're just reinventing assembly. Whitespace is only sometimes significant in YAML. On the JSON constructs, it's not significant. On certain (all?) of the non-JSON YAML-specific things, then it's significant. Yikes. So whitespace is neither always significant nor always insignificant, but *sometimes* significant? Then I have to agree with Jonathan that YAML is truly evil! I don't know about evil. While I don't like indent syntax (it is one of the things I dislike about Python), YAML does at least make it *possible* to avoid both the syntax-fiesta of JSON and the anti-DRYness of XML, and YAML files *can* be constructed to be very easy to read. So I think that alone means YAML deserves at least *some* respect. However, the problem I have with YAML is that it's complex, with various difficult-to-remember syntaxes. I'd feel that I'd have to be very, very careful when writing, editing or generating YAML. And yea, the sometimes significant indenting is certainly part of that complexity. Fun fact: Did you know Python's indenting is only sometimes significant, too?
Re: std.d.lexer: pre-voting review / discussion
On 9/11/2013 10:10 PM, deadalnix wrote: See my comment, it is possible, with increased parser complexity, to handle many cases where you don't know what you are parsing yet. Doing so, lookahead is only required to find matching closing token. I suspect that a fast path in the lexer for that precise use case may be faster than buffering tokens, as it allow to save one branch per token. I don't believe that, because you can see about anything for tokens in lookahead and so have to duplicate nearly the whole lexer anyway for the 'fast path', but you're free to try it out and prove me wrong.
Re: std.d.lexer: pre-voting review / discussion
On Thursday, 12 September 2013 at 03:31:42 UTC, H. S. Teoh wrote: On Wed, Sep 11, 2013 at 10:06:11PM -0400, Jonathan M Davis wrote: On Thursday, September 12, 2013 03:37:06 deadalnix wrote: Int, Function, Scope, Import are all valid identifiers. All of which violate Phobos' naming conventions for enum values (they must start with a lowercase letter), which is why we went with adding an _ on the end. And it's pretty much as close as you can get to the keyword without actually using the keyword, which is a plus IMHO (though from the sounds of it, H.S. Teoh would consider that a negative due to possible confusion with the keyword). [...] Actually, the main issue I have is that some of the enum values end with _ while others don't. This is inconsistent. I'd rather have consistency than superficial resemblance to the keywords as typed. Either *all* of the enum values should end with _, or *none* of them should. Having a mixture of both is an eyesore, and leads to people wondering, should I add a _ at the end or not? If people insist that the 'default' keyword absolutely must be represented as TokenType.default_ (I really don't see why), then *all* TokenType values should end with _. But honestly, I find that really ugly. Writing something like kwDefault, or tokenTypeDefault, would be far better. Sigh, Andrei was right. Once the bikeshed is up for painting, even the rainbow won't suffice. :-P T Delphi would use TokenType ttDefault MyType mtDefault
Re: dub: should we make it the de jure package manager for D?
On 2013-09-12 00:50, H. S. Teoh wrote: But this is JS we're talking about. What happens, boys and gals, when the JS interpreter encounters an uncaught error? That's right, it logs an error message to the error console (which is hidden by default in most browsers), and silently fails without any warning, and then the rest of the site's scripts (hooked to various event handlers) CONTINUE RUNNING AS IF NOTHING HAPPENED. My experience is that if a JavaScript fails, somehow, JavaScript fill stop working all together on the site. But that's perhaps not the case if an exception is thrown. But rather if you're trying to use a variable or similar that doesn't exist. -- /Jacob Carlborg
Re: std.d.lexer: pre-voting review / discussion
On 2013-09-12 00:02, H. S. Teoh wrote:
Constructors with many arguments are evil, because you can't just modify
one setting and use defaults for the rest; you have to specify all of
them up to the last default argument you have to change.
Currently this is possible:
LexerConfig config = { iterStyle: IterationStyle.everything, tokenStyle:
TokenStyle.source };
I would really like the following to be possible as well:
void foo (LexerConfig config);
foo({ iterStyle: IterationStyle.everything, tokenStyle:
TokenStyle.source });
--
/Jacob Carlborg
Re: Move VisualD to github/d-programming-language ?
On Wed, 11 Sep 2013 08:57:59 +0200 PauloPinto pj...@progtools.org wrote: On Tuesday, 10 September 2013 at 21:25:41 UTC, Walter Bright wrote: On 9/9/2013 11:35 AM, Russel Winder wrote: C++11 has revitalized C++ in ways that are only just showing themselves. That's true. This is a threat to D gaining traction. I'm less sure about that. I think it presents an opportunity for us. Driving the C++ resurgence is: 1. demand for high performance computing 2. turning back towards native languages 3. recognition of the value of functional-style programming techniques 4. recognition of the value of safety, encapsulation, etc. Your 1-4 points are already covered by existing languages for traditional line of business applications, specially given the fact that even current VM based languages have native compilers available. Putted another way, how well do the 1 - 4 bullet points stand against Java/C#/Scala/Clojure native compilers ? Points #1 and #2 (performance computing and the draw towards native languages) are *both* about these two things: A. Lack of the VM middleman sucking up resources. B. Low-level capabilities. The native compilers for VM languages (With the possible exception of C#) can only address point A at best. And even C# is a bit awkward at point B. Giving a VM language a native compiler is only going half-way. The language itself is geared towards, and therefore limited by, the need to be runnable in a VM. That places inherent limitations on the potential benefits of native compilation. So while it's technically native-compiled, it's just bolted-on as an afterthought. Just because I add a turbocharger to a sedan doesn't mean it's comparable to a McLaren or a Bugatti.
Re: std.d.lexer: pre-voting review / discussion
On 2013-09-11 22:37, Brian Schott wrote: It's possible, but I fear that it would make the code a mess. Really? It seems you only need to change two places in the code: https://github.com/Hackerpilot/phobos/blob/master/std/d/lexer.d#L1724 And https://github.com/Hackerpilot/phobos/blob/master/std/d/lexer.d#L400 Although I just did a quick search for line. -- /Jacob Carlborg
Re: std.d.lexer: pre-voting review / discussion
On 2013-09-11 21:57, Piotr Szturmaj wrote: Delphi designers realized this problem years ago and they came up with a solution: http://docwiki.embarcadero.com/RADStudio/XE4/en/Fundamental_Syntactic_Elements#Extended_Identifiers Basically, Delphi allows escaping reserved identifiers with a ''. I wonder how D solves that problem when interfacing to COM classes if they have for example a function named scope. Scala does it as well: `keyword` if I recall correctly. Seems like you can put basically anything between the backticks in Scala. -- /Jacob Carlborg
Re: std.d.lexer: pre-voting review / discussion
On 2013-09-12 00:36, Martin Nowak wrote: Also a convenience function that reads a file and processes UTF BOM marks would be nice (see toUtf8 https://github.com/dawgfoto/lexer/blob/master/dlexer/dlexer.d#L1429), but that could as well fit somewhere else into phobos. Sounds like that would fit in std.file or similar. -- /Jacob Carlborg
Re: Not with the scene…
On Wednesday, 11 September 2013 at 12:00:19 UTC, Nick Sabalausky wrote: There's a relatively recent derivative of the dub music genre called dubstep https://en.wikipedia.org/wiki/Dubstep. More recently, dubstep begat a derived genre of...random noises...called brostep. The quintessential brostep recording person is probably Skrillex. This is what Skrillex sounds like, along with the reaction it typically gets: https://www.youtube.com/watch?v=A6Mti91l2Sc That's probably one of his *better* tracks, actually. Basically, take the random noises that radio stations like to stick between songs and extend it to fill a CD. If you accidentally stumble upon the beginnings of a beat or melody, cease immediately. That's basically it. Apparently he won a Grammy for that. Still, at least it's not Taylor Swift. Dubstep is old news. *Monkstep* is where it's at. http://www.youtube.com/watch?v=MgHclJGCOPY
Re: Not with the scene…
On Wednesday, 11 September 2013 at 12:00:19 UTC, Nick Sabalausky wrote: There's a relatively recent derivative of the dub music genre called dubstep https://en.wikipedia.org/wiki/Dubstep. More recently, dubstep begat a derived genre of...random noises...called brostep. The quintessential brostep recording person is probably Skrillex. This is what Skrillex sounds like, along with the reaction it typically gets: https://www.youtube.com/watch?v=A6Mti91l2Sc That's probably one of his *better* tracks, actually. Basically, take the random noises that radio stations like to stick between songs and extend it to fill a CD. If you accidentally stumble upon the beginnings of a beat or melody, cease immediately. That's basically it. Apparently he won a Grammy for that. Still, at least it's not Taylor Swift. Dubstep is old news. *Monkstep* is where it's at. http://www.youtube.com/watch?v=MgHclJGCOPY
Re: std.d.lexer: pre-voting review / discussion
On 2013-09-11 17:01, Dicebot wrote:
std.d.lexer is standard module for lexing D code, written by Brian Schott
Finally :)
* How does it handler errors, just returns TokenType.invalid?
* Personally I think the module is too big. I would go with:
- std.d.lexer.token
- std.d.lexer.tokentype
- std.d.lexer.lexer - contains the rest
- std.d.lexer.config - IterationStyle, TokenStyle, LexerConfig
- CircularRange, StringCache, possibly put somewhere else. I assume this
can be used for other things than lexing?
- Trie related code, same as above
* I see that errorMessage throws an exception. Do we really want that? I
would except it just returns an invalid token.
If we do decide it should throw, it should absolutely _not_ throw a
plain Exception. Create a new type, LexException or similar. I hate when
code throws plain Exceptions, it makes it useless to catch them.
I would also expect this LexException to contain a Token. It shouldn't
be needed to parse the exception message to get line and column information.
* I like that you overall use clear and descriptive variable and
function names. Except sbox:
https://github.com/Hackerpilot/phobos/blob/master/std/d/lexer.d#L3265
* Could we get some unit tests for string literals, comments and
identifies out side of the ASCII table
* I would like to see a short description for each unit test, what it's
testing. Personally I have started with this style:
@describe(byToken)
{
@context(valid string literal)
{
@it(should return a token with the type
TokenType.stringLiteral) unittest
{
// test
}
@it(should return a token with the correct lexeme) unittest
{
// test
}
}
}
Better formatted: http://pastebin.com/Dx78Vw6r
People here might think that would be a bit too verbose. The following
would be ok as well:
@describe(short description of the unit test) unittest { }
* Could you remove debug code and other code that is commented out:
- 344
- 1172
- 1226, is that needed?
- 3165-3166
- 3197-3198
- 3392
- 3410
- 3434
Spell errors:
* forwarad - 292
* commemnt - 2031
* sentenels - 299
* messsage - 301
* underliying - 2454
* alloctors - 3230
* strightforward - 2276
--
/Jacob Carlborg
Re: std.d.lexer: pre-voting review / discussion
On Thursday, 12 September 2013 at 06:17:04 UTC, Walter Bright wrote: On 9/11/2013 10:10 PM, deadalnix wrote: See my comment, it is possible, with increased parser complexity, to handle many cases where you don't know what you are parsing yet. Doing so, lookahead is only required to find matching closing token. I suspect that a fast path in the lexer for that precise use case may be faster than buffering tokens, as it allow to save one branch per token. I don't believe that, because you can see about anything for tokens in lookahead and so have to duplicate nearly the whole lexer anyway for the 'fast path', but you're free to try it out and prove me wrong. I plan to, but you know what it is, the best optimization is the one that go from non working to working state.
Re: std.d.lexer: pre-voting review / discussion
I got some time to work on the lexer this evening. Changeset here: https://github.com/Hackerpilot/phobos/commit/9bdb7f97bb8021f3b0d0291896b8fe21a6fead23#std/d/lexer.d The DDoc page has moved here: http://hackerpilot.github.io/experimental/std_lexer/phobos/std_d_lexer.html * There are a few more unit tests now * bitAnd renamed to amp * slice rename to dotdot * Much more cross-referencing in the doc comments * Start line and column can be specified in the lexer config
Re: std.d.lexer: pre-voting review / discussion
On Thursday, 12 September 2013 at 01:39:52 UTC, Walter Bright wrote: On 9/11/2013 6:30 PM, deadalnix wrote: Indeed. What solution do you have in mind ? The solution dmd uses is to put in an intermediary layer that saves the lookahead tokens in a linked list. I think this is the right approach. It can probably be another function, we can put into std.range and reuse for other lexer/parsers. The lexer or the parser should not be made more complex for this.
Re: Not with the scene…
On 12 September 2013 08:31, Meta jared...@gmail.com wrote: On Wednesday, 11 September 2013 at 12:00:19 UTC, Nick Sabalausky wrote: There's a relatively recent derivative of the dub music genre called dubstep https://en.wikipedia.org/wiki/Dubstep. More recently, dubstep begat a derived genre of...random noises...called brostep. The quintessential brostep recording person is probably Skrillex. This is what Skrillex sounds like, along with the reaction it typically gets: https://www.youtube.com/watch?v=A6Mti91l2Sc That's probably one of his *better* tracks, actually. Basically, take the random noises that radio stations like to stick between songs and extend it to fill a CD. If you accidentally stumble upon the beginnings of a beat or melody, cease immediately. That's basically it. Apparently he won a Grammy for that. Still, at least it's not Taylor Swift. Dubstep is old news. *Monkstep* is where it's at. Monkstep? Pfft... Folkstep is the way forward. ;-) http://www.youtube.com/watch?v=S3fTw_D3l10 -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: std.d.lexer: pre-voting review / discussion
just an idea regarding your string[TokenType.max + 1] in immutable(string[TokenType.max + 1]) tokenValues = [..] it seems that you try to reduce the memory usage wouldn't it be a nice idea to generate a combined imutable string at compiletime like this one ...pragmaexportpackageprivate... and generated string slice accesses? imuteable string big_one = generated_from(toke_list); imutable string export_token = big_one[10..6];
Re: std.d.lexer: pre-voting review / discussion
Am 12.09.2013 10:15, schrieb Brian Schott: I got some time to work on the lexer this evening. Changeset here: https://github.com/Hackerpilot/phobos/commit/9bdb7f97bb8021f3b0d0291896b8fe21a6fead23#std/d/lexer.d The DDoc page has moved here: http://hackerpilot.github.io/experimental/std_lexer/phobos/std_d_lexer.html * There are a few more unit tests now * bitAnd renamed to amp * slice rename to dotdot * Much more cross-referencing in the doc comments * Start line and column can be specified in the lexer config problem: many occurences of the same string you should use constants for the tokens (and others) string asm_token = asm; ... immutable(string[TokenType.max + 1]) tokenValues = [ ... asm_token ... ] and reuse these constants in your optimization maybe you can replace these lines with something getting feed with asm_token and give the same result but without 'a' and sm as splitted and different magic values - maybe a nice template or subrange... case 'a': if (input[1..$].equal(sm)) return TokenType.asm_; else ... break; and in your unit tests for example on auto expected =
Re: std.d.lexer: pre-voting review / discussion
On Wednesday, 11 September 2013 at 15:02:00 UTC, Dicebot wrote: Most important goal of this review is to determine any API / design problems. Any internal implementation tweaks may happen after inclusion to Phobos but it is important to assure that no breaking changes will be required any time soon after module will get wider usage. One quick remark : we need some kind of value provider to reuse across different lexing, and can be used outside the lexer. If I process a module and have to kick in a new lexing phase because of mixin, I want to generate identifiers out of the same pool.
Re: finding errors with templates without instantiating them
On Wednesday, 11 September 2013 at 22:22:46 UTC, Jonathan M Davis wrote: On Wednesday, September 11, 2013 15:15:39 Andrei Alexandrescu wrote: I'm not against it, but I find it a very small improvement. Its user base is exactly people who want to write and deploy a template without ever testing it at all, ever. I have a hard time catering to that hypothetical crowd. That crowd deserves whatever bugs they get. - Jonathan M Davis *cough* Phobos is that crowd *cough* Seriously. Most new code is tested, however, there is a ton of older code that is not only not-tested, but not even *compiled*: https://github.com/D-Programming-Language/phobos/pull/1514 Not that I'm necessarily *for* said feature, but I think we need to re-evaluate who this hypothetical crowd is, before looking down on it.
Re: std.d.lexer: pre-voting review / discussion
On 09/12/2013 12:09 AM, Manfred Nowak wrote: Walter Bright wrote: Since the very beginning. One example is determining if something is a declaration or an expression. I see now, that you wrote about parsing---not about lexing. Btw. I wrote an LALR(1)-parser for an early version of D. This means a lookahead of one was sufficient---or I made terrible mistakes. -manfred I had problems with it, especially with /IdentifierList: Identifier . IdentifierList TemplateInstance. IdentifierList And Bison also complaint. /
Re: Move VisualD to github/d-programming-language ?
On Thursday, 12 September 2013 at 06:58:53 UTC, Nick Sabalausky wrote: On Wed, 11 Sep 2013 08:57:59 +0200 PauloPinto pj...@progtools.org wrote: On Tuesday, 10 September 2013 at 21:25:41 UTC, Walter Bright wrote: On 9/9/2013 11:35 AM, Russel Winder wrote: C++11 has revitalized C++ in ways that are only just showing themselves. That's true. This is a threat to D gaining traction. I'm less sure about that. I think it presents an opportunity for us. Driving the C++ resurgence is: 1. demand for high performance computing 2. turning back towards native languages 3. recognition of the value of functional-style programming techniques 4. recognition of the value of safety, encapsulation, etc. Your 1-4 points are already covered by existing languages for traditional line of business applications, specially given the fact that even current VM based languages have native compilers available. Putted another way, how well do the 1 - 4 bullet points stand against Java/C#/Scala/Clojure native compilers ? Points #1 and #2 (performance computing and the draw towards native languages) are *both* about these two things: A. Lack of the VM middleman sucking up resources. B. Low-level capabilities. The native compilers for VM languages (With the possible exception of C#) can only address point A at best. And even C# is a bit awkward at point B. Giving a VM language a native compiler is only going half-way. The language itself is geared towards, and therefore limited by, the need to be runnable in a VM. That places inherent limitations on the potential benefits of native compilation. So while it's technically native-compiled, it's just bolted-on as an afterthought. Just because I add a turbocharger to a sedan doesn't mean it's comparable to a McLaren or a Bugatti. I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? How it is different from compiling Apple/Object/Turbo/Think Pascal, Delphi, Modula-2, Modula-3, Ada, OCaml, Oberon, Haskell, D, Go, Rust to native code? There is no VM about it, other than implementation details. Is the lack of access to processor resources what makes some of them VM languages? Then even ANSI C is a VM language, given that what gives the language lower hardware access capabilities are all language extensions. -- Paulo
Re: dub: should we make it the de jure package manager for D?
On Thu, Sep 12, 2013 at 08:39:11AM +0200, Jacob Carlborg wrote: On 2013-09-12 00:50, H. S. Teoh wrote: But this is JS we're talking about. What happens, boys and gals, when the JS interpreter encounters an uncaught error? That's right, it logs an error message to the error console (which is hidden by default in most browsers), and silently fails without any warning, and then the rest of the site's scripts (hooked to various event handlers) CONTINUE RUNNING AS IF NOTHING HAPPENED. My experience is that if a JavaScript fails, somehow, JavaScript fill stop working all together on the site. But that's perhaps not the case if an exception is thrown. But rather if you're trying to use a variable or similar that doesn't exist. [...] Well, you may be right, but at the time I was working on a YUI-based framework, and I discovered that either YUI, or the way the framework initializes YUI (I didn't write the base framework code so I don't know for sure), has some kind of default exception-catching code that catches such exceptions and then proceeds to *ignore* it. It does terminate the current execution thread, mind you, but all the other event handlers and hooks are still active, and the rest of the code attached to them will continue running despite the fact that something has catastrophically failed. Which, of course, means that now some object(s) are in an invalid state due to the previous failure, but the code is completely unaware of this situation and continue barging ahead and doing stuff, until it encounters the bad objects, and then random failures happen (which all get swallowed by the default catcher, thus promulgating the problem). But either way, the behaviour is equally bad. If JS stops executing upon encountering an exception, then you just randomly get a blank page (when the bug is triggered) with no helpful indication whatsoever what went wrong. If it continues executing, then you get random failures for no apparent reason. Both are equally hard to debug, and both could've been avoided had JS had *sane* handling of errors in the first place. T -- Gone Chopin. Bach in a minuet.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-12 15:37, H. S. Teoh wrote: Well, you may be right, but at the time I was working on a YUI-based framework, and I discovered that either YUI, or the way the framework initializes YUI (I didn't write the base framework code so I don't know for sure), has some kind of default exception-catching code that catches such exceptions and then proceeds to *ignore* it. It does terminate the current execution thread, mind you, but all the other event handlers and hooks are still active, and the rest of the code attached to them will continue running despite the fact that something has catastrophically failed. Which, of course, means that now some object(s) are in an invalid state due to the previous failure, but the code is completely unaware of this situation and continue barging ahead and doing stuff, until it encounters the bad objects, and then random failures happen (which all get swallowed by the default catcher, thus promulgating the problem). But either way, the behaviour is equally bad. If JS stops executing upon encountering an exception, then you just randomly get a blank page (when the bug is triggered) with no helpful indication whatsoever what went wrong. If it continues executing, then you get random failures for no apparent reason. Both are equally hard to debug, and both could've been avoided had JS had *sane* handling of errors in the first place. Absolutely, I agree. I have seen JavaScript errors on Github from time to time. But it seems like everything continue to work as expected. -- /Jacob Carlborg
Re: std.d.lexer: pre-voting review / discussion
On Thu, Sep 12, 2013 at 07:00:09AM +0200, deadalnix wrote:
On Thursday, 12 September 2013 at 03:37:55 UTC, H. S. Teoh wrote:
I still don't understand why backtracking is necessary in the first
place. I would've thought a modern parser should be well able to
encode seen tokens in its state so that backtracking is never
necessary. Or does D grammar have tricky bits that cannot be handled
this way, that I'm unaware of?
The problem is that it can cause a exponential (and I literally mean
exponential here) amount of complexity.
In some cases, the complexity is manageable, but in other that don't
make any sense (it has to be noted that even full lexing don't make
any sens here). For instance :
int foo()() {}
^
When you are at the caret position, you don't know if you face a
function declaration or a template declaration. You could go for
some ambiguous parsing, but each template argument can itself be a
type, an expression or a symbol.
[...]
This can be handled by using an intermediate grammar rule. Reduce all
(...) into an intermediate type, say ArgList, so the reduction
happens something like this:
int foo () () {}
Type Ident ArgList ArgList ^
Then have the rule:
CompileTimeArgs ::= ArgList
RuntimeArgs ::= ArgList
TemplateDecl ::= Type Ident CompileTimeArgs RuntimeArgs '{' ...
FuncDecl ::= Type Ident RuntimeArgs '{' ...
So first, all (...) gets parsed to ArgList, but it's not yet fixed
whether they are compile-time arguments or runtime arguments. It's only
after you see the next '(' or '{' that you decide whether ArgList should
reduce to CompileTimeArgs or RuntimeArgs.
ArgList itself, of course, will accept all possible parameters (both
runtime and compile-time): types, expressions, symbols. Then when you
reduce it to RuntimeArgs, you reject stuff that can't be interpreted as
parameter declarations.
T
--
There are two ways to write error-free programs; only the third one works.
Thread-Multiplexing for D
Hello, I once had a look at Go and D and can't see a reason to choose Go except for language built-in CSP and thread-multiplexing. But thread-multiplexing in Go is really a killer feature, especially when developing server-side applications that need to scale with the number of requests (from the Internet). Several startups chose therefore Go. It could have been D. So my point is that it would be good to have thread multiplexing in D as well. It exists for Java, see http://hawtdispatch.fusesource.org/. And then there is the original from Apple: http://en.wikipedia.org/wiki/Grand_Central_Dispatch. I think server-side applications that need to scale and are not for enterprise computing (aka company internal Java clusters) is a new kind of application for which people could also decide to choose D for just like Go. Putting in some thread-multiplexing into D is not easy and I won't have the time for doing that. But I would like to suggest to start an effort in that direction or think it over whether that would be something to have. Regards, Bienlein
Re: std.d.lexer: pre-voting review / discussion
Robert Schadek wrote: especially with IdentifierList: I see the shift/reduce conflict if you was indeed trying to syntactically differantiate between template identifiers and other identifiers. -manfred
Re: std.d.lexer: pre-voting review / discussion
deadalnix wrote: When you are at the caret position, you don't know If one ever reaches that position one uses petty lexer/grammar definitions. -manfred
Re: Thread-Multiplexing for D
Am 12.09.2013 16:12, schrieb Bienlein: Hello, I once had a look at Go and D and can't see a reason to choose Go except for language built-in CSP and thread-multiplexing. But thread-multiplexing in Go is really a killer feature, especially when developing server-side applications that need to scale with the number of requests (from the Internet). Several startups chose therefore Go. It could have been D. So my point is that it would be good to have thread multiplexing in D as well. It exists for Java, see http://hawtdispatch.fusesource.org/. And then there is the original from Apple: http://en.wikipedia.org/wiki/Grand_Central_Dispatch. I think server-side applications that need to scale and are not for enterprise computing (aka company internal Java clusters) is a new kind of application for which people could also decide to choose D for just like Go. Putting in some thread-multiplexing into D is not easy and I won't have the time for doing that. But I would like to suggest to start an effort in that direction or think it over whether that would be something to have. Regards, Bienlein There is an existing implementation already: http://vibed.org/ I didn't hear the term thread-multiplexing before, though. Maybe that should be added somewhere to make it more searchable.
Re: std.d.lexer: pre-voting review / discussion
On 09/11/2013 08:49 PM, Walter Bright wrote:
3. I assumed [an instance of] TokenType is a type.
(is(TokenType) is true).
But it's not, it's an enum.
Even the document says it's a 'type', but it's not a type.
It's a type tag. The tag uniquely determines the type. (As in 'the type
of a token', as opposed to 'the type of an expression'.)
4. When naming tokens like .. 'slice', it is giving it a
syntactic/semantic name rather than a token name. This would be awkward
if .. took on new meanings in D. Calling it 'dotdot' would be clearer.
Ditto for the rest. For example that is done better, '*' is called
'star', rather than 'dereference'.
FWIW, I use Tok!... I.e. a UDL for specifying kinds of tokens when
interfacing with the parser. Some other kinds of tokens get a canonical
representation. Eg. Tok!i is the kind of identifier tokens, Tok!0 is
the kind of signed integer literal tokens etc.
5. The LexerConfig initialization should be a constructor rather than a
sequence of assignments.
Using the { a:2, b:3 }-style initialization syntax?
6. No clue how lookahead works with this.
Eg. use a CircularBuffer adapter range. I have an implementation
currently coupled with my own lexer implementation. If there is
interest, I could factor it out.
Lookahead is realized as follows in the parser:
(assume 'code' is the circular buffer range.)
auto saveState(){muteerr++; return code.pushAnchor();} // saves the
state and mutes all error messages until the state is restored
void restoreState(Anchor state){ muteerr--; code.popAnchor(state); }
The 'Anchor' is a trivial wrapper around a size_t. The circular buffer
grows automatically to keep around tokens still reachable by an anchor.
(The range only needs small constant space besides the buffer to support
this functionality, though it is unable to detect usage errors.)
This approach is typically more efficient than using a free list on
contemporary architectures.
Re: dub: should we make it the de jure package manager for D?
We're considering making dub the official package manager for D. What do you all think? Please let's do it. At first I was like 'why would I need that?'. But soon I ported all my (alive) projects to dub and never came back. Since it can also generate projects for IDEs, it's also a great simplifier of projects. I've noted that it dramatically increased my likeliness to use and discover other's packages.
Re: Thread-Multiplexing for D
Ah! Cool :-). They are saying that they are using libevent, see http://vibed.org/features#performance. I see... Although only shitty software comes from my country such as SAP, this is a company that seems to develop some cool stuff. So I hope that D will also pop up in job ads over here some day ;-). -- Bienlein
Re: Thread-Multiplexing for D
About thread-multiplexing... You find a lot in Google when searching for socket multplexing, but not when searching for thread-multiplexing. Maybe I coined the term myself (don't know any more) when reading the section here: http://golang.org/doc/effective_go.html#goroutines Goroutines are multiplexed onto multiple OS threads so if one should block, such as while waiting for I/O, others continue to run. Their design hides many of the complexities of thread creation and management. -- Bienlein
Re: std.d.lexer: pre-voting review / discussion
On Thu, Sep 12, 2013 at 08:10:18PM +0400, Dmitry Olshansky wrote: 12-Sep-2013 19:39, Timon Gehr пишет: On 09/11/2013 08:49 PM, Walter Bright wrote: 4. When naming tokens like .. 'slice', it is giving it a syntactic/semantic name rather than a token name. This would be awkward if .. took on new meanings in D. Calling it 'dotdot' would be clearer. Ditto for the rest. For example that is done better, '*' is called 'star', rather than 'dereference'. FWIW, I use Tok!... I.e. a UDL for specifying kinds of tokens when interfacing with the parser. Some other kinds of tokens get a canonical representation. Eg. Tok!i is the kind of identifier tokens, Tok!0 is the kind of signed integer literal tokens etc. I like this. Not only this has the benefit of not colliding with keywords. I also imagine that it could be incredibly convenient to get back the symbolic representation of a token (when token used as parameter to AST-node say BinaryExpr!(Tok!+)). And truth be told we all know how tokens look in symbolic form so learning a pack of names for them feels pointless. +1. This is superior to both the ad hoc _ suffix and my ad hoc prefixing approach. Tok!default is maximally readable, and requires no silly convolutions like _ or 'kw' / 'tokenType' prefixes. I vote for Tok!... to denote token types. Question: what's the implementation of Tok? Does it fit into an enum? What's the underlying representation? I imagine some kind of canonical mapping into an integral type would be desired, to maximize runtime performance. T -- There are three kinds of people in the world: those who can count, and those who can't.
Re: std.d.lexer: pre-voting review / discussion
12-Sep-2013 12:05, Jacob Carlborg пишет: On 2013-09-11 17:01, Dicebot wrote: std.d.lexer is standard module for lexing D code, written by Brian Schott Finally :) * How does it handler errors, just returns TokenType.invalid? * Personally I think the module is too big. I would go with: - std.d.lexer.token - std.d.lexer.tokentype These could be one module. There is really no meaningful way to use token type separately from token. - std.d.lexer.lexer - contains the rest - std.d.lexer.config - IterationStyle, TokenStyle, LexerConfig Contrary I see this break down pointless - do you really want to use config without the lexer? - CircularRange, StringCache, possibly put somewhere else. I assume this can be used for other things than lexing? - Trie related code, same as above No good public interface defined is the reason. Basically the same as with Trie in the new std.uni module - needs its own review. * I see that errorMessage throws an exception. Do we really want that? I would except it just returns an invalid token. If we do decide it should throw, it should absolutely _not_ throw a plain Exception. Create a new type, LexException or similar. I hate when code throws plain Exceptions, it makes it useless to catch them. I would also expect this LexException to contain a Token. It shouldn't be needed to parse the exception message to get line and column information. Better yet to have a std exception hierarchy... so that all parsing modules can be tied to ParseException. So this needs to be resolved in a forward-compatible way. -- Dmitry Olshansky
Re: std.d.lexer: pre-voting review / discussion
12-Sep-2013 19:39, Timon Gehr пишет:
On 09/11/2013 08:49 PM, Walter Bright wrote:
4. When naming tokens like .. 'slice', it is giving it a
syntactic/semantic name rather than a token name. This would be awkward
if .. took on new meanings in D. Calling it 'dotdot' would be clearer.
Ditto for the rest. For example that is done better, '*' is called
'star', rather than 'dereference'.
FWIW, I use Tok!... I.e. a UDL for specifying kinds of tokens when
interfacing with the parser. Some other kinds of tokens get a canonical
representation. Eg. Tok!i is the kind of identifier tokens, Tok!0 is
the kind of signed integer literal tokens etc.
I like this.
Not only this has the benefit of not colliding with keywords. I also
imagine that it could be incredibly convenient to get back the symbolic
representation of a token (when token used as parameter to AST-node say
BinaryExpr!(Tok!+)). And truth be told we all know how tokens look in
symbolic form so learning a pack of names for them feels pointless.
6. No clue how lookahead works with this.
Eg. use a CircularBuffer adapter range. I have an implementation
currently coupled with my own lexer implementation. If there is
interest, I could factor it out.
Lookahead is realized as follows in the parser:
(assume 'code' is the circular buffer range.)
auto saveState(){muteerr++; return code.pushAnchor();} // saves the
state and mutes all error messages until the state is restored
void restoreState(Anchor state){ muteerr--; code.popAnchor(state); }
The 'Anchor' is a trivial wrapper around a size_t. The circular buffer
grows automatically to keep around tokens still reachable by an anchor.
(The range only needs small constant space besides the buffer to support
this functionality, though it is unable to detect usage errors.)
This approach is typically more efficient than using a free list on
contemporary architectures.
This ^^ is how. In fact std.d.lexer internally does similar thing with
non-RA ranges of bytes.
--
Dmitry Olshansky
Re: Move VisualD to github/d-programming-language ?
On Thursday, 12 September 2013 at 11:30:57 UTC, PauloPinto wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? Compiling such language to native code require horribly convoluted code generation. For instance, an helloworld in java compilled natively with gcj gives you a 50Mb (!) binary blob.
Re: std.d.lexer: pre-voting review / discussion
On 09/11/2013 05:01 PM, Dicebot wrote: std.d.lexer is standard module for lexing D code, written by Brian Schott Input Code: https://github.com/Hackerpilot/phobos/tree/master/std/d Documentation: http://hackerpilot.github.io/experimental/std_lexer/phobos/lexer.html ... (Commenting on what's visible in the documentation only for now.) auto config = ... ... .byToken(config) ... Seems to be a natural candidate for manual partial specialization. enum config = ... ... .byToken!config() ... uint line; ushort column; // is there overflow checking? Check to see if the token is of the same type and has the same string representation as the given token. Tokens with the same string representation always are of the same type, so this seems redundant. Furthermore, I'd expect (!a.opCmp(b)) === (a == b). Why provide the operator overloads at all? They don't implement essential or natural functionality. includeSpecialTokens. It's not clear what this flag does. If the input range supports slicing, the caching layer aliases itself away and the lexing process is much more efficient. It might be more sensible to require the user to manually wrap his range. pure nothrow bool isOperator(const TokenType t); pure nothrow bool isOperator(ref const Token t); pure nothrow bool isKeyword(const TokenType t); pure nothrow bool isKeyword(ref const Token t); ... IMO we should get rid of these. TokenType naming seems inconsistent. eg: is amp, = is assign, == is equal, but = is bitAndEqual and is logicAnd IMO better: is and, = is assign, = is andAssign and is andAnd. Of course, it might be best to use a template instead. Tok!, Tok!= and Tok!.
Re: std.d.lexer: pre-voting review / discussion
On Thursday, 12 September 2013 at 14:09:43 UTC, H. S. Teoh wrote:
This can be handled by using an intermediate grammar rule.
Reduce all
(...) into an intermediate type, say ArgList, so the reduction
happens something like this:
int foo () () {}
Type Ident ArgList ArgList ^
Then have the rule:
CompileTimeArgs ::= ArgList
RuntimeArgs ::= ArgList
TemplateDecl ::= Type Ident CompileTimeArgs RuntimeArgs '{' ...
FuncDecl ::= Type Ident RuntimeArgs '{' ...
So first, all (...) gets parsed to ArgList, but it's not yet
fixed
whether they are compile-time arguments or runtime arguments.
It's only
after you see the next '(' or '{' that you decide whether
ArgList should
reduce to CompileTimeArgs or RuntimeArgs.
ArgList itself, of course, will accept all possible parameters
(both
runtime and compile-time): types, expressions, symbols. Then
when you
reduce it to RuntimeArgs, you reject stuff that can't be
interpreted as
parameter declarations.
And then you got to backtrack the parsing instead of the lexing.
You just moved the problem around. You'll have to create some
temporary ast nodes that then will fix into what they really are.
Re: Move VisualD to github/d-programming-language ?
On Thursday, 12 September 2013 at 15:55:26 UTC, deadalnix wrote: On Thursday, 12 September 2013 at 11:30:57 UTC, PauloPinto wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? Compiling such language to native code require horribly convoluted code generation. For instance, an helloworld in java compilled natively with gcj gives you a 50Mb (!) binary blob. GCJ also doesn't offer improved performance over the JVM for non-trivial code. It will reduce the startup time, since no JIT is needed, but beyond that it doesn't really offer benefits. Not sure about other VM-native compilation
Re: Move VisualD to github/d-programming-language ?
On Thu, 12 Sep 2013 09:22:18 -0700, Trent a...@nope.avi wrote: On Thursday, 12 September 2013 at 15:55:26 UTC, deadalnix wrote: On Thursday, 12 September 2013 at 11:30:57 UTC, PauloPinto wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? Compiling such language to native code require horribly convoluted code generation. For instance, an helloworld in java compilled natively with gcj gives you a 50Mb (!) binary blob. GCJ also doesn't offer improved performance over the JVM for non-trivial code. It will reduce the startup time, since no JIT is needed, but beyond that it doesn't really offer benefits. Not sure about other VM-native compilation Microsoft built the Singularity OS using a special full-native compiler for C# called Sing#. Very cool piece of technology that I really wish they would release to the public: http://en.wikipedia.org/wiki/Sing_Sharp -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: std.d.lexer: pre-voting review / discussion
On Thu, Sep 12, 2013 at 06:09:41PM +0200, deadalnix wrote:
On Thursday, 12 September 2013 at 14:09:43 UTC, H. S. Teoh wrote:
This can be handled by using an intermediate grammar rule. Reduce all
(...) into an intermediate type, say ArgList, so the reduction
happens something like this:
int foo () () {}
Type Ident ArgList ArgList ^
Then have the rule:
CompileTimeArgs ::= ArgList
RuntimeArgs ::= ArgList
TemplateDecl ::= Type Ident CompileTimeArgs RuntimeArgs '{' ...
FuncDecl ::= Type Ident RuntimeArgs '{' ...
So first, all (...) gets parsed to ArgList, but it's not yet fixed
whether they are compile-time arguments or runtime arguments. It's
only after you see the next '(' or '{' that you decide whether
ArgList should reduce to CompileTimeArgs or RuntimeArgs.
ArgList itself, of course, will accept all possible parameters (both
runtime and compile-time): types, expressions, symbols. Then when you
reduce it to RuntimeArgs, you reject stuff that can't be interpreted
as parameter declarations.
And then you got to backtrack the parsing instead of the lexing. You
just moved the problem around. You'll have to create some temporary
ast nodes that then will fix into what they really are.
No. You can just use ArgListItem for both runtime args and compile-time
args. Once you decided which one it is, wrong arguments are rejected at
semantic time (which you have to do anyway).
Let's take a concrete example. Say we're parsing this invalid code:
int foo(alias A)(alias B) {}
You'd go through these steps:
1) Parse initial prefix of declaration:
int foo(alias A)(alias B) {}
^
AST:
FuncDecl
|--RetType: int
|--Ident: foo
\--[ being built ]
2) Parse first (...):
int foo(alias A)(alias B) {}
^
AST:
FuncDecl
|--RetType: int
|--Ident: foo
|--ArgList
| \-- AliasArg
|\-- ident: A
\--[ being built ]
I'm skipping the intermediate steps here, it's obvious how to
construct AliasArg from the usual parsing process.
3) Parse second (...):
int foo(alias A)(alias B) {}
^
AST:
FuncDecl
|--RetType: int
|--Ident: foo
|--ArgList
| \-- AliasArg
|\-- ident: A
|--ArgList
| \-- AliasArg
|\-- ident: B
\--[ being built ]
4) At this point, you now know the first ArgList is CompileTimeArgList,
and the second is RuntimeArgList, so you can just change the type
fields (along with narrowing FuncDecl to TemplateFuncDecl):
AST:
TemplateFuncDecl (was: FuncDecl)
|--RetType: int
|--Ident: foo
|--CompileTimeArgList (was: ArgList)
| \-- AliasArg
|\-- ident: A
|--RuntimeArgList (was: ArgList)
| \-- AliasArg
|\-- ident: B
\--[ being built ]
Since you're still constructing FuncDecl, your current parsing context
should still have a direct reference to the partially-constructed
FuncDecl node, which in turn has a direct reference to both ArgList
child nodes. So this is just dereferencing a couple of pointers. No
backtracking.
5) Finish parsing the declaration:
int foo(alias A)(alias B) {}
^
AST:
TemplateFuncDecl
|--RetType: int
|--Ident: foo
|--CompileTimeArgList (was: ArgList)
| \-- AliasArg
|\-- ident: A
|--RuntimeArgList (was: ArgList)
| \-- AliasArg
|\-- ident: B
\--FuncBody
\-- CompoundStatement
\-- [empty body]
6) Run semantic:
- Create local symbol table for foo, etc..
- Run semantic on CompileTimeArgList:
- Check AliasArg for validity
- Run semantic on AliasArg: add A to function's local symbol
table, etc.
- Run semantic on RuntimeArgList:
- Check AliasArg for validity: ERROR: cannot have alias parameter
at runtime.
- (Add B to local symbol table)(skipped due to previous error)
- (Run semantic on FuncBody)(skipped due to previous error)
- (Run semantic on RetType (verify return type match, etc.))(skipped
due to previous error)
- (Add function to parent scope symbol table)(skipped due to previous
error)
So, no backtracking is necessary.
Of course, it sounds like DMD's parser doesn't work this way, but that's
a limitation of DMD's parser, not an *inherent* need for backtracking.
T
--
I see that you JS got Bach.
Re: Move VisualD to github/d-programming-language ?
Am 12.09.2013 17:55, schrieb deadalnix: On Thursday, 12 September 2013 at 11:30:57 UTC, PauloPinto wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? Compiling such language to native code require horribly convoluted code generation. For instance, an helloworld in java compilled natively with gcj gives you a 50Mb (!) binary blob. gcj is a lame example. The project is dead since 2009 and they hardly invested anything into optimizing compiler. Just blindly translating bytecodes directly to processor instructions. Native code generation has *nothing* to do with binary size. Have you tried to statically compile similar examples in other languages? Lets use Oberon as an example for my native point. GC enabled systems programming language created to write operating systems. So same space as D, but also same space as Java and Go because it does not allow disabling the GC. Wirt used it to write Native Oberon, originally targeting the Ceres workstation at Zurich Technical University. System that was used for a few years by students and professors, even for daily desktop tasks. At ETHZ, there were native code compilers for Oberon, interpreters and they even played with the idea of JIT compiling dynamic modules on load using a kernel level JIT. So does Oberon use require VM ? If one wants to misuse the VM term to mean runtime library, then all high level languages have a VM even ANSI C. -- Paulo
Re: std.d.lexer: pre-voting review / discussion
On 2013-09-12 18:21, Dmitry Olshansky wrote: Contrary I see this break down pointless - do you really want to use config without the lexer? std.d.lexer.config might be a bit unnecessary. But yes, in general I would like to see smaller modules. -- /Jacob Carlborg
Re: finding errors with templates without instantiating them
Tudor Andrei Cristian Alexandrescu: I'm not against it, but I find it a very small improvement. Its user base is exactly people who want to write and deploy a template without ever testing it at all, ever. I have a hard time catering to that hypothetical crowd. This is an interesting discussion topic. Mistakes and bugs are mechanically caught at several different moments, sometimes even before compile-time, thanks to squiggly red lines shown by the IDEs where there's a syntax error. Then bugs are found during compilations, by contracts and unittests at run-time, etc. If you have a significant amount of template code already written, with unittests, etc, then the small amount of bugs spotted by this enhancement request is not much significant, because at that stage of the development of the code you are supposed to catch the remaining bugs using a high coverage ratio of unittests, etc. So in this case I agree with you this feature is not much useful, or it could even be harmful if programmers trust it too much. But there is a another situation, when you don't have much new code written. You have just written 5-15 lines of templated code, and you have not yet written unittests on it (I write lot of unittests, but usually after writing the code), at this early stage I think it's useful to have a D compiler that performs simple analysis of the template code, to catch the simple mistakes very quickly. This helps me focus on the less obvious errors, and then I can write the unittests to catch the semantic mistakes of the code. So I think this feature is not useful the seasoned code you seem to refer to; it's useful to catch as soon as possible silly mistakes inside just written templated code, that allows the programmer to focus on the less obvious bugs and on writing the unittests with high coverage. Bye, bearophile
Re: std.d.lexer: pre-voting review / discussion
On 2013-09-12 17:39, Timon Gehr wrote:
Using the { a:2, b:3 }-style initialization syntax?
Unfortunately that's currently not possible to pass to functions, see my
other post:
http://forum.dlang.org/thread/jsnhlcbulwyjuqcqo...@forum.dlang.org?page=6#post-l0ro6h:249mk:241:40digitalmars.com
--
/Jacob Carlborg
Re: std.d.lexer: pre-voting review / discussion
On 09/12/2013 06:40 PM, H. S. Teoh wrote:
I vote for Tok!... to denote token types.
Question: what's the implementation of Tok? Does it fit into an enum?
What's the underlying representation? I imagine some kind of canonical
mapping into an integral type would be desired, to maximize runtime
performance.
This is just a quick hack. One would maybe want to avoid the unwarranted
copies and linear searches at compile time:
import std.algorithm;
private enum symbols = [i,.,..,,,0,/+...+/];
struct TokenType{
private uint _tag; // or smaller type
private this(int tag){_tag=tag;}
string toString(){ // (optional)
static array(R)(R s){ string[] r; foreach(x;s) r~=x; return r; }
static immutable strs=array(symbols.map!((a)=`Tok!`~a~``));
return strs[_tag];
}
}
template Tok(string name)if(symbols.countUntil(name)!=-1){
enum Tok=TokenType(cast(uint)symbols.countUntil(name));
}
import std.stdio;
void main(){
enum x = Tok!i;
writeln(x);
writeln(Tok!0);
}
Re: Thread-Multiplexing for D
On Sep 12, 2013, at 8:34 AM, Bienlein jeti...@web.de wrote: About thread-multiplexing... You find a lot in Google when searching for socket multplexing, but not when searching for thread-multiplexing. Maybe I coined the term myself (don't know any more) when reading the section here: http://golang.org/doc/effective_go.html#goroutines Goroutines are multiplexed onto multiple OS threads so if one should block, such as while waiting for I/O, others continue to run. Their design hides many of the complexities of thread creation and management. The trick in D is that because statics are thread-local by default, any multiplexed app like this that expects its static data to remain consistent across calls is likely to fail. I've mentioned fiber-local storage here in the past, but it's a tricky problem. But I think it's one that we will need to sort out for things like this to work as the user expects them to.
Re: Move VisualD to github/d-programming-language ?
Am 12.09.2013 18:45, schrieb Adam Wilson: On Thu, 12 Sep 2013 09:22:18 -0700, Trent a...@nope.avi wrote: On Thursday, 12 September 2013 at 15:55:26 UTC, deadalnix wrote: On Thursday, 12 September 2013 at 11:30:57 UTC, PauloPinto wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? Compiling such language to native code require horribly convoluted code generation. For instance, an helloworld in java compilled natively with gcj gives you a 50Mb (!) binary blob. GCJ also doesn't offer improved performance over the JVM for non-trivial code. It will reduce the startup time, since no JIT is needed, but beyond that it doesn't really offer benefits. Not sure about other VM-native compilation Microsoft built the Singularity OS using a special full-native compiler for C# called Sing#. Very cool piece of technology that I really wish they would release to the public: http://en.wikipedia.org/wiki/Sing_Sharp You mean this? http://singularity.codeplex.com/ -- Paulo
Re: Move VisualD to github/d-programming-language ?
Am 12.09.2013 18:22, schrieb Trent: On Thursday, 12 September 2013 at 15:55:26 UTC, deadalnix wrote: On Thursday, 12 September 2013 at 11:30:57 UTC, PauloPinto wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? Compiling such language to native code require horribly convoluted code generation. For instance, an helloworld in java compilled natively with gcj gives you a 50Mb (!) binary blob. GCJ also doesn't offer improved performance over the JVM for non-trivial code. It will reduce the startup time, since no JIT is needed, but beyond that it doesn't really offer benefits. Not sure about other VM-native compilation It is a waste of time to bring up gcj as an example, it is an outdated compiler, frozen in time (2009), where not much effort was spent in optimizing code. If one wants to compare performance of Java native compilers, Aonix, Excelsior JET and Websphere Real Time JVM are better examples. Again there is no such thing as VM - native compilation, it is always an implementation decision. I really hate all this VM/Managed code concepts introduced by Sun/Microsoft. Before we used to discuss implementation techniques for programming languages, compiled/interpreted/jitted, not VM vs native. -- Paulo
Re: std.d.lexer: pre-voting review / discussion
On 09/12/2013 08:17 AM, Walter Bright wrote: I don't believe that, because you can see about anything for tokens in lookahead and so have to duplicate nearly the whole lexer anyway for the 'fast path', but you're free to try it out and prove me wrong. Me neither and if you have to duplicate the code for this it's a double loss. I think it would be more fruitful to batch lexing, i.e. fill a hole memory page (4Kb) with tokens the go back to parsing until you need more.
Some Clang warnings
A blog post that shows some of the warnings of the Clang compiler:
http://blog.llvm.org/2013/09/clang-warnings.html
Below I copy from the blog post some of the most interesting of
them, with quotations.
-
enum SortType {
unknown = 0,
min_invalid = 3,
bubble = 1,
quick,
insert
};
In this enum, a few non-valid values are defined, then the valid
enums listed. Valid enums use the auto increment to get their
values. However, min_invalid and insert both have value 3.
Luckily, -Wduplicate-enum will identify enums in this situation
and point them out. GCC will not warn on this.
D doesn't warn on this. Perhaps it's good to accept duplicated
enum values only when the user adds some kind of annotation that
denotes a desire for a duplicated value.
-
On to sort.cc
Class constructor:
Sort::Sort(int vec[], int size, bool sorted)
: sorted_(sorted_), vec_(vec), size_(size) {
Members from sort.h:
int* vec_;
bool sorted_;
int size_;
Checking the only constructor of the class, numerous problems can
be seen here. [...] sorted_ is initialized with itself instead of
with sorted. This leads to uninitialized value in sorted_, which
is caught by the aptly named -Wuninitialized. For this case, GCC
has -Wself-assign and -Wself-init.
In bugzilla I have asked for similar tests, like (that is closed):
http://d.puremagic.com/issues/show_bug.cgi?id=4407
Self-assignment is a common mistake that I'd like the D compiler
to catch.
-
for (int i = 0; i size_; ++i) {
for (int j = 1; j size_; ++i) {
...
}
}
This double nested loop gives bubble sort its n2 running time.
Rather, in this case, an infinite running time. Note the
increment in both of the loops happen on i, even in the inner
loop. j is never touched, either here or inside the loop.
-Wloop-analysis will give a warning when all the variables inside
a for loop conditional does not change during the loop iteration.
Only in Clang.
D has foreach that avoids this bug, but sometimes in D you have
to use for loops (like when the increment is not 1)
void main() {
enum size_ = 5;
for (int i = 0; i size_; i += 2) {
for (int j = 1; j size_; i += 2) {}
}
}
See:
http://d.puremagic.com/issues/show_bug.cgi?id=11018
Bye,
bearophile
Re: dub: should we make it the de jure package manager for D?
I am all in favor of making dub *the* D package manager to use. pypi is awesome, and we've got something which can be just as awesome. Let's do it. On the subject of YAML. I've tried it out a few times. The syntax for JSON can be described in 1-2 pages, on json.org's front page. The syntax for YAML requires a 50-100 page document to describe. That should be enough to tell you not to use it.
Re: std.d.lexer: pre-voting review / discussion
On 09/12/2013 05:39 PM, Timon Gehr wrote:
Lookahead is realized as follows in the parser:
(assume 'code' is the circular buffer range.)
auto saveState(){muteerr++; return code.pushAnchor();} // saves the
state and mutes all error messages until the state is restored
void restoreState(Anchor state){ muteerr--; code.popAnchor(state); }
The 'Anchor' is a trivial wrapper around a size_t. The circular buffer
grows automatically to keep around tokens still reachable by an anchor.
(The range only needs small constant space besides the buffer to support
this functionality, though it is unable to detect usage errors.)
Do you think it's possible to use CircularBufferRange itself as anchor.
Then calling save() would return a CircularBufferRange and you canould
scratch the two functions above. I had some promising experiments in
that direction, but the implicit save on copy is annoying because you
end up with anchors from temporary copies.
Re: dub: should we make it the de jure package manager for D?
On Thursday, September 12, 2013 22:02:15 w0rp wrote: On the subject of YAML. I've tried it out a few times. The syntax for JSON can be described in 1-2 pages, on json.org's front page. The syntax for YAML requires a 50-100 page document to describe. That should be enough to tell you not to use it. While, I agree in principle, I would point out that what's on json.org is oversimplified and is not a valid spec. You really need to read the RFC if you want to get it right, and that has about 9 pages of actual content: http://www.ietf.org/rfc/rfc4627.txt?number=4627 So, you're comparing a basic explanation of JSON to a full-on spec for YAML. Now, the YAML 1.2 spec does have about 77 pages of actual content: http://yaml.org/spec/1.2/spec.pdf So, you're not completely off base in your comparison (YAML _is_ way more complicated), but to properly specify JSON requires a lot more pages than your post indicates. - Jonathan M Davis
Re: finding errors with templates without instantiating them
On Thursday, September 12, 2013 11:59:52 monarch_dodra wrote: On Wednesday, 11 September 2013 at 22:22:46 UTC, Jonathan M Davis wrote: On Wednesday, September 11, 2013 15:15:39 Andrei Alexandrescu wrote: I'm not against it, but I find it a very small improvement. Its user base is exactly people who want to write and deploy a template without ever testing it at all, ever. I have a hard time catering to that hypothetical crowd. That crowd deserves whatever bugs they get. - Jonathan M Davis *cough* Phobos is that crowd *cough* Seriously. Most new code is tested, however, there is a ton of older code that is not only not-tested, but not even *compiled*: https://github.com/D-Programming-Language/phobos/pull/1514 Not that I'm necessarily *for* said feature, but I think we need to re-evaluate who this hypothetical crowd is, before looking down on it. I don't think that there's any question that we've had some fairly poor testing of a lot of templates in the past, but you have to do that testing regardless if you want to make sure that your code is valid. So, best case, this feature would point out that you were too negligent to even bother testing beyond maybe the most basic of tests. I really don't think that it's worth the extra complication in the compiler and the additional cost to the compiler developers' time, particularly when it's likely to introduce new bugs that would have to be fixed when we already have plenty of way more important issues that still need to be fixed. I think that if this sort of thing is to be considered, it needs to be considered after all of the far more major issues have been resolved. - Jonathan M Davis
Re: finding errors with templates without instantiating them
On 9/12/13 9:53 AM, bearophile wrote: But there is a another situation, when you don't have much new code written. You have just written 5-15 lines of templated code, and you have not yet written unittests on it (I write lot of unittests, but usually after writing the code), at this early stage I think it's useful to have a D compiler that performs simple analysis of the template code, to catch the simple mistakes very quickly. This helps me focus on the less obvious errors, and then I can write the unittests to catch the semantic mistakes of the code. So I think this feature is not useful the seasoned code you seem to refer to; it's useful to catch as soon as possible silly mistakes inside just written templated code, that allows the programmer to focus on the less obvious bugs and on writing the unittests with high coverage. I don't buy this. All that work for a fleeting moment. Reminds me of that comedic bit - hands-free sanitizer dispensers are for the fraction of a second between your hand pushing on the button and your hand getting cleaned. Andrei
About contract programming
Some slides about contract-based programming, etc: http://www.slideshare.net/DanielPrager/introduction-to-contracts-and-functional-contracts Reddit discussion: http://www.reddit.com/r/programming/comments/1lt7zi/introduction_to_contracts_and_functional_contracts/ Bye, bearophile
Re: std.d.lexer: pre-voting review / discussion
On 09/12/2013 04:21 AM, Martin Nowak wrote: On 09/12/2013 03:39 AM, Walter Bright wrote: On 9/11/2013 6:30 PM, deadalnix wrote: Indeed. What solution do you have in mind ? The solution dmd uses is to put in an intermediary layer that saves the lookahead tokens in a linked list. Linked list sounds bad. Do you have a rough idea how often lookahead is needed, i.e. is it performance relevant? If so it might be worth tuning. Maybe some fixed size stack vector with 64 elements or so and some linked list for the unusual case would help ..
Re: std.d.lexer: pre-voting review / discussion
13-Sep-2013 00:46, Robert Schadek пишет: On 09/12/2013 04:21 AM, Martin Nowak wrote: On 09/12/2013 03:39 AM, Walter Bright wrote: On 9/11/2013 6:30 PM, deadalnix wrote: Indeed. What solution do you have in mind ? The solution dmd uses is to put in an intermediary layer that saves the lookahead tokens in a linked list. Linked list sounds bad. Do you have a rough idea how often lookahead is needed, i.e. is it performance relevant? If so it might be worth tuning. Maybe some fixed size stack vector with 64 elements or so and some linked list for the unusual case would help .. And an extra branch to detect which one is currently the case? No, thanks ;) -- Dmitry Olshansky
Re: std.d.lexer: pre-voting review / discussion
On 09/12/2013 10:03 PM, Martin Nowak wrote:
On 09/12/2013 05:39 PM, Timon Gehr wrote:
Lookahead is realized as follows in the parser:
(assume 'code' is the circular buffer range.)
auto saveState(){muteerr++; return code.pushAnchor();} // saves the
state and mutes all error messages until the state is restored
void restoreState(Anchor state){ muteerr--; code.popAnchor(state); }
The 'Anchor' is a trivial wrapper around a size_t. The circular buffer
grows automatically to keep around tokens still reachable by an anchor.
(The range only needs small constant space besides the buffer to support
this functionality, though it is unable to detect usage errors.)
Do you think it's possible to use CircularBufferRange itself as anchor.
Well, this implementation is geared towards the usage pattern found in
the top-down parser. If the first anchor is not restored last, it will
not generally work. This does not conform to the range interface.
Implementing .save requires a more advanced and less efficient
implementation.
Then calling save() would return a CircularBufferRange and you could
scratch the two functions above. I had some promising experiments in
that direction,
What did you do? The way I think I'd approach it would be to maintain a
binary min-heap using a few pointers in each range instance.
but the implicit save on copy is annoying because you
end up with anchors from temporary copies.
Is this referring to suboptimal efficiency? I guess that is rather hard
to address in a safe way. Basically, you'd want to forget about anchor
management in case it can be proven that there is another range on the
same buffer that stays at most as progressed during the whole lifetime
of the range under consideration.
Of course, it is always possible to do it in unsafely.
Re: finding errors with templates without instantiating them
Jonathan M Davis: I think that if this sort of thing is to be considered, it needs to be considered after all of the far more major issues have been resolved. In general this a well know fallacious point of view. I D there are large issues open since several years. If you apply your idea to this situation along the years, then you don't improve the compiler much In most jobs you use pipelining: you don't wait for the largest jobs to finish before doing small jobs. You try to cram as many jobs in parallel as possible, filling all the empty cracks, to increase throughput. This is why people are improving small things in dmd even if large compiler issues are still present. Bye, bearophile
Re: Move VisualD to github/d-programming-language ?
On Thu, 12 Sep 2013 13:30:55 +0200 PauloPinto pj...@progtools.org wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? How it is different from compiling Apple/Object/Turbo/Think Pascal, Delphi, Modula-2, Modula-3, Ada, OCaml, Oberon, Haskell, D, Go, Rust to native code? There is no VM about it, other than implementation details. Is the lack of access to processor resources what makes some of them VM languages? Then even ANSI C is a VM language, given that what gives the language lower hardware access capabilities are all language extensions. Let me try to clarify my main point, since I may have been a bit unclear: I don't really mean to debate VM vs native here; I'm aware (as you are) that a normally-VMed language can be made to be every bit as fast and powerful as any normally-native-compiled language. Heck, all you need is a VM that interprets/JITs the LLVM's bytecode, and then bam, all of a sudden C/C++ are VM languages. That VM vs native isn't what I was really trying to address. I was only trying to address a couple very specific points in your and Walter's discussion about D vs languages like Java/C#/Scala/Clojure. And not *all* normally-VM languages in general, but specifically ones along those particular lines (frankly, the more popular ones). Walter had said: I think [the C++ resurgence] presents an opportunity for [D]. Driving the C++ resurgence is: 1. demand for high performance computing 2. turning back towards native languages 3. recognition of the value of functional-style programming techniques 4. recognition of the value of safety, encapsulation, etc. I'll concede to your point that #3 and #4 are addressed not only by D but also by several normally-VMed languages (to varying levels of success). However, and this is the core of what I was trying to say: I'm disputing that most of those other popular normally-VMed languages address Walter's #1 and #2 *as effectively* as D does. My reasoning for that goes like this: - Walter's point #1, demand for high performance computing is *partly* about avoiding the runtime/startup costs of interpretation/JIT, but it's *also* about being able to reach down to the low-level when necessary (pointers, reinterpret casts, manual memory management, etc.) - Walter's point #2, turning back towards native languages has much the same duality: It's *partly* about performance, but *also* about being able to access the hardware (ex: drivers, OS-level stuff, certain embedded systems, reduced electricity usage, reduced memory footprint (ex: for phones/tablets with little or no virtual mem), etc.) - While it's certainly *possible* for a normally-VMed language to offer full low-level abilities, most of them don't (or at least most of the popular ones don't), and the ones that do (ex: C#, AIUI) don't usually (ever?) do it on a level that's on par with C/C++/D. And I think the fact that most normally-VMed languages lack, or skimp on, low-level abilities is no coincidence: There's a natural tendency for that specifically because two of the main reasons for using a VM in the first place are A, the safety of being banned from low-level access and B, the increased cross-platform portability achieved by not accessing low-level. Again, it's not that normally-VMed languages can't/never get around that (C#/.NET found ways around it), but that they *typically* don't, and even when they do it's typically (if ever?) not up-to-par with C/C++/D.
Re: std.d.lexer: pre-voting review / discussion
On 09/12/2013 11:03 PM, Dmitry Olshansky wrote: Maybe some fixed size stack vector with 64 elements or so and some linked list for the unusual case would help .. And an extra branch to detect which one is currently the case? No, thanks ;) I would think that branching and using the vector is faster than using the constructing and linking nodes in the LL
Re: std.d.lexer: pre-voting review / discussion
On Thursday, September 12, 2013 23:55:23 Robert Schadek wrote: On 09/12/2013 11:03 PM, Dmitry Olshansky wrote: Maybe some fixed size stack vector with 64 elements or so and some linked list for the unusual case would help .. And an extra branch to detect which one is currently the case? No, thanks ;) I would think that branching and using the vector is faster than using the constructing and linking nodes in the LL Well, it sounds like there are several ideas to try out in this thread. I guess that only benchmarking and profiling will really tell us which (if any) are better though. - Jonathan M Davis
Re: finding errors with templates without instantiating them
On 9/12/2013 3:16 PM, bearophile wrote: Jonathan M Davis: I think that if this sort of thing is to be considered, it needs to be considered after all of the far more major issues have been resolved. In general this a well know fallacious point of view. I D there are large issues open since several years. If you apply your idea to this situation along the years, then you don't improve the compiler much In most jobs you use pipelining: you don't wait for the largest jobs to finish before doing small jobs. You try to cram as many jobs in parallel as possible, filling all the empty cracks, to increase throughput. This is why people are improving small things in dmd even if large compiler issues are still present. Except that this is a large job, with a high likelihood of causing other unanticipated issues.
Re: std.d.lexer: pre-voting review / discussion
On 9/12/2013 1:15 AM, Brian Schott wrote: I got some time to work on the lexer this evening. Changeset here: https://github.com/Hackerpilot/phobos/commit/9bdb7f97bb8021f3b0d0291896b8fe21a6fead23#std/d/lexer.d The DDoc page has moved here: http://hackerpilot.github.io/experimental/std_lexer/phobos/std_d_lexer.html Great! * There are a few more unit tests now I strongly recommend running the unit tests with -cov. A lexer should be able to get near 100% coverage with the unit tests. * bitAnd renamed to amp * slice rename to dotdot * Much more cross-referencing in the doc comments * Start line and column can be specified in the lexer config
Re: std.d.lexer: pre-voting review / discussion
On Thursday, 12 September 2013 at 23:40:55 UTC, Walter Bright wrote: I strongly recommend running the unit tests with -cov. A lexer should be able to get near 100% coverage with the unit tests. Some of the code is only present to be used at compile-time to generate switch statements inside of the lexer. -cov doesn't show code that's executed at compile-time to be covered, and it couldn't show meaningful line numbers on code that's generated and mixed in. That being said, it's currently 70% covered. I'll be making another pass over the code later to fill in some of the gaps.
Re: std.d.lexer: pre-voting review / discussion
On 9/12/2013 4:57 PM, Brian Schott wrote: On Thursday, 12 September 2013 at 23:40:55 UTC, Walter Bright wrote: I strongly recommend running the unit tests with -cov. A lexer should be able to get near 100% coverage with the unit tests. Some of the code is only present to be used at compile-time to generate switch statements inside of the lexer. -cov doesn't show code that's executed at compile-time to be covered, and it couldn't show meaningful line numbers on code that's generated and mixed in. That's right. That being said, it's currently 70% covered. I'll be making another pass over the code later to fill in some of the gaps. Thanks.
Re: Move VisualD to github/d-programming-language ?
On Thu, 12 Sep 2013 11:47:02 -0700, Paulo Pinto pj...@progtools.org wrote: Am 12.09.2013 18:45, schrieb Adam Wilson: On Thu, 12 Sep 2013 09:22:18 -0700, Trent a...@nope.avi wrote: On Thursday, 12 September 2013 at 15:55:26 UTC, deadalnix wrote: On Thursday, 12 September 2013 at 11:30:57 UTC, PauloPinto wrote: I don't get the point, what there is VM like when I compile Java, Scala, F#, C# native code? Compiling such language to native code require horribly convoluted code generation. For instance, an helloworld in java compilled natively with gcj gives you a 50Mb (!) binary blob. GCJ also doesn't offer improved performance over the JVM for non-trivial code. It will reduce the startup time, since no JIT is needed, but beyond that it doesn't really offer benefits. Not sure about other VM-native compilation Microsoft built the Singularity OS using a special full-native compiler for C# called Sing#. Very cool piece of technology that I really wish they would release to the public: http://en.wikipedia.org/wiki/Sing_Sharp You mean this? http://singularity.codeplex.com/ -- Paulo Indeed. Would be really nice to revive Sing# and make it into a first class too. To bad it's license to so restrictive as to be unusable -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: finding errors with templates without instantiating them
On Thursday, September 12, 2013 15:58:05 Walter Bright wrote: On 9/12/2013 3:16 PM, bearophile wrote: Jonathan M Davis: I think that if this sort of thing is to be considered, it needs to be considered after all of the far more major issues have been resolved. In general this a well know fallacious point of view. I D there are large issues open since several years. If you apply your idea to this situation along the years, then you don't improve the compiler much In most jobs you use pipelining: you don't wait for the largest jobs to finish before doing small jobs. You try to cram as many jobs in parallel as possible, filling all the empty cracks, to increase throughput. This is why people are improving small things in dmd even if large compiler issues are still present. Except that this is a large job, with a high likelihood of causing other unanticipated issues. Yeah. This sounds like exactly the sort of thing that's going to result in a lot of additional bugs, and it's arguably going to yield only a minor benefit. - Jonathan M Davis
Re: extern(C) declaration inside function
On Tuesday, 10 September 2013 at 13:41:14 UTC, Luís Marques wrote: Is this a bug? *bump*
Re: std.d.lexer: pre-voting review / discussion
We are talking about parameters here, not arguments. But overall, that isn't the point. If I have int a = 3 as an argument in the first set of (), I still have no clue if it is a runtime or compile time parameter. But ref int a = 3 ? Doing so, you ends up with a lot of ambiguous node (like FunctionArgumentOrValueTemplateParameter) and have to manage that complexity, or have to backtrack on the parsing and fix ambiguous node when you have enough information.
Re: Compiling the hello world example fails (2)
On Thursday, 12 September 2013 at 11:23:51 UTC, Jacob Carlborg wrote: On 2013-09-12 12:31, Anton Alexeev wrote: OK, thanks for the links! First noob question: I need an icon for the window. The Shell class has a method setImage(Image image). In Java there is a SWTResourceManager available: shell.setImage(SWTResourceManager.getImage(/home/virtualbox/favicon.png)); I've looked around a bit in *.di files and found an Image and an ImageLoader classes. Maybe it could work but I don't know how to create a Java String... ImageLoader imageLoader = new ImageLoader(); Image image = new Image(); image.init_(imageLoader.load(/home/virtualbox/favicon.png)); shell.setImage(image); SWTResourceManager is not part of the standard SWT. It's seems to be a part of WindowBuilder. Here's an example of how to set the icon of a window: http://www.java2s.com/Tutorial/Java/0280__SWT/Setleftuppercornerimage.htm That site contains many other SWT tutorials as well. Thanks a lot! One more question: how do I work with events?
Re: Compiling the hello world example fails (2)
Found out how to handle events:
class DisposeListenerImpl:DisposeListener{
public void widgetDisposed(DisposeEvent e) {
writeln(Disposed);
}
}
shell.addDisposeListener(new DisposeListenerImpl);
Is this the right way?
Re: may gc free malloced memory?
On Thursday, 12 September 2013 at 05:36:31 UTC, Alexandr Druzhinin wrote: Some C function malloc-ed memory. This memory should be freeed much later. I don't want to manually call C function to free this memory in some point later, so may I in some way ask gc to free this memory using something like addRoot(for instance) or else or the true way is to copy malloc-ed memory to gc-allocated memory and free malloc-ed memory at once? Like: ubyte data* = cfunction_allocates_memory(); auto gcmemory = data[0..length(data)]; cfunction_frees_memory(data); // work with gcmemory only or ubyte data* = cfunction_allocates_memory(); GC.someUnknownToMeFunction(data); // now gc will control this memory No. Only free can be used with malloc. The memory comes from distinct pools. Another option could be to use GC.malloc, and memcpy your old mmory into your new memory, free the old memory, and use your new block. GC.malloc, as the name suggests, is a malloc, but done by the GC.
Re: Greedy memory handling
On 2013-09-11 10:06, monarch_dodra wrote: I have a function that will *massively* benefit from having a persistent internal buffer it can re-use (and grow) from call to call, instead of re-allocating on every call. What I don't want is either of: 1. To set a fixed limitation of size, if the user ends up making repeated calls to something larger to my fixed size. 2. For a single big call which will allocate a HUGE internal buffer that will consume all my memory. What I need is some sort of lazy buffer. Basically, the allocation holds, but I don't want the to prevent the GC from collecting it if it deems it has gotten too big, or needs more memory. Any idea on how to do something like that? Or literature? How about keeping a stack or static buffer. If that gets too small use a new buffer. When you're done with the new buffer set it to null to allow the GC to collect it. Then repeat. -- /Jacob Carlborg
Re: may gc free malloced memory?
On Thursday, 12 September 2013 at 05:59:33 UTC, monarch_dodra wrote: On Thursday, 12 September 2013 at 05:36:31 UTC, Alexandr Druzhinin wrote: Some C function malloc-ed memory. This memory should be freeed much later. I don't want to manually call C function to free this memory in some point later, so may I in some way ask gc to free this memory using something like addRoot(for instance) or else or the true way is to copy malloc-ed memory to gc-allocated memory and free malloc-ed memory at once? Like: ubyte data* = cfunction_allocates_memory(); auto gcmemory = data[0..length(data)]; cfunction_frees_memory(data); // work with gcmemory only or ubyte data* = cfunction_allocates_memory(); GC.someUnknownToMeFunction(data); // now gc will control this memory No. Only free can be used with malloc. The memory comes from distinct pools. Another option could be to use GC.malloc, and memcpy your old mmory into your new memory, free the old memory, and use your new block. GC.malloc, as the name suggests, is a malloc, but done by the GC. You could also use some kind of helper class. Perhaps Phobos has a facility for this, but to illustrate the idea: http://dpaste.dzfl.pl/805a61c0 However note that the memory isn't guaranteed to be freed this way. Only if the GC heap gets full and the collector runs.
Re: may gc free malloced memory?
12.09.2013 14:45, Rene Zwanenburg пишет: On Thursday, 12 September 2013 at 05:59:33 UTC, monarch_dodra wrote: On Thursday, 12 September 2013 at 05:36:31 UTC, Alexandr Druzhinin wrote: Some C function malloc-ed memory. This memory should be freeed much later. I don't want to manually call C function to free this memory in some point later, so may I in some way ask gc to free this memory using something like addRoot(for instance) or else or the true way is to copy malloc-ed memory to gc-allocated memory and free malloc-ed memory at once? Like: ubyte data* = cfunction_allocates_memory(); auto gcmemory = data[0..length(data)]; cfunction_frees_memory(data); // work with gcmemory only or ubyte data* = cfunction_allocates_memory(); GC.someUnknownToMeFunction(data); // now gc will control this memory No. Only free can be used with malloc. The memory comes from distinct pools. Another option could be to use GC.malloc, and memcpy your old mmory into your new memory, free the old memory, and use your new block. GC.malloc, as the name suggests, is a malloc, but done by the GC. You could also use some kind of helper class. Perhaps Phobos has a facility for this, but to illustrate the idea: http://dpaste.dzfl.pl/805a61c0 However note that the memory isn't guaranteed to be freed this way. Only if the GC heap gets full and the collector runs. Ok. I just think that copying from one pool to another is excessive and may be there is a some way to avoid it. Thanks for the answers!
Re: VisualD import
On Wednesday, 11 September 2013 at 22:15:07 UTC, Rainer Schuetze
wrote:
On 11.09.2013 23:42, Lemonfiend wrote:
On Wednesday, 11 September 2013 at 20:36:39 UTC, Rainer
Schuetze wrote:
On 11.09.2013 18:13, Lemonfiend wrote:
Oops, I forgot to say what I actually did.
I added derelict to Compiler-General-Additional Imports.
The code is just this:
module main;
import std.stdio;
import derelict.opengl3.gl3;
void main()
{
writeln(Hello D-World!);
}
And the build output is a symbol undefined linker issue:
-- Rebuild All started: Project: Test, Configuration:
Debug Win32
--
Building Debug\Test.exe...
OPTLINK (R) for Win32 Release 8.00.12
Copyright (C) Digital Mars 1989-2010 All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
Debug\Test.obj(Test)
Error 42: Symbol Undefined
_D8derelict7opengl33gl312__ModuleInfoZ
Building Debug\Test.exe failed!
Details saved as
file://C:\D\Test\Test\Debug\Test.buildlog.html
Build time: 3 s
Solution build stopped.
Build has been canceled.
Did you add the derelict library/libraries as linker inputs?
I haven't compiled derelict to a lib, I'm using the source
directly.
With rdmd I can simply do -Ipath\to\derelict\source
The compilation model of rdmd is not supported by Visual D. You
might be able to use rdmd as other compiler in the project
options with additional options --build-only.
On the other hand, you can also create a library from the
project templates, then drag the source folder into the project
to add all files. Then set a project dependency of your
application to the library.
You can also create a dub package and generate visuald project
from dub.
override and package
Code:
import std.stdio;
class T1 {
protected:
void _apply() {
writeln(Call T1);
}
}
class T2 : T1 {
public:
override void _apply() {
writeln(Call T2);
}
}
class T3 : T1 {
protected:
override void _apply() {
writeln(Call T3);
}
}
class T4 : T1 {
package:
void _apply() { /// -- [1]
writeln(Call T4);
}
}
void main()
{
T1 t1 = new T1();
T2 t2 = new T2();
T3 t3 = new T3();
T4 t4 = new T4();
t1._apply();
t2._apply();
t3._apply();
t4._apply();
}
Produce the correct output:
Call T1
Call T2
Call T3
Call T4
If I remove 'override' from T3 (or also T2) I get the correct
deprecation message:
/d172/f194.d(19): Deprecation: overriding base class function
without using override attribute is deprecated (f194.T3._apply
overrides f194.T1._apply)
But if I try to write 'override' before [1], I get this error
message:
Error: function T4._apply cannot override a non-virtual function
This seems inconsistent. I really overwrite the method, and then
I put it in a package label.
Re: override and package
Same with private, of course.
Re: Adding libraries to an executable
On Thursday, 12 September 2013 at 01:16:28 UTC, Mike Parker wrote: On 9/12/2013 6:20 AM, Anton Alexeev wrote: On Tuesday, 10 September 2013 at 11:06:04 UTC, Jacob Carlborg wrote: On 2013-09-10 11:04, Anton Alexeev wrote: Can be but not libphobos2 Link statically with it, which is does by default. Just compile with: dmd -L-lcurl test.d The point is: http://pastebin.com/0VkYgFix Apparently you need to install the libcurl dev package. The linker can't find libcurl.a. I've installed libcurl4-openssl-dev 7.29.0-1ubuntu3.1 before I asked for help here. Else I can't even compile with that command: dmd -L-lphobos2 -L-lcurl test.d
Re: override and package
On 2013-09-12 11:28, Namespace wrote:
But if I try to write 'override' before [1], I get this error message:
Error: function T4._apply cannot override a non-virtual function
This seems inconsistent. I really overwrite the method, and then I put
it in a package label.
I think the error message is pretty clear. You cannot override a
function that isn't virtual. Private and package methods are not
virtual. Example:
class Base
{
void foo ()
{
writeln(Base.foo);
}
}
class Sub : Base
{
package void foo ()
{
writeln(Sub.foo);
}
}
void main ()
{
auto sub = new Sub;
sub.foo(); // prints Sub.foo as expected
Base base = sub;
base.foo(); // prints Base.foo
}
--
/Jacob Carlborg
Re: Adding libraries to an executable
On 2013-09-12 12:14, Anton Alexeev wrote: I've installed libcurl4-openssl-dev 7.29.0-1ubuntu3.1 before I asked for help here. Else I can't even compile with that command: dmd -L-lphobos2 -L-lcurl test.d How does your dmd.conf file look like? -- /Jacob Carlborg
Re: override and package
On Thursday, 12 September 2013 at 11:29:22 UTC, Jacob Carlborg
wrote:
On 2013-09-12 11:28, Namespace wrote:
But if I try to write 'override' before [1], I get this error
message:
Error: function T4._apply cannot override a non-virtual
function
This seems inconsistent. I really overwrite the method, and
then I put
it in a package label.
I think the error message is pretty clear. You cannot override
a function that isn't virtual. Private and package methods are
not virtual. Example:
class Base
{
void foo ()
{
writeln(Base.foo);
}
}
class Sub : Base
{
package void foo ()
{
writeln(Sub.foo);
}
}
void main ()
{
auto sub = new Sub;
sub.foo(); // prints Sub.foo as expected
Base base = sub;
base.foo(); // prints Base.foo
}
Obvious. But what happend? Is the original _apply hidden?
