Re: Dpaste - online compiler and collaboration tool dedicated to D Programming Language.
nazriel: I would like to share with you, Beta version of http://dpaste.dzfl.pl/ I have tried that page some more. Two screen grabs: http://derp.co.uk/35b8d 1) In the first part of the image the background colors are disabled. For such situations I suggest to make the green and red boxes with v and x to be visible with disabled colors too. 1b) I also suggest to write down segmentation fault somewhere, instead just setting it a tooltip of those boxes. 2) Second part of the image: why I am not seeing a link to create a new paste? 3) I have tried to register myself, but it always refuse my CAPTCHA :-( Bye, bearophile
Re: Dpaste - online compiler and collaboration tool dedicated to D Programming Language.
On 5 July 2012 14:47, bearophile bearophileh...@lycos.com wrote: nazriel: I would like to share with you, Beta version of http://dpaste.dzfl.pl/ I have tried that page some more. Two screen grabs: http://derp.co.uk/35b8d 1) In the first part of the image the background colors are disabled. For such situations I suggest to make the green and red boxes with v and x to be visible with disabled colors too. 1b) I also suggest to write down segmentation fault somewhere, instead just setting it a tooltip of those boxes. 2) Second part of the image: why I am not seeing a link to create a new paste? 3) I have tried to register myself, but it always refuse my CAPTCHA :-( Try logging in using OpenID. :~) -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Dpaste - online compiler and collaboration tool dedicated to D Programming Language.
On Thursday, 5 July 2012 at 13:47:42 UTC, bearophile wrote: nazriel: I would like to share with you, Beta version of http://dpaste.dzfl.pl/ I have tried that page some more. Two screen grabs: http://derp.co.uk/35b8d 1) In the first part of the image the background colors are disabled. For such situations I suggest to make the green and red boxes with v and x to be visible with disabled colors too. 1b) I also suggest to write down segmentation fault somewhere, instead just setting it a tooltip of those boxes. 2) Second part of the image: why I am not seeing a link to create a new paste? 3) I have tried to register myself, but it always refuse my CAPTCHA :-( Bye, bearophile Thank you bearophile for your testings! There is so much to do that I had no time yet to make some better tests in browsers like links, or with JavaScript disabled. 1) Yes, its good idea, I need to think about it. Its nice look vs utility on browsers without colors (how many people uses browsers without colors? :D) 1b) Hmm, maybe good idea, yea, will think about it too. 2) Dunno tbh, could you give me some details about your web-browser. Its looks like Bootstrap fail so its has to be really exotic browser. 3) It maybe problematic, I had to introduce some kind of anti-spam mechanism. You can sing in using Github, Google, Facebook, OpenID accounts if that will fail for you. Also there may be case, that anti-spam mechanism triggers while you send code snippet. I think that after all I should make it clear for users that Images are required for website full functionality. Thanks again! Will try to solve your problems tommorow. Best Regards, Damian Ziemba
Re: Dpaste - online compiler and collaboration tool dedicated to D Programming Language.
nazriel: Thank you bearophile for your testings! Thank you for the very elegant D paste site. I'm good at hitting ways to _not_ be able to use software and sites :o) Sometimes even people like Mister Beam are useful. In that site sometimes I'd like an way to compile with -d or -debug or -version, this is probably doable. Offering an option to see the produced assembly is less easy, because usually D produces very large asm listings... maybe this is doable showing only the asm of the single or few specific functions selected by the user, this gives a manageable sized output. There is so much to do that I had no time yet to make some better tests in browsers like links, or with JavaScript disabled. In all tests I've done I was keeping JavaScript enabled, and I think images too were enabled. I have just disabled colors in one test. (how many people uses browsers without colors? :D) I don't know. It's a standard feature of Firefox, well visible in the Options/Contents page. I sometimes disable colors when I program or write, to look for info faster, and cause less disruption to my flow. A friend of mine keeps them always disabled to reduce distractions. And I know another person that keeps them disabled because he's deeply color blind, and finds colors confusing (he sometimes removes part of the colors from presentations created by other people). 2) Dunno tbh, could you give me some details about your web-browser. Its looks like Bootstrap fail so its has to be really exotic browser. It's a normal Firefox 13.0.1, but it's not easy to give a complete answer to this question. Maybe we have to meet on IRC when you modify the site :-) Bye, bearophile
Re: dfmt - D source code formatter
On 7/4/2012 10:22 PM, Jonathan M Davis wrote: On Wednesday, July 04, 2012 21:46:29 Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? Doesn't that need a lexer and parser for D first (which I'd _love_ to do but just haven't had time to get around to)? Yes. Or it could be written based on the .di generation logic, i.e. as part of dmd.
Re: Proposal: takeFront and takeBack
On 07/05/2012 01:26 AM, Jonathan M Davis wrote: I wonder how the results look on gdc when using the improved popFront, given how surprising they were with consumeFront. Improved popFront, gdc as before (-frelease -finline-functions -fweb -O3): ascii 22.69%: old [2 secs, 449 ms, 248 μs, and 7 hnsecs], new [555 ms, 718 μs, and 3 hnsecs] uni 42.52%: old [3 secs, 879 ms, 380 μs, and 7 hnsecs], new [1 sec, 649 ms, 387 μs, and 4 hnsecs] For comparison, on the same machine with dmd -release -O -inline, ascii 118.23%: old [1 sec, 754 ms, 449 μs, and 2 hnsecs], new [2 secs, 74 ms, 197 μs, and 8 hnsecs] uni 86.82%: old [4 secs, 413 ms, 167 μs, and 8 hnsecs], new [3 secs, 831 ms, 569 μs, and 1 hnsec] (The machine is an old 32-bit Athlon XP 3000+ (Barton)) When gdc finishes building on my 64-bit box I can run timings on that, but it's a crappy underpowered emachine/acer, so any results may say less about any 32/64 bit differences and more about what happens when you buy a $250 computer at Walmart. --Ed
Re: dfmt - D source code formatter
On Thursday, 5 July 2012 at 06:27:55 UTC, Walter Bright wrote: On 7/4/2012 10:22 PM, Jonathan M Davis wrote: On Wednesday, July 04, 2012 21:46:29 Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? Doesn't that need a lexer and parser for D first (which I'd _love_ to do but just haven't had time to get around to)? Yes. Or it could be written based on the .di generation logic, i.e. as part of dmd. Personally I would rather see it as a separate tool. D already gets a bit of bad publicity for having the compiler do too much stuff that should be relegated to separate tools. This is again another use case that would benefit from the compiler as library.
Re: D in the cloud with cassandra ?
Hi guys. Thank for the comments. Nice to know that someone have had the D-Cassandra combo working. Knud
Re: Proposal: takeFront and takeBack
On 04-Jul-12 19:17, Andrei Alexandrescu wrote:
On 7/4/12 8:33 AM, deadalnix wrote:
Le 04/07/2012 13:04, Roman D. Boiko a écrit :
On Wednesday, 4 July 2012 at 10:54:41 UTC, deadalnix wrote:
I do agree with that. However, the current proposal have the same
drawback but on the code that use the range.
Both solutions are messy IMO.
What is error-prone in current client code?
If particular algorithm wants to take advantage of a potential speed-up,
it may decide to check whether hasConsume!Range, and call consumeFront
instead of front + popFront. Nobody is obliged to do so.
The only problem I can see is potential code duplication, which is lower
priority in performance-critical part of code, IMO.
You have to maintain 2 version of you algorithm. This is more work to
test, to maintain, and it is likely to introduce more bugs.
More code == more bugs. More NPATH = more bugs and harder to test and to
maintain.
In addition, this will result in practice in less generic code, because
one may not need the version of the algorithm without consume primitives
in his/her own code and not code it at all.
I agree. Can't get behind this proposal.
First, it solves the problem of double decoding for UTF-8 and UTF-16
strings. Before that, we need to estimate how bad the problem is and how
we can optimize front/popFront() to alleviate it. For all I can tell, in
ASCII-heavy strings we're looking at optimizing away ONE redundant
comparison in a sequence that involves a bounds check, two word
assignments, and another comparison.
It's about optimizing away the whole sequence. In current state it does
this twice in decode and in stride. (front calls decode that does shrink
temporary range)
Off the top of my head I don't
think the gains are going to be impressive.
I hate to say this but if we look at ASCII heavy then this is should be
a baseline to race against:
dchar decode(ref char[] a)
{
dchar ch = a[0];
a = a[1..$];
return ch;
}
Now it's abundantly clear that removing a comparison redundant
assignment is a big win, the extra ops per _iteration_ :
a = a[std.utf.stride(a, 0) .. $];
1) a = ... is redundant as decode is called in front
2) the following is redundant as we already know the length when do decode:
immutable c = str[index];
if (c 0x80)
return 1;
memory access, comparison, 2 word assignments and a bounds check.
And with DMD it may also miss inlining thus adding extra call to the table.
In theory all of these redundant operations can be done away with
optimization techniques, but probably some aren't. So we should look
first at optimizing this flow heavily before looking at an API addition
that has disadvantages in other dimensions.
It is optimized but there is a limit to where it can go. Previously (say
one year ago) it was far slower.
--
Dmitry Olshansky
Re: dfmt - D source code formatter
Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? I'm using uncrustify (http://uncrustify.sourceforge.net/). It does most of the time what I want. Jens
Re: D SFML2 derelict
compiled with: dmd hello.d rdmd hello.d
Re: A delegate problem, create delegation in loop
lijie:
import std.stdio;
void main() {
void delegate()[] functions;
foreach (i; 0 .. 5) {
functions ~= {
printf(%d\n, i);
};
}
foreach (func; functions) {
func();
}
}
import std.stdio;
void main() {
void delegate()[] functions;
foreach (i; 0 .. 5)
functions ~= ((int j) = { printf(%d\n, j); })(i);
foreach (func; functions)
func();
}
Bye,
bearophile
Re: More Front End Vector Support
On 4 July 2012 22:43, Walter Bright newshou...@digitalmars.com wrote: On 7/4/2012 7:19 AM, Iain Buclaw wrote: Morning, I've noticed that 256bit vector support has been added to the D frontend during the development stages of 2.060 whilst was looking around in druntime core. https://github.com/D-Programming-Language/druntime/commit/fcc91588e89fa48b699f0efe0cdfb8c23e2bb4ae Is anyone willing to object if I raise a pull request to add 64bit Vector support in the D frontend too for architectures that support? This includes i386/x86_64 using MMX extensions, and ARM using NEON extensions. Not sure how well DMD would cope with the type in it's backend though, but can always reject it in its backend with an error. 64 bit MMX extensions for x86 are, as far as I can tell, quite obsolete. The CPUs support those instructions for backwards compatibility, but I cannot see any reason for D to add new support for it. Fair enough. Only asked as if we do Y and Z, why not X? GCC backend already supported the use of __vector[N] sizes long before D support was added, but then again only less than of a handful of architectures actually __support__ vector operations (as I said, I think only MMX and NEON would benefit) - the rest just slowly emulate it. Regards -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: A delegate problem, create delegation in loop
05.07.2012 9:54, lijie пишет:
Hi,
My test code:
import std.stdio;
void main() {
void delegate()[] functions;
foreach (i; 0 .. 5) {
functions ~= {
printf(%d\n, i);
};
}
foreach (func; functions) {
func();
}
}
output:
5
5
5
5
5
This program behaves as expected. Just like C# program that will give
the same output:
---
delegate void MyFunc();
class Program
{
static void Main()
{
var funcs = new MyFunc[5];
for (int i = 0; i funcs.Length; ++i)
funcs[i] = new MyFunc(() = System.Console.WriteLine(i));
foreach (var f in funcs)
f();
}
}
---
because i is the same for every iteration. Different situation is for
such C# loop:
---
for (int i = 0; i funcs.Length; ++i)
{
int t = i;
funcs[i] = new MyFunc(() = System.Console.WriteLine(t));
}
---
where t is local for scope. Here C# behaves correctly, but D doesn't.
This D loop
---
foreach(i; 0 .. 5) {
int t = i;
functions ~= { printf(%d\n, t); };
}
---
prints 4 five times. It's Issue 2043:
http://d.puremagic.com/issues/show_bug.cgi?id=2043
I'm not posting workaround here because bearophile already did it.
--
Денис В. Шеломовский
Denis V. Shelomovskij
Re: Proposal: takeFront and takeBack
On Wednesday, 4 July 2012 at 22:02:28 UTC, deadalnix wrote: Le 04/07/2012 21:45, Jonathan M Davis a écrit : On Wednesday, July 04, 2012 12:19:15 Jonathan M Davis wrote: But at present, I'm seeing a performance improvement of approximately 70 - 80% in iterating over strings with consumeFront rather than front and popFront (depending on the compiler flags and strings used). I should clarify that. It's taking 70 - 80% as long to use consumeFront to iterate over a string than it is to iterate using popFront and getting front on every iteration. The way I worded that could imply that it was taking 20 - 30% as much time, which would be a _much_ larger improvement than I'm actually seeing. - Jonathan M Davis The win is significant indeed. I'm not sure. Let's speculate a bit with some almost random numbers. E.g., suppose that some test without user code in a loop takes 70 (or 80) ms (per iteration) instead of 100. Or, assuming that some optimization of front / popFront would make it proportionally twice faster, 35 instead of 50. It may be significant, if user code inside the loop is relatively fast, but that is not often the case. Let's assume that it takes 30 ms (that looks like pretty fast). So overall it would be (70 + 30) vs (100 + 30) before front / popFront optimization, and (35 + 30) vs (50 + 30) after (huge optimization). The latter is 65 vs 80, which is about 81 vs 100 (instead of 70 vs 100 before optimization). If user code was slower, impact of front / popFront optimization would be relatively larger, and vice verse. I make no conclusions, because have not run any benchmarks to estimate how complex the code should be to take those 30 ms. Such benchmarks would be valuable for discussion.
Re: Proposal: takeFront and takeBack
If you really don't need the value, you could devise a justPop method that does not return (by the way, overloading by return type would be an amazing feature here). The idea is not we should return a value everytime we pop, but we should pop when we return a value. -- Christophe
Re: Proposal: takeFront and takeBack
On Thursday, 5 July 2012 at 08:34:29 UTC, Roman D. Boiko wrote: I make no conclusions, because have not run any benchmarks to estimate how complex the code should be to take those 30 ms. Such benchmarks would be valuable for discussion. That has been written before I saw all benchmarks, so my post expired before being created.
Re: dfmt - D source code formatter
On Thursday, 5 July 2012 at 06:27:55 UTC, Walter Bright wrote: On 7/4/2012 10:22 PM, Jonathan M Davis wrote: On Wednesday, July 04, 2012 21:46:29 Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? Doesn't that need a lexer and parser for D first (which I'd _love_ to do but just haven't had time to get around to)? Yes. Or it could be written based on the .di generation logic, i.e. as part of dmd. Somehow my reply seems to have been lost. I find it nice, but would rather see it as a separate tool. This are the type of projects that would benefit from having the compiler available as library. -- Paulo
Re: dfmt - D source code formatter
On Thursday, July 05, 2012 11:11:28 Paulo Pinto wrote: This are the type of projects that would benefit from having the compiler available as library. It will be eventually, but someone (or several someones) will have to take the time to do it. Once I find the time, I intend to port the lexer (and later parser) from dmd's frontend to D, and I intended to have it done quite some time ago, but I haven't had the time, so it hasn't happened yet, and no one else has taken the time to do that (or if they have, they haven't finished the job). - Jonathan M Davis
Re: Proposal: takeFront and takeBack
On Wednesday, 4 July 2012 at 22:08:18 UTC, Jonathan M Davis wrote:
C++'s iterators definitely do _not_ return an element when you
increment them.
- Jonathan M Davis
Just a thought, but C++'s input iterator very often do increment
when being _dereferenced_, and then they make the _increment_
operator a no-op.
So technically, while they do not return a value when you
increment/popFront(), they do exactly the contrary: they
increment/popFront() when you dereference/front()
Maybe instead of trying to write a consumeFront function, it
would be easier/more manageable to enforce this semantic on input
ranges?
for example:
import std.utf;
import std.range;
import std.stdio;
import std.algorithm;
struct stringAsInputRange
{
dchar front()
{
size_t index = 0;
dchar c = decode(m_data, index);
m_data = m_data[index .. $];
return c;
}
void popFront() {}
typeof(this) Save(){return this;}
bool empty() {return m_data.empty;}
private: string m_data;
}
void main()
{
string s = héllö;
stringAsInputRange sir = stringAsInputRange(s);
foreach(c; sir)
writeln(c);
writeln(std.algorithm.count(sir, 'l'));
}
Voilà! As you can see, I just exploited efficient traversal of a
utf8 string, both with foreach and an algorithm, without them
being none the wiser ;)
The sweet part is that in theory, this can work with any
algorithm that is compatible with input ranges, and does not
mutate their data: The bulk of the algorithms that could use
consumeFront anyways...
Such algorithms that operate on input ranges *should* never call
front more than once per loop anyways.
For those few algorithms that work on bidirRange, we'd need a
garantee that they don't ever front/back the same item twice. We
*could* achieve this by defining a bidirectionalInputRange class
of range.
inputRange- *bidirectionalInputRange*
| |
v v
ForwardRange - bidirectionalRange
But this would be more work, just to get those last few cases...
And I'm not sure this diamond classification isn't dangerous
somehow...
Re: dfmt - D source code formatter
2012/7/5 Jens Mueller jens.k.muel...@gmx.de: Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? I'm using uncrustify (http://uncrustify.sourceforge.net/). It does most of the time what I want. Jens I'm using uncrustify too although it has the annoying habbit of rewriting = to = causing all functions using the new lambda syntax to break.
Re: Proposal: takeFront and takeBack
monarch_dodra , dans le message (digitalmars.D:171175), a écrit : For those few algorithms that work on bidirRange, we'd need a garantee that they don't ever front/back the same item twice. We *could* achieve this by defining a bidirectionalInputRange class of range. filter does that. If you want to call front only oncem you have to cache the results or... pop as you take the front value. popFrontN and drop will crash too.
Re: dfmt - D source code formatter
maarten van damme wrote: 2012/7/5 Jens Mueller jens.k.muel...@gmx.de: Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? I'm using uncrustify (http://uncrustify.sourceforge.net/). It does most of the time what I want. Jens I'm using uncrustify too although it has the annoying habbit of rewriting = to = causing all functions using the new lambda syntax to break. Have you tried adding an issue? I had different problems. Most where fixed. Jens
Re: dfmt - D source code formatter
On 2012-07-05 06:46, Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? It's a great idea but as others have said, I see no point in creating this until we have a D compiler available as a library. -- /Jacob Carlborg
Re: Proposal: takeFront and takeBack
Le 05/07/2012 01:06, Jonathan M Davis a écrit :
Another option would be to create a wrapper range for strings to be used when
they already have to be wrapped in another range. Functions which want to make
string processing more efficient can already specialize on strings (and often
do), so they can deal with the consumeFront issue directly if they need to
(though their code might be a bit more concise with it in some cases). The
problem is when a string is wrapped in another range (e.g the result of map or
filter), you can't reasonably specialize on them anymore, and you're forced to
deal with the inefficiency of using both front and popFront.
So, if we created a wrapper range whose front caches the index of the next
code unit for a subsequent call to popFront, and functions such as filter and
map wrapped strings in that before wrapping them in their own range types,
then the wrapped calls to popFront will be more efficient (at least in the case
when front was called - the call to popFront would be slightly less efficient in
the cases where front wasn't used). So, we could have something like this:
struct StringCache(C)
if(is(Unqual!C == char) || is(Unqual!C == wchar))
{
public:
this(C[] str)
{
_str = str;
}
@property bool empty() { return _str.empty; }
@property dchar front()
{
import std.utf;
_nextIndex = 0;
return decode(_str, _nextIndex);
}
void popFront()
{
if(_nextIndex)
{
_str = _str[_nextIndex .. $];
_nextIndex = 0;
}
else
_str.popFront();
}
private:
C[] _str;
size_t _nextIndex = 0;
}
auto stringCache(C)(C[] str)
if(isSomeChar!C)
{
static if(is(Unqual!C == dchar))
return str;
else
return StringCache!C(str);
}
Obviously, it would need the range functions for forward range and
bidirectional range added, but that gives the basic idea. And of course a name
other than StringCache could be used (it's not great, but it's the best that I
could come up with in the few minutes that I thought about it and the name
isn't really the main issue here - the design is).
And the performance in comparison to consumeFront is interesting. Without any
optimizations turned on (not even -release), consumeFront takes about 72 - 79%
as long as front and popFront separately, whereas StringCache takes about 107
- 120% as long as front and popFront (so it's actually slower without any
optimizations, unlike consumeFront). But with all optimizations on (-release -
O -inline), consumeFront takes about 66 - 80% as long, and StringCache takes
69 - 75% as long. So, with optimizations, StringCache is roughly the same as
consumeFront. The times are rough and the various runs of the same test vary a
bit, so clearly stuff like the CPU cache and context switching are having an
impact, making exact comparisons difficult, but I think that StringCache is fast
enough to be comparable to consumeFront in speed, and it's a less invasive
change to start using StringCache than it would be to use hasConsume and
consumeFront. And actually, if stringCache were changed to operate on _any_
range and then just wrap arrays of char[] and wchar[], then it would be even
_less_ invasive, because it wouldn't even require special casing on the part
of the caller. For instance, filter's return statement could go from
return FilteredRange(rs);
to
return FilteredRange(stringCache(rs));
and suddenly you get the optimization for anything using filter (though such
usage definitely seems to indicate that a better name than stringCache should
be found).
Clearly we need to look at improving the performance of front and popFront as
much as possible (and the StringCache implementation would benefit from those
as well), but given the disgreements over consumeFront, it would seem that
StringCache would be a better solution (certainly, it would directly impact
less code), and unless front and popFront can be optimized to the point that
StringCache adds no real measurable benefit (which I doubt will happen), I
think that it would be worthwhile to add StringCache and have functions like
filter and map start using it.
I guess that the first thing that we need to do though is look at what can be
done to improve the performance of front and popFront - the first thing
probably being optimizations relating to the fact that they're always
operating from index 0.
Still, what do you folks think of this StringCache idea? Is it more acceptable
than consumeFront, or do some of you still think that it's not worth having?
- Jonathan M Davis
This is a way better idea IMO.
Let's stop parser Hell
There are more and more projects requiring parsing D code (IDE plugins, DCT and dfmt now). We definitely need a _standard_ fast D parser (suitable as tokenizer). We already have a good one at least in Visual D. Maybe dmd's parser is faster. If so, it can be (easily?) rewritten in D. We also already have some other parsers (in C# from Mono-D etc.). What about to get one and call it standard? -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 12:11:33 UTC, Denis Shelomovskij wrote: There are more and more projects requiring parsing D code (IDE plugins, DCT and dfmt now). We definitely need a _standard_ fast D parser (suitable as tokenizer). We already have a good one at least in Visual D. Maybe dmd's parser is faster. If so, it can be (easily?) rewritten in D. We also already have some other parsers (in C# from Mono-D etc.). What about to get one and call it standard? Visual-D is not Boost-licensed (I think this would be possible to change) Mono-D is written in C#, as you mentioned Pegged may eventually become standard, if it will be performance optimized and a bit more customizable Dscanner(https://github.com/Hackerpilot/Dscanner) from Brian Schott is pretty good, too. SDC is another nice option DIL (http://code.google.com/p/dil/) is very nice but GPL I plan to try using Pegged inside my DCT project. Probably that will require huge modifications though... Some more links from Pegged readme: Hisayuki Mima's CTPG(https://github.com/youkei/ctpg), very similar, also done in D. Have a look! Nick Sabalausky's Goldie (http://www.dsource.org/projects/goldie). Benjamin Shropshire's dparser (http://dsource.org/projects/scrapple/browser/trunk/dparser). Martin Nowak put these gists on the D newsgroup: https://gist.github.com/1255439 - lexer generator https://gist.github.com/1262321 - complete and fast D lexer
Re: Let's stop parser Hell
Forgot to add DDMD, which also has been forked and redesigned recently by someone (thus two more different compiler frontends). But overall I doubt that any project could become standard very soon.
Re: Let's stop parser Hell
On 05-Jul-12 16:11, Denis Shelomovskij wrote: There are more and more projects requiring parsing D code (IDE plugins, DCT and dfmt now). We definitely need a _standard_ fast D parser (suitable as tokenizer). Then do it. It's all about having something so obviously good, fast and flexible that other stuff can be considered only as toys. I might do one, but I'd rather just help other folks make it faster ;) -- Dmitry Olshansky
Re: Proposal: takeFront and takeBack
On 7/5/12 1:26 AM, Jonathan M Davis wrote: In any case, I guess that this shows that what we can get with popFront is so close to what consumeFront or StringCache would do that we might as well not bother with them, which is a _big_ surpise to me. It does pay to benchmark code though. Good point. I think predicting what optimization does is an area in which I (probably with others too) have remained staunchly incompetent over years. Andrei
Re: Let's stop parser Hell
05.07.2012 16:30, Roman D. Boiko пишет: Forgot to add DDMD, which also has been forked and redesigned recently by someone (thus two more different compiler frontends). But overall I doubt that any project could become standard very soon. Why? Even were they all almost equal we can select any one. The point is to select something (anything) to guide a coder who want to do something with this task to a good/up-to-date/supported parser. -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote: Then do it. It's all about having something so obviously good, fast and flexible that other stuff can be considered only as toys. I might do one, but I'd rather just help other folks make it faster ;) Would you help to make Pegged faster?
Re: Let's stop parser Hell
On 05-Jul-12 17:04, Roman D. Boiko wrote: On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote: Then do it. It's all about having something so obviously good, fast and flexible that other stuff can be considered only as toys. I might do one, but I'd rather just help other folks make it faster ;) Would you help to make Pegged faster? Well why not. But first I'll need to deliver some stuff on my GSOC project. -- Dmitry Olshansky
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 12:53:02 UTC, Denis Shelomovskij wrote: 05.07.2012 16:30, Roman D. Boiko пишет: Forgot to add DDMD, which also has been forked and redesigned recently by someone (thus two more different compiler frontends). But overall I doubt that any project could become standard very soon. Why? Even were they all almost equal we can select any one. The point is to select something (anything) to guide a coder who want to do something with this task to a good/up-to-date/supported parser. Well, I guess we'd like to select one and not change it later, don't we? I'm not sure we can decide which is the best currently and will stay the best in the future for all major use cases. Anyway I propose to enumerate major use cases first.
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 13:05:41 UTC, Dmitry Olshansky wrote: On 05-Jul-12 17:04, Roman D. Boiko wrote: Well why not. But first I'll need to deliver some stuff on my GSOC project. I bet that after you finish with GSOC optimizing Pegged will not be less relevant than it is now :) And as for DCT, it will take another half a year (at least) until it will become usable for any real needs (except the most trivial).
Re: Let's stop parser Hell
On 7/5/12 9:05 AM, Dmitry Olshansky wrote: On 05-Jul-12 17:04, Roman D. Boiko wrote: On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote: Then do it. It's all about having something so obviously good, fast and flexible that other stuff can be considered only as toys. I might do one, but I'd rather just help other folks make it faster ;) Would you help to make Pegged faster? Well why not. But first I'll need to deliver some stuff on my GSOC project. Good call :o). Note that we can discuss tweaking the scope of the project after the midterm. Ping me if you have some ideas. Andrei
Re: Let's stop parser Hell
Is the actual grammar available somewhere? The online language reference is all we got I guess? DMD doesn't seem to be using yacc/bison either. On Thu, Jul 5, 2012 at 7:11 AM, Denis Shelomovskij verylonglogin@gmail.com wrote: There are more and more projects requiring parsing D code (IDE plugins, DCT and dfmt now). We definitely need a _standard_ fast D parser (suitable as tokenizer). We already have a good one at least in Visual D. Maybe dmd's parser is faster. If so, it can be (easily?) rewritten in D. We also already have some other parsers (in C# from Mono-D etc.). What about to get one and call it standard? -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Let's stop parser Hell
On 2012-07-05 15:08, Roman D. Boiko wrote: Anyway I propose to enumerate major use cases first. Haven't we already done that a couple of times. -- /Jacob Carlborg
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 15:40:53 UTC, Jacob Carlborg wrote: On 2012-07-05 15:08, Roman D. Boiko wrote: Anyway I propose to enumerate major use cases first. Haven't we already done that a couple of times. Well, we did something like that for DCT... but I doubt that it would fit general needs. If we had, why haven't they been analyzed, classified, discussed, etc.? Or have they?
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 15:42:22 UTC, Caligo wrote: Is the actual grammar available somewhere? The online language reference is all we got I guess? DMD doesn't seem to be using yacc/bison either. In PEG format, yes (not necessarily correct): https://github.com/roman-d-boiko/Pegged/blob/dct/pegged/examples/dgrammar.d I don't know about any others, though.
Re: Let's stop parser Hell
On 2012-07-05 18:04, Roman D. Boiko wrote: Well, we did something like that for DCT... but I doubt that it would fit general needs. Why wouldn't it. If we had, why haven't they been analyzed, classified, discussed, etc.? Or have they? I don't know. Here is what I wrote for DCT: * IDE integration * Refactoring tool * Static analysis * Compiler * Doc generating * Build tool * DI generating In general, use cases that can span several compile phases, i.e. lexing, parsing, semantic analysis and so on. Some of these use cases can be broken in to several new use cases at a lower level. Some examples: IDE integration: * Syntax highlighting * Code completion * Showing lex, syntax and semantic errors * Formatter Refactoring: * Cross-referencing symbols * Renaming of symbols * Extract local variable to instance variable * Extract variable to function/method * Extract a piece of code/method into a new class Build tool: * Tracking module dependencies Doc generating: * Associate a declaration and its documentation Original post: http://www.digitalmars.com/d/archives/digitalmars/D/DCT_use_cases_-_draft_168106.html#N168141 -- /Jacob Carlborg
Re: Let's stop parser Hell
On 05-Jul-12 18:29, Andrei Alexandrescu wrote: On 7/5/12 9:05 AM, Dmitry Olshansky wrote: On 05-Jul-12 17:04, Roman D. Boiko wrote: On Thursday, 5 July 2012 at 12:32:19 UTC, Dmitry Olshansky wrote: Then do it. It's all about having something so obviously good, fast and flexible that other stuff can be considered only as toys. I might do one, but I'd rather just help other folks make it faster ;) Would you help to make Pegged faster? Well why not. But first I'll need to deliver some stuff on my GSOC project. Good call :o). Note that we can discuss tweaking the scope of the project after the midterm. Ping me if you have some ideas. It's great that you are not opposed to adjusting scope of project. I have a ton of ideas, but let's discuss them after midterm. -- Dmitry Olshansky
Re: Let's stop parser Hell
On Thursday, July 05, 2012 18:04:11 Roman D. Boiko wrote: On Thursday, 5 July 2012 at 15:40:53 UTC, Jacob Carlborg wrote: On 2012-07-05 15:08, Roman D. Boiko wrote: Anyway I propose to enumerate major use cases first. Haven't we already done that a couple of times. Well, we did something like that for DCT... but I doubt that it would fit general needs. If we had, why haven't they been analyzed, classified, discussed, etc.? Or have they? It's been discussed before, but there are some obvious use cases such as syntax highlighting, code formatting, and manipulation of D source files (e.g. to strip out the unit tests). We need to have a lexer in Phobos which parsers D code and generates a range of tokens, and we need to have a parser in Phobos which operates on a range of tokens. As discussed previously, there is desire to have the lexer and parser ported from dmd's frontend to D for Phobos so that what we get is easily maintainable alongside dmd's frontend and produces the same results (and is fast). It's _also_ desirable that we get a lexer/parser generator into Phobos for generating lexers/parsers for whatever language you might want to generate them for. Pegged is a good example of what can be done, and I think that Andrei was trying to get Philippe to prepare a submission to Phobos from it (though I'm not sure), but regarldess of whether pegged (or anything based on it) makes it into Phobos, we definitely want something similar. So, we have a fair idea of some of the stuff that we want, but it's a question of time and effort. I keep intending to port dmd's lexer to D for submission to Phobos, but I've been too busy to do it. At least a couple of other people have also expressed interest in doing it, but no one has submitted anything for Phobos. So, it remains undone, and anything which would need to lex or parse D code has to find its own solution. As with most everything around here, it's a question of people being having the time and being willing to put in the effort to do it. It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos. - Jonathan M Davis
Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! Andrei
Re: Proposal: takeFront and takeBack
On 07/05/2012 02:35 AM, Ed McCardell wrote: When gdc finishes building on my 64-bit box I can run timings on that, There also seems to be a speed improvement for consumeFront on 64-bit gdc, with both standard and Andrei's improved popFront: standard popfront: ascii 35.95%: old [3 secs, 831 ms, 261 μs, and 5 hnsecs], new [1 sec, 377 ms, 299 μs, and 7 hnsecs] uni 49.48%: old [4 secs, 860 ms, 874 μs, and 7 hnsecs], new [2 secs, 405 ms, 372 μs, and 7 hnsecs] improved popfront: ascii 62.74%: old [2 secs, 388 ms, 776 μs, and 5 hnsecs], new [1 sec, 498 ms, 736 μs, and 2 hnsecs] uni 61.26%: old [4 secs, 58 ms, 522 μs, and 5 hnsecs], new [2 secs, 486 ms, 451 μs, and 7 hnsecs] --Ed
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 16:14:27 UTC, Jacob Carlborg wrote: Original post: http://www.digitalmars.com/d/archives/digitalmars/D/DCT_use_cases_-_draft_168106.html#N168141 OK, fairly complete. Let it be the starting point. Why not put it on some wiki, then add some more, discuss, vote, etc.? Meanwhile create a matrix of features implemented in each of interesting standardization candidates. And pick up one of them as standard or reference parser. My vote would be for Pegged, I guess.
Re: dfmt - D source code formatter
On 2012-07-05 06:46, Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? I just tried ddmd-clean and it already does some form of source code formatting. It will format the following file: https://github.com/zachthemystic/ddmd-clean/blob/master/main.d To something like this: http://pastebin.com/JUauQDvb https://github.com/zachthemystic/ddmd-clean -- /Jacob Carlborg
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On 2012-07-05 18:26, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! Andrei That is really cool. -- /Jacob Carlborg
Re: Let's stop parser Hell
On 2012-07-05 18:32, Roman D. Boiko wrote: My vote would be for Pegged, I guess. Aren't you voting on your own project :) -- /Jacob Carlborg
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote: It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos. - Jonathan M Davis Resume: everybody is welcome to join effort of translating DMD front end, and improving Pegged. Also I would like to invite those interested in DCT project to help me with it. Right now I'm trying to understand whether it is possible to incorporate Pegged inside without losing anything critical (and I think it is very likely possible), and how exactly to do that. Dmitry proposed to help improve Pegged (or some other compiler's) speed. Anyone else?
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 16:38:27 UTC, Jacob Carlborg wrote: On 2012-07-05 18:32, Roman D. Boiko wrote: My vote would be for Pegged, I guess. Aren't you voting on your own project :) Well, I'm going to base parsing on Pegged, after tweaking it to my needs.
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On Thu, Jul 05, 2012 at 12:26:01PM -0400, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! [...] Won't that be open to abuse? Like if somebody wrote a fork bomb and tried to run it... Unless the backend server has tight resource control over the code sample executor, of course T -- Nothing in the world is more distasteful to a man than to take the path that leads to himself. -- Herman Hesse
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On 5 July 2012 17:51, H. S. Teoh hst...@quickfur.ath.cx wrote: On Thu, Jul 05, 2012 at 12:26:01PM -0400, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! [...] Won't that be open to abuse? Like if somebody wrote a fork bomb and tried to run it... Unless the backend server has tight resource control over the code sample executor, of course If it's using the same engine as dpaste ( http://dpaste.dzfl.pl ), then it is fairly locked down. -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On Thu, 05 Jul 2012 09:51:50 -0700, H. S. Teoh wrote: On Thu, Jul 05, 2012 at 12:26:01PM -0400, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! [...] Won't that be open to abuse? Like if somebody wrote a fork bomb and tried to run it... Unless the backend server has tight resource control over the code sample executor, of course T It appears to at the very least strip out calls to shell(). I just tried adding: writeln(shell(whoami)); and just got a blank line.
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
Great tool! Just a small layout bug: On Firefox 3.6.4 (on Mac) the [your code here] tags is misplaced after clicking the Run button. It then overlaps the appearing output box. Cheers, André On Thursday, 5 July 2012 at 16:59:33 UTC, Iain Buclaw wrote: On 5 July 2012 17:51, H. S. Teoh hst...@quickfur.ath.cx wrote: On Thu, Jul 05, 2012 at 12:26:01PM -0400, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! [...] Won't that be open to abuse? Like if somebody wrote a fork bomb and tried to run it... Unless the backend server has tight resource control over the code sample executor, of course If it's using the same engine as dpaste ( http://dpaste.dzfl.pl ), then it is fairly locked down.
Re: dfmt - D source code formatter
On Thursday, 5 July 2012 at 04:47:29 UTC, Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? I'm already working on adding formatting to my general-purpose D tool. https://github.com/Hackerpilot/Dscanner
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On Thursday, 5 July 2012 at 16:26:02 UTC, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! Andrei Great!
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On 05-Jul-12 20:26, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! Wonderful! It's fast and fluid, looks good. Still I would request adding interactive console input. Some magic with WebSockets some server daemon on worker machines should do the trick. And being able to run for some time if network client is active. Browsers without WebSockets can just use non-interactive input with some text area which contents are fed to the program. -- Dmitry Olshansky
Re: Let's stop parser Hell
05.07.2012 20:14, Jacob Carlborg пишет: On 2012-07-05 18:04, Roman D. Boiko wrote: Well, we did something like that for DCT... but I doubt that it would fit general needs. Why wouldn't it. If we had, why haven't they been analyzed, classified, discussed, etc.? Or have they? I don't know. Here is what I wrote for DCT: * IDE integration * Refactoring tool * Static analysis * Compiler * Doc generating * Build tool * DI generating Just want to mention I'm talking about parser only. Things like Refactoring have to work perfectly or are unusable so it require a full compiler at least as finished as current dmd. -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Let's stop parser Hell
05.07.2012 20:28, Jonathan M Davis пишет: It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos. I didn't want for someone to code anything at all! I on the contrary want them to stop writing parsers because it results in the only consequence: one have to spend more time to find a better parser. -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Let's stop parser Hell
On 2012-07-05 18:32, Roman D. Boiko wrote: My vote would be for Pegged, I guess. As much as I'm flattered by that, my current impression is Pegged is very far from being performant. As a proof-of-concept that, in D, it's possible to parse a string and create a parse tree at compile-time and then generate code from this, it's also successful. Go D! As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn by coding. Hey, I just received the Dragon Book (International Edition), I'm sure I'll learn many things in there. So, if anyone is willing to change the code generated by Pegged, I'm game. The results you showed me on keyword parsing are very interesting! But, my impression is that the need for a 'D'-only parser and lexer is far greater and much more imediate that the need for a parser generator. All the reasons advanced upthread ask for a D parser, not a generic generator. Parser generators are for those of us interested in having DSLs or macros in D. So Pegged or any other generator should *not* get the community focus right now. My plan would be as follow: 1- assemble a group of people knowing parsing. I don't think I'm exactly knowledgeable, but I'm ready to be part of such a group. 2- create a github process. 3- translate an existing parser / adapt a D parser for Phobos. I'm ready to be part of this (I'm sure I'll learn in the process) 4- spend 1-2 years fighting over LR parsing and such :) (Just kidding) 5- submit it to Phobos and have it adopted. much later: 6- see the way the parser code is organized and tweak a code generator (Pegged is a possibility if recursive parsing is OK) to produce an equivalent code when fed the D grammar. side-effect: maybe a std.tree or std.collection.tree to deal with trees in a generic way. Philippe
Re: Let's stop parser Hell
On Thursday, July 05, 2012 22:23:00 Denis Shelomovskij wrote: 05.07.2012 20:28, Jonathan M Davis пишет: It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos. I didn't want for someone to code anything at all! I on the contrary want them to stop writing parsers because it results in the only consequence: one have to spend more time to find a better parser. Well, until a lexer and parser for D make it into Phobos, more people are going to keep writing them, and even if/when they _do_ make it into Phobos, people will keep writing them, because some people like to write that kind of thing. Honestly though, if your complaint is that there's too much choice, I don't have much sympathy for you. In general, we have too little D code out there, not too much. If there's a problem with more parsers being written, I think that it's almost purely an issue of some people's time probably being better spent on other projects, but it's their right to work on whatever they feel like working on. - Jonathan M Davis
Re: Let's stop parser Hell
On 7/5/12 12:39 PM, Roman D. Boiko wrote: On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote: It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos. - Jonathan M Davis Resume: everybody is welcome to join effort of translating DMD front end, and improving Pegged. Also I would like to invite those interested in DCT project to help me with it. Right now I'm trying to understand whether it is possible to incorporate Pegged inside without losing anything critical (and I think it is very likely possible), and how exactly to do that. Dmitry proposed to help improve Pegged (or some other compiler's) speed. Anyone else? I'd really want to create a task force on this, it is of strategic importance to D. In Walter's own words, no new feature is going to push us forward since we're not really using the great goodies we've got, and CTFE technology is the most important. I also am actively opposed to a project of just translating D's front-end to D and dropping it into Phobos because it would smother (a) work on generic parser generators, and (b) strong, dependable formalization of D's syntax. Andrei
Re: Let's stop parser Hell
On 7/5/12 2:16 PM, Philippe Sigaud wrote: As much as I'm flattered by that, my current impression is Pegged is very far from being performant. As a proof-of-concept that, in D, it's possible to parse a string and create a parse tree at compile-time and then generate code from this, it's also successful. Go D! As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn by coding. Hey, I just received the Dragon Book (International Edition), I'm sure I'll learn many things in there. I'll be glad to buy for you any book you might feel you need for this. Again, there are few things more important for D right now than exploiting its unmatched-by-competition features to great ends. I don't want the lack of educational material to hold you down. Please continue working on this and let me know of what you need. So, if anyone is willing to change the code generated by Pegged, I'm game. The results you showed me on keyword parsing are very interesting! But, my impression is that the need for a 'D'-only parser and lexer is far greater and much more imediate that the need for a parser generator. I very strongly disagree. This is our chance to get things right instead of having a limited product that destroys competition (much like lex and yacc have been for years in the parser generator field). All the reasons advanced upthread ask for a D parser, not a generic generator. Parser generators are for those of us interested in having DSLs or macros in D. So Pegged or any other generator should *not* get the community focus right now. Pegged should be the focus. My plan would be as follow: 1- assemble a group of people knowing parsing. I don't think I'm exactly knowledgeable, but I'm ready to be part of such a group. 2- create a github process. 3- translate an existing parser / adapt a D parser for Phobos. I'm ready to be part of this (I'm sure I'll learn in the process) 4- spend 1-2 years fighting over LR parsing and such :) (Just kidding) 5- submit it to Phobos and have it adopted. Sounds good. Replace 1-2 years with 1-2 months. Andrei
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 18:17:06 UTC, Philippe Sigaud wrote: On 2012-07-05 18:32, Roman D. Boiko wrote: My vote would be for Pegged, I guess. As much as I'm flattered by that, my current impression is Pegged is very far from being performant. As a proof-of-concept that, in D, it's possible to parse a string and create a parse tree at compile-time and then generate code from this, it's also successful. Go D! As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn by coding. Hey, I just received the Dragon Book (International Edition), I'm sure I'll learn many things in there. So, if anyone is willing to change the code generated by Pegged, I'm game. The results you showed me on keyword parsing are very interesting! But, my impression is that the need for a 'D'-only parser and lexer is far greater and much more imediate that the need for a parser generator. All the reasons advanced upthread ask for a D parser, not a generic generator. Parser generators are for those of us interested in having DSLs or macros in D. So Pegged or any other generator should *not* get the community focus right now. I'm sure it can generate **much** faster code. I'm going to focus on its part that generates D parser (i.e., to make it significantly faster and able to efficiently parse-as-you-type). Actually, I'm sure it will be able to beat any other parser with respect to performance. :) 1. So my plan is the following: invite whoever would want to help. 2. Prove my claims above in practice. :-)
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 18:28:21 UTC, Andrei Alexandrescu wrote: On 7/5/12 2:16 PM, Philippe Sigaud wrote: So Pegged or any other generator should *not* get the community focus right now. Pegged should be the focus. +10 (can I vote ten times?) My plan would be as follow: 1- assemble a group of people knowing parsing. I don't think I'm exactly knowledgeable, but I'm ready to be part of such a group. 2- create a github process. 3- translate an existing parser / adapt a D parser for Phobos. I'm ready to be part of this (I'm sure I'll learn in the process) 4- spend 1-2 years fighting over LR parsing and such :) (Just kidding) 5- submit it to Phobos and have it adopted. Sounds good. Replace 1-2 years with 1-2 months. Well, probably with 3-4 months... :)
Re: Let's stop parser Hell
On 05-Jul-12 22:23, Denis Shelomovskij wrote: 05.07.2012 20:28, Jonathan M Davis пишет: It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos. I didn't want for someone to code anything at all! I on the contrary want them to stop writing parsers because it results in the only consequence: one have to spend more time to find a better parser. It doesn't work like that in OpenSource. No matter what you do people keep writing code ;) -- Dmitry Olshansky
Re: Let's stop parser Hell
On 05-Jul-12 22:22, Andrei Alexandrescu wrote: On 7/5/12 12:39 PM, Roman D. Boiko wrote: On Thursday, 5 July 2012 at 16:28:57 UTC, Jonathan M Davis wrote: It's all too common for someone to suggest that we should do something or implement something without ever attempting to do it themselves, and in general, stuff around here gets done because someone really wants it done, takes the time to do it, and sees it through until its done and in Phobos. - Jonathan M Davis Resume: everybody is welcome to join effort of translating DMD front end, and improving Pegged. Also I would like to invite those interested in DCT project to help me with it. Right now I'm trying to understand whether it is possible to incorporate Pegged inside without losing anything critical (and I think it is very likely possible), and how exactly to do that. Dmitry proposed to help improve Pegged (or some other compiler's) speed. Anyone else? I'd really want to create a task force on this, it is of strategic importance to D. In Walter's own words, no new feature is going to push us forward since we're not really using the great goodies we've got, and CTFE technology is the most important. Count me as interested. CTFE needs more correctness speed though. So to put it blantly - no it's not possible right NOW. BUT it doesn't prevent us from planing and doing a proof of concept. Pegged seems a good starting point even if we end up re-writing it from scratch. I also am actively opposed to a project of just translating D's front-end to D and dropping it into Phobos because it would smother (a) work on generic parser generators, and (b) strong, dependable formalization of D's syntax. Well put. It shouldn't stop people from doing parsers, IMO the more the merrier. -- Dmitry Olshansky
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 18:33:50 UTC, Dmitry Olshansky wrote: Count me as interested. CTFE needs more correctness speed though. So to put it blantly - no it's not possible right NOW. BUT it doesn't prevent us from planing and doing a proof of concept. Pegged seems a good starting point even if we end up re-writing it from scratch. IMO, first priority is to make code generated by Pegged work fast, and second priority - make code generation itself fast.
Re: Let's stop parser Hell
On 7/5/12 2:38 PM, Roman D. Boiko wrote: On Thursday, 5 July 2012 at 18:33:50 UTC, Dmitry Olshansky wrote: Count me as interested. CTFE needs more correctness speed though. So to put it blantly - no it's not possible right NOW. BUT it doesn't prevent us from planing and doing a proof of concept. Pegged seems a good starting point even if we end up re-writing it from scratch. IMO, first priority is to make code generated by Pegged work fast, and second priority - make code generation itself fast. Well I'd say there are things like supporting large, realistic grammars without blowing up or taking long compilation times is the first priority. Andrei
Re: Let's stop parser Hell
On 7/5/12 2:33 PM, Dmitry Olshansky wrote: On 05-Jul-12 22:22, Andrei Alexandrescu wrote: I'd really want to create a task force on this, it is of strategic importance to D. In Walter's own words, no new feature is going to push us forward since we're not really using the great goodies we've got, and CTFE technology is the most important. Count me as interested. CTFE needs more correctness speed though. So to put it blantly - no it's not possible right NOW. BUT it doesn't prevent us from planing and doing a proof of concept. Pegged seems a good starting point even if we end up re-writing it from scratch. Excellent point. Walter is 100% behind the general strategy and we can bring him to work on specific bug reports and enhancement requests if we make a case they are important for Pegged. I also am actively opposed to a project of just translating D's front-end to D and dropping it into Phobos because it would smother (a) work on generic parser generators, and (b) strong, dependable formalization of D's syntax. Well put. It shouldn't stop people from doing parsers, IMO the more the merrier. Exactly. Andrei
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
Nice. Should probably remove the references to local files when compilation fails. Not very user friendly to see: /home/jail/compileme369.d(14): expression expected, not '}' Would probably suffice just to switch the filename with something less distracting.
Re: dfmt - D source code formatter
On 7/4/2012 9:46 PM, Walter Bright wrote: It would be nice to have a D source code formatter. But it needs a champion. Who's up for it? I think that formatting the code is actually rather easy - the hard part will be dealing with the comments in a reasonable way.
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
2012/7/5 André nos...@spambog.com: Great tool! Just a small layout bug: On Firefox 3.6.4 (on Mac) the [your code here] tags is misplaced after clicking the Run button. It then overlaps the appearing output box. Cheers, André same bug with chrome
Re: Let's stop parser Hell
On Thu, Jul 5, 2012 at 8:28 PM, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: I'll be glad to buy for you any book you might feel you need for this. Again, there are few things more important for D right now than exploiting its unmatched-by-competition features to great ends. I don't want the lack of educational material to hold you down. Please continue working on this and let me know of what you need. That's nice of you, if a bit extreme for a public mailing list :) Andrei, money is no problem :) Interest in the field of parsing is no problem. Interest in D future is no problem. Having a demanding job, and three children, is a problem. No, scratch that, you know what I mean. But hey, Roman is doing interesting things on keyword parsing right now, and changing the parser generated by Pegged is not difficult. We will see where this thread lead. (Roman, you should send your results here, because I'm still taken aback by the built-in AA speed compared to linear array look-up for 100 keywords). As Dmitry said, we might hit a CTFE wall: memory consumption, computation speed, ... (*channelling Andrei*: then we will correct whatever makes CTFE a problem. Right) Philippe (Hesitating between 'The Art of the Metaobject Protocol' and 'Compilers, Techniques and Tools', right now)
Re: Let's stop parser Hell
On 2012-07-05 20:22, Andrei Alexandrescu wrote: I also am actively opposed to a project of just translating D's front-end to D and dropping it into Phobos because it would smother (a) work on generic parser generators, and (b) strong, dependable formalization of D's syntax. I don't see why you are against this. I'm seeing this as basically the only change for DMD every being written in D. Sure I would much rather have a completely new compiler written in D, with a module approach and designed from scratch to be used as a library, but I'm trying to realistic here. -- /Jacob Carlborg
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 18:17:06 UTC, Philippe Sigaud wrote: As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn by coding. Hey, I just received the Dragon Book (International Edition), If you are interested in parsing, than I wouldn't recommend the dragon book, but this one http://dickgrune.com/Books/PTAPG_2nd_Edition/Additional.html It really is an awesome book.
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 19:54:39 UTC, Philippe Sigaud wrote: On Thu, Jul 5, 2012 at 8:28 PM, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: I'll be glad to buy for you any book you might feel you need for this. Again, there are few things more important for D right now than exploiting its unmatched-by-competition features to great ends. I don't want the lack of educational material to hold you down. Please continue working on this and let me know of what you need. That's nice of you, if a bit extreme for a public mailing list :) Andrei, money is no problem :) Interest in the field of parsing is no problem. Interest in D future is no problem. Having a demanding job, and three children, is a problem. No, scratch that, you know what I mean. I have four, from 1 to 7 years old... Wouldn't call them a problem, though :))) But hey, Roman is doing interesting things on keyword parsing right now, and changing the parser generated by Pegged is not difficult. We will see where this thread lead. (Roman, you should send your results here, because I'm still taken aback by the built-in AA speed compared to linear array look-up for 100 keywords). Well, I wouldn't call those results yet. Just some benchmarks. Here they are: isKeyword_Dummy (baseline): 427 [microsec] total, 28 [nanosec / lookup]. isKeyword_Dictionary: 1190 [microsec] total, 214 [nanosec / lookup]. isKeyword_SwitchByLengthThenByChar: 466 [microsec] total, 83 [nanosec / lookup]. isKeyword_BinaryArrayLookup: 2722 [microsec] total, 490 [nanosec / lookup]. isKeyword_LinearArrayLookup: 13822 [microsec] total, 2490 [nanosec / lookup]. isKeyword_UnicodeTrie: 1317 [microsec] total, 237 [nanosec / lookup]. isKeyword_UnicodeTrieBoolLookup: 1072 [microsec] total, 193 [nanosec / lookup]. Total: 22949 identifiers + 5551 keywords. isKeyword_Dummy (baseline): 2738 [microsec] total, 50 [nanosec / lookup]. isKeyword_Dictionary: 4247 [microsec] total, 242 [nanosec / lookup]. isKeyword_SwitchByLengthThenByChar: 1593 [microsec] total, 91 [nanosec / lookup]. isKeyword_BinaryArrayLookup: 14351 [microsec] total, 820 [nanosec / lookup]. isKeyword_LinearArrayLookup: 59564 [microsec] total, 3405 [nanosec / lookup]. isKeyword_UnicodeTrie: 4167 [microsec] total, 238 [nanosec / lookup]. isKeyword_UnicodeTrieBoolLookup: 3466 [microsec] total, 198 [nanosec / lookup]. Total: 104183 identifiers + 17488 keywords. As Dmitry said, we might hit a CTFE wall: memory consumption, computation speed, ... (*channelling Andrei*: then we will correct whatever makes CTFE a problem. Right) Philippe (Hesitating between 'The Art of the Metaobject Protocol' and 'Compilers, Techniques and Tools', right now)
Re: Let's stop parser Hell
isKeyword_Dummy (baseline): 2738 [microsec] total, 50 [ns / lookup]. This one calculates a sum of all identifier code units. Included for comparison. isKeyword_Dictionary: 4247 [microsec] total, 242 [ns / lookup]. Check whether an identifier is a keyword using AA (dictionary) lookup. isKeyword_SwitchByLengthThenByChar: 1593 [microsec] total, 91 [ns / lookup]. Switch by identifier length at the top, then recursively switch by each character. Clearly a winner, but I think it can be improved further. isKeyword_BinaryArrayLookup: 14351 [microsec] total, 820 [ns / lookup]. Binary search in an ordered array of keywords. isKeyword_LinearArrayLookup: 59564 [microsec] total, 3405 [ns / lookup]. Ditto, search is linear. isKeyword_UnicodeTrie: 4167 [microsec] total, 238 [ns / lookup]. Lookup a keyword in a trie, created by Dmitry. This will be improved. isKeyword_UnicodeTrieBoolLookup: 3466 [microsec] total, 198 [ns / lookup]. The same, but only determines whether an identifier is a keyword, not which exactly. Total: 104183 identifiers + 17488 keywords. Analyzed the largest Phobos file (DateTime? I didn't check the name.) Results are consistent for other files, too.
Re: Let's stop parser Hell
On Thu, Jul 5, 2012 at 10:00 PM, Tobias Pankrath tob...@pankrath.net wrote: On Thursday, 5 July 2012 at 18:17:06 UTC, Philippe Sigaud wrote: As a parser proper, Pegged is awful :-) Nothing I'm ashamed of, as I learn by coding. Hey, I just received the Dragon Book (International Edition), If you are interested in parsing, than I wouldn't recommend the dragon book, but this one http://dickgrune.com/Books/PTAPG_2nd_Edition/Additional.html It really is an awesome book. Hey, good catch! I saw this one a few months ago and forgot about it. Having half a book being annotated bibliography (IIUC) scared me. Hmm 72 € by Springer, 55 € on Amazon. Something is not right. Paperback vs perfect bound maybe? Is Modern Compiler Design by the same authors any good?
Re: Let's stop parser Hell
On Thu, Jul 5, 2012 at 10:02 PM, Roman D. Boiko r...@d-coding.com wrote (on children) I have four, from 1 to 7 years old... Wouldn't call them a problem, though :))) Better not telling my wife. She's making noises about having a fourth.
Re: Let's stop parser Hell
On 06-Jul-12 00:16, Roman D. Boiko wrote: isKeyword_Dummy (baseline): 2738 [microsec] total, 50 [ns / lookup]. This one calculates a sum of all identifier code units. Included for comparison. isKeyword_Dictionary: 4247 [microsec] total, 242 [ns / lookup]. Check whether an identifier is a keyword using AA (dictionary) lookup. isKeyword_SwitchByLengthThenByChar: 1593 [microsec] total, 91 [ns / lookup]. Switch by identifier length at the top, then recursively switch by each character. Clearly a winner, but I think it can be improved further. I'd stress the fact that it's a fully unrolled hard-coded switch that takes a lot of pages (~72Kbytes). It's easily be perfect. Sorry, couldn't resist ;) And I'm not sure how much it could be optimized maybe some measly 10-30%. Total: 104183 identifiers + 17488 keywords. Analyzed the largest Phobos file (DateTime? I didn't check the name.) Results are consistent for other files, too. It is std.datetime as I've been running this benchmark against it and seen the same numbers :) -- Dmitry Olshansky
Re: Let's stop parser Hell
On 7/5/12 3:54 PM, Philippe Sigaud wrote: (Hesitating between 'The Art of the Metaobject Protocol' and 'Compilers, Techniques and Tools', right now) Former sux latter rox. Andrei
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 20:28:32 UTC, Philippe Sigaud wrote: Hmm 72 € by Springer, 55 € on Amazon. Something is not right. Paperback vs perfect bound maybe? http://www.komkon.org/~sher/books/parsing_techniques_2008.pdf Not sure that it is legal, but definitely free.
Re: dfmt - D source code formatter
On Thursday, 5 July 2012 at 19:22:56 UTC, Walter Bright wrote: I think that formatting the code is actually rather easy - the hard part will be dealing with the comments in a reasonable way. Eclipse has had a LOT of time and energy put into it, and it still makes my javadoc uglier than it was before formatting. My thought is to properly indent/align comments, but other than that leave them unmodified. There are too many things that can go wrong with re-wrapping them (I'm imagining accidentally ruining somebody's carefully-crafted ASCII art diagram).
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On 7/5/12 12:26 PM, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! Updated to a much nicer shape. http://dlang.org. Thanks Damian! He'll work soon on enabling such compilation for all code examples in the Phobos pages. Andrei
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On Thursday, 5 July 2012 at 16:59:33 UTC, Iain Buclaw wrote: On 5 July 2012 17:51, H. S. Teoh hst...@quickfur.ath.cx wrote: On Thu, Jul 05, 2012 at 12:26:01PM -0400, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! [...] Won't that be open to abuse? Like if somebody wrote a fork bomb and tried to run it... Unless the backend server has tight resource control over the code sample executor, of course If it's using the same engine as dpaste ( http://dpaste.dzfl.pl ), then it is fairly locked down. Yea, its using dpaste backend :~)
Re: std.hash: More questions
On 04-Jul-12 18:58, Johannes Pfau wrote: Code: https://github.com/D-Programming-Language/phobos/pull/646 Docs: http://dl.dropbox.com/u/24218791/d/phobos/std_hash_hash.html http://dl.dropbox.com/u/24218791/d/phobos/std_hash_crc.html http://dl.dropbox.com/u/24218791/d/phobos/std_hash_md.html http://dl.dropbox.com/u/24218791/d/phobos/std_hash_sha.html I just had another look at my initial std.hash design, and I realized that the API could be simplified a little: There's a reset function that's implemented in every hash. For sha1, md5, crc32 it only forwards to the start function though. So I'm not sure how useful this function is or if it should be dropped. Advantages of keeping it: * 'reset' better documents what's done than 'start' if the hash has already processed data * Are there hashes which can implement a reset function in a faster way than calling start again? Cons: * Adds an additional function which probably isn't necessary The start function is probably not needed as well. Tango doesn't have a start function or something similar, but it could use constructors for this (I only looked at docs, not code). The only thing I can think of that would require start function is using unconventional initial vectors. We can't use constructors, so a start function would be necessary for advanced initialization. But do we actually need that advanced initialization? SHA1, MD5 and CRC32 just do a this = typeof(this).init so a start function isn't necessary here. Advantages of keeping it: * Are there hash algorithms which need some sort of complex initialization which can't be done with .init / default values? * If we drop both start and reset the only way to reset the internal state is calling finish. This might be a little less efficient than a start/reset method. Advantages of dropping it: * Using hashes is easier, no need to call 'start' before hashing data I think someone more familiar with hash functions than me needs to answer the do we need start/reset functions questions. API question: CRC32 sums are usually presented as a uint, not a ubyte[4]. To fit the rest of the API ubyte[4] is used. Now there's a small annoying detail: The CRC32 should be printed in LSB-first order. You probably meant MSB first. When printing an uint like this, that works well: writefln(%#x, 4157704578); //0xf7d18982 but this doesn't: toHexString(*cast(ubyte[4]*)4157704578); //8289D1F7 There is no problem it's just order of printing that at fault. So I suggest to *stop* doing a bswap. It's just that printing something as an array of ubytes does it from least significant byte to most significant. You could try to add MSB/LSB first options to toHexString. I can't change toHexString as it's used for all hashes and it's correct for SHA1, MD5, ... So I currently use bswap in the CRC32 finish() implementation to fix this issue. no-no-no see the above ;) Now the question is should I provide an additional finishUint function which avoids the bswap? Implementation issue: The current implementation of SHA1 and MD5 uses memcpy which doesn't work in CTFE IIRC and which also prevents the code from being pure. I could replace those memcpy calls with array copying but I'm not sure if memcpy was used for performance, so I'd like to keep it as long as we have no performance tests. Replace memcpy with and array ops: ptr1[x..y] = ptr2[x2..y2]; note that it's better to have them be pointers as it avoid bounds check D runtime magic. If need be I can provide benchmarks but I'm certain from the days of optimizing std.regex that it's faster or on par with memcpy. -- Dmitry Olshansky
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On Thursday, 5 July 2012 at 19:10:57 UTC, Peter Alexander wrote: Nice. Should probably remove the references to local files when compilation fails. Not very user friendly to see: /home/jail/compileme369.d(14): expression expected, not '}' Would probably suffice just to switch the filename with something less distracting. Partially fixed. Thanks!
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On Thursday, 5 July 2012 at 17:56:34 UTC, Dmitry Olshansky wrote: On 05-Jul-12 20:26, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! Wonderful! It's fast and fluid, looks good. Still I would request adding interactive console input. Some magic with WebSockets some server daemon on worker machines should do the trick. And being able to run for some time if network client is active. Browsers without WebSockets can just use non-interactive input with some text area which contents are fed to the program. That would be really nice, but I am afraid it's currently not doable with current design of whole infrastructure. Although I will think about it, dpaste probably could benefit from this too.
Re: Editable and runnable code sample on dlang.org by Damian Ziemba (nazriel)
On 06-Jul-12 01:28, nazriel wrote: On Thursday, 5 July 2012 at 17:56:34 UTC, Dmitry Olshansky wrote: On 05-Jul-12 20:26, Andrei Alexandrescu wrote: Check this out: on http://dlang.org you can actually click in the code example and edit it, then click Run and pronto, you see the output! Damian is actively working on the UI as I'm writing this. Feel free to chime in with feedback! Wonderful! It's fast and fluid, looks good. Still I would request adding interactive console input. Some magic with WebSockets some server daemon on worker machines should do the trick. And being able to run for some time if network client is active. Browsers without WebSockets can just use non-interactive input with some text area which contents are fed to the program. That would be really nice, but I am afraid it's currently not doable with current design of whole infrastructure. Although I will think about it, dpaste probably could benefit from this too. The truth be told I'd love to get this kind of infrastructure for a personal use. I've seen firsthand Claud9 IDE with node.js working on a very tiny device and, of course, I got jealous. I thought: such a waste of cycles, it would be so much better if it was D running on it :) -- Dmitry Olshansky
Re: Proposal: takeFront and takeBack
(grain of salt, I'm new to D.)
I'd vote for consumeFront being always available, because it's
distinctly more convenient to call one function instead of two,
especially when you expect that making a copy of front is cheap
(e.g. a collection of pointers, numbers or slices). Ranges where
'front' returns a pointer to a buffer that popFront destroys
(overwrites) are surely in the minority, right? So I hope they
would be retrofitted to support consumeFront.
But, given that popFront is allowed to be destructive to the
value of front, by re-using the same buffer (and that the
proposed consumeFront might also be implemented with 'delayed
destruction' to front), I am wondering how one is supposed to
implement generic code correctly when this is unacceptable, e.g.:
void append(Range1,Range2)(Range1 input, ref Range2 output)
{
// Usually works, unless input.popFront happens to be
destructive?
foreach(e; input) output ~= e;
}
Re: Let's stop parser Hell
Le 05/07/2012 18:32, Roman D. Boiko a écrit : On Thursday, 5 July 2012 at 16:14:27 UTC, Jacob Carlborg wrote: Original post: http://www.digitalmars.com/d/archives/digitalmars/D/DCT_use_cases_-_draft_168106.html#N168141 OK, fairly complete. Let it be the starting point. Why not put it on some wiki, then add some more, discuss, vote, etc.? Meanwhile create a matrix of features implemented in each of interesting standardization candidates. And pick up one of them as standard or reference parser. My vote would be for Pegged, I guess. Why not program instead of doing bureaucracy ?
Re: Let's stop parser Hell
On Thursday, 5 July 2012 at 22:11:41 UTC, deadalnix wrote: Why not program instead of doing bureaucracy ? To avoid programming things which are not needed or don't fit. I've thrown away several implementations already... time to reflect a little :) But, actually, for DCT I do know what I need to do. That suggestion was with respect to any candidate for becoming standard. Everybody understands that their way (likely differently than others).
Re: Let's stop parser Hell
Le 05/07/2012 18:28, Jonathan M Davis a écrit : On Thursday, July 05, 2012 18:04:11 Roman D. Boiko wrote: On Thursday, 5 July 2012 at 15:40:53 UTC, Jacob Carlborg wrote: On 2012-07-05 15:08, Roman D. Boiko wrote: Anyway I propose to enumerate major use cases first. Haven't we already done that a couple of times. Well, we did something like that for DCT... but I doubt that it would fit general needs. If we had, why haven't they been analyzed, classified, discussed, etc.? Or have they? It's been discussed before, but there are some obvious use cases such as syntax highlighting, code formatting, and manipulation of D source files (e.g. to strip out the unit tests). We need to have a lexer in Phobos which parsers D code and generates a range of tokens, and we need to have a parser in Phobos which operates on a range of tokens. As discussed previously, there is desire to have the lexer and parser ported from dmd's frontend to D for Phobos so that what we get is easily maintainable alongside dmd's frontend and produces the same results (and is fast). You never looked at dmd frontend soure code don't you ? It is a horror museum, and if we want to have something in D, I certainly wouldn't copy that.
Re: dfmt - D source code formatter
On Thursday, 5 July 2012 at 22:25:15 UTC, Walter Bright wrote: I agree that whatever is inside the comment should be left alone. I was more talking about lining up comment blocks, etc. Why would that be more difficult to do than code formatting? I'm wondering.
Re: dfmt - D source code formatter
On 7/5/2012 1:52 PM, Brian Schott wrote: On Thursday, 5 July 2012 at 19:22:56 UTC, Walter Bright wrote: I think that formatting the code is actually rather easy - the hard part will be dealing with the comments in a reasonable way. Eclipse has had a LOT of time and energy put into it, and it still makes my javadoc uglier than it was before formatting. My thought is to properly indent/align comments, but other than that leave them unmodified. There are too many things that can go wrong with re-wrapping them (I'm imagining accidentally ruining somebody's carefully-crafted ASCII art diagram). I agree that whatever is inside the comment should be left alone. I was more talking about lining up comment blocks, etc.
Re: More Front End Vector Support
On 7/5/2012 12:55 AM, Iain Buclaw wrote: Fair enough. Only asked as if we do Y and Z, why not X? GCC backend already supported the use of __vector[N] sizes long before D support was added, but then again only less than of a handful of architectures actually __support__ vector operations (as I said, I think only MMX and NEON would benefit) - the rest just slowly emulate it. I don't think D should do emulation - it should give a compiler error on vector sizes and operations that are not supported. The reason is the user may not expect the (very) slow emulation, and gets no indication of when it happens. By giving a compiler error on them, the user has the opportunity to decide what to do about it. After all, the only reason he is even using vector ops is for performance.
Re: More Front End Vector Support
On Thursday, 5 July 2012 at 22:28:21 UTC, Walter Bright wrote: I don't think D should do emulation - it should give a compiler error on vector sizes and operations that are not supported. The reason is the user may not expect the (very) slow emulation, and gets no indication of when it happens. By giving a compiler error on them, the user has the opportunity to decide what to do about it. It's called a warning. ;)
Re: Let's stop parser Hell
Le 06/07/2012 00:21, Roman D. Boiko a écrit : On Thursday, 5 July 2012 at 22:11:41 UTC, deadalnix wrote: Why not program instead of doing bureaucracy ? To avoid programming things which are not needed or don't fit. I've thrown away several implementations already... time to reflect a little :) But, actually, for DCT I do know what I need to do. That suggestion was with respect to any candidate for becoming standard. Everybody understands that their way (likely differently than others). Well you did the mistake to introduce an over complex mechanism in log(n) to get back to source code when an obvious one in O(1) exists. Let me doubt.
