Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On Tuesday, 14 May 2013 at 19:57:12 UTC, Guillaume Chatelet wrote: Dicebot, Sönke, how about a vibe.d based compiler ? :P I don't think networking part of of distributed compiler will be that demanding :) It is more about good pipe-lining, like described in Robert presentation. Concurrency level is always low and network speed tend to be much faster than compilation speed. I'd like someone more proficient in this topic to correct me, but once compiler is daemonized and decent data model / pipe line is defined, adding distributed networking functionality on top is one of the easiest parts.
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
P.S. And even easier if std.concurrency starts to work over network sockets :P
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On 5/13/2013 10:19 PM, dennis luehring wrote: Am 13.05.2013 14:01, schrieb Andrei Alexandrescu: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Andrei it seems that the http://www.phoronix.com/ guys miss the dconf completely - they write about rust, go, D in gcc, kernel udpates etc. - but absolutely nothing about dconf :( Email them about it!
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
Am 14.05.2013 08:18, schrieb Walter Bright: On 5/13/2013 10:19 PM, dennis luehring wrote: Am 13.05.2013 14:01, schrieb Andrei Alexandrescu: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Andrei it seems that the http://www.phoronix.com/ guys miss the dconf completely - they write about rust, go, D in gcc, kernel udpates etc. - but absolutely nothing about dconf :( Email them about it! is there something like an overview page with slides, videos etc.? dconf.org - could someone change the main page to reflect the success of the conference :)
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On Monday, 13 May 2013 at 12:01:54 UTC, Andrei Alexandrescu wrote: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Andrei On promising consequence of such compiler daemonization I have always had in mind since got familiar with the concept is that you can have convenience of separate compilation system AND speed of single step compilation at the same time, even if no stuff is actually distributed network-wise.
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On 05/14/2013 12:25 AM, dennis luehring wrote: is there something like an overview page with slides, videos etc.? The schedule page has links to both the slides and the videos: http://dconf.org/schedule/index.html Individual presentation pages too: http://dconf.org/talks/schadek.html Ali
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
13-May-2013 16:01, Andrei Alexandrescu пишет: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Nice one! Reminds me the gory days of our compiler technologies course :) Our professor seemed obsessed with tabulated methods as well (Dragon book etc.) And above all - reverse polish notation as the one and holy AST representation. Don't you try to say recursive descent parser in the class... Speaking of flat AST point in the talk - reverse-polish notation is kind of neat in that it does represent flat AST and is easy to walk. So one need not to have 2-array AST (childs + map to where are these). Another advantage - you can slice off a sub-tree as simple as a chunk of an array! It should be good for CPU caches (all related stuff is in one block, no pointer chasing), but I never was close enough to a real compiler to run any meaningful benchmarks. Let's see the idea of AST as polish-notation array applied to the tree in the talk. I show the forward one as it easier to read and less tricky to walk. Using the following notation for each node: arity, array of offsets * arity, type code+contents | subtree(s) Data is laid out byte-by-byte in raw-binary. Offsets are in bytes starting right after type+contents. So e.g. leaf node of int would be: 0, int No offsets nor sub-trees, content/type tag etc. is marked as int Then whole tree (tried to align for reading): 1, 0, S | 1, 0, DeclDefs | 3, 0, x, y, Declarator | 1, 0, BasicType | 0, int 1, 0, Identifier | 0, main 1, 0, DeclDefs | 1, 0, DeclDef | 1, 0 ReturnStatement | 0, 1 x and y are sizes of sub-trees and can easily be calculated at construction. Insertion is simple array insertion + fix ups of offsets up the tree (you have to know your path in nodes anyway to back up). Giving up on random access to child nodes would allow us to drop offset table completely. I have limited experience on semantic analysis to tell if it makes sense to require or drop it. Will comment on the DFA Unicode some time later, it's a neat topic in its own right. -- Dmitry Olshansky
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On promising consequence of such compiler daemonization I have always had in mind since got familiar with the concept is that you can have convenience of separate compilation system AND speed of single step compilation at the same time, even if no stuff is actually distributed network-wise. Dicebot, Sönke, how about a vibe.d based compiler ? :P Compilation stages could be pipelined/distributed through localhost or distant url calls (of course it shouldn't go through the network stack in case of local calls). We could even make use of resource caching to speedup the compilation. Maybe it's a crazy idea but I see vibe.d as an excellent framework for a parallel compiler (local and distributed). Also it becomes almost trivial to distribute over a farm if you see a compilation stage as a web resource. Lexing could be streamed to a Parsing url. Semantic analysis could be done in parallel and load balanced, Linking of course... well... linking is the brutally sequential part - nothing fancy here. I'm not familiar enough with vibe.d to see if it could work, it's just an idea. The RestFul model act as an interface and allows for thinking in terms of services (or components) and abstracts away the execution location as well as the implementation. Each url could be seen as a pure function, allowing for caching (memoizing). And with no state (restful) : distribution becomes almost free. It's a very simplistic view of a compiler and Walter will surely have myriads of objections, but I wanted to share my thoughts. This kind of infrastructure could pay on massive codebase compilation ( like at Facebook or Google ) -- Guillaume
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On Tue, 14 May 2013 17:43:46 +0400 Dmitry Olshansky dmitry.o...@gmail.com wrote: 13-May-2013 16:01, Andrei Alexandrescu пишет: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Nice one! Reminds me the gory days of our compiler technologies course :) Yea, I found it to be a fascinating talk, too. Our professor seemed obsessed with tabulated methods as well (Dragon book etc.) I'm quite partial to Crafting a Compiler http://www.pearsonhighered.com/educator/product/Crafting-A-Compiler/9780136067054.page. I'm not sure it's quite as complete as the dragon book, but it's somewhat more programmer-friendly and much easier to understand, whereas the dragon book (as well as all CS theory books that I've seen) is geared specifically to CS researchers and mathemeticians. So e.g. leaf node of int would be: 0, int No offsets nor sub-trees, content/type tag etc. is marked as int Then whole tree (tried to align for reading): 1, 0, S | 1, 0, DeclDefs | 3, 0, x, y, Declarator | 1, 0, BasicType | 0, int 1, 0, Identifier | 0, main 1, 0, DeclDefs | 1, 0, DeclDef | 1, 0 ReturnStatement | 0, 1 Interesting. Will comment on the DFA Unicode some time later, it's a neat topic in its own right. Somewhat of a naive approach, but transitioning on character ranges instead of characters will drastically reduce the storage requirements for generalized unicode lexer, at least for any remotely typical lexer (there can be some pretty bad worst-case-scenarios, but nobody ever defines a token like [acegikmoq...]+). Although I imagine it's not nearly as fast as a pure table since you have those extra conditionals for each character of input. FWIW.
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On 5/14/13 4:12 PM, Nick Sabalausky wrote: [snip] This is a great discussion - I encourage you all to hold future discussions on DConf topics to the associated reddit pages for greater visibility and participation. Tomorrow morning comes another one - Ben's first talk! Thanks, Andrei
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On 14 May 2013 22:16, Andrei Alexandrescu seewebsiteforem...@erdani.orgwrote: On 5/14/13 4:12 PM, Nick Sabalausky wrote: [snip] This is a great discussion - I encourage you all to hold future discussions on DConf topics to the associated reddit pages for greater visibility and participation. Tomorrow morning comes another one - Ben's first talk! Ben did not talk as if it was his first talk... :o) -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On 2013-05-13 14:01, Andrei Alexandrescu wrote: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Awesome :) -- /Jacob Carlborg
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
Is there a way to find all the reddit links to these (not a frequent reddit user, but I'm curious to look over the discussions each vid gets when I have the chance). --bb On Mon, May 13, 2013 at 5:01 AM, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Watch, discuss, vote up! http://reddit.com/r/**programming/comments/1e8mwq/** dconf_2013_day_1_talk_3_**distributed_caching/http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Andrei
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On Monday, 13 May 2013 at 12:01:54 UTC, Andrei Alexandrescu wrote: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Andrei Is the download speed too low because the video is popular? :)
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On Mon, 13 May 2013 10:01:03 -0700 Bill Baxter wbax...@gmail.com wrote: Is there a way to find all the reddit links to these (not a frequent reddit user, but I'm curious to look over the discussions each vid gets when I have the chance). Added Reddit links here: http://semitwist.com/download/misc/dconf2013/ Also added torrents for this new video.
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
Excellent. Thanks! On Mon, May 13, 2013 at 12:09 PM, Nick Sabalausky seewebsitetocontac...@semitwist.com wrote: On Mon, 13 May 2013 10:01:03 -0700 Bill Baxter wbax...@gmail.com wrote: Is there a way to find all the reddit links to these (not a frequent reddit user, but I'm curious to look over the discussions each vid gets when I have the chance). Added Reddit links here: http://semitwist.com/download/misc/dconf2013/ Also added torrents for this new video.
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On 2013-05-13, 21:09, Nick Sabalausky wrote: On Mon, 13 May 2013 10:01:03 -0700 Bill Baxter wbax...@gmail.com wrote: Is there a way to find all the reddit links to these (not a frequent reddit user, but I'm curious to look over the discussions each vid gets when I have the chance). Added Reddit links here: http://semitwist.com/download/misc/dconf2013/ Also added torrents for this new video. The linky for original MP4 keynote torrent seems to be dead. -- Simen
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On Mon, 13 May 2013 23:27:53 +0200 Simen Kjaeraas simen.kja...@gmail.com wrote: On 2013-05-13, 21:09, Nick Sabalausky wrote: Added Reddit links here: http://semitwist.com/download/misc/dconf2013/ Also added torrents for this new video. The linky for original MP4 keynote torrent seems to be dead. Thanks. Fixed.
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
On Monday, 13 May 2013 at 12:01:54 UTC, Andrei Alexandrescu wrote: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Andrei Very cool presentation. Not sure if Robert visits news group, but maybe someone else have experience in this subject - I wonder how solution shown in presentation scales compared to similar products like ccache+distcc+make and j flag combo. If it is much faster, maybe there would be value in bringing such functionality to our current compilers? DMD, GDC, LDC troika. Best Regards
Re: DConf 2013 Day 1 Talk 3: Distributed Caching Compiler for D by Robert Schadek
Am 13.05.2013 14:01, schrieb Andrei Alexandrescu: Watch, discuss, vote up! http://reddit.com/r/programming/comments/1e8mwq/dconf_2013_day_1_talk_3_distributed_caching/ Andrei it seems that the http://www.phoronix.com/ guys miss the dconf completely - they write about rust, go, D in gcc, kernel udpates etc. - but absolutely nothing about dconf :(
