Re: Slides share: DMesos - Not only a re-implementation of Mesos
On Saturday, 15 July 2017 at 15:57:18 UTC, 鲜卑拓跋枫 wrote: On Friday, 14 July 2017 at 05:07:11 UTC, Dsby wrote: On Monday, 10 July 2017 at 17:29:01 UTC, 鲜卑拓跋枫 wrote: [...] 我也希望能用D做出来。 我们也在研究过raft， akka这些技术。 raft算法的翻译我同事也有个port到D版本的。只是具体什么样，我没参与也没关心。 我们也在上海，我看链接里介绍，你现在也在上海工作，在EMC？ 只是不知道你们开始做没？ 做DMesos的目的一是Just for Fun, 二是在实战中提高D语言的功力。 D语言若能有更多一些大项目，则能更好地扩大其影响力并促进进一步的发展:)。 D 的确缺少大点的项目。 那个raft实现具体我没关心。 而且最近我在公司推nogc呢。
Re: proposed @noreturn attribute
On 7/13/2017 5:18 PM, Andrei Alexandrescu wrote: On 7/13/17 2:37 PM, Timon Gehr wrote: On Thursday, 13 July 2017 at 17:25:18 UTC, Timon Gehr wrote: Anyway, my assertion that Bottom cannot be a subtype of all other types was actually incorrect: the compiler does not need to generate code for implicit conversion from Bottom to some other type, so it can be treated as a subtype. ... (Actually, there are some complications like the .sizeof property. Anyway, it is clear what the semantics of Bottom are, no matter whether it is subtyping or implicit conversion.) I wonder if sizeof could be made size_t.max. -- Andrei I thought bottom.sizeof would be 0.
Re: Slides share: DMesos - Not only a re-implementation of Mesos
On Saturday, 15 July 2017 at 16:16:01 UTC, Iain Buclaw wrote: On Saturday, 15 July 2017 at 15:40:38 UTC, 鲜卑拓跋枫 wrote: On Thursday, 13 July 2017 at 19:32:07 UTC, Iain Buclaw wrote: On Monday, 10 July 2017 at 17:29:01 UTC, 鲜卑拓跋枫 wrote: Dear all, I am a D-language amateur from China, and just want to share you with a slides from me that post on MesosCon Asia 2017(Beijing): https://mesosconasia2017.sched.com/event/AZc6/dmesos-not-only-a-re-implementation-of-mesos-ce-feng-li-emc# I do really wanna to implement the design or "dream" as described in this slides, your help and suggestion are very welcome! Thanks！ GDC: - complete support armel, armhf - partial or bare-metal only support aarch64 Only bare-metal? As far as I'm concerned, all targets have full compiler support. The druntime and phobos libraries less so on that front. ;-) Iain. Such declaration is from the official D compilers wiki: https://wiki.dlang.org/Compilers It says the aarch64 support in GDC is not "Complete"... Wikis tend to be a little behind, in any case I think baremetal probably doesn't quite give that list justice. None of the packages available on Debian (as listed by the wiki) are baremetal targets, they all for Linux. There's a few more levels of language support that aren't made obvious: - Has a compiler. - Can build druntime. - Passes all compilable tests in the testsuite. - Passes druntime unittester - Can build phobos. - Passes all runnable tests in the testsuite. - Passes phobos unittester Granted that the top marks only goes to x86 and Arm. The others are not far behind, and have at least the first three covered. Since D front-end for GCC 8 is on the way, maybe we could replace GDC with GCC soon... D frontend for GCC and GDC are the same thing. ;-) Iain. Thank you very much for the clarification:)!
Re: newCTFE Status July 2017
On Saturday, 15 July 2017 at 17:43:04 UTC, Stefan Koch wrote: On Saturday, 15 July 2017 at 11:31:30 UTC, Tourist wrote: On Saturday, 15 July 2017 at 09:02:02 UTC, Stefan Koch wrote: On Saturday, 15 July 2017 at 07:50:28 UTC, Dmitry wrote: ... It would indeed be nice to have a GitHub issue (or similar) with progress checkboxes of what works, what's in progress, and what is yet to be done. here is the current newCTFE test. /UplinkCoder/dmd/blob/newCTFE_on_master/test/compilable/ctfeTest.d Eveything in there is supported by newCTFE . Modulo regressions. Nice, but it's not as clear, and doesn't specify what's left to be done.
Re: Function attribute deduction depends on compile invocation
On 7/14/2017 4:51 PM, Walter Bright wrote: On 7/14/2017 9:53 AM, Johan Engelen wrote: What happens in that in one kind of compilation, @nogc is deduced for a function. But in another compilation, it isn't. Thus references to the function will have the wrong mangling, and linker errors happen: https://issues.dlang.org/show_bug.cgi?id=17541 At the moment you can work around it by explicitly adding the annotation. https://github.com/dlang/dmd/pull/6995
Re: Compilation times and idiomatic D code
On Saturday, 15 July 2017 at 15:58:12 UTC, Jonathan M Davis wrote: On Saturday, July 15, 2017 11:10:32 Enamex via Digitalmars-d wrote: [...] The issue has to do with how each invocation of a range-based function tends to result in a new template instantiation, and it's common practice in D to chain a bunch of templated function calls together. For instance, if you have [...] The type is getting progressively longer and more complicated, because when the function template is instantiated, it's instantiated with the type from the previous function call, so it's wrapping the previous type, and you get a new type that has the name of the type of its argument embedded in it. It's like if you keep wrapping a container inside another container. [...] - Jonathan M Davis This was quite insightful, thank you. All that time I'd assumed that 'symbols' as linkers used them were constant length :T Just to be clear: 100% of that bloat resides in the symbol table? So the current proposed remedy is to hash symbols above a length threshold? Besides the symbol problem though, does the template instantiation explosion problem imply as many duplicated function bodies corresponding to the new type symbols?
Re: dmd and Archlinux
On Tuesday, 11 July 2017 at 12:00:51 UTC, Seb wrote: On Tuesday, 11 July 2017 at 00:23:55 UTC, Marco Leise wrote: Am Sun, 09 Jul 2017 18:35:09 + schrieb Antonio Corbi : Hi! Are there any news about the status of packaging dmd for archlinux? The last dmd compiler packaged is 2.074.0 and since the last batch of updated packages in archlinux, dmd generated objects fail to link with libphobos with erros like these: /usr/bin/ld: /usr/lib/libphobos2.a(object_a_66e.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile con -fPIC /usr/bin/ld: /usr/lib/libphobos2.a(object_b_58c.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile con -fPIC /usr/bin/ld: /usr/lib/libphobos2.a(object_c_7f4.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile con -fPIC /usr/bin/ld: /usr/lib/libphobos2.a(object_d_a07.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile con ... A. Corbi The linker gold (since binutils 2.27) is problematic with DMD and would emit such errors. (Is ld a symlink to ld.gold in this case?) Also binutils 2.28 breaks how DMD's druntime loads shared libraries. You have to either downgrade binutils or use the hack of adding '-fPIC' when compiling executable that link against shared libraries. binutils 2.26 should be the most compatible version for the time being. @mleise: OP is using the testing repos where the PIE enforcement already landed [1], but libphobos.a isn't built with -fPIC on Arch yet. The Arch developers, however, are aware of the need to rebuild libphobos [2], but maybe we can simply build Phobos with -fPIC by default on Posix [3]. [1] https://www.archlinux.org/todo/pie-rebuild [2] https://bugs.archlinux.org/task/54749 [3] https://github.com/dlang/phobos/pull/5586 Bumping this as the PIE changes have been moved to core and as the maintainers haven't rebuilt Phobos with PIE yet, one will see messages like: /usr/local/bin/ld: error: /usr/lib/libphobos2.a(encoding_4bd_6d6.o): requires dynamic R_X86_64_32 reloc which may overflow at runtime; recompile with -fPIC A quick fix is: curl https://i.dlang.io | bash -s dmd-2.074.0 sudo cp ~/dlang/dmd-2.074.0/linux/lib64/libphobos2.a /usr/lib The official releases [1] are built with PIE=1 [1] http://dlang.org/download.html
Re: D easily overlooked?
On Saturday, 15 July 2017 at 17:10:56 UTC, Joakim wrote: To answer Mark's original question, the corporates get interested when there are competitors eating their lunch with new tech. They don't actively scout out all the new tech, they're far too lazy for that. But when Sociomantic or Weka or some new company _you_ build with D starts putting them out of business, their ears perk up. ;) Point taken. :)
Re: Weka launched out of stealth mode a couple days ago
On Saturday, 15 July 2017 at 18:00:12 UTC, Seb wrote: On Saturday, 15 July 2017 at 11:38:59 UTC, Joakim wrote: [...] How about reposting this to announce? Maybe linking it with Liran's excellent DConf talk: http://dconf.org/2016/talks/zvibel.html Feel free, I wasn't sure if it belonged in there, since none of the recent Weka links mention D.
Re: Weka launched out of stealth mode a couple days ago
On Saturday, 15 July 2017 at 11:38:59 UTC, Joakim wrote: Here are a bunch of write-ups taken from their twitter feed: http://searchstorage.techtarget.com/news/450422665/New-WekaIO-file-system-incorporates-flash-object-store https://www.theregister.co.uk/2017/07/13/wekaio_surfaces_after_swimming_submerged_against_the_current/ https://gestaltit.com/exclusive/stephen/wekaio-distributed-storage-cloud/ https://storageswiss.com/2017/07/12/solve-the-unstructured-data-paradox-wekaio/ http://www.computerweekly.com/news/450422600/WekaIO-brings-brings-NAS-software-defined-storage https://insidebigdata.com/2017/07/13/wekaio-eliminates-need-legacy-external-storage-cloud-scale-storage-software/ Congrats to the Weka team, you can follow their twitter here: https://twitter.com/WekaIO How about reposting this to announce? Maybe linking it with Liran's excellent DConf talk: http://dconf.org/2016/talks/zvibel.html
Re: newCTFE Status July 2017
On Saturday, 15 July 2017 at 11:31:30 UTC, Tourist wrote: On Saturday, 15 July 2017 at 09:02:02 UTC, Stefan Koch wrote: On Saturday, 15 July 2017 at 07:50:28 UTC, Dmitry wrote: ... It would indeed be nice to have a GitHub issue (or similar) with progress checkboxes of what works, what's in progress, and what is yet to be done. here is the current newCTFE test. /UplinkCoder/dmd/blob/newCTFE_on_master/test/compilable/ctfeTest.d Eveything in there is supported by newCTFE . Modulo regressions.
Re: D easily overlooked?
On Saturday, 15 July 2017 at 17:36:50 UTC, aberba wrote: On Saturday, 15 July 2017 at 17:10:56 UTC, Joakim wrote: On Saturday, 15 July 2017 at 16:52:51 UTC, Russel Winder wrote: [...] Sociomantic, Weka, EMSI, and a handful of others. None is as humongous as google or Apple, but then it's not like those companies write everything in Go and Swift. Those languages are small parts of those giant corporations. To answer Mark's original question, the corporates get interested when there are competitors eating their lunch with new tech. They don't actively scout out all the new tech, they're far too lazy for that. But when Sociomantic or Weka or some new company ... The no. 1 thing that will help D is people writing great software with it and showing how wonderful D is. Who is building the killer app? Maybe it's Weka? http://forum.dlang.org/thread/pjogugkxvvntleoqc...@forum.dlang.org Maybe it's Adam. :) You never know where it will come from. Nobody knew who the VisiCalc guys were and they made the Apple II: https://en.wikipedia.org/wiki/VisiCalc The important part is that we need many people to try building great stuff with D, and out of some combination of good application design, users who really want that ability, and luck, something will rise up.
Re: D easily overlooked?
On Saturday, 15 July 2017 at 17:10:56 UTC, Joakim wrote: On Saturday, 15 July 2017 at 16:52:51 UTC, Russel Winder wrote: [...] Sociomantic, Weka, EMSI, and a handful of others. None is as humongous as google or Apple, but then it's not like those companies write everything in Go and Swift. Those languages are small parts of those giant corporations. To answer Mark's original question, the corporates get interested when there are competitors eating their lunch with new tech. They don't actively scout out all the new tech, they're far too lazy for that. But when Sociomantic or Weka or some new company ... The no. 1 thing that will help D is people writing great software with it and showing how wonderful D is. Who is building the killer app?
Re: Toward Go 2 (or D needs to collect experience reports)
On Saturday, 15 July 2017 at 02:55:42 UTC, Seb wrote: Very interesting article: https://blog.golang.org/toward-go2 Good read, thanks for posting. I also thought the discussion under "Explaining Problems" was really well done. A couple of lines from that section: Convincing others that a problem is significant is an essential step. ... When we disagree about whether to adopt a particular solution, we're often actually disagreeing about the significance of the problem being solved. The author is trying to motivate the importance of experience reports, but is also describing a common and challenging problem in software development environments. --Jon
Re: D easily overlooked?
On Saturday, 15 July 2017 at 16:52:51 UTC, Russel Winder wrote: D has ??? D has me. Do not be too proud of the corporate terror the others have constructed. The power to wave around a million dollars is insignificant next to the power of the Nerdiness.
Re: D easily overlooked?
On Saturday, 15 July 2017 at 16:52:51 UTC, Russel Winder wrote: On Sat, 2017-07-15 at 11:22 +, Mark via Digitalmars-d wrote: […] Well, at one point Andrei said that what is missing to make D's growth explosive is a strong corporate sponser [1]. This seemed sensible to me at the time and still seems sensible today. But I don't know what (if anything) can be done to increase the corporate community's interest in D. Java has IBM, Oracle, Red Hat, many companies, and the JCP C++ has Microsoft, Intel, many companies, and ISO/IEC JTC1/SC22/WG21 Rust has Mozilla, and lots of companies thinking it beats C++. Go has Google, Cloudflare, and many companies, mostly Web focussed. Swift has Apple, and the entire iOS community. D has ??? Sociomantic, Weka, EMSI, and a handful of others. None is as humongous as google or Apple, but then it's not like those companies write everything in Go and Swift. Those languages are small parts of those giant corporations. To answer Mark's original question, the corporates get interested when there are competitors eating their lunch with new tech. They don't actively scout out all the new tech, they're far too lazy for that. But when Sociomantic or Weka or some new company _you_ build with D starts putting them out of business, their ears perk up. ;) The no. 1 thing that will help D is people writing great software with it and showing how wonderful D is. More than docs, fixing bugs, marketing, anything. It doesn't matter if the language still has flaws, the influx of use, attention, and money after that will help patch those holes up, look at Rails. You don't have to have a huge budget to write that D software, the guy who wrote Tilix did it in his spare time and it's the top-starred D project on github after dmd: https://github.com/search?l=D&q=stars%3A>1&s=stars&type=Repositories
Re: D easily overlooked?
On Sat, 2017-07-15 at 11:22 +, Mark via Digitalmars-d wrote: […] > > Well, at one point Andrei said that what is missing to make D's > growth explosive is a strong corporate sponser [1]. This seemed > sensible to me at the time and still seems sensible today. But I > don't know what (if anything) can be done to increase the > corporate community's interest in D. Java has IBM, Oracle, Red Hat, many companies, and the JCP C++ has Microsoft, Intel, many companies, and ISO/IEC JTC1/SC22/WG21 Rust has Mozilla, and lots of companies thinking it beats C++. Go has Google, Cloudflare, and many companies, mostly Web focussed. Swift has Apple, and the entire iOS community. D has ??? -- Russel. = Dr Russel Winder t:+44 20 7585 2200 voip:sip: russel.win...@ekiga.net 41 Buckmaster Road m:+44 7770 465 077 xmpp:rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype:russel_winder signature.asc Description: This is a digitally signed message part
Re: Toward Go 2 (or D needs to collect experience reports)
On Sat, 2017-07-15 at 04:29 +, Matt via Digitalmars-d wrote: > The one thing I got from that blog post is that the HN and reddit > discussion was overwhelmingly about generics, and how the Go > leadership seems to not give a damn about what its user community > wants... On the other hand, I wonder how many practicing Go programmers actually care about things such as HN and Reddit? On the Go mailing lists, generics comes up every so often and the thread dies a death quite quickly. I think the view that "there is no sensible semantics for generics that work with Go, and we have interface anyway" has been accepted by the Go community at large. There is also go generate. -- Russel. = Dr Russel Winder t:+44 20 7585 2200 voip:sip: russel.win...@ekiga.net 41 Buckmaster Road m:+44 7770 465 077 xmpp:rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype:russel_winder signature.asc Description: This is a digitally signed message part
Re: Slides share: DMesos - Not only a re-implementation of Mesos
On Saturday, 15 July 2017 at 15:40:38 UTC, 鲜卑拓跋枫 wrote: On Thursday, 13 July 2017 at 19:32:07 UTC, Iain Buclaw wrote: On Monday, 10 July 2017 at 17:29:01 UTC, 鲜卑拓跋枫 wrote: Dear all, I am a D-language amateur from China, and just want to share you with a slides from me that post on MesosCon Asia 2017(Beijing): https://mesosconasia2017.sched.com/event/AZc6/dmesos-not-only-a-re-implementation-of-mesos-ce-feng-li-emc# I do really wanna to implement the design or "dream" as described in this slides, your help and suggestion are very welcome! Thanks！ GDC: - complete support armel, armhf - partial or bare-metal only support aarch64 Only bare-metal? As far as I'm concerned, all targets have full compiler support. The druntime and phobos libraries less so on that front. ;-) Iain. Such declaration is from the official D compilers wiki: https://wiki.dlang.org/Compilers It says the aarch64 support in GDC is not "Complete"... Wikis tend to be a little behind, in any case I think baremetal probably doesn't quite give that list justice. None of the packages available on Debian (as listed by the wiki) are baremetal targets, they all for Linux. There's a few more levels of language support that aren't made obvious: - Has a compiler. - Can build druntime. - Passes all compilable tests in the testsuite. - Passes druntime unittester - Can build phobos. - Passes all runnable tests in the testsuite. - Passes phobos unittester Granted that the top marks only goes to x86 and Arm. The others are not far behind, and have at least the first three covered. Since D front-end for GCC 8 is on the way, maybe we could replace GDC with GCC soon... D frontend for GCC and GDC are the same thing. ;-) Iain.
Re: Slides share: DMesos - Not only a re-implementation of Mesos
On Friday, 14 July 2017 at 05:07:11 UTC, Dsby wrote: On Monday, 10 July 2017 at 17:29:01 UTC, 鲜卑拓跋枫 wrote: Dear all, I am a D-language amateur from China, and just want to share you with a slides from me that post on MesosCon Asia 2017(Beijing): https://mesosconasia2017.sched.com/event/AZc6/dmesos-not-only-a-re-implementation-of-mesos-ce-feng-li-emc# I do really wanna to implement the design or "dream" as described in this slides, your help and suggestion are very welcome! Thanks！ BRs, Koo 我也希望能用D做出来。 我们也在研究过raft， akka这些技术。 raft算法的翻译我同事也有个port到D版本的。只是具体什么样，我没参与也没关心。 我们也在上海，我看链接里介绍，你现在也在上海工作，在EMC？ 只是不知道你们开始做没？ 做DMesos的目的一是Just for Fun, 二是在实战中提高D语言的功力。 D语言若能有更多一些大项目，则能更好地扩大其影响力并促进进一步的发展:)。
Re: Compilation times and idiomatic D code
On Saturday, July 15, 2017 11:10:32 Enamex via Digitalmars-d wrote: > - What type information are being kept because of UFCS chains? > Doesn't that mechanism simply apply overload resolution then > choose between the prefix and .method forms as appropriate, > rewriting the terms? > Then it's a problem of function invocation. I don't get > what's happening here still. Does this tie to the Voldemort types > problem? (=> are many of the functions in the chain returning > custom types?) It doesn't make sense, especially if, from your > indirection workaround, it looks like it would work around the > same but without the bloat problem if we unlinked the chain into > many intermediate temporary parts. So how is this a type > information issue? UFCS is irrelevant to all of this. That just so happens to be how he's calling the functions. You'd get the same issue if you called all of the functions in a chain with the traditional function call syntax rather than treating them as a bunch of member functions. The issue has to do with how each invocation of a range-based function tends to result in a new template instantiation, and it's common practice in D to chain a bunch of templated function calls together. For instance, if you have int[] a; auto b = a.map!(a => a / 2)(); pragma(msg, typeof(b)); then it prints out MapResult!(__lambda1, int[]) If you have int[] a; auto b = a.map!(a => a / 2)().map!(a => a)(); pragma(msg, typeof(b)); then it prints out MapResult!(__lambda2, MapResult!(__lambda1, int[])) If you have int[] a; auto b = a.map!(a => a / 2)().map!(a => a)().filter!(a => a < 7)(); pragma(msg, typeof(b)); then it prints out FilterResult!(__lambda3, MapResult!(__lambda2, MapResult!(__lambda1, int[]))) The type is getting progressively longer and more complicated, because when the function template is instantiated, it's instantiated with the type from the previous function call, so it's wrapping the previous type, and you get a new type that has the name of the type of its argument embedded in it. It's like if you keep wrapping a container inside another container. Array!int a; Array!(Array!int) b; Array!(Array!(Array!int)) c; The first level or two may not be so bad, but pretty quickly, it gets quite ridiculous. The fact that we use auto and/or the name of the template parameter in range-based code completely hides the fact that the type we're operating on has a very long and hideous name. So, for the most part, you don't care, but you do get ridiculously long symbol names underneath the hood, and the compiler and linker have to deal with them, and it becomes a performance problem. And as bad as this example is, map doesn't actually use Voldemort types. MapResult is declared outside of map as a private struct. The extra information that gets encoded in the symbol name with a Voldemort type makes them _much_ worse, though interestingly enough, the compiler doesn't print all of that out. e.g. int[][] a; auto b = a.joiner([5]); pragma(msg, typeof(b)); just prints Result which doesn't look bad even though things are apparently much uglier underneath the hood. But the fact that working with ranges results in a whole bunch of chained invocations of function templates tends to result in a much larger symbol sizes than you'd get with code that wasn't so generic. And the fact that Voldemort types have been pushed as a great way to encapsulate the range types that result from function calls and really shouldn't be used by name makes the situation that much worse. So, idiomatic D code currently results in very large symbol names, and while in many cases, you don't notice, some people (like H.S. Teoh) _are_ noticing, and it's becoming a problem. As I understand it, Weka has quite a few problems with this. Fortunately, a fix to condense symbol names should be along shortly, which will help the situation considerably, but in general, we really need to look at how we can make some of these common D idioms do better in terms of symbol size. - Jonathan M Davis
Re: Slides share: DMesos - Not only a re-implementation of Mesos
On Friday, 14 July 2017 at 05:07:11 UTC, Dsby wrote: On Monday, 10 July 2017 at 17:29:01 UTC, 鲜卑拓跋枫 wrote: Dear all, I am a D-language amateur from China, and just want to share you with a slides from me that post on MesosCon Asia 2017(Beijing): https://mesosconasia2017.sched.com/event/AZc6/dmesos-not-only-a-re-implementation-of-mesos-ce-feng-li-emc# I do really wanna to implement the design or "dream" as described in this slides, your help and suggestion are very welcome! Thanks！ BRs, Koo 我也希望能用D做出来。 我们也在研究过raft， akka这些技术。 raft算法的翻译我同事也有个port到D版本的。只是具体什么样，我没参与也没关心。 我们也在上海，我看链接里介绍，你现在也在上海工作，在EMC？ 只是不知道你们开始做没？ 有更多人关注并实现基于D语言的数据中心真是太好了！ 若你们能公开相关Raft算法的D语言实现那更好了：）！ 我近期已经离开EMC和上海了，不过还是经常有机会去上海的。 就像上文和幻灯片中描述地那样：目前DMesos的开发者只有我一个，DLMDB我在写，希望能在1，2个月内公开出来。但我主要精力会放在底层实现上面（操作系统和运行时级别的重构），希望能有更多人能帮助实现DMesos的user space部分，因为这部分的工作量非常大。 让我们一起努力吧：）！
Re: Slides share: DMesos - Not only a re-implementation of Mesos
On Thursday, 13 July 2017 at 19:32:07 UTC, Iain Buclaw wrote: On Monday, 10 July 2017 at 17:29:01 UTC, 鲜卑拓跋枫 wrote: Dear all, I am a D-language amateur from China, and just want to share you with a slides from me that post on MesosCon Asia 2017(Beijing): https://mesosconasia2017.sched.com/event/AZc6/dmesos-not-only-a-re-implementation-of-mesos-ce-feng-li-emc# I do really wanna to implement the design or "dream" as described in this slides, your help and suggestion are very welcome! Thanks！ GDC: - complete support armel, armhf - partial or bare-metal only support aarch64 Only bare-metal? As far as I'm concerned, all targets have full compiler support. The druntime and phobos libraries less so on that front. ;-) Iain. Such declaration is from the official D compilers wiki: https://wiki.dlang.org/Compilers It says the aarch64 support in GDC is not "Complete"... Since D front-end for GCC 8 is on the way, maybe we could replace GDC with GCC soon...
Re: Vibe.d - very low performance
On 2017-07-14 11:55, Marek wrote: So why Ruby or Python frameworks are much faster in this benchmark? They scale better since, at least Ruby on Rails applications, are run using multiple processes. -- /Jacob Carlborg
Re: Types: The Next Generation (Was: Why is phobos so wack?)
On Sunday, 9 July 2017 at 20:22:16 UTC, Nick Sabalausky
(Abscissa) wrote:
So, why not encapsulate much of that stuff we merely *describe*
in signatures for generic functions into genuine
honest-to-goodness types?
There would be user-defined symbols, such as "InputRange" or
"SomeString", or "RandomAccessRange!SomeString", which the type
system sees as actual types. And naturally there would be some
mechanism for actually defining those types. Then, defining a
function like this:
SomeString fizzbar(RandomAccessRange!SomeNumeric r) {...}
...would automatically imply to the compiler all (or at least a
lot of) the pluming we usually clutter the function declaration
with: Defining the templated types and calling the appropriate
if(isBlahBlah) constraints. About the only things remaining
would be additional constraints not already defined by the
next-gen types, and restrictions on how the parameters relate
to each other.
Even better, having all that encapsulated into genuine types
should make it easier for the compiler to provide better error
messages.
This reminds me of the "RAII vs. scope" arguments that came up on
the forum a few times (see e.g. [1]). Many of the features in D
(and other programming languages) can be theoretically (and
practically, in some languages) implemented using the type
system: resource management, template constraints, in/out
contracts, function attributes (pure, nothrow, etc.), I/O
streams, error handling, and probably others. This is sometimes
profitable and sometimes not so much.
If you're using some concept all over your code then
encapsulating it in a type can reduce code duplication, improve
readability and let you exploit various features of the type
system to your benefit (e.g. subtyping: A RandomAccessRange being
a subtype of InputRange, exception hierarchies, etc.). But if
you're only using it once or twice then it's probably not worth
the hassle. That is, the issue is scale.
My general impression is that C++ tries (and often succeeds,
pyrrhically) to do pretty much everything using the type system
(specifically, classes): Resources, exceptions, iterators,
interfaces, initialization lists, ... Almost everything is
encapsulated in a type. Concepts (Lite) seem to me like a way to
further enrich the type system, although concepts aren't exactly
types.
My opinion, in a nutshell, is that "ad-hocish" facilities like
template constraints, the scope statement, contracts, etc. don't
scale up, while more formal facilities like concepts, RAII, etc.
don't scale *down*. As such, there is a need for both of them.
With respect to the specific issue of ranges, I don't know Phobos
well enough to have an opinion as to whether or not they should
be made more formal (as types/concepts/...). The error messages
should be more clear and precise, but I'm not convinced that an
overhaul of the system is necessary to achieve that.
[1]
https://forum.dlang.org/post/tlsrpqvfeepcfbyfq...@forum.dlang.org
Re: Function attribute deduction depends on compile invocation
On Saturday, 15 July 2017 at 12:29:09 UTC, Johan Engelen wrote: On Friday, 14 July 2017 at 23:51:24 UTC, Walter Bright wrote: On 7/14/2017 9:53 AM, Johan Engelen wrote: What happens in that in one kind of compilation, @nogc is deduced for a function. But in another compilation, it isn't. Thus references to the function will have the wrong mangling, and linker errors happen: https://issues.dlang.org/show_bug.cgi?id=17541 At the moment you can work around it by explicitly adding the annotation. Cannot add something to a compiler internal function (e.g. __fieldPostblit), @nogc and pure are infectious, the templates are instantiated with different types where some instantiations require deducing @nogc/pure/..., others require @gc/impure/..., annotations cannot be applied conditionally (at least I don't know how), cannot add @gc, cannot add @impure, etc. Not that it helps you at the moment, but thus is addressed by my attributes DIP. I suppose you could abuse @allocSize to force @gc, but thats very ugly.
Re: Function attribute deduction depends on compile invocation
On Friday, 14 July 2017 at 23:51:24 UTC, Walter Bright wrote: On 7/14/2017 9:53 AM, Johan Engelen wrote: What happens in that in one kind of compilation, @nogc is deduced for a function. But in another compilation, it isn't. Thus references to the function will have the wrong mangling, and linker errors happen: https://issues.dlang.org/show_bug.cgi?id=17541 At the moment you can work around it by explicitly adding the annotation. Cannot add something to a compiler internal function (e.g. __fieldPostblit), @nogc and pure are infectious, the templates are instantiated with different types where some instantiations require deducing @nogc/pure/..., others require @gc/impure/..., annotations cannot be applied conditionally (at least I don't know how), cannot add @gc, cannot add @impure, etc. My current workarounds force deduction one way or another, but it's definitely not nice and it is very time consuming to figure out what tricks the compiler into doing the right thing. Each error needs it's own special work around. For example, working around the __fieldPostblit deduction problem involved searching all member fields (and their members, etc.) to try to find which this(this) should be annotated to fix things. Not so nice to do for a somewhat complex struct in a 9MB D-code base, sprinkled with conditional compilation, templates, CTFE... Each trial build (incremental) takes approx a minute. Fun times. Fortunately, it looks like I've "fixed" all issues and we can now link; kicked of the testing, let's hope there are not too many regression bugs at runtime. -Johan
Weka launched out of stealth mode a couple days ago
Here are a bunch of write-ups taken from their twitter feed: http://searchstorage.techtarget.com/news/450422665/New-WekaIO-file-system-incorporates-flash-object-store https://www.theregister.co.uk/2017/07/13/wekaio_surfaces_after_swimming_submerged_against_the_current/ https://gestaltit.com/exclusive/stephen/wekaio-distributed-storage-cloud/ https://storageswiss.com/2017/07/12/solve-the-unstructured-data-paradox-wekaio/ http://www.computerweekly.com/news/450422600/WekaIO-brings-brings-NAS-software-defined-storage https://insidebigdata.com/2017/07/13/wekaio-eliminates-need-legacy-external-storage-cloud-scale-storage-software/ Congrats to the Weka team, you can follow their twitter here: https://twitter.com/WekaIO
Re: newCTFE Status July 2017
On Saturday, 15 July 2017 at 09:02:02 UTC, Stefan Koch wrote: On Saturday, 15 July 2017 at 07:50:28 UTC, Dmitry wrote: ... All I have are the CTFE status threads, where one can see what I consider a working feature set. What is planed is simple to state: "Re-implement the full functionality of the current ctfe-interpreter in the new IR-based system" It is hard for me to say in advance which features will be working, or indeed regress next. Since the interactions between features form a complex system which by definition is very hard to predict. ... It would indeed be nice to have a GitHub issue (or similar) with progress checkboxes of what works, what's in progress, and what is yet to be done.
Re: D easily overlooked?
On Friday, 14 July 2017 at 08:57:17 UTC, Wulfklaue wrote: https://blog.sourced.tech/post/language_migrations/ A recent article where github programming languages popularity and migration got analysed was very interesting but it showed one noticeable thing: A total lack of D even mentioned!!! When looking at other language ranking sites, D always scores better then Rust. Yet, Rust gets included in the ranking but D is ... nowhere to be seen. It gets even a bit annoying when its always Rust, Rust, Rust ... that keeps popping up. Seen it more and more how Rust is simply trampling over any D messaging. D... It really has no very unique feature that makes it noticeable. * No Galactic overlord ( C#, Go, ... ) * no GC language that people can push until people there ears bleed ( Rust) * no really unique features that people care about to set it aside from C/C++, ... * It has the kitchen and sink but nobody talk about the kitchen and sink. I know people will jump onboard and start yelling how D has very unique features but from the "outside world" its always the same response. While more people are downloading D and trying it out, i barely see any independent D language blogs. Not to be a downer but D really in my eyes is missing that "unique" feature that people care about, that allows people to blog about the language... Well, at one point Andrei said that what is missing to make D's growth explosive is a strong corporate sponser [1]. This seemed sensible to me at the time and still seems sensible today. But I don't know what (if anything) can be done to increase the corporate community's interest in D. [1] CPPcast, Oct. 27, 2015: http://cppcast.com/2015/10/andrei-alexandrescu/ (at 25:30)
Re: Compilation times and idiomatic D code
On Saturday, 15 July 2017 at 11:10:32 UTC, Enamex wrote: On Friday, 14 July 2017 at 22:45:44 UTC, H. S. Teoh wrote: [...] I have some stupid questions: - What does everyone mean when they say 'symbol' here? I'm probably misunderstanding symbols gravely or it's something that DMD in particular handles in a strange way. [...] A symbol is anything which has an address and can/could be picked up by the linker. In other words a symbol is anything which has a name. (or could have a name)
Re: newCTFE Status July 2017
On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote: [ ... ] I fixed my struct ABI issues for now. The problem was the lowering of the IR types. Since before everything was expressible as pointer, it was fine to convert everything to integers and treat them as pointer. Since structs treated are value-types now we can no longer just point to them. However due to the premature conversion we would stick random intgeres with the pointer values inside the struct. This is what should happen: *(cast (struct_t*)targetMemory) = *(cast(struct_t*) sourceMemory); And this is what actually did happen: *(cast(size_t*) targetMemory) = cast(size_t) sourceMemory; Due to bizarre coincidences this wrong behavior would actually pass a few tests which were supposed to catch this mistake :) long story short. This bug is now fixed. I am assuming that there are still a few issues with arrayLiterals. But I will fix those another day. I am spent. Have a nice day, Stefan.
Re: Compilation times and idiomatic D code
On Friday, 14 July 2017 at 22:45:44 UTC, H. S. Teoh wrote: Here's a further update to the saga of combating ridiculously large symbol sizes. So yesterday I wrote a new module that also heavily uses UFCS chains. My initial draft of the module, once I linked it with the main program, particularly with a UFCS chain that has led to the 600MB executable sizes seen above, caused another explosion in symbol size that actually managed to reach 100MB in *one* symbol, triggering a DMD termination complaining about possible infinite template recursion. :-D Funnier still, temporarily simplifying part of the chain to bring symbol sizes down, I eventually got it below 100MB but ended up with linker segfaults and ELF errors because the huge symbol was too ridiculously huge. Eventually, it drove me to refactor two Phobos functions that are used heavily in my code: std.range.chunks and std.algorithm.joiner, using the same "horcrux" technique (see Phobos PRs #5610 and #5611). This, together with some further refactoring in my own code, eventually brought things down to the 20MB range of executable sizes. Then an idea occurred to me: the reason these symbol sizes got so large, was because the UFCS chain preserves *all* type information about every component type used to build the final type. So, by necessity, the type name has to somehow encode all of this information in an unambiguous way. Now, arguably, DMD's current mangling scheme is at fault because it contains too many repeating components, but even if you disregard that, the fact remains that if you have 50+ components in your overall UFCS chain, the symbol length cannot be less than 50*n where n is the average length of a single component's type name, plus some necessary overhead to account for the mangling scheme syntax. Let's say n is on average 20-25 characters, say round it up to 35 for mangling syntax, so you're still looking at upwards of 1700+ characters *minimum*. That, coupled with the current O(n^2) / O(n^3) mangling scheme, you easily reach megabytes of symbol length. We can compress the symbols all we want, but there's a limit as to how much compression will help. At the end of the day, you still have to represent those 50+ components *somehow*. But what if most of this type information is actually *unnecessary*? To use a range example, if all you care about at the end of the chain is that it's a forward range of ubyte, then why even bother with multi-MB symbols encoding type information that's never actually used? Maybe a little type-erasure would help, via hiding those 50+ component types behind an opaque runtime OO polymorphic interface. Phobos does have the facilities of this, in the form of the InputRange, ForwardRange, etc., interfaces in std.range.interfaces. In my case, however, part of the chain uses another generic type (a kind of generalization of 2D arrays). But either way, the idea is simple: at the end of the UFCS chain, wrap it in a class object that inherits from a generic interface that encodes only the primitives of the 2D array concept, and the element type. The class object itself, of course, still must retain knowledge of the full-fledged type, but the trick is that if we insert this type erasure in the middle of the chain, then later components don't have to encode the type names of earlier components anymore. T I have some stupid questions: - What does everyone mean when they say 'symbol' here? I'm probably misunderstanding symbols gravely or it's something that DMD in particular handles in a strange way. - What type information are being kept because of UFCS chains? Doesn't that mechanism simply apply overload resolution then choose between the prefix and .method forms as appropriate, rewriting the terms? Then it's a problem of function invocation. I don't get what's happening here still. Does this tie to the Voldemort types problem? (=> are many of the functions in the chain returning custom types?) It doesn't make sense, especially if, from your indirection workaround, it looks like it would work around the same but without the bloat problem if we unlinked the chain into many intermediate temporary parts. So how is this a type information issue? Thanks!
Re: Request for a more powerful template specialization feature
On Saturday, 15 July 2017 at 10:34:13 UTC, Enamex wrote: But specializations are quite different from constraints, no? Constraints wouldn't help when the template name is overloaded and passes the constraint checks of several different template implementations; specializations narrow things down for overload resolution. Unless I'm misunderstanding the OP or everyone else in the thread somehow :/ That's exactly right, you've probably put it better than I have! Specializations are quite different from constraints and I have found as you said that specializations are a great way to narrow things down for overload resolution. I use them to dispatch methods and they are the easiest way to specify a known set of kinds on a method set. When you narrow to your specialization set like that it will always work in the same way - no rouge types will mess things up.
Re: Some GC and emulated TLS questions (GDC related)
On Friday, 14 July 2017 at 09:13:26 UTC, Johannes Pfau wrote: Another solution could be to enhance libgcc emutls to allow custom allocators, then have a special allocation function in druntime for all D emutls variables. As far as I know there is no GC heap that is scanned, but not automatically collected? I believe that's what's done with the TLS ranges now, they're scanned but not collected, though they're not part of the GC heap. I'd need a way to completely manually manage GC.malloc/GC.free memory without the GC collecting this memory, but still scanning this memory for pointers. Does something like this exist? It doesn't have to be GC.malloc/GC.free, right? The current DMD-style emutls simply mallocs and frees the TLS data itself and only expects the GC to scan it. Another option is simply using the DMD-style emutls. But as far as I can see the DMD implementation never supported dynamic loading of shared libraries? This is something the GCC emutls support is quite good at: It doesn't have any platform dependencies (apart from mutexes and some way to store one thread specific pointer+destructor) and should work with all kinds of shared library combinations. DMD style emutls also does not allow sharing TLS variables between D and other languages. Yes, DMD's emutls was never made to work with loading multiple shared libraries. As for sharing with other languages without copying the TLS data over, that seems a rare scenario. So I was thinking, if DMD style emutls really doesn't support shared libraries, maybe we should just clone a GCC-style, compiler and OS agnostic emutls implementation into druntime? A D implementation could simply allocate all internal arrays using the GC. This should be just as efficient as the C implementation for variable access and interfacing to the GC is trivial. It gets somewhat more complicated if we want to use this in betterC though. We also lose C/C++ compatibility though by using such a custom implementation. It would be a good alternative to have, and you're not going to care in betterC mode, since there's no druntime or GC. You'd have to be careful how you called TLS data from C/C++, but it could still be done. The rest of this post is a description of the GCC emutls code. Someone can use this specification to implement a clean-room design D emutls clone. Source code can be found here, but beware of the GPL license: https://github.com/gcc-mirror/gcc/blob/master/libgcc/emutls.c [...] There is also this llvm implementation, available under permissive licenses and actually documented somewhat: https://github.com/llvm-mirror/compiler-rt/blob/master/lib/builtins/emutls.c
Re: Request for a more powerful template specialization feature
On Friday, 14 July 2017 at 19:49:05 UTC, Andrei Alexandrescu
wrote:
On 7/14/17 2:19 PM, data pulverizer wrote:
template Construct(R: Union{double, int}, W: Union{string,
char, dchar})
template Construct(R, W)
if ((is(R == double) || is(R == int))
&& (is(W == string) || is(W == char) || is(W == dchar))
It would be good to get comments and suggestions before I
write a DIP.
An effective way of improving the state of affairs would be to
create a PR that makes the constraint easier to read and write,
e.g.:
among!(R, double, int) && among!(W, string, char, dchar)
In fact it's surprising it hasn't been proposed yet.
Andrei
But specializations are quite different from constraints, no?
Constraints wouldn't help when the template name is overloaded
and passes the constraint checks of several different template
implementations; specializations narrow things down for overload
resolution.
Unless I'm misunderstanding the OP or everyone else in the thread
somehow :/
Re: newCTFE Status July 2017
On Saturday, 15 July 2017 at 07:50:28 UTC, Dmitry wrote:
On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote:
...
Hi. Have you any public roadmap (or somethilng like this) of
newCTFE?
Will be useful to see what planned, what finished, etc.
All I have are the CTFE status threads, where one can see what I
consider a working feature set.
What is planed is simple to state: "Re-implement the full
functionality of the current ctfe-interpreter in the new IR-based
system"
It is hard for me to say in advance which features will be
working, or indeed regress next.
Since the interactions between features form a complex system
which by definition is very hard to predict.
My current goal is to make complex structs work.
Meaning structs with other structs inside it.
For example
struct SimpleStruct
{
float c;
uint a;
ulong b;
}
struct complexStruct {
SimpleStruct[2] a;
complexStruct* next;
}
The complex struct will currently produce a vaild IR type.
But will generate invalid code, since the ABI is currently in the
process of changing.
And different parts of the code will interpret the data
differently.
On top of that I have to rebuild a few of the expression handling
routines to switch from the old reference based ABI to the new
hybrid ABI.
Re: newCTFE Status July 2017
On Thursday, 13 July 2017 at 12:45:19 UTC, Stefan Koch wrote: ... Hi. Have you any public roadmap (or somethilng like this) of newCTFE? Will be useful to see what planned, what finished, etc.
