Re: char array weirdness
On Monday, March 28, 2016 16:29:50 H. S. Teoh via Digitalmars-d-learn wrote: > On Mon, Mar 28, 2016 at 04:07:22PM -0700, Jonathan M Davis via > Digitalmars-d-learn wrote: [...] > > > The range API considers all strings to have an element type of dchar. > > char, wchar, and dchar are UTF code units - UTF-8, UTF-16, and UTF-32 > > respectively. One or more code units make up a code point, which is > > actually something displayable but not necessarily what you'd call a > > character (e.g. it could be an accent). One or more code points then > > make up a grapheme, which is really what a displayable character is. > > When Andrei designed the range API, he didn't know about graphemes - > > just code units and code points, so he thought that code points were > > guaranteed to be full characters and decided that that's what we'd > > operate on for correctness' sake. > > [...] > > Unfortunately, the fact that the default is *not* to use graphemes makes > working with non-European language strings pretty much just as ugly and > error-prone as working with bare char's in European language strings. Yeah. Operating at the code point level instead of the code unit level is correct for more text than just operating at the code unit level (especially if you're dealing with char rather than wchar), but ultimately, it's definitely not correct, and there's plenty of text that will be processed incorrectly as code points. > I argue that auto-decoding, as currently implemented, is a net minus, > even though I realize this is unlikely to change in this lifetime. It > charges a constant performance overhead yet still does not guarantee > things will behave as the user would expect (i.e., treat the string as > graphemes rather than code points). I totally agree, and I think that _most_ of the Phobos devs agree at this point. It's Andrei that doesn't. But we have the twin problems of figuring out how to convince him and how to deal with the fact that changing it would break a lot of code. Unicoding is disgusting to deal with if you want to deal with it correctly _and_ be efficient about it, but hiding it doesn't really work. I think that the first steps are to make it so that the algorithms in Phobos will operate just fine on ranges of char and wchar in addition to dchar and move towards making it irrelevant wherever we can. Some functions (like filter) are going to have to be told what level to operate at and would be a serious problem if/when we switched away from auto-decoding, but many others (such as find) can be made not to care while still operating on Unicode correctly. And if we can get the amount code impacted low enough (at least as far as Phobos goes), then maybe we can find a way to switch away from auto-decoding. Ultimately though, I fear that we're stuck with it and that we'll just have to figure out how to make it work well for those who know what they're doing while minimizing the performance impact of auto-decoding on those who don't know what they're doing as much as we reasonably can. - Jonathan M Davis
Joining AliasSeq/TypeTuple s
Hi, i want to join two or more tupples in to one, with mixing the indexes like roundRobin but in compile time. unittest{ import std.meta; alias first=AliasSeq!(int, string,bool); alias second=AliasSeq!("abc","def","ghi"); alias third=... static assert(third==AliasSeq!(int, "abc", string, "def", bool, "ghi")); } How to obtain this kind of functionality ? Is there maybe some builin template for this in phobos ? I couldn't find this. Thank you Voitech.
Re: string and char[] in Phobos
On Friday, 18 March 2016 at 20:06:27 UTC, Jonathan M Davis wrote: When a function accepts const(char)[] than it can accept char[], const(char)[], const(char[]), immutable(char)[], and immutable(char[]), which, whereas if it accepts string, then all it accepts are immutable(char)[] and immutable(char[]). So, it's more So I need to use const(char)[] in my function definitions instead of in char[]? restrictive, but if you need to return a slice of the array you passed in, if your function accepts const rather than mutable or immutable, then the slice has to be const, and you've lost the type information, which is why inout exists - Well, I never got inout until now, thanks! [...] I don't know what you're using in Phobos that takes string and returns char[]. That implies an allocation, and if the function is pure, char[] may have been selected, because it could be implicitly converted to string thanks to the fact that the compiler could prove that the char[] being returned had to have been allocated in the function and that there could be no other references to that array. But without knowing exactly which functions you're talking about, I can't really say. In general though, the solution that we've gone with is to templatize functions that operate on strings, and a function that's taking a string explicitly is most likely storing it, in which case, it needs an explicit type, and using an immutable value ensures that it doesn't change later. I just got this feeling from using functions in the std.file module, like dirEntries and File constructor itself. After reading your explaination, it makes sense now. And with a second look up, most functions there ARE alread templatized. Thanks for your clarification. On a side note, I'd strongly argue against using "in" on function arguments that aren't delegates. in is equivalent to const scope, and scope currently does nothing for any types other than delegates - but it might later, in which case, you could be forced to change your code, depending on the exact semantics of scope for non-delegates. But it does _nothing_ now with non-delegate types regardless, so it's a meaningless attribute that might change meaning later, which makes using it a very bad idea IMHO. Just use const if you want const and leave scope for delegates. I'd actually love to see in deprecated, because it adds no value to the language (since it's equivalent to const scope, which you can use explicitly), and it hides the fact that scope is used. Well, this is too complicated level for me now. I'll get to that later when I learn more with the language. My take away from your post: - when the function is pure for the stringlike, use 'const(char)[]' or 'inout(char)[]' when neccessary. - when the argument is stored in the function, use string. - manually convert stringlike objects to string with to!string when calling those functions. are those above correct? - Jonathan M Davis
Re: Joining AliasSeq/TypeTuple s
On Tuesday, 29 March 2016 at 09:33:40 UTC, Voitech wrote: Hi, i want to join two or more tupples in to one, with mixing the indexes like roundRobin but in compile time. unittest{ import std.meta; alias first=AliasSeq!(int, string,bool); alias second=AliasSeq!("abc","def","ghi"); alias third=... static assert(third==AliasSeq!(int, "abc", string, "def", bool, "ghi")); } How to obtain this kind of functionality ? Is there maybe some builin template for this in phobos ? I couldn't find this. Thank you Voitech. import std.meta : AliasSeq; template RR(A...) if (!(A.length & 1)) { static if (A.length == 0) alias RR = AliasSeq!(); else { alias Left = A[0 .. $ / 2]; alias Right = A[$ / 2 .. $]; alias RR = AliasSeq!(Left[0], Right[0], RR!(Left[1 .. $], Right[1 .. $])); } } struct S(A...) {} // needed to reliably compare AliasSeq's for equality unittest { alias first = AliasSeq!(int, string, bool); alias second = AliasSeq!("abc", "def", "ghi"); alias third = RR!(first, second); static assert(is(S!third == S!(int, "abc", string, "def", bool, "ghi"))); }
infer type argument in classe constructor?
Hi, I'm writing a generic class: ```d struct Message { ... } class Decoder(MsgSrc) { } ``` When using it, I'd have to include the type of its argument: ``` void main() { Message[] src = ...; auto decoder = new Decoder!(Message[])(src); ... } ``` Can it be inferred so that I only need to write? ```d auto decoder = new Decoder(src); // you can infer the type from src. ```
Re: infer type argument in classe constructor?
On Tuesday, 29 March 2016 at 10:13:28 UTC, Puming wrote: Hi, I'm writing a generic class: ```d struct Message { ... } class Decoder(MsgSrc) { } ``` When using it, I'd have to include the type of its argument: ``` void main() { Message[] src = ...; auto decoder = new Decoder!(Message[])(src); ... } ``` Can it be inferred so that I only need to write? ```d auto decoder = new Decoder(src); // you can infer the type from src. ``` Nope. To see why, consider a class like this: class A(T) { T data; this(int n) { } } void main() { auto a = new A(3); // What is T? } The common solution is a simple 'create' function: Decoder!T decoder(T)(T msg) { return new Decoder!T(msg); } -- Simen
Re: infer type argument in classe constructor?
On Tuesday, 29 March 2016 at 10:13:28 UTC, Puming wrote: Hi, I'm writing a generic class: ```d struct Message { ... } class Decoder(MsgSrc) { } ``` When using it, I'd have to include the type of its argument: ``` void main() { Message[] src = ...; auto decoder = new Decoder!(Message[])(src); ... } ``` Can it be inferred so that I only need to write? ```d auto decoder = new Decoder(src); // you can infer the type from src. ``` You can't directly. This is (AFAIK) because this()() can also be templated, making it impossible to just derive. The common way in D to deal with this/work around it is to create a helper function that can infer it: ```D auto decoder(T)(T src) { return new Decoder!T(src); } auto dec = decoder(src) ``` This pattern is widely used in phobos (e.g. tuple and Tuple)
Re: parsing fastq files with D
On Thursday, 24 March 2016 at 06:34:51 UTC, rikki cattermole wrote: void popFront() { import std.string : indexOf; if (source is null) { isEmpty = true; return; } void tidyInput() { foreach(i, c; source) { switch(c) { case 0: .. case ' ': break; default: source = source[i .. $]; return; } } source = null; } tidyInput(); Do you mind to explain what is really going on at popFront() and tidyInput() functions?
Re: infer type argument in classe constructor?
On Tuesday, 29 March 2016 at 10:29:46 UTC, Simen Kjaeraas wrote: On Tuesday, 29 March 2016 at 10:13:28 UTC, Puming wrote: Hi, I'm writing a generic class: ```d struct Message { ... } class Decoder(MsgSrc) { } ``` When using it, I'd have to include the type of its argument: ``` void main() { Message[] src = ...; auto decoder = new Decoder!(Message[])(src); ... } ``` Can it be inferred so that I only need to write? ```d auto decoder = new Decoder(src); // you can infer the type from src. ``` Nope. To see why, consider a class like this: class A(T) { T data; this(int n) { } } void main() { auto a = new A(3); // What is T? } Sorry I don't see it. In this case, I don't see an ambiguity? `int n` and T are not connected, so invoking A(3) means you are only setting the argument n to 3, so T can not be infered, and the compiler could just complain 'generic type T is not provided'. The common solution is a simple 'create' function: Decoder!T decoder(T)(T msg) { return new Decoder!T(msg); } -- Simen
Re: parsing fastq files with D
On 30/03/2016 1:46 AM, eastanon wrote: On Thursday, 24 March 2016 at 06:34:51 UTC, rikki cattermole wrote: void popFront() { import std.string : indexOf; if (source is null) { isEmpty = true; return; } void tidyInput() { foreach(i, c; source) { switch(c) { case 0: .. case ' ': break; default: source = source[i .. $]; return; } } source = null; } tidyInput(); Do you mind to explain what is really going on at popFront() and tidyInput() functions? popFront is a special function used with input ranges. It has support in the language for e.g. foreach loops. You also need empty and front to go along with it however. With front returning a value and empty if there is any more items. The if statement in popFront is just to make empty set correctly (delayed). Now tidyInput basically just trims (on the left) the source and sets it to null if there is no more of it. That way things like " @Myid" will still work fine. If you need an explanation about the rest of popFront, let me know!
Error: template instance does not match template declaration
OS: Win 8.1 Pro DMD: v2.070.0 cmd: dmd main.d -ofconsole-app.exe -debug -unittest -wi Code is successfully compiled until I uncomment test1 function call in the main. If it's uncommented I get the following compile-time error: main.d(58): Error: template instance impl!(string, bool, ["y":true, "n":false]) does not match template declaration impl(S, T, Color[string] dict)(S arg) main.d(62): Error: template instance mainentry.test1!string error instantiating What have I done wrong? Thank you. import std.stdio; enum Color { RED, GREEN } private void impl(S, T, T[string] dict)(S arg) { writeln("S: ", S.stringof); writeln("T: ", T.stringof); writeln("dict: ", dict); } unittest { enum COLOR_MAP = [ "r": Color.RED, "g": Color.GREEN ]; void test0(S)(S arg) { impl!(S, Color, COLOR_MAP)(arg); } test0("000"); } void test1(S)(S arg) { impl!(S, bool, ["y": true, "n": false])(arg); } void main(string[] args) { // test1("111"); // if you uncomment this line you'll get a compile-time error }
Re: Error: template instance does not match template declaration
On 03/29/2016 11:21 AM, ref2401 wrote: > private void impl(S, T, T[string] dict)(S arg) { > writeln("S: ", S.stringof); > writeln("T: ", T.stringof); > writeln("dict: ", dict); So, dict is a template value parameter of type T[string]. I don't think you can use an associative array as a template value parameter. (Can we?) However, it is possible to use anything as an alias template parameter. If it's acceptable in your case, the following works: private void impl(S, T, alias dict)(S arg) if (is (typeof(dict) == T[S])) { (Or T[string]). Ali
Re: Error: template instance does not match template declaration
On Tuesday, 29 March 2016 at 18:29:27 UTC, Ali Çehreli wrote: On 03/29/2016 11:21 AM, ref2401 wrote: So, dict is a template value parameter of type T[string]. I don't think you can use an associative array as a template value parameter. (Can we?) However, it is possible to use anything as an alias template parameter. If it's acceptable in your case, the following works: private void impl(S, T, alias dict)(S arg) if (is (typeof(dict) == T[S])) { (Or T[string]). Ali alias works for me, thank you.
Re: How to be more careful about null pointers?
On Tuesday, 29 March 2016 at 06:21:49 UTC, Ali Çehreli wrote: parent.prep.bind is translated to the following by the compiler: "Call bind() for the object at address... let's calculate... Wherever parent is, we should add the offset of prep inside that object." Okay, that's helpful actually. So, when accessing members of an object, it won't trigger a NPE, because the object is not dereferenced immediately, but the expected offset to the member is dereferenced. D puts off resolving pointer addresses as much as possible, so instead of bind((*parent).(*prep)) it's more like bind(parentlocation + prepoffset) and only (this.)member resolves to (*(parentlocation + prepoffset).member). I thought it had an extra layer of pointer indirection, like if class object A has address 1234, and member A.b has offset 0x20, then you first dereference address 1234 (to get 4321 say) and then add 0x20 to that. But actually a class object is... already dereferenced, and is an integer representing the address to the object, not a pointer to that address. Variables representing fixed, definite memory locations really throws me sometimes. You can't assign to a const, because that assigns to const marked memory, and doesn't just move the variable somewhere else. And NPE doesn't occur with a class, because setting it to NULL, and accessing a member is more like this in C: intptr_t a = 0; (struct B*)(a+0x20).member and less like this in C: intptr_t* a = 0; (struct B*)(*a+0x20).member
Re: How to be more careful about null pointers?
On 03/29/2016 11:57 AM, cy wrote: > On Tuesday, 29 March 2016 at 06:21:49 UTC, Ali Çehreli wrote: >> parent.prep.bind is translated to the following by the compiler: >> >> "Call bind() for the object at address... let's calculate... Wherever >> parent is, we should add the offset of prep inside that object." > > Okay, that's helpful actually. So, when accessing members of an object, > it won't trigger a NPE, because the object is not dereferenced > immediately, but the expected offset to the member is dereferenced. D > puts off resolving pointer addresses as much as possible, so instead of > bind((*parent).(*prep)) it's more like bind(parentlocation + prepoffset) > and only (this.)member resolves to (*(parentlocation + prepoffset).member). > > I thought it had an extra layer of pointer indirection, like if class > object A has address 1234, The class object would be sitting at that address. Our access to it is through a class variable: Foo var = new Foo(); Although you don't see a pointer in sight, behind scenes 'var' is nothing but a pointer. > and member A.b has offset 0x20, then you > first dereference address 1234 (to get 4321 say) and then add 0x20 to > that. No, there is no dereferencing of 'var'. However, you could have 'Foo*', which would be the equivalent of a "pointer to pointer" in C. > But actually a class object is... already dereferenced, and is an > integer representing the address to the object, not a pointer to that > address. Correct. > Variables representing fixed, definite memory locations really throws me > sometimes. You can't assign to a const, because that assigns to const > marked memory The object *may* really be on const-marked memory but not necessarily. You can have const variables anywhere; the const access is checked at compile time. > and doesn't just move the variable somewhere else. That does not happen for various reasons. Firstly, if we could mutate, what would 'const' mean? Then, the compiler would have to have a complete view of the program to update all other references, which would be slow and the semantics would be difficult. > And > NPE doesn't occur with a class, because setting it to NULL, and > accessing a member is more like this in C: > > intptr_t a = 0; > (struct B*)(a+0x20).member > > and less like this in C: > > intptr_t* a = 0; > (struct B*)(*a+0x20).member Yes. Ali
Re: How to be more careful about null pointers?
Am Tue, 29 Mar 2016 06:00:32 + schrieb cy : > struct Database { >string derp; >Statement prepare(string s) { > return Statement(1234); >} > } > > struct Statement { >int member; >void bind(int column, int value) { > import std.stdio; > writeln("derp",member); >} > > } > > > class Wrapper { >Database something; >Statement prep; >this() { > something = Database("..."); > prep = something.prepare("..."); >} > } > > Wrapper oops; > void initialize() { >oops = new Wrapper(); > } > > class Entry { >Wrapper parent; >this(Wrapper parent) { > //this.parent = parent; > //oops > parent = parent; >} >void usefulmethod() { > parent.prep.bind(1,42); > //parent.prep.execute(); > //parent.prep.reset(); >} > } > > void main() { >initialize(); >auto entry = new Entry(oops); >entry.usefulmethod(); > } > > That program causes a segmentation fault on my machine. Somehow > despite never initializing Entry.parent, a class object (whose > default init is a null pointer), I can still call methods on it, > access members on it, and call methods on those members. No > warnings or errors. The segfault doesn't happen until the bind() > method. Dlang aborts programs that run into invalid states. Common invalid states are failing contract assertions, out of memory conditions or in this case a null-pointer dereference. When designing Dlang, Walter decided that null pointers don't require checks as the operating system will eventually abort the program and debuggers are designed to assist you in this situation. Some people with a different programming background feel that a modern language should wrap every pointer access in a check and possibly throw a recoverable Exception here or make null-pointer opt-in to begin with. The topic is still open for discussion: http://wiki.dlang.org/Language_issues#Null_References -- Marco
Re: I need some help benchmarking SoA vs AoS
On Saturday, 26 March 2016 at 17:43:48 UTC, maik klein wrote: On Saturday, 26 March 2016 at 17:06:39 UTC, ag0aep6g wrote: On 26.03.2016 18:04, ag0aep6g wrote: https://gist.github.com/aG0aep6G/a1b87df1ac5930870ffe/revisions PS: Those enforces are for a size of 100_000 not 1_000_000, because I'm impatient. Thanks, okay that gives me more more reliable results. for 1_000_000 benchmarking complete access AoS: 1 sec, 87 ms, 266 μs, and 4 hnsecs SoA: 1 sec, 491 ms, 186 μs, and 6 hnsecs benchmarking partial access AoS: 7 secs, 167 ms, 635 μs, and 8 hnsecs SoA: 1 sec, 20 ms, 573 μs, and 1 hnsec This is sort of what I expected. I will do a few more benchmarks now. I probably also randomize the inputs. Can you benchmark my implementation (http://dpaste.dzfl.pl/3de1e18756f8) against yours? It should have roughly the same API, except that mine doesn't support pre-allocation (reserving capacity) - I wanted to keep it minimalistic. During access it should have the same performance, however the insertion behavior should be noticeably different. I'm interested to see if one larger allocation on insertion would be faster than a number of small ones (or vice-versa). Though, AoS should be the fastest in this regard (growth performance).
Re: Joining AliasSeq/TypeTuple s
On 03/29/2016 02:33 AM, Voitech wrote: Hi, i want to join two or more tupples in to one, with mixing the indexes like roundRobin but in compile time. unittest{ import std.meta; alias first=AliasSeq!(int, string,bool); alias second=AliasSeq!("abc","def","ghi"); alias third=... static assert(third==AliasSeq!(int, "abc", string, "def", bool, "ghi")); } Just a reminder that if you needed to concatenate, then it would be trivial due to automatic expansion of AliasSeq elements: alias third = AliasSeq!(first, second); Ali
Re: char array weirdness
Am Mon, 28 Mar 2016 16:29:50 -0700 schrieb "H. S. Teoh via Digitalmars-d-learn" : > […] your diacritics may get randomly reattached to > stuff they weren't originally attached to, or you may end up with wrong > sequences of Unicode code points (e.g. diacritics not attached to any > grapheme). Using filter() on Korean text, even with autodecoding, will > pretty much produce garbage. And so on. I'm on the same page here. If it ain't ASCII parsable, you *have* to make a conscious decision about whether you need code units or graphemes. I've yet to find out about the use cases for auto-decoded code-points though. > So in short, we're paying a performance cost for something that's only > arguably better but still not quite there, and this cost is attached to > almost *everything* you do with strings, regardless of whether you need > to (e.g., when you know you're dealing with pure ASCII data). An unconscious decision made by the library that yields the least likely intended and expected result? Let me think ... mhhh ... that's worse than iterating by char. No talking back :p. -- Marco
Part of D available to run at compile time
is there documentation on which parts of D are available to compile time execution?
Re: Part of D available to run at compile time
On 03/29/2016 03:29 PM, Carl Sturtivant wrote: is there documentation on which parts of D are available to compile time execution? This seems to be the official spec: https://dlang.org/spec/function.html#interpretation See you at DConf! ;) Ali
Re: How to be more careful about null pointers?
On Monday, 28 March 2016 at 21:01:19 UTC, cy wrote: I finally found the null pointer. It took a week. I was assigning "db = db" when I should have been assigning "this.db = db". Terrible, I know. But... I invoked db.find_chapter.bindAll(8,4), when db was a null pointer. There was no null pointer error. No exception raised for dereferencing a null. I'm not in release mode. Assertions are enabled. Shouldn't that have raised a null pointer exception? Instead, it accesses db as if db were not null, producing a garbage structure in find_chapter, which bindAll chokes on, then causes the whole program to segfault. I realize enforce(db).find_chapter would work, but... I thought D was more careful about null pointers? At least enough to die on dereferencing them? You can use a lint to warn you about silly mistakes, check out dscanner and possibly others.
Re: char array weirdness
On Monday, 28 March 2016 at 22:34:31 UTC, Jack Stouffer wrote: void main () { import std.range.primitives; char[] val = ['1', '0', 'h', '3', '6', 'm', '2', '8', 's']; pragma(msg, ElementEncodingType!(typeof(val))); pragma(msg, typeof(val.front)); } prints char dchar Why? I've seen you so many time as a reviewer on dlang that I belive this Q is a joke. Even if obviously nobody can know everything... https://www.youtube.com/watch?v=l97MxTx0nzs seriously you didn't know that auto decoding is on and that it gives you a dchar...
Re: char array weirdness
On Tuesday, 29 March 2016 at 23:15:26 UTC, Basile B. wrote: I've seen you so many time as a reviewer on dlang that I belive this Q is a joke. Even if obviously nobody can know everything... https://www.youtube.com/watch?v=l97MxTx0nzs seriously you didn't know that auto decoding is on and that it gives you a dchar... It's not a joke. This is the first time I've run into this problem in my code. I just started using D more and more in my work and I've never written anything that was really string heavy.
Debugging D DLL from C# app with C linkage for native Unity 5 plugin
Apologies if this has been discussed before, but I wasn't able to find anything similar on the forums or web. I can't seem to figure out how to debug a D DLL from a C# EXE. (My actual purpose here is to use D to build native plugins for Unity 5, but Unity and Mono aren't necessary to repro the problem I'm running into.) The D DLL and C# are both built as 64-bit. (C# is not set to AnyCPU). Using VS 2015 and Visual D, I can drive the D DLL from the C# EXE without any problems, but the debugger doesn't load the D DLL symbols. The DLL and PDB are in the same folder as the C# EXE. Everything behaves the same if I link to the DLL statically using P/Invoke or dynamically using LoadLibrary. Everything from the D DLL is exposed as extern(C), listed in the .def, etc. When I drive the DLL from a D EXE, I can break into the debugger easily, whether I attach at launch or attach to the process when it's already running. I'm building the DLL using: dmd dllmain.d dll.def -w -wi -g -map -ofLogic.dll -m64 -debug -shared Anyone know what I should try next? Am I missing something simple? :) thanks! Thalamus
Re: char array weirdness
On Tue, Mar 29, 2016 at 11:15:26PM +, Basile B. via Digitalmars-d-learn wrote: > On Monday, 28 March 2016 at 22:34:31 UTC, Jack Stouffer wrote: > >void main () { > >import std.range.primitives; > >char[] val = ['1', '0', 'h', '3', '6', 'm', '2', '8', 's']; > >pragma(msg, ElementEncodingType!(typeof(val))); > >pragma(msg, typeof(val.front)); > >} > > > >prints > > > >char > >dchar > > > >Why? > > I've seen you so many time as a reviewer on dlang that I belive this Q > is a joke. > Even if obviously nobody can know everything... > > https://www.youtube.com/watch?v=l97MxTx0nzs > > seriously you didn't know that auto decoding is on and that it gives > you a dchar... Believe it or not, it was only last year (IIRC, maybe the year before) that Walter "discovered" that Phobos does autodecoding, and got pretty upset over it. If even Walter wasn't aware of this for that long... I used to be in favor of autodecoding, but more and more, I'm seeing that it was a bad choice. It's a special case to how ranges normally work, and this special case has caused a ripple of exceptional corner cases to percolate throughout all Phobos code, leaving behind a string(!) of bugs over the years that, certainly, eventually got addressed, but nevertheless it shows that something didn't quite fit in. It also left behind a trail of additional complexity to deal with these special cases that made Phobos harder to understand and maintain. It's a performance bottleneck for string-processing code, which is a pity because D could have stood the chance to win against C/C++ string processing (due to extensive need to call strlen and strdup). But in spite of this heavy price we *still* don't guarantee correctness. On the spectrum of speed (don't decode at all) vs. correctness (segment by graphemes, not by code units or code points) autodecoding lands in the anemic middle where you get neither speed nor full correctness. The saddest part of it all is that this is unlikely to change because people have gotten so uptight about the specter of breaking existing code, in spite of the repeated experiences of newbies (and not-so-newbies like Walter himself!) wondering why strings have ElementType == dchar instead of char, usually followed by concerns over the performance overhead. T -- Designer clothes: how to cover less by paying more.
Re: char array weirdness
On 3/29/16 7:42 PM, H. S. Teoh via Digitalmars-d-learn wrote: On Tue, Mar 29, 2016 at 11:15:26PM +, Basile B. via Digitalmars-d-learn wrote: On Monday, 28 March 2016 at 22:34:31 UTC, Jack Stouffer wrote: void main () { import std.range.primitives; char[] val = ['1', '0', 'h', '3', '6', 'm', '2', '8', 's']; pragma(msg, ElementEncodingType!(typeof(val))); pragma(msg, typeof(val.front)); } prints char dchar Why? I've seen you so many time as a reviewer on dlang that I belive this Q is a joke. Even if obviously nobody can know everything... https://www.youtube.com/watch?v=l97MxTx0nzs seriously you didn't know that auto decoding is on and that it gives you a dchar... Believe it or not, it was only last year (IIRC, maybe the year before) that Walter "discovered" that Phobos does autodecoding, and got pretty upset over it. If even Walter wasn't aware of this for that long... Phobos treats narrow strings (wchar[], char[]) as ranges of dchar. It was discovered that auto decoding strings isn't always the smartest thing to do, especially for performance. So you get things like this: https://github.com/D-Programming-Language/phobos/blob/master/std/algorithm/searching.d#L1622 That's right. Phobos insists that auto decoding must happen for narrow strings. Except that's not the best thing to do so it inserts lots of exceptions -- for narrow strings. Mind blown? -Steve
key in aa.keys, but aa[key] give range violation?
My code looks something like this: bool[ulong][ulong] edge; foreach(u; from) foreach(v; to_) edge[u][v] = true; foreach(u; edge.keys) { auto adj = edge[u]; // } And sometimes edge[u] would give Range violation error.
Re: char array weirdness
On Wednesday, 30 March 2016 at 00:05:29 UTC, Steven Schveighoffer wrote: On 3/29/16 7:42 PM, H. S. Teoh via Digitalmars-d-learn wrote: On Tue, Mar 29, 2016 at 11:15:26PM +, Basile B. via Digitalmars-d-learn wrote: On Monday, 28 March 2016 at 22:34:31 UTC, Jack Stouffer wrote: void main () { import std.range.primitives; char[] val = ['1', '0', 'h', '3', '6', 'm', '2', '8', 's']; pragma(msg, ElementEncodingType!(typeof(val))); pragma(msg, typeof(val.front)); } prints char dchar Why? I've seen you so many time as a reviewer on dlang that I belive this Q is a joke. Even if obviously nobody can know everything... https://www.youtube.com/watch?v=l97MxTx0nzs seriously you didn't know that auto decoding is on and that it gives you a dchar... Believe it or not, it was only last year (IIRC, maybe the year before) that Walter "discovered" that Phobos does autodecoding, and got pretty upset over it. If even Walter wasn't aware of this for that long... Phobos treats narrow strings (wchar[], char[]) as ranges of dchar. It was discovered that auto decoding strings isn't always the smartest thing to do, especially for performance. So you get things like this: https://github.com/D-Programming-Language/phobos/blob/master/std/algorithm/searching.d#L1622 That's right. Phobos insists that auto decoding must happen for narrow strings. Except that's not the best thing to do so it inserts lots of exceptions -- for narrow strings. Mind blown? -Steve https://www.youtube.com/watch?v=JKQwgpaLR6o Listen to this then it'll be more clear.
Re: key in aa.keys, but aa[key] give range violation?
On Wednesday, 30 March 2016 at 00:26:49 UTC, Yuxuan Shui wrote: My code looks something like this: bool[ulong][ulong] edge; foreach(u; from) foreach(v; to_) edge[u][v] = true; foreach(u; edge.keys) { auto adj = edge[u]; // } And sometimes edge[u] would give Range violation error. I guess I'm not supposed to do aa.remove() while iterate through it?
Re: key in aa.keys, but aa[key] give range violation?
On 03/29/2016 05:29 PM, Yuxuan Shui wrote: On Wednesday, 30 March 2016 at 00:26:49 UTC, Yuxuan Shui wrote: My code looks something like this: bool[ulong][ulong] edge; foreach(u; from) foreach(v; to_) edge[u][v] = true; foreach(u; edge.keys) { auto adj = edge[u]; // } And sometimes edge[u] would give Range violation error. I guess I'm not supposed to do aa.remove() while iterate through it? That was my first guess. No, you cannot mutate the container itself while iterating over it. However, the sample that you've shown should be fine as edge.keys is an eagerly generated array of keys. In that case, you can remove from the aa. Perhaps your actual code is different? Ali
Re: I need some help benchmarking SoA vs AoS
On Tuesday, 29 March 2016 at 21:27:15 UTC, ZombineDev wrote: On Saturday, 26 March 2016 at 17:43:48 UTC, maik klein wrote: On Saturday, 26 March 2016 at 17:06:39 UTC, ag0aep6g wrote: On 26.03.2016 18:04, ag0aep6g wrote: https://gist.github.com/aG0aep6G/a1b87df1ac5930870ffe/revisions PS: Those enforces are for a size of 100_000 not 1_000_000, because I'm impatient. Thanks, okay that gives me more more reliable results. for 1_000_000 benchmarking complete access AoS: 1 sec, 87 ms, 266 μs, and 4 hnsecs SoA: 1 sec, 491 ms, 186 μs, and 6 hnsecs benchmarking partial access AoS: 7 secs, 167 ms, 635 μs, and 8 hnsecs SoA: 1 sec, 20 ms, 573 μs, and 1 hnsec This is sort of what I expected. I will do a few more benchmarks now. I probably also randomize the inputs. Can you benchmark my implementation (http://dpaste.dzfl.pl/3de1e18756f8) against yours? It should have roughly the same API, except that mine doesn't support pre-allocation (reserving capacity) - I wanted to keep it minimalistic. During access it should have the same performance, however the insertion behavior should be noticeably different. I'm interested to see if one larger allocation on insertion would be faster than a number of small ones (or vice-versa). Though, AoS should be the fastest in this regard (growth performance). Yes I'll do it tomorrow, though I am not sure how much you can rely on the data. I think it really depends on how fragmented the heap currently is. I will also benchmark memory access, just to see if there is any difference. I recently asked a question about it on SO https://stackoverflow.com/questions/36296960/pros-and-cons-of-one-big-buffer-vs-many-smaller-buffer? Doesn't seem to be well received though.
Re: Part of D available to run at compile time
On 30/03/2016 11:29 AM, Carl Sturtivant wrote: is there documentation on which parts of D are available to compile time execution? Rule of thumb, if it can be pure and @safe, you're good to go.
Re: char array weirdness
On Tuesday, 29 March 2016 at 23:42:07 UTC, H. S. Teoh wrote: Believe it or not, it was only last year (IIRC, maybe the year before) that Walter "discovered" that Phobos does autodecoding, and got pretty upset over it. If even Walter wasn't aware of this for that long... The link (I think this is what you're referring to) to that discussion: http://forum.dlang.org/post/lfbg06$30kh$1...@digitalmars.com It's a shame Walter never got his way. Special casing ranges like this is a huge mistake.
Re: char array weirdness
On Wed, Mar 30, 2016 at 03:22:48AM +, Jack Stouffer via Digitalmars-d-learn wrote: > On Tuesday, 29 March 2016 at 23:42:07 UTC, H. S. Teoh wrote: > >Believe it or not, it was only last year (IIRC, maybe the year > >before) that Walter "discovered" that Phobos does autodecoding, and > >got pretty upset over it. If even Walter wasn't aware of this for > >that long... > > The link (I think this is what you're referring to) to that > discussion: http://forum.dlang.org/post/lfbg06$30kh$1...@digitalmars.com > > It's a shame Walter never got his way. Special casing ranges like this > is a huge mistake. To be fair, one *could* make a case of autodecoding, if it was done right, i.e., segmenting by graphemes, which is what is really expected by users when they think of "characters". This would allow users to truly think in terms of characters (in the intuitive sense) when they work with strings. However, segmenting by graphemes is, in general, quite expensive, and few algorithms actually need to do this. Most don't need to -- a pretty large part of string processing consists of looking for certain markers, mostly punctuation and control characters, and treating the stuff in between as opaque data. If we didn't have autodecoding, would be a simple matter of searching for sentinel substrings. This also indicates that most of the work done by autodecoding is unnecessary -- it's wasted work since most of the string data is treated opaquely anyway. The unfortunate situation in Phobos currently is that we are neither doing it right (segmenting by graphemes), *and* we're inefficient because we're constantly decoding all that data that the application is mostly going to treat as opaque data anyway. It's the worst of both worlds. I wish we could get consensus for implementing Walter's plan to phase out autodecoding (as proposed in the linked thread above). T -- Freedom of speech: the whole world has no right *not* to hear my spouting off!
Re: char array weirdness
On Tue, Mar 29, 2016 at 08:05:29PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: [...] > Phobos treats narrow strings (wchar[], char[]) as ranges of dchar. It > was discovered that auto decoding strings isn't always the smartest > thing to do, especially for performance. > > So you get things like this: > https://github.com/D-Programming-Language/phobos/blob/master/std/algorithm/searching.d#L1622 > > That's right. Phobos insists that auto decoding must happen for narrow > strings. Except that's not the best thing to do so it inserts lots of > exceptions -- for narrow strings. > > Mind blown? [...] Mind not blown. Mostly because I've seen many, many instances of similar code in Phobos. It's what I was alluding to when I said that special-casing strings has caused a ripple of exceptional cases to percolate throughout Phobos, increasing code complexity and making things very hard to maintain. I mean, honestly, just look at that code as linked above. Can anyone honestly claim that this is maintainable code? For something so trivial as linear search of strings, that's some heavy hackery just to make strings work, as contrasted with, say, the one-line call to simpleMindedFind(). Who would have thought linear string searching would require a typecast, a @trusted hack, and templates named "force" just to make things work? It's code like this -- and its pervasive ugliness throughout Phobos -- that slowly eroded my original pro-autodecoding stance. It's becoming clearer and clearer to me that it's just not pulling its own weight against the dramatic increase in Phobos code complexity, nevermind the detrimental performance consequences. T -- Obviously, some things aren't very obvious.
Re: Debugging D DLL from C# app with C linkage for native Unity 5 plugin
On 30.03.2016 01:41, Thalamus wrote: Apologies if this has been discussed before, but I wasn't able to find anything similar on the forums or web. I can't seem to figure out how to debug a D DLL from a C# EXE. (My actual purpose here is to use D to build native plugins for Unity 5, but Unity and Mono aren't necessary to repro the problem I'm running into.) The D DLL and C# are both built as 64-bit. (C# is not set to AnyCPU). Using VS 2015 and Visual D, I can drive the D DLL from the C# EXE without any problems, but the debugger doesn't load the D DLL symbols. The DLL and PDB are in the same folder as the C# EXE. Everything behaves the same if I link to the DLL statically using P/Invoke or dynamically using LoadLibrary. Everything from the D DLL is exposed as extern(C), listed in the .def, etc. When I drive the DLL from a D EXE, I can break into the debugger easily, whether I attach at launch or attach to the process when it's already running. I'm building the DLL using: dmd dllmain.d dll.def -w -wi -g -map -ofLogic.dll -m64 -debug -shared Anyone know what I should try next? Am I missing something simple? :) thanks! Thalamus I haven't tried debugging a C# application, but you might have to disable "Just My Code" in the global debugger options. Also make sure to allow native debugging (JIT options). When debugging Visual D (a D DLL), I set VS (devenv.exe, a mixed C++/C# app) as the program to debug in the project debugger options. This ensures a native debugger engine is used and is the simplest way to use the mago debugger. With the "Mixed mode" engine, you can debug both C# and D/C++ in the same session.