Re: string file = __FILE__ considered harmful (and solution)
On Wednesday, 30 May 2018 at 14:40:50 UTC, Steven Schveighoffer
wrote:
On 5/30/18 4:27 AM, FeepingCreature wrote:
There's a very common idiom where in order to report line
numbers of an error or a log line at the callsite of a
function, you pass __FILE__ and __LINE__ as default parameters:
void foo(string file = __FILE__, size_t line = __LINE__);
What's wrong with this?
Say you add a string parameter, such as
void foo(string msg, string file = __FILE__, size_t line =
__LINE__);
foo("Hello World");
Now when you accidentally grab an old version of the library,
your new code will still run, but it will believe that it's
being called from file "Hello World", line 15. Not good.
Luckily there's a fix. Just stick this in some common header
file in your project:
struct CallerInfo
{
string file;
size_t line;
}
void foo(string msg, CallerInfo caller = CallerInfo(__FILE__,
__LINE__));
Now you cannot accidentally invoke foo with a string, or in
fact any type except another instance of CallerInfo.
Awesome idea! Unfortunately, it doesn't work. The __FILE__ and
__LINE__ there are not from the caller, but from the line that
defines foo.
See here: https://run.dlang.io/is/siz9YZ
At which point we can shorten this to CallerInfo caller =
__CALLER__, and be forward compatible for additional
information about the callsite, such as, say, attributes of
the calling function.
Hm.. I don't like this too much. Adding more magic to the
compiler seems unnecessary.
But if we fixed the behavior that causes your idea not to work,
then we could probably easily define a function like so:
CallerInfo __CALLER__(string file = __FILE__, size_t line =
__LINE__)
{
return CallerInfo(file, line);
}
-Steve
Instead of these hack keywords.
Perhaps a __traits() in the compiler with the information would
be better suited like:
void foo()
{
enum caller = __traits(getCaller);
}
getCaller would return a compile-time struct with additional
information about the current module and the module/function it
was called from.
Alternatively you can use the following traits.
true/false as secondary argument for whether it should be its
current module or the call module/function etc. This argument
should be optional and when omitted should default to the callee.
__FILE__ -- __traits(getFile);
__FILE_FULL_PATH__ -- __traits(getFilePath);
__MODULE__ -- __traits(getModule);
__LINE__ -- __traits(getLine);
__FUNCTION__ -- __traits(getFunction);
__PRETTY_FUNCTION__ -- __traits(getPrettyFunction);
Re: Default hashing function for AA's
On 2017-10-10 15:22:05 +, Steven Schveighoffer said: AA uses typeid(Key).getHash. [1] For objects, this calls the virtual function `toHash`. [2] Please keep in mind that all you are hashing is the class reference pointer, as that is the default comparison for `opEquals`. It might make sense to shuffle those bits a bit, since the bucket algorithm only looks at the lower bits of the hash, and this basically guarantees keys with the default hash will be empty in the first few buckets, since class objects are aligned on a power-of-2 boundary. But I'm not sure running a full blown hash on the pointer is necessary. Perhaps just xor-ing the upper bits with the lower bits makes more sense. Alternatively, you could change the default `opEquals` but that may break things more than expected. What you *can't* do is change what the hash is based on without changing `opEquals`. Note that I wouldn't recommend using an Object as a key without defining toHash and opEquals anyway. -Steve [1] https://github.com/dlang/druntime/blob/master/src/rt/aaA.d#L310 [2] https://github.com/dlang/druntime/blob/master/src/object.d#L66 Not sure this thread fits or a new one would be better. Anyway, here is an interesting article about a very fast hashtable and maybe it's a good candidate for the default AA implementation in D. IMO providing a very performant AA implementation is a big asset: https://probablydance.com/2018/05/28/a-new-fast-hash-table-in-response-to-googles-new-fast-hash-table/ -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: dub subpckages and how to depend on them internally
On Tuesday, 29 May 2018 at 23:41:59 UTC, aliak wrote:
Hi, I'm trying to get dub working with subpackages and I just
can't quite seem to hit the nail on the head. Any help would be
greatly appreciated.
This is the current setup is like this, and there's a shared
source folder as well called "common" and "sub2" depends on
"sub1".
lib
|-- dub.json
|-- source/
| -- sub1/
| -- package.d
| -- dub.json
| -- sub2/
| -- package.d
| -- dub.json
| -- common/
[...]
Halp!
I had a similar struggle, may be the version is the missing hint:
"dependencies": {
"diet-ng": "~>1.4",
,
mylib":{
"versions": "~master",
"path": "/home/mt/d/mylib"
},
}
Try to place "versions": "~master", beside your path.
Re: Comparing the c ffi overhead on various programming languages
On 28/05/2018 10:19 PM, rikki cattermole wrote: On 28/05/2018 10:16 PM, Robert M. Münch wrote: Might be interesting to checkout to find some optimization potential: https://github.com/dyu/ffi-overhead https://github.com/dyu/ffi-overhead/issues/5 With ldc, we are closer to zig now (equal/better), but c++ is still 8ms better than us. Either way we are better than Rust :)
Re: string file = __FILE__ considered harmful (and solution)
On 5/31/18 7:14 AM, bauss wrote:
Instead of these hack keywords.
Perhaps a __traits() in the compiler with the information would be
better suited like:
void foo()
{
enum caller = __traits(getCaller);
}
getCaller would return a compile-time struct with additional information
about the current module and the module/function it was called from.
How can this be possible? When inside the function, you have no idea
where you were called from.
-Steve
Re: dub subpckages and how to depend on them internally
On Tuesday, 29 May 2018 at 23:41:59 UTC, aliak wrote: Hi, I'm trying to get dub working with subpackages and I just can't quite seem to hit the nail on the head. Any help would be greatly appreciated. Move your sub packages out of source. And each package will have it's own src folder, which you may have.
Re: cycle dependencies
On 5/31/18 2:14 AM, Simen Kjærås wrote: On Wednesday, 30 May 2018 at 20:57:32 UTC, DigitalDesigns wrote: Why is this a runtime issue? It is not as if the execution of static this are non-deterministic. The compiler and linker must order the calls in some way. Maybe this is what you mean by own object/linker? But even then, they would only have to be checked once so why check every time the exe is ran when once it is ran it must remain statically oriented for all future. Because of separate compilation, the compiler can't do it. Because the generic linker doesn't do that sort of thing, the linker can't do it. The first part is essentially intractable - e.g. module A's static this uses a global variable in module B that's set by module C. Module A may be compiled separately from module C, so the compiler can't see the dependency. If the linker is to do it, the compiler needs to encode the information in the object file, and the linker must be made specially to support this. Maybe this could be put in some optional section in the object file, and linkers that don't support it would just ignore the information. If a non-compliant linker is used, the runtime needs to have a fallback, so even this won't get us entirely out of the woods. Only supporting special linkers comes with its own set of problems. Yes, this is really the crux of it. I want to stress that the cycle detection is not really for the sake of detecting cycles, it's because you need to sort the modules in the order their static ctors should be called. When you can't sort them in an order, you have to call them in an arbitrary order (probably just in the order they were linked). It is really a shame we have to do this at runtime, but that's due to the compilation model we are stuck with, and the linker tools we deal with. However, once the binary is formed, it's possible we could *edit* the binary to hard-code the order -- at that point, everything is known. I have thrown that out there in terms of a nice project someone with the correct skills could do: https://forum.dlang.org/post/nl3du7$1as5$1...@digitalmars.com One
Re: dub subpckages and how to depend on them internally
On Thursday, 31 May 2018 at 12:55:10 UTC, Martin Tschierschke
wrote:
I had a similar struggle, may be the version is the missing
hint:
"dependencies": {
"diet-ng": "~>1.4",
,
mylib":{
"versions": "~master",
"path": "/home/mt/d/mylib"
},
}
Try to place "versions": "~master", beside your path.
Unfortunately that did not work :(
Re: dub subpckages and how to depend on them internally
On Thursday, 31 May 2018 at 13:54:07 UTC, Jesse Phillips wrote: On Tuesday, 29 May 2018 at 23:41:59 UTC, aliak wrote: Hi, I'm trying to get dub working with subpackages and I just can't quite seem to hit the nail on the head. Any help would be greatly appreciated. Move your sub packages out of source. And each package will have it's own src folder, which you may have. Do you mean dub can't have source folder as a top level source directory if you use subpackages? So do I have to do this? root |-- sub1/source |-- sub2/source
Bad codegen, crashing (win64)
Can anyone that knows how help me with this: https://issues.dlang.org/show_bug.cgi?id=18928 This is a really bad look for my recent evaluation here :/ If a prompt fix is possible, it might not put a bullet in my evaluation, but this rattles peoples confidence.
Re: dub subpckages and how to depend on them internally
On 31.05.2018 20:56, aliak wrote: On Thursday, 31 May 2018 at 13:54:07 UTC, Jesse Phillips wrote: On Tuesday, 29 May 2018 at 23:41:59 UTC, aliak wrote: Hi, I'm trying to get dub working with subpackages and I just can't quite seem to hit the nail on the head. Any help would be greatly appreciated. Move your sub packages out of source. And each package will have it's own src folder, which you may have. Do you mean dub can't have source folder as a top level source directory if you use subpackages? So do I have to do this? root |-- sub1/source |-- sub2/source Yes
Re: Tiny D suitable for embedded JIT
On Wednesday, 30 May 2018 at 00:05:52 UTC, rikki cattermole wrote: https://github.com/dlang/dmd/blob/master/src/dmd/glue.d Hi - not really as I don't know what this does. In any case my understanding is the interface between the front-end and GDC/LDC is at the level of ASTs. The input is the AST, the output to the backend is some form of IR in essence. It just maps one understanding of the code to another form, that's all. Okay - I was trying to understand if there was some sort of IR that is intermediate stage before codegen - but I couldn't see this. Also the code is quite hard to follow without enough documentation of what's going on. Plus lots of global state I think which is fine for a command line tool but not a JIT engine. But really I looked only for a short while so please correct me if I am wrong. I decided to use a cut-down version of Eclipse OMR - the backend is much smaller than LLVM, although not as small as I would like. But I hope to create a more trimmed version in due course. (https://github.com/dibyendumajumdar/nj) Regards
Re: dub subpckages and how to depend on them internally
On Thursday, 31 May 2018 at 17:56:57 UTC, aliak wrote: On Thursday, 31 May 2018 at 13:54:07 UTC, Jesse Phillips wrote: On Tuesday, 29 May 2018 at 23:41:59 UTC, aliak wrote: Hi, I'm trying to get dub working with subpackages and I just can't quite seem to hit the nail on the head. Any help would be greatly appreciated. Move your sub packages out of source. And each package will have it's own src folder, which you may have. Do you mean dub can't have source folder as a top level source directory if you use subpackages? So do I have to do this? root |-- sub1/source |-- sub2/source Do that. You can have a top level source folder, but this is going to be code separate from your subpackage. The subpackage can depend on this top-level project, or the top-level project can depend on the subpackage.
Re: cycle dependencies
On Thursday, 31 May 2018 at 15:15:44 UTC, Steven Schveighoffer wrote: On 5/31/18 2:14 AM, Simen Kjærås wrote: On Wednesday, 30 May 2018 at 20:57:32 UTC, DigitalDesigns wrote: Why is this a runtime issue? It is not as if the execution of static this are non-deterministic. The compiler and linker must order the calls in some way. Because of separate compilation, the compiler can't do it. Because the generic linker doesn't do that sort of thing, the linker can't do it. The first part is essentially intractable - e.g. module A's static this uses a global variable in module B that's set by module C. Module A may be compiled separately from module C, so the compiler can't see the dependency. Yes, this is really the crux of it. I want to stress that the cycle detection is not really for the sake of detecting cycles, it's because you need to sort the modules in the order their static ctors should be called. When you can't sort them in an order, you have to call them in an arbitrary order (probably just in the order they were linked). It is really a shame we have to do this at runtime, but that's due to the compilation model we are stuck with, and the linker tools we deal with. Thinking about this problem for a while I can up with something that could both reduce the work the runtime had to do and allow us to remove the error in the OP. But now I see most of it was suggested in that pull request. [1] Would the best way to implement this be to extend ModuleInfo and include a getter that loads the dependencies like importedModules(), or should the ctor/dtor stuff be moved to a new tables? Also we might be able to get the compiler to insert a cluster_hash that's unique for each compiler run and a pre-ordered cluster_index. Then use this to speed up sorting in the runtime. [1] https://github.com/dlang/druntime/pull/1602#issuecomment-231527759
std.digest can't CTFE?
"CTFE Digests do not work in CTFE" That's an unfortunate limitation... why is, those things? :(
Re: std.digest can't CTFE?
On Thursday, May 31, 2018 14:29:13 Manu via Digitalmars-d wrote: > "CTFE > Digests do not work in CTFE" > > > That's an unfortunate limitation... why is, those things? :( If I had to guess without looking at the code? I would guess that it's doing various casts to hash stuff, and no kind of reintpret cast is allowed in CTFE. But really, you'd have to actually run the code, see what works and what doesn't, and look at each error you get when it doesn't to see what it's doing that can't be done during CTFE. Depending on what it's doing, it may be possible to make it work during CTFE, or it may not. CTFE can do a lot, but there's also a lot that it can't do - especially if you start doing anything low level. - Jonathan M Davis
Re: cycle dependencies
On 5/31/18 5:12 PM, David Bennett wrote: On Thursday, 31 May 2018 at 15:15:44 UTC, Steven Schveighoffer wrote: On 5/31/18 2:14 AM, Simen Kjærås wrote: On Wednesday, 30 May 2018 at 20:57:32 UTC, DigitalDesigns wrote: Why is this a runtime issue? It is not as if the execution of static this are non-deterministic. The compiler and linker must order the calls in some way. Because of separate compilation, the compiler can't do it. Because the generic linker doesn't do that sort of thing, the linker can't do it. The first part is essentially intractable - e.g. module A's static this uses a global variable in module B that's set by module C. Module A may be compiled separately from module C, so the compiler can't see the dependency. Yes, this is really the crux of it. I want to stress that the cycle detection is not really for the sake of detecting cycles, it's because you need to sort the modules in the order their static ctors should be called. When you can't sort them in an order, you have to call them in an arbitrary order (probably just in the order they were linked). It is really a shame we have to do this at runtime, but that's due to the compilation model we are stuck with, and the linker tools we deal with. Thinking about this problem for a while I can up with something that could both reduce the work the runtime had to do and allow us to remove the error in the OP. But now I see most of it was suggested in that pull request. [1] Would the best way to implement this be to extend ModuleInfo and include a getter that loads the dependencies like importedModules(), or should the ctor/dtor stuff be moved to a new tables? It's really an interesting problem, probably one of the most puzzling algorithmic problems I've had to solve. However, a lot of this is complicated because we don't do it at compile-time, but at runtime instead. We have way more time and memory and whatever we want to throw at it if we can move the sorting and cycle check to build time instead of runtime. If we want to make the check more fine grained, we are talking metadata for every function call, and which static ctor-initialized items it may access, and which lines actually initialize them. Also we might be able to get the compiler to insert a cluster_hash that's unique for each compiler run and a pre-ordered cluster_index. Then use this to speed up sorting in the runtime. Yeah, I don't think the runtime speed is a huge problem. Initializing the hash doesn't take much time at all. Hm... I just had a crazy idea: what if D had a switch that allowed it to store a dependency graph of constructors into a json file, and then when you link, there is a wrapper which consumes all these files, runs the cycle detection and sort, and then compiles a perfectly sorted moduleinfo section to be included in the binary (obviously, written in D and compiled by the compiler). This doesn't affect the linker, and uses all the existing tools we have to create object files and binaries. I'll have to bring this out into its own thread so it's not buried. -Steve
Re: cycle dependencies
On Thursday, 31 May 2018 at 23:17:20 UTC, Steven Schveighoffer
wrote:
Hm... I just had a crazy idea: what if D had a switch that
allowed it to store a dependency graph of constructors into a
json file, and then when you link, there is a wrapper which
consumes all these files, runs the cycle detection and sort,
and then compiles a perfectly sorted moduleinfo section to be
included in the binary (obviously, written in D and compiled by
the compiler).
This is how languages get custom object formats and custom
linkers.
A fly in the ointment is .di files. This works today and does the
right thing:
// bar.d
import modulewithstaticctor;
shared static this() {}
string something = modulewithstaticctor.someValue;
// bar.di
// no static ctor, no imports
string something;
// foo.d
import bar, std.stdiod;
void main()
{
writefln(something);
}
So mixing that dependency graph with .di files is "here be
dragons" territory.
If the dependency data were inserted into the object files
instead (which it should be? that's what ModuleInfo is), then the
compiler could potentially read this out before linking, and we
could be sure that the resulting order is correct. Like, insert a
weak symbol into each normal object file saying there's no
dependency graph, and then the compiler can run on a set of
object files to produce a new object file with the complete
dependency graph as a strong symbol.
But that's more work.
Re: std.digest can't CTFE?
Hashing's not low-level. It would be great if these did CTFE; generating compile-time hashes is a thing that would be really useful! Right here, I have a string class that carries a hash around with it for comparison reasons. Such string literals would prefer to have CT hashes. On 31 May 2018 at 15:40, Jonathan M Davis via Digitalmars-d wrote: > On Thursday, May 31, 2018 14:29:13 Manu via Digitalmars-d wrote: >> "CTFE >> Digests do not work in CTFE" >> >> >> That's an unfortunate limitation... why is, those things? :( > > If I had to guess without looking at the code? I would guess that it's doing > various casts to hash stuff, and no kind of reintpret cast is allowed in > CTFE. But really, you'd have to actually run the code, see what works and > what doesn't, and look at each error you get when it doesn't to see what > it's doing that can't be done during CTFE. Depending on what it's doing, it > may be possible to make it work during CTFE, or it may not. CTFE can do a > lot, but there's also a lot that it can't do - especially if you start doing > anything low level. > > - Jonathan M Davis >
Re: std.digest can't CTFE?
On 05/31/2018 06:40 PM, Jonathan M Davis wrote: On Thursday, May 31, 2018 14:29:13 Manu via Digitalmars-d wrote: "CTFE Digests do not work in CTFE" That's an unfortunate limitation... why is, those things? :( If I had to guess without looking at the code? I would guess that it's doing various casts to hash stuff, and no kind of reintpret cast is allowed in CTFE. But really, you'd have to actually run the code, see what works and what doesn't, and look at each error you get when it doesn't to see what it's doing that can't be done during CTFE. Depending on what it's doing, it may be possible to make it work during CTFE, or it may not. CTFE can do a lot, but there's also a lot that it can't do - especially if you start doing anything low level. - Jonathan M Davis I know the SHA1 implementation uses some inline asm. Although, now that I think about it, I think that might just be one specialization of the implementation, not the only implementation. Regardless, I do know that std.digest dates back a long time to when CTFE was WAY more limited that it is today. Most likely, CTFE at the time probably just wasn't quite up to the task, so it was simply left as runtime-only. A lot of Phobos was like that back then. Heck, I wouldn't even be surprised if that note in the docs turned out to be outdated and it had magically started working at CTFE at some point. But even if not, I'd say it's almost certainly just a classic case of "Nobody's needed it badly enough yet to get it working."
Re: std.digest can't CTFE?
On 31 May 2018 at 18:14, Nick Sabalausky (Abscissa) via Digitalmars-d wrote: > On 05/31/2018 06:40 PM, Jonathan M Davis wrote: >> >> On Thursday, May 31, 2018 14:29:13 Manu via Digitalmars-d wrote: >>> >>> "CTFE >>> Digests do not work in CTFE" >>> >>> >>> That's an unfortunate limitation... why is, those things? :( >> >> >> If I had to guess without looking at the code? I would guess that it's >> doing >> various casts to hash stuff, and no kind of reintpret cast is allowed in >> CTFE. But really, you'd have to actually run the code, see what works and >> what doesn't, and look at each error you get when it doesn't to see what >> it's doing that can't be done during CTFE. Depending on what it's doing, >> it >> may be possible to make it work during CTFE, or it may not. CTFE can do a >> lot, but there's also a lot that it can't do - especially if you start >> doing >> anything low level. >> >> - Jonathan M Davis >> > > I know the SHA1 implementation uses some inline asm. Although, now that I > think about it, I think that might just be one specialization of the > implementation, not the only implementation. > > Regardless, I do know that std.digest dates back a long time to when CTFE > was WAY more limited that it is today. Most likely, CTFE at the time > probably just wasn't quite up to the task, so it was simply left as > runtime-only. A lot of Phobos was like that back then. > > Heck, I wouldn't even be surprised if that note in the docs turned out to be > outdated and it had magically started working at CTFE at some point. But > even if not, I'd say it's almost certainly just a classic case of "Nobody's > needed it badly enough yet to get it working." It doesn't work. I suspect a blunt pointer cast where a shift&mask could be is the culprit. The demo effect always seems to plague D every time I try and introduce new people >_<
Re: string file = __FILE__ considered harmful (and solution)
On 5/30/2018 2:45 PM, John Colvin wrote: https://run.dlang.io/is/oMe7KQ Less elegant, but solves the problem of accidental argument adding (CallerFile acts as a barrier). Unfortunately, while it works in theory, in practice the compiler crashes LOL Please post bug reports when you find compiler crashes. Thanks!
Re: string file = __FILE__ considered harmful (and solution)
On 5/30/2018 1:27 AM, FeepingCreature wrote:
There's a very common idiom where in order to report line numbers of an error or
a log line at the callsite of a function, you pass __FILE__ and __LINE__ as
default parameters:
void foo(string file = __FILE__, size_t line = __LINE__);
What's wrong with this?
Say you add a string parameter, such as
void foo(string msg, string file = __FILE__, size_t line = __LINE__);
foo("Hello World");
Now when you accidentally grab an old version of the library, your new code will
still run, but it will believe that it's being called from file "Hello World",
line 15.
A solution:
enum E { reserved }
void foo(E e = E.reserved, string file = __FILE__, size_t line = __LINE__);
void foo(string msg, E e = E.reserved, string file = __FILE__, size_t line =
__LINE__);
Re: std.digest can't CTFE?
On Thursday, May 31, 2018 21:14:18 Nick Sabalausky via Digitalmars-d wrote: > On 05/31/2018 06:40 PM, Jonathan M Davis wrote: > > On Thursday, May 31, 2018 14:29:13 Manu via Digitalmars-d wrote: > >> "CTFE > >> Digests do not work in CTFE" > >> > >> > >> That's an unfortunate limitation... why is, those things? :( > > > > If I had to guess without looking at the code? I would guess that it's > > doing various casts to hash stuff, and no kind of reintpret cast is > > allowed in CTFE. But really, you'd have to actually run the code, see > > what works and what doesn't, and look at each error you get when it > > doesn't to see what it's doing that can't be done during CTFE. > > Depending on what it's doing, it may be possible to make it work during > > CTFE, or it may not. CTFE can do a lot, but there's also a lot that it > > can't do - especially if you start doing anything low level. > > > > - Jonathan M Davis > > I know the SHA1 implementation uses some inline asm. Although, now that > I think about it, I think that might just be one specialization of the > implementation, not the only implementation. > > Regardless, I do know that std.digest dates back a long time to when > CTFE was WAY more limited that it is today. Most likely, CTFE at the > time probably just wasn't quite up to the task, so it was simply left as > runtime-only. A lot of Phobos was like that back then. > > Heck, I wouldn't even be surprised if that note in the docs turned out > to be outdated and it had magically started working at CTFE at some > point. But even if not, I'd say it's almost certainly just a classic > case of "Nobody's needed it badly enough yet to get it working." Yeah, unless there's something about the implementation that actually requires something low level that can't be done during CTFE, or it requires a C function, then it's almost certainly just a matter of someone taking the time to make it work during CTFE (even if that involves using __ctfe). Presumably, either no one has had enough of a need to make it work, or the few who did didn't want to go to the effort of making it work. I have no idea how much the need to use std.digest at compile-time actually comes up. I would have thought that in most cases, you'd be hashing a file or something else that's read at runtime, but I don't know. Personally, I think that I've used std.digest all of once. - Jonathan M Davis
