Re: Fake IFTI-compatible struct constructors
On Saturday, 1 May 2021 at 23:21:33 UTC, Basile B. wrote: On Saturday, 1 May 2021 at 21:57:54 UTC, Chad Joan wrote: ... Rather, this setup doesn't work at all, because IFTI doesn't seem to work on function-templates with alias parameters. Yes your observation is correct. That should work but this is currently not implemented and referenceced as [issue 20877](https://issues.dlang.org/show_bug.cgi?id=20877) Good to know. Thank you!
Fake IFTI-compatible struct constructors
I came up with a couple techniques for making it seem like
templated structs deduce their template parameters from
constructor invocations. These are given further down in the post.
This has been a profitable exercise. However, the techniques I
came up with have some drawbacks.
Thus, I have some questions for the community:
Is there a better way?
Do any of you have other constructor-IFTI-faking techniques to
share?
Maybe one of you has encountered this already and come up
different ways to approach this. Please share if you can!
### Background:
I ended up distracted with trying to make IFTI (Implicit Function
Template Instantiation) work for a struct constructor.
I'm already aware of the normal factory function technique (ex:
have `foo(T)(T x)` construct `struct Foo(T)`) and have looked
into some of the history of this issue, like this thread from
April of 2013:
https://forum.dlang.org/thread/lkyjyhmirazaonbvf...@forum.dlang.org
I updated to DMD v2.096.1 to see if IFTI for constructors had
been implemented recently.
But it seems not, because code like this fails to compile:
```D
struct Foo(StrT)
{
StrT payload;
this(StrT text)
{
this.payload = text;
}
}
void main()
{
import std.stdio;
auto foo = Foo("x");
writeln(foo.payload);
}
```
Compilation results:
```
xfail.d(13): Error: struct `xfail.Foo` cannot deduce function
from argument types `!()(string)`, candidates are:
xfail.d(1):`Foo(StrT)`
```
What followed was a series of experiments to see if I could get
most of what I wanted by (ab)using existing D features.
### Technique 1: Mutually Exclusive Constraints
This technique was the first one I came up with.
It's disadvantage is that it only works for templated types with
parameter lists that meet specific criteria:
* At least one of the parameters must be a string or array type.
* There can't be any other versions of the template that take the
element's type at the same position.
So if I have a templated struct that parameterizes on string
types, then this works for that.
The advantage of this method (over the later method I discovered)
is that it doesn't regress the long or free-standing (non-IFTI)
form of template instantiation in any way that I've noticed. As
long as the above parameter list criteria can be met, it's
strictly an upgrade.
It looks like this:
```D
// Fake IFTI constructor for templates with at least one string
parameter.
auto Foo(Char)(Char[] text)
// The constraint basically just ensures
// that `Char` isn't a string or array.
if ( !is(Char T : T[]) )
{
import std.stdio;
Foo!(Char[]) foo;
foo.payload = text;
pragma(msg, "function Char.stringof == "~Char.stringof);
return foo;
}
struct Foo(StrT)
// This constraint ensures the opposite:
// that `StrT` is a string|array.
// It also declares `CharT` as it's
// element type, but that's quickly discarded.
if ( is(StrT CharT : CharT[]) )
{
StrT payload;
pragma(msg, "struct StrT.stringof == " ~ StrT.stringof);
/+
// These fail to compile.
// Presumably, `CharT` isn't declared /in this scope/.
CharT ch;
pragma(msg, "struct CharT.stringof == " ~ CharT.stringof);
// (It's not a big deal though. It's easy enough
// to get the element type from an array without
// the help of the template's parameter list.)
+/
}
void main()
{
import std.stdio;
// Test IFTI-based instantiation.
auto foo1 = Foo("1"); // IFTI for Foo constructor!
writeln(foo1.payload); // Prints: 1
// Test normal instantiation.
// (In other words: we try to avoid regressing this case.)
Foo!string foo2;
foo2.payload = "2";
writeln(foo2.payload); // Prints: 2
/// Accidental instantiations of the wrong template are
/// prevented by the `!is(Char T : T[])` template constraint.
// Foo!char foo3; // Error: `Foo!char` is used as a type
// foo3.payload = "3";
}
```
### Technique 2: Explicit Instantiation
I wanted a way to do this with templates that don't deal with
strings or arrays, and with templates that might accept either an
array or its element in the same parameter (or parameter
position).
So I did that, but it has a notable drawback: free-standing
instantiations like `Bar!int bar;` won't work without adding
additional template specializations for every type that might
need to be used that way.
This drawback isn't very severe for project internals, but I
think it's no good for library APIs.
I really hope someone has a way to do something like this, but
without this drawback.
Here's what this technique looks like:
```D
// This is like the usual helper function that returns
// an instance of the desired templated type.
// (AKA: a
Re: safety and auto vectorization
On Sunday, 2 August 2020 at 17:31:45 UTC, Bruce Carneal wrote:
import std;
void f0(int[] a, int[] b, int[] dst) @safe {
dst[] = a[] + b[];
}
void f1(int[] a, int[] b, int[] dst) @trusted {
const minLen = min(a.length, b.length, dst.length);
dst[0..minLen] = a[0..minLen] + b[0..minLen];
assert(dst.length == minLen);
}
I was surprised that f0 ran just fine with a.length and
b.length geq dst.length. Is that a bug or a feature?
Assuming it's a feature, are f0 and f1 morally equivalent? I
ask because f1 auto-vectorizes in ldc while f0 does not. Not
sure why. As a guess I'd say that the front end doesn't hoist
bounds checks in f0 or at least doesn't convey the info to the
back end in a comprehensible fashion. Non-guesses welcome.
I don't know what's going on auto-vectorization-wise, but to
address the behavioral issues, the next thing I would do if I
were in your shoes is something like this:
import std.stdio;
int[100] a, b, dst;
a[] = 2;
b[] = 3;
dst[] = 42;
f0(a[0..$], b[0..$], dst[0..50]); // Notice: dst is a smaller
slice.
writefln("dst[49] == %d", dst[49]); // Should be 5.
writefln("dst[50] == %d", dst[50]); // 42 or 5?
On my machine (Linux 64-bit DMD v2.093.0) it prints this:
dst[49] == 5
dst[50] == 42
Which suggests that it is doing the minimum-length calculation,
as the dst[] values outside of the lesser-sized slice were
untouched.
This was DMD, so it's going to be worth trying on your compiler
to see what you get.
Re: D on lm32-CPU: string argument on stack instead of register
On Saturday, 1 August 2020 at 08:58:03 UTC, Michael Reese wrote:
[...]
So the compiler knows how to use r1 and r2 for arguments. I
checked again in the lm32 manual
(https://www.latticesemi.com/view_document?document_id=52077),
and it says:
"As illustrated in Table 3 on page 8, the first eight function
arguments are
passed in registers. Any remaining arguments are passed on the
stack, as
illustrated in Figure 12."
So strings and structs should be passed on the stack and this
seems to be more an issue of the gcc lm32 backend than a D
issue.
Nice find!
Though if the compiler is allowed to split a single uint64_t into
two registers, I would expect it to split struct/string into two
registers as well. At least, the manual doesn't seem to
explicitly mention higher-level constructs like structs. It does
suggest a one-to-one relationship between arguments and registers
(up to a point), but GCC seems to have decided otherwise for
certain uint64_t's. (Looking at Table 3...) It even gives you two
registers for a return value: enough for a string or an array.
And if the backend/ABI weren't up for it, it would be
theoretically possible to have the frontend to lower strings
(dynamic arrays) and small structs into their components before
function calls and then also insert code on the other side to
cast them back into their original form. I'm not sure if anyone
would want to write it, though. o.O
But I just found a workaround using a wrapper function.
void write_to_host(in string msg) {
write_to_hostC(msg.ptr, msg.length);
}
I checked the assembly code on the caller side, and the call
write_to host("Hello, World!\n") is inlined. There is only one
call to write_to_hostC. This is still not nice, but I think I
can live with that for now. Now I have to figure out how make
the cast back from from pointer-length pair into a string. I'm
sure I read that somewhere before, but forgot it and was unable
to find it now on a quick google search...
That's pretty clever. I like it.
Getting from pointer-length to string might be pretty easy:
string foo = ptr[0 .. len];
D allows pointers to be indexed, like in C. But unlike C, D has
slices, and pointers can be "sliced". The result of a slice
operation, at least for primitive arrays+pointers, is always an
array (the "dynamic" ptr+length kind). Hope that helps.
And since this is D: is there maybe some CTFE magic that allows
to create these wrappers automatically? Somthing like
fix_stack!write_to_host("Hello, World!\n");
It ended up being a little more complicated than I thought it
would be. Hope I didn't ruin the fun. ;)
https://pastebin.com/y6e9mxre
Also, that part where you mentioned a 64-bit integer being passed
as a pair of registers made me start to wonder if unions could be
(ab)used to juke the ABI:
https://pastebin.com/eGfZN0SL
Good luck with your lm32/FPGA coding. That sounds like cool
stuff!
I'm doing this mainly to improve my understanding of how
embedded processors work, and how to write linker scripts for a
given environment. Although I do have actual hardware where I
can see if everything runs in the real world, I mainly use
simulations. The coolest thing in my opinion is, nowadays it
can be done using only open source tools (mainly ghdl and
verilator, the lm32 source code is open source, too). The
complete system is simulated and you can look at every logic
signal in the cpu or in the ram while the program executes.
Thanks for the insights; I've done just a little hobby electrical
stuff here-and-there, and having some frame of reference for tool
and component choice makes me feel good, even if I don't plan on
buying any lm32s or FPGAs anytime soon :) Maybe I can Google
some of that later and geek out at images of other people's
debugging sessions or something. I'm curious how they manage the
complexity that happens when circuits and massive swarms of logic
gates do their, uh, complexity thing. o.O
Re: D on lm32-CPU: string argument on stack instead of register
On Friday, 31 July 2020 at 10:22:20 UTC, Michael Reese wrote:
Hi all,
at work we put embedded lm32 soft-core CPUs in FPGAs and write
the firmware in C.
At home I enjoy writing small projects in D from time to time,
but I don't consider myself a D expert.
Now, I'm trying to run some toy examples in D on the lm32 cpu.
I'm using a recent gcc-elf-lm32. I succeeded in compiling and
running some code and it works fine.
But I noticed, when calling a function with a string argument,
the string is not stored in registers, but on the stack.
Consider a simple function (below) that writes bytes to a
peripheral (that forwards the data to the host computer via
USB). I've two versions, an ideomatic D one, and another
version where pointer and length are two distinct function
parameters.
I also show the generated assembly code. The string version is
4 instructions longer, just because of the stack manipulation.
In addition, it is also slower because it need to access the
ram, and it needs more stack space.
My question: Is there a way I can tell the D compiler to use
registers instead of stack for string arguments, or any other
trick to reduce code size while maintaining an ideomatic D
codestyle?
Best regards
Michael
// ideomatic D version
void write_to_host(in string msg) {
// a fixed address to get bytes to the host via usb
char *usb_slave = cast(char*)BaseAdr.ft232_slave;
foreach(ch; msg) {
*usb_slave = ch;
}
}
// resulting assembly code (compiled with -Os) 12 instructions
_D10firmware_d13write_to_hostFxAyaZv:
addi sp, sp, -8
addi r3, r0, 4096
sw (sp+4), r1
sw (sp+8), r2
add r1, r2, r1
.L3:
be r2,r1,.L1
lbu r4, (r2+0)
addi r2, r2, 1
sb (r3+0), r4
bi .L3
.L1:
addi sp, sp, 8
bra
// C-like version
void write_to_hostC(const char *msg, int len) {
char *ptr = cast(char*)msg;
char *usb_slave = cast(char*)BaseAdr.ft232_slave;
while (len--) {
*usb_slave = *ptr++;
}
}
// resulting assembly code (compiled with -Os) 8 instructions
_D10firmware_d14write_to_hostCFxPaiZv:
add r2, r1, r2
addi r3, r0, 4096
.L7:
be r1,r2,.L5
lbu r4, (r1+0)
addi r1, r1, 1
sb (r3+0), r4
bi .L7
.L5:
bra
Hi Michael!
Last time I checked, D doesn't have any specific type attributes
or special ways to force variables to enregister. But I could be
poorly informed. Maybe there are GDC-specific hints or something.
I hope that if anyone else knows better, they will toss in an
answer.
THAT SAID, I think there are things to try and I hope we can get
you what you want.
If you're willing to entertain more experimentation, here are my
thoughts:
---
(1) Try writing "in string" as "in const(char)[]" instead:
// ideomatic D version
void write_to_host(in const(char)[] msg) {
// a fixed address to get bytes to the host via usb
char *usb_slave = cast(char*)BaseAdr.ft232_slave;
foreach(ch; msg) {
*usb_slave = ch;
}
}
Explanation:
The "string" type is an alias for "immutable(char)[]".
In D, "immutable" is a stronger guarantee than "const". The
"const" modifier, like in C, tells the compiler that this
function shall not modify the data referenced by this
pointer/array/whatever. The "immutable" modifier is a bit
different, as it says that NO ONE will modify the data referenced
by this pointer/array/whatever, including other functions that
may or may not be concurrently executing alongside the one you're
in. So "const" constraints the callee, while "immutable"
constrains both the callee AND the caller. This makes it more
useful for some multithreaded code, because if you can accept the
potential inefficiency of needing to do more copying of data (if
you can't modify, usually you must copy instead), then you can
have more deterministic behavior and sometimes even much better
total efficiency by way of parallelization. This might not be a
guarantee you care about though, at which point you can just toss
it out completely and see if the compiler generates better code
now that it sees the same type qualifier as in the other example.
I'd actually be surprised if using "immutable" causes /less/
efficient code in this case, because it should be even /safer/ to
use the argument as-is. But it IS a difference between the two
examples, and one that might not be benefiting your cause (though
that's totally up to you).
---
(2) Try keeping the string argument, but make the function more
closely identical in semantics:
// ideomatic D version
void write_to_host(string msg) {
// a fixed address to get bytes to the host via usb
char
Re: Why does stringof not like functions with arguments?
On Thursday, 10 August 2017 at 14:51:22 UTC, Meta wrote:
On Wednesday, 9 August 2017 at 01:39:07 UTC, Jason Brady wrote:
Why does the following code error out with:
app.d(12,10): Error: function app.FunctionWithArguments (uint
i) is not callable using argument types ()
Code:
import std.stdio;
void FunctionWithoutArguments() {
}
void FunctionWithArguments(uint i) {
}
void main()
{
writeln(FunctionWithoutArguments.stringof);
writeln(FunctionWithArguments.stringof);
}
Welcome to optional parentheses hell. Please enjoy your stay.
[...]
Muahahaha it's necromancy time!
... meaning I just ran into this problem. Again. And it sucked.
And I found this thread. Again. Now it's time for me to be less
of a dummy and post my solution.
This seems to have different solutions depending on what you want
out of the function symbol. The advice already given in this
thread is great if you want to print the function's name (and
maybe a couple other things I already forgot).
But what I needed was to print the function's *signature*.
Basically, I want to
writeln(FunctionWithArguments.stringof);
and get this output:
void FunctionWithArguments(uint i)
I didn't quite get there. I got this far:
void(uint i)
But for what I'm doing right now, that's good enough.
Alright here's how it's done:
writeln(typeof(FunctionWithArguments).stringof);
So it was ultimately really easy. At least, for this one very
specific use-case. I just about kicked myself.
The previous example then becomes this:
import std.stdio;
void FunctionWithoutArguments() {
}
void FunctionWithArguments(uint i) {
}
void main()
{
writeln(typeof(FunctionWithoutArguments).stringof);
writeln(typeof(FunctionWithArguments).stringof);
}
I needed this when writing a program that checks for whether
functions visible from alias-this statements are included in the
results of __traits(getOverloads,...).
Here is the end result:
https://pastebin.com/yj3idDhp
And no. No they are not. :3
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 09:58:25 UTC, Jonathan M Davis wrote: For two types to be compared, they must be the the same type - or they must be implicitly convertible to the same type, in which case, they're converted to that type and then compared. So, as far as D's design of comparison goes, there is no need worry about comparing differing types. At most, you need to worry about what implicit type conversions exist, and D isn't big on implicit type conversions, because they tend to cause subtle bugs. So, while they definitely affect comparison, they don't affect anywhere near as much as they would in a language like C++. In general, you're not going to get very far if you're trying to make it possible to compare a user-defined type against other types in D without explicitly converting it first. - Jonathan M Davis That makes sense, but requiring types to be explicitly converted before comparisons kinda throws sand on the cake when I'm ostensibly trying to make things that interact seamlessly with existing types. "alias this" is still awesome, so it's usually fine regardless :) Thanks for the explanation.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 13:23:15 UTC, Adam D. Ruppe wrote: On Thursday, 27 September 2018 at 05:04:09 UTC, Chad Joan wrote: As above, I think this might be a very clean and effective solution for a different class of use-cases :) I'll keep it in mind though. Yeah. And I did make one mistake: the tupleof assignment trick wouldn't work well for references, so lol it isn't much of a deep copy. You'd want to deep copy any arrays too probably. But sounds like you are already doing that, yay. You're right though, if I end up adding boilerplate anyways, I may as well have a good shallow copy to begin with.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 14:23:48 UTC, Steven
Schveighoffer wrote:
On 9/27/18 10:20 AM, Steven Schveighoffer wrote:
typeid sometimes gives you a more derived type than TypeInfo.
Including for classes and structs.
In the past, .classinfo gave you a different thing than
typeid(obj), but now it is the same thing:
auto obj = new Object;
// classinfo and typeid are the same object
assert(obj.classinfo is typeid(obj));
// and the same type
static assert(is(typeof(obj.classinfo) ==
typeof(typeid(obj;
I wouldn't use classinfo any more, I generally use typeid.
I should add that typeid does give you the derived
TypeInfo_Class, not the concrete one.
that is:
Object obj; // = null
//typeid(obj); // segfault, can't dereference null pointer
class C {}
obj = new C;
static assert(is(typeof(obj) == Object));
writeln(typeid(obj)); // C, not Object
So it really is a drop-in replacement for classinfo. I think
classinfo is still there to avoid breaking existing code that
uses it.
-Steve
Interesting! That's yet another thing I hadn't realized had
changed. Good to know.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 13:16:58 UTC, Adam D. Ruppe wrote: On Thursday, 27 September 2018 at 05:18:14 UTC, Chad Joan wrote: How does this work? The language reference states that typeid(Type) returns "an instance of class TypeInfo corresponding to Type". (https://dlang.org/spec/expression.html#typeid_expressions) But then the TypeInfo class doesn't seem to have a .create() method, or at least not one in the documentation: https://dlang.org/phobos/object.html#.TypeInfo I forgot about the create method, lol, that is what you want. But typeid(obj) - pass it an existing object of the type btw - when obj is a class will return TypeInfo_Class - a subclass of TypeInfo. It is *that* which has the create method. https://dlang.org/phobos/object.html#.TypeInfo_Class.create (or less horrible docs http://dpldocs.info/experimental-docs/object.TypeInfo_Class.html ) Yep, that solves the mystery! Thanks!
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 18:37:27 UTC, Chad Joan wrote: I will probably end up going with the latter suggestion and have Extended use the Root's T. That would probably make sense for what I'm doing. In my case, the T allows the caller to configure what kind of output the thing provides... IIRC (it's been a while since I touched this code o.O). Thanks for pointing that out. Wait, actually the T is the element type of the input range, like "char" if the parser is to parse UTF8 strings. I'd already figured that out too. (*ﾟｰﾟ)ゞ
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 08:56:22 UTC, Alex wrote:
On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan
wrote:
class Root(T)
{
T x;
}
class Extended(T) : Root!T
{
T y;
}
Sorry for a technical aside, but would this be something for
you?
https://forum.dlang.org/post/vtaxcxpufrovwfrkb...@forum.dlang.org
I mean... In either case, there is something curious in the
Extended/Root usage, as they both are bound to the same type.
And if so, you could get rid of the templatization in the
Extended class, either by templating Root on the Extended or
use the T of Root in Extended.
Perhaps it's half some form of unintended neural network
fabrication that happened as I wrote the example and half that
the original code isn't that well thought out yet. The original
code looks more like this:
template Nodes(T)
{
class Root
{
T x;
}
class Extended : Root
{
T y;
}
}
I will probably end up going with the latter suggestion and have
Extended use the Root's T. That would probably make sense for
what I'm doing. In my case, the T allows the caller to configure
what kind of output the thing provides... IIRC (it's been a while
since I touched this code o.O).
Thanks for pointing that out.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 08:19:41 UTC, Jonathan M Davis
wrote:
On Thursday, September 27, 2018 1:41:23 AM MDT Chad Joan via
Digitalmars-d- learn wrote:
On Thursday, 27 September 2018 at 05:12:06 UTC, Jonathan M
Davis
This is also reminding me of how it's always bugged me that
there isn't a way to operator overload opEquals with a static
method (or even a free function?), given that it would allow
the class/struct implementer to guard against (or even
interact intelligently with) null values:
That's not really an issue with D. With classes, when you have
a == b, it
doesn't lower to a.opEquals(b). Rather, it lowers to
opEquals(a, b), and the
free function opEquals is defined as
bool opEquals(Object lhs, Object rhs)
{
// If aliased to the same object or both null => equal
if (lhs is rhs) return true;
// If either is null => non-equal
if (lhs is null || rhs is null) return false;
// If same exact type => one call to method opEquals
if (typeid(lhs) is typeid(rhs) ||
!__ctfe && typeid(lhs).opEquals(typeid(rhs)))
/* CTFE doesn't like typeid much. 'is' works, but
opEquals
doesn't
(issue 7147). But CTFE also guarantees that equal
TypeInfos are
always identical. So, no opEquals needed during
CTFE. */
{
return lhs.opEquals(rhs);
}
// General case => symmetric calls to method opEquals
return lhs.opEquals(rhs) && rhs.opEquals(lhs);
}
/
* Returns true if lhs and rhs are equal.
*/
bool opEquals(const Object lhs, const Object rhs)
{
// A hack for the moment.
return opEquals(cast()lhs, cast()rhs);
}
So, it already takes care of checking for null or even if the
two references point to the same object. For structs, a == b,
does lower to a.opEquals(b), but for better or worse, structs
are designed so that their init values need to be valid, or
you're going to have problems in general. Trying to work around
that is fighting a losing battle.
The spec seems to have the homogeneous cases covered: classes
with classes or structs with structs. What I'm more worried
about is stuff like when you have a class compared to a struct or
builtin type, or maybe a struct compared to a builtin type
(especially more complicated builtin types like arrays!). The
homogeneous cases are important for making a type consistent with
itself, but the other cases are important for integrating a type
with everything else in the ecosystem.
Notably, "alias this" is awesome and has more or less solved that
for me in the pedestrian cases I tend to encounter. I can write
a struct and alias this to some reference variable that will be
representative of my struct's "nullness" or other states of
existence.
But I wouldn't be surprised if there are corner-cases I haven't
encountered yet (actually I think I just remembered that this bit
me a little bit once or twice) where having a single alias-this
isn't sufficient to cover all of the possible things my
struct/class could be compared to (ex: if the type's null-state
corresponded to int.max for ints and float.nan for floats, and
you can't just use opEquals, such as when the type is a class and
could be precisely null).
Wouldn't it be helpful to have a root class type just to have
a "Top" type at runtime, even if it had no members? Ex: so
you could do things like make an array ProtoObject[] foo; that
can contain any runtime polymorphic variables.
Maybe? It's not something that I've personally found to be
particularly
useful. Once you can templatize code, the need to have a common
base class
gets pretty hard to argue for, but I don't know that it's
non-existent.
Also, for better or worse, you can already get it with void* -
and cover
more types no less (albeit less @safely). But from what I
understand of what
Andrei is intending, ProtoObject will end up being the new root
class for
all D classos, so having ProtoObject[] would work for all
extern(D) classes.
Of course, that still potentially leaves exern(C++) classes and
interfaces
(which could be extern(C++) or COM even right now and aren't
derived from
Object). So, things are already a bit weird with classes when
you start
interacting with other languages through D. is(T : Object) and
is(T == class) do _not_ mean quite the same thing even though
you'd think
that they would. And I haven't done enough with extern(C++) or
COM in D to
claim to understand all of the subtleties. If you're not
messing with them
directly or writing generic code that's going to mess with
them, it doesn't
really matter though.
-Jonathan M Davis
Gotcha. Quite a rabbit hole :)
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 05:12:06 UTC, Jonathan M Davis
wrote:
On Wednesday, September 26, 2018 10:20:58 PM MDT Chad Joan via
Digitalmars- d-learn wrote:
...
That's interesting! Thanks for mentioning.
If you don't mind, what are the complaints regarding Object?
Or can you link me to discussions/issues/documents that point
out the shortcomings/pitfalls?
I've probably run into a bunch of them, but I realize D has
come a long way since that original design and I wouldn't be
surprised if there's a lot more for me to learn here.
I can point you to the related DIP, though it's a WIP in
progress
https://github.com/andralex/DIPs/blob/ProtoObject/DIPs/DIP.md
There are also these enhancement requests for removing the
various member functions from Object (though they're likely to
be superceded by the DIP):
https://issues.dlang.org/show_bug.cgi?id=9769
https://issues.dlang.org/show_bug.cgi?id=9770
https://issues.dlang.org/show_bug.cgi?id=9771
https://issues.dlang.org/show_bug.cgi?id=9772
Basically, the problems tend to come in two areas:
1. Because of how inheritance works, once you have a function
on a class, you're forcing a certain set of attributes on that
function - be it type qualifiers like const or shared or scope
classes like pure or @safe. In some cases, derived classes can
be more restricted when they override the function (e.g. an
overide can be @safe when the original is @system), but that
only goes so far, and when you use the base class API, you're
stuck with whatever attributes it has. Regardless, derived
classes can't be _less_ restrictive. In fact, the only reason
that it's currently possible to use == with const class
references in D right now is because of a hack. The free
function opEquals that gets called when you use == on two class
references actually casts away const so that it can then call
the member function opEquals (which doesn't work with const).
So, if the member function opEquals mutates the object, you
actuall get undefined behavior. And because Object.opEquals
defines both the parameter and invisible this parameter as
mutable, derived classes have to do the same when they override
it; otherwise, they'd be overloading it rather than overriding
it.
You're right, I wouldn't be caught dead wearing that.
:)
But yeah, thanks for pointing that out. Now I know not to mutate
things in an opEquals, even if it makes sense from the class's
point of view, just in case. At least until this all gets sorted
out and code gets updated to not inherit from Object.
Object and its member functions really come from D1 and predate
all of the various attributes in D2 - including const. But even
if we could just add all of the attributes that we thought
should be there without worrying about breaking existing code,
there would be no right answer. For instance, while in the vast
majority of cases, opEquals really should be const, having it
be const does not work with types that lazily initialize some
members (since unlike in C++, D does not have backdoors for
const - when something is const, it really means const, and
it's undefined behavior to cast away const and mutate the
object). So, having Object.opEquals be const might work in 99%
of cases, but it wouldn't work in all. The same could be said
for other attributes such as pure or nothrow. Forcing a
particular set of attributes on these functions on everyone is
detrimental. And honestly, it really isn't necessary.
Having them on Object comes from a Java-esque design where you
don't have templates. With proper templates like D2 has, there
normally isn't a reason to operate on an Object. You templatize
the code rather than relying on a common base class. So,
there's no need to have Object.toString in order have toString
for all classes or Object.opEquals to have opEquals for all
classes. Each class can define it however it sees fit. Now,
once a particular class in a hierarchy has defined a function
like opEquals or toString, that affects any classes derived
from it, but then only the classes derived from it are
restricted by those choices, not every single class in the
entire language as has been the case with Object.
That makes sense. Also, compile-time inheritance/duck-typing
FTW, again.
This is also reminding me of how it's always bugged me that there
isn't a way to operator overload opEquals with a static method
(or even a free function?), given that it would allow the
class/struct implementer to guard against (or even interact
intelligently with) null values:
import std.stdio;
class A
{
int payload;
bool opEquals(int rhs)
{
if ( rhs == int.max )
return false;
else
return this.payload == rhs;
}
}
class B
{
int payload;
static bool opEquals(B lhs, int rhs)
{
if ( lhs is null &&
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Wednesday, 26 September 2018 at 21:25:07 UTC, Steven Schveighoffer wrote: ... Object.factory is a really old poorly supported type of reflection. I would not depend on it for anything. Roger that. Will avoid :) You are better off using your own registration system. As far as choosing the design for your problem, you can use: auto obj = typeid(obj).create(); which is going to work better, and doesn't require a linear search through all modules/classes like Object.factory. How does this work? The language reference states that typeid(Type) returns "an instance of class TypeInfo corresponding to Type". (https://dlang.org/spec/expression.html#typeid_expressions) But then the TypeInfo class doesn't seem to have a .create() method, or at least not one in the documentation: https://dlang.org/phobos/object.html#.TypeInfo It looks like what I want, so it might be very helpful. If it were me, I'd probably instead implement a clone virtual method. This way if any custom things need to occur when copying the data, it can be handled. -Steve I'm thinking I want to avoid the virtual method in this case, for reasons I wrote about in my response to Adam (https://forum.dlang.org/post/zovficijurwhuurrr...@forum.dlang.org). But I think it's probably a good suggestion in most cases; I suspect that most of the time wanting to write a "deepCopy" method is going to be in response to some problem that will respond well to just virtualizing the method. Thanks for the advice!
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Wednesday, 26 September 2018 at 21:24:07 UTC, Adam D. Ruppe
wrote:
On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan
wrote:
I'm implementing a deep-copy method for a tree of templated
class instances. As part of this, I need some way to copy
each node.
[...]
that isn't already handled by their deepCopy method.
I would strongly suggest just using that virtual method and
having the child classes override it, then you call it from any
of them and get the right result.
The tree nodes are potentially very diverse, but the tree
structure itself will be very homogeneous. I'm writing a parser
generator backend and the tree is expressions of various
operators (Sequence, OrderedChoice, UnorderedChoice, Repetition,
etc). I'm trying to keep it simple: everything is an expression,
and expressions can contain other expressions (though some are
always leaves in the tree). At some level, if I let things
implement their own deepCopy, then it means there are potentially
other classes and state out there to iterate that the rest of the
code doesn't know about. That could be bad, and expressions
shouldn't contain anything besides expressions!
This probably contrasts a lot with other use-cases, like
serialization. And I wouldn't be surprised if things change
later on and I end up with some kind of auxiliary virtual copy
function that does what you suggest, but is specifically for
handling special out-of-band mutable reference data that the
expressions might need to carry someday.
I suppose I've never considered just how hard/impossible it is to
have a generic way to copy things. Well, maybe a little bit at
various points, but not this bad ;) There are so many dimensions
to the problem and it seems like the context and requirements
will always be really important. So it can be made simple under
the right constraints (ex: everything is immutable!), but the
constraints are always different depending on the requirements
(ex: ... but in this hypothetical, we need mutability, so then
it's gotta happen a different way).
Object.factory kinda sux and I'd actually like to remove it
(among other people). There's no plan to actually do that, but
still, just on principle I want to turn people away.
But even as you can see, the implementation is lacking and it
isn't great design anyway - the interface with virtual methods
does better work. It also wouldn't work in ctfe anyway,
object.factory relies on runtime stuff.
Good to know!
I don't think I've even used it much, if at all. I suppose I
won't miss it if it goes ;)
If Object.factory is incapable of this, is there some other
CTFE-friendly way to copy templated class instances?
I think you can copy typeinfo().init and then call
typeinfo().defaultConstructor - this is iirc what
Object.factory does, but once you already have the typeinfo you
can use it directly and bypass the string lookup.
I'm having trouble looking this up. Could you link me to the
docs for this?
But you'd really be better off with a virtual copy method. I
say those string factory things should only be used if you are
starting with a string, like deserialization.
interface Copyable {
Copyable copy();
}
class Whatever(T) : Copyable {
Whatever!T copy() {
auto c = new Whatever!T();
c.tupleof = this.tupleof;
return c;
}
}
that kind of method. the template implements the interface so
little boilerplate and it works and can be customized etc and
fits in well. If you call it from the interface, you get an
instance of the interface (tho note since it is virtual, the
underlying type is still what you need). If you call from the
child static type, you get it right back. Yay, fits liskov and
works!
As above, I think this might be a very clean and effective
solution for a different class of use-cases :) I'll keep it in
mind though.
If I have to, I can probably make these things register
themselves in some list of delegates that can be used to
instantiate the correct class. Or something like that. But I
am hoping that there is a better way that involves less
boilerplate.
that's not a terrible idea if you need delegates keyed to
strings...
Right. At some level I just need a function that I can call like
this:
auto newThing = makeAnother(originalThing);
and perhaps makeAnother(...) can just lookup originalThing's
classname in an associative array of delegates. Or maybe I can
just hash some part of originalThing's type information. I can
put all of the ugly registration boilerplate into a mixin
template and somehow force that to always be mixed-into any
descendant classes. OK, it's probably getting too far into the
weeds now, but it seems doable and I'll reach for that if I need
to.
...
Things at least seem much more clear already. Thanks a bunch!
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Wednesday, 26 September 2018 at 23:32:36 UTC, Jonathan M Davis wrote: On Wednesday, September 26, 2018 3:24:07 PM MDT Adam D. Ruppe via Digitalmars-d-learn wrote: Object.factory kinda sux and I'd actually like to remove it (among other people). There's no plan to actually do that, but still, just on principle I want to turn people away. While there may not currently be plans to be remove it, as there _are_ plans to add ProtoObject as the new root class underneath Object, at some point here, it's likely that a large percentage of classes won't have anything to do with Object, so relying on Object.factory to be able to construct class Objects in general isn't likely to be a viable path in the long term - though presumably it would work for a code base that's written specifically with it in mind. Personally, I'm hoping that we eventually get to the point where Walter and Andrei are willing to outright deprecate Object itself, but I expect that ProtoObject will have to have been in use for a while before we have any chance of that happening. Either way, I think that it's clear that most code bases should go with a solution other than Object.factory if at all reasonably possible. - Jonathan M Davis That's interesting! Thanks for mentioning. If you don't mind, what are the complaints regarding Object? Or can you link me to discussions/issues/documents that point out the shortcomings/pitfalls? I've probably run into a bunch of them, but I realize D has come a long way since that original design and I wouldn't be surprised if there's a lot more for me to learn here.
Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
Hi all,
I'm implementing a deep-copy method for a tree of templated class
instances. As part of this, I need some way to copy each node.
I want to avoid code that does things like casting objects into
byte arrays and then copying raw bytes; I want all operations to
be memory safe things that I can use at compile-time. So I
planned to make all of these have default constructors and use
Object.factory to at least create the correct instance type at
the destination. The classes can implement auxiliary copy
methods if they need to copy anything that isn't already handled
by their deepCopy method.
But I ran into a problem: Object.factory doesn't seem to be
compatible with templated classes.
Here is an example:
import std.stdio;
class Root(T)
{
T x;
}
class Extended(T) : Root!T
{
T y;
}
void main()
{
Root!int foo = new Extended!int();
auto name = foo.classinfo.name;
writefln("foo's name is '%s'", name);
// foo's name is 'main.Extended!int.Extended'
Object obj = Object.factory(name);
writefln("Is obj null? %s", obj is null);
Root!int bar = cast(Root!int)obj; // Still going to be null.
writefln("Is bar null? %s", obj is null);
//bar.x = 3; // crash!
}
I had a look at Object.factory. It seems very simple. Perhaps
too simple. I think this might be a dead end. Have I missed
something? Can it actually handle templates somehow, but I just
don't know how to calculate the correct string to hand it?
If Object.factory is incapable of this, is there some other
CTFE-friendly way to copy templated class instances?
If I have to, I can probably make these things register
themselves in some list of delegates that can be used to
instantiate the correct class. Or something like that. But I am
hoping that there is a better way that involves less boilerplate.
Thanks!
- Chad
Re: Going on std.regex & std.uni bug-fixing hunt
On Sunday, 10 September 2017 at 11:47:02 UTC, Dmitry Olshansky wrote: On Sunday, 10 September 2017 at 00:16:10 UTC, Chad Joan wrote: I was working on std.regex a bit myself, so I created this bug report to capture some of the findings/progress: https://issues.dlang.org/show_bug.cgi?id=17820 It seems like something you might be interested in, or might even have a small chance of fixing in the course of other things. Yeah, well known problem. Solution is to keep a bit of memory cached eg in TLS variable. Indeed. Is there another issue I can mark it as a duplicate of? [...] -- The Captures struct does not specify what value is returned for submatches that were in the branch of an alternation that wasn't taken or in a repetition that matched 0 or more than 1 times. As every engine out there the value is "", empty string. I usually don't refer to other libraries while using a library. If an API doesn't define something, then it is, by definition, undefined behavior, and thus quite undesirable to rely upon. This one seems pretty easy to fix though. I will probably make a documentation PR at some point. -- The Captures struct does not seem to have a way to access all of the strings matched by a submatch in repetition context, not to mention nested repetition contexts. Just like any other regex library. Counterexample: https://msdn.microsoft.com/en-us/library/system.text.regularexpressions.group.captures(v=vs.110).aspx#code-snippet-3 I actually have a strong interest in this. And not because I need to write regular expressions that extract lists of patterns all the time (well, it might've happened). More importantly: this would make it easier to integrate Phobos' regex engine into a parser generator framework. Current plans involve regular expression + parsing expression grammars. I'm pretty sure it is possible to mechanically convert a subset of PEGs into Regexes and gain some useful optimizations, but this requires granular control over regular expression captures to be able to extract the text matched by the original PEG symbols. I'm not sure how much those mentions help without proper bug reports, but at least I got it off my chest (for the time being) without having to spend my whole weekend writing bug reports ;) Well they are warmly welcome shouldypu get to it. Thanks! ... Dmitry, I appreciate your working towards making the regex module easier to work on. Thanks. ... I'm curious what you're thinking about when you mention something ambitious like writing a new GC :) (like this https://imgur.com/cWa4evD) I can't help but fantasize about cheap ways to get GC allocations to parallelize well and not end up writing an entire generational collector! ThreadCache can go a long way to help that. Google didn't help me with this one. Any chance I could get a link? But I doubt I'll ever have the opportunity to work on such things. I hope your GC attempt works out! Me too. It's won't be trivial effort though. Good luck!
Re: Going on std.regex & std.uni bug-fixing hunt
On Tuesday, 5 September 2017 at 10:50:46 UTC, Dmitry Olshansky wrote: It's been tough time on D front for me, going down from about ~1 week of activity during July to ~2-3 days in August. One thing I realised is that doing a new GC is going to be a long battle. Before I'm too deep down this rabbit hole I decided to first address the long-standing backlog of issues of std.regex and std.uni. My burndown list for std.regex: https://issues.dlang.org/buglist.cgi?bug_status=UNCONFIRMED_status=NEW_status=ASSIGNED_status=REOPENED_status=VERIFIED=phobos_id=216638=D_format=advanced=---_desc=regex_desc_type=allwordssubstr ... I was working on std.regex a bit myself, so I created this bug report to capture some of the findings/progress: https://issues.dlang.org/show_bug.cgi?id=17820 It seems like something you might be interested in, or might even have a small chance of fixing in the course of other things. ... There are other regex improvements I might be interested in, but I'm not sure I have time to make bug reports for them right now. I might be convinced to fast track them if someone wants to make legitimate effort towards fixing them, otherwise I'll eventually get around to writing the reports and/or making PRs someday. Examples: -- Calls to malloc in the CTFE path cause some regexes to fail at compile time. I suspect this happens due to the Captures (n > smallString) condition when the number of possible captures is greater than 3, but I haven't tested it (time consuming...). -- I remember being unable to iterate over named captures. But I'm not confident that I'm remembering this correctly, and I'm not sure if it's still true. -- The Captures struct does not specify what value is returned for submatches that were in the branch of an alternation that wasn't taken or in a repetition that matched 0 or more than 1 times. -- The Captures struct does not seem to have a way to access all of the strings matched by a submatch in repetition context, not to mention nested repetition contexts. I'm not sure how much those mentions help without proper bug reports, but at least I got it off my chest (for the time being) without having to spend my whole weekend writing bug reports ;) ... Dmitry, I appreciate your working towards making the regex module easier to work on. Thanks. ... I'm curious what you're thinking about when you mention something ambitious like writing a new GC :) (like this https://imgur.com/cWa4evD) I can't help but fantasize about cheap ways to get GC allocations to parallelize well and not end up writing an entire generational collector! But I doubt I'll ever have the opportunity to work on such things. I hope your GC attempt works out!
Re: Supporting musl libc
On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote: As an alternative to glibc there's a C standard library called musl [1]. [...] The issue is that musl doesn't support the functions defined by execinfo.h: backtrace, backtrace_symbols_fd and backtrace_symbols, since these are glibc extensions. As far as I can see, these functions are used in two places in druntime: src/rt/backtrace/dwarf.d [3] and src/core/runtime.d [4]. The imports of execinfo is guarded by version(CRuntime_Glibc). I see that CRuntime_Glibc is a predefined version identifier defined by the compiler on Linux. [...] I just ran into these problems while trying to get D running on a hardened Gentoo system using musl libc. Using a patched gdc-4.9.4 I've been able to compile and run a simple D "Hello world". That made me very happy! Here are the binaries: http://www.chadjoan.com/d/gdc-4.9.4-hardened-musl.tar.gz I have forked the dlang overlay in the hope that it can provide the necessary information if someone tries to compile the thing for themselves: https://github.com/chadjoan/dlang This is pretty recent work, so I haven't had time to use it for anything besides "Hello world". I'm also not sure how to run regression tests for this thing, if there even are any. So this is all very untested. I'm sharing a minimally useful milestone, and hopefully it is more than that. Background: Earlier, I modified the "dmd-2.067.1-r2" ebuild from the dlang overlay and actually managed to get it to compile and emit executables. The BSD execinfo did not even manage to run its own test program on my system without segfaulting. So I used libbacktrace for backtraces and had to do a bunch of other patching to get dmd/druntime/phobos to build on this system; it involved a lot of trial-and-error. Although it emitted executables, these executables would segfault or overstep security bounds enforced by the kernel (to the point where removing all PAX restrictions would not help it). I remember something about text relocations, but at this point it's been a while, so my memory of details is lacking. Diving into codegen is far too time-consuming in my situation, so that was a dead-end. I started over with gdc, because I knew that gcc could already generate working executables on the system. Most of the pain in this process involved two things: (1) coercing portage into building gdc in its own directory so that it didn't take over my system compiler and (2) fixing gcc/gdc build scripts. It was important to use portage and not just manually configure gdc, because I wanted to ensure that the resulting gdc build would be configured and patched in a way closely matching what my system compiler does (and thus having higher success chances). Trying to configure and patch it by hand, thus repeating a lot of hard work already done by Gentoo maintainers, is... not practical. As for the build scripts: I have no idea what conventions gcc maintainers use for these things, and thus how I might do basic things like use an environment variable from outside of the build process to affect what flags are passed to a compiler. There be hacks in my patching! As an example, it will unconditionally compile the D frontend with -fPIC; if you end up building this on another system and you didn't want that, then it's just too bad ;) So it took more trial-and-error, and maybe a little compromise, but eventually it got there. I hope this helps someone.
Re: Best way to manage non-memory resources in current D, ex: database handles.
Awesome, thank you! On Thursday, 9 March 2017 at 00:47:48 UTC, Adam D. Ruppe wrote: Now, if you forget to scope(exit), it is OK, the garbage collector WILL get around to it eventually, and it is legal to work with C handles and functions from a destructor. It is only illegal to call D's garbage collector's functions or to reference memory managed by D's GC inside the destructor. C pointers are fine. It's good to have this confirmed. I'm always a bit trepidatious around destructors. Oooh, and it looks like there is more information in the language spec about @disable on struct constructors and postblits now (compared to the previous time I tried to learn about that feature). So between that and your example, I think I have a feel for how to use that. Thanks. Have a wonderful day!
How do I get names of regex captures during iteration? Populate AAs with captures?
Is there a way to get the name of a named capture when iterating
over captures from a regular expression match? I've looked at
the std.regex code and it seems like "no" to my eyes, but I
wonder if others here have... a way.
My original problem is this: I need to populate an associative
array (AA) with all named captures that successfully matched
during a regex match (and none of the captures that failed). I
was wondering what the best way to do this might be.
Thanks!
Please see comments in the below program for details and my
current progress:
void main()
{
import std.compiler;
import std.regex;
import std.range;
import std.stdio;
writefln("Compiler name:%s", std.compiler.name);
writefln("Compiler version: %s.%s", version_major,
version_minor);
writeln("");
enum pattern = `(?P\w+)\s*=\s*(?P\d+)?;`;
writefln("Regular expression: `%s`", pattern);
writeln("");
auto re = regex(pattern);
auto c = matchFirst("a = 42;", re);
reportCaptures(re, c);
c = matchFirst("a = ;", re);
reportCaptures(re, c);
}
void reportCaptures(Regex, RegexCaptures)(Regex re, RegexCaptures
captures)
{
import std.range;
import std.regex;
import std.stdio;
writefln("Captures from matched string '%s'",
captures[0]);
string[string] captureList;
// I am trying to read the captures from a regular
expression match
// into the above AA.
//
// ...
//
// This kind of works, but requires a string lookup for
each capture
// and using it in practice relies on undocumented
behavior regarding
// the return value of std.regex.Capture's
opIndex[string] method
// when the string index is a valid named capture that
was not actually
// captured during the match (ex: the named capture was
qualified with
// the ? operator or the * operator in the regex and
never appeared in
// the matched string).
foreach( captureName; re.namedCaptures )
{
auto capture = captures[captureName];
if ( capture is null )
writefln(" captures[%s] is null",
captureName);
else if ( capture.empty )
writefln(" captures[%s] is empty",
captureName);
else
{
writefln(" captures[%s] is '%s'",
captureName, capture);
captureList[captureName] = capture;
}
}
writefln("Total captures: %s", captureList);
/+
// I really want to do something like this, instead:
foreach( capture; captures )
captureList[capture.name] = capture.value;
// And, in reality, it might need to be more like this:
foreach( capture; captures )
foreach ( valueIndex, value; capture.values )
captureList[format("%s-%s",capture.name,valueIndex)] = value;
// Because, logically, named captures qualified with the
// *, +, or {} operators in regular expressions may
capture
// multiple slices.
writefln("Total captures: %s", captureList);
+/
writeln("");
}
//Output, DMD64 D Compiler v2.073.1:
//---
//
//Compiler name:Digital Mars D
//Compiler version: 2.73
//
//Regular expression: `(?P\w+)\s*=\s*(?P\d+)?;`
//
//Captures from matched string 'a = 42;'
// captures[value] is '42'
// captures[var] is 'a'
//Total captures: ["value":"42", "var":"a"]
//
//Captures from matched string 'a = ;'
// captures[value] is empty
// captures[var] is 'a'
//Total captures: ["var":"a"]
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 23:30:52 UTC, Chad Joan wrote:
On Tuesday, 21 February 2017 at 22:43:15 UTC, H. S. Teoh wrote:
Parsing strings at program startup is ugly and slow. What
about parsing at compile-time with CTFE into an array literal
and transforming that into an AA at startup, which should be a
lot faster?
// Warning: untested code
pure string[2][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[2][] result;
foreach (record;
csvReader!(Tuple!(string,string))(inputCsv)) {
result ~= [record[0], record[1]];
}
return result;
}
immutable string[string] dataLookup;
static this()
{
enum halfCookedData =
parseTwoColumnCsv(import("some_data.csv"));
foreach (p; halfCookedData) {
dataLookup[p[0]] = p[1];
}
}
T
Hi,
That makes a lot of sense and it had crossed my mind.
In my case the data sets are super small, so I'm probably going
to be lazy/productive and leave it the way it is.
Anyone else from the internet copying these examples: try to
just do it H.S. Teoh's way from the start ;)
Thanks for the suggestion.
- Chad
OK I couldn't help it:
---
pure private string[][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[][] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result ~= [record[0],record[1]];
return result;
}
pure private string[string] twoColumnArrayToAA(const string[][]
arr)
{
string[string] result;
foreach ( pair; arr )
result[pair[0]] = pair[1];
return result;
}
pure private string[string] importTwoColumnCsv(string csvFile)()
{
// Force the parse to happen at compile time.
immutable string[][] tempArray =
import(csvFile).parseTwoColumnCsv();
// Convert the parsed array into a runtime Associative Array and
return it.
return tempArray.twoColumnArrayToAA();
}
immutable string[string] dataLookup;
immutable string[string] dataLookup1;
immutable string[string] dataLookup2;
static this()
{
import std.range;
dataLookup1 = importTwoColumnCsv!"some_data1.csv";
dataLookup2 = importTwoColumnCsv!"some_data2.csv";
foreach( pair; chain(dataLookup1.byKeyValue,
dataLookup2.byKeyValue) )
dataLookup[pair.key] = pair.value;
}
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
This example also shows joining associative arrays.
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 22:43:15 UTC, H. S. Teoh wrote:
Parsing strings at program startup is ugly and slow. What
about parsing at compile-time with CTFE into an array literal
and transforming that into an AA at startup, which should be a
lot faster?
// Warning: untested code
pure string[2][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[2][] result;
foreach (record; csvReader!(Tuple!(string,string))(inputCsv))
{
result ~= [record[0], record[1]];
}
return result;
}
immutable string[string] dataLookup;
static this()
{
enum halfCookedData =
parseTwoColumnCsv(import("some_data.csv"));
foreach (p; halfCookedData) {
dataLookup[p[0]] = p[1];
}
}
T
Hi,
That makes a lot of sense and it had crossed my mind.
In my case the data sets are super small, so I'm probably going
to be lazy/productive and leave it the way it is.
Anyone else from the internet copying these examples: try to just
do it H.S. Teoh's way from the start ;)
Thanks for the suggestion.
- Chad
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 22:26:01 UTC, Chad Joan wrote:
On Tuesday, 21 February 2017 at 21:58:07 UTC, Stefan Koch wrote:
On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote:
Hello all,
I'm trying to make this work:
[...]
You cannot create AA's at ctfe and carry them over to runtime
use.
You'd have to use a costum dictionary-type.
I think the vibe.d project has one you could use.
I wondered if this might be the case.
Well, I won't worry about it then. I'll have to find another
way around.
Thanks a bunch!
For the sake of the rest of the internet, here is what I'm
probably going to stick with:
---
pure string[string] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[string] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result[record[0]] = record[1];
return result;
}
immutable string[string] dataLookup;
static this()
{
dataLookup = parseTwoColumnCsv(import("some_data.csv"));
}
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
In this case the AA isn't actually coded into the executable; but
at least the configuration from some_data.csv will be in the
executable as a string. The program will construct the AA at
startup. It's not as "cool", but it should get the job done.
HTH.
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 21:58:07 UTC, Stefan Koch wrote: On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote: Hello all, I'm trying to make this work: [...] You cannot create AA's at ctfe and carry them over to runtime use. You'd have to use a costum dictionary-type. I think the vibe.d project has one you could use. I wondered if this might be the case. Well, I won't worry about it then. I'll have to find another way around. Thanks a bunch!
How do I use CTFE to generate an immutable associative array at compile time?
Hello all,
I'm trying to make this work:
---
pure string[string] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[string] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result[record[0]] = record[1];
return result;
}
immutable string[string] dataLookup =
parseTwoColumnCsv(import("some_data.csv"));
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
But (with DMD 2.073.1) I am getting this error:
main.d(14): Error: non-constant expression [['a', 'b', 'c']:['x',
'y', 'z'], ['1', '2', '3']:['4', '5', '6']]
The case with normal (non-associative) arrays seems to work fine,
but is not what I needed:
---
pure string[][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[][] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result ~= [record[0],record[1]];
return result;
}
immutable string[][] dataLookup =
parseTwoColumnCsv(import("some_data.csv"));
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
Any idea how I can get this working?
I have tried a couple other things, like having the
parseTwoColumnCsv function return an immutable string[string] and
casting 'result' to that in the return statement, and I also
tried just casting the value returned from parseTwoColumnCsv as
it appears in the declaration of 'dataLookup'. I tried
std.exception's assumeUnique, but the function/template signature
doesn't seem to support associative arrays.
Thanks.
Re: Another new io library
On Friday, 19 February 2016 at 01:29:15 UTC, Steven Schveighoffer wrote: On 2/18/16 6:52 PM, Chad Joan wrote: ... This is why I think it will be much more important to have at least these two interfaces take front-and-center: A. The presence of a .popAs!(...) operation (mentioned by Wyatt in this thread, IIRC) for simple deserialization, and maybe for other miscellaneous things like structured user interaction. To me, this is a higher-level function. popAs cannot assume to know how to read what it is reading. If you mean something like reading an entire struct in binary form, that's not difficult to do. I think I understand what you mean. We are entering the problem domain of serializing and deserializing arbitrary types. I think what I'd expect is to have the basic language types (ubyte, int, char, string, etc) all covered, and to provide some way (or ways) to integrate with serialization code provided by other types. So you can do ".popAs!int" out of the box, but ".popAs!MyType" will require MyType to provide a .deserialize member function. Understandably, this may require some thought (ex: what if MyType is already under constraints from some other API that expects serialization? what does this look like if there are multiple serialization frameworks? etc etc). I don't have the answer right now and I don't expect it to be solved quickly ;) B. The ability to attach parsers to streams easily. This might be as easy as coercing the input stream into the basic encoding that the parser expects (ex: char/wchar/dchar Ranges for compilers, or maybe ubyte Ranges for our PostgreSQL client's network layer), though it might need (A) to help a bit first if the encoding isn't known in advance (text files can be represented in sooo many ways! isn't it fabulous!). This is the fundamental goal for my library -- enabling parsers to read data from a "stream" efficiently no matter how that data is sourced. I know your time is limited, but I would invite you to take a look at the convert program example that I created in my library. In it, I handle converting any UTF format to any other UTF format. https://github.com/schveiguy/iopipe/blob/master/examples/convert/convert.d Awesome! I understand that most unsuspecting programmers will arrive at a stream library expecting to immediately see an InputRange interface. This /probably/ is not what they really want at the end of the day. So, I think it will be very important for any such library to concisely and convincingly explain the design methodology and rationale early and aggressively. Neglect to do this, and the library and it's documentation will become a frustration and a violation of expectations (an "astonishment"). Do it right, and the library's documentation will become a teaching tool that leaves visitors feeling enlightened and empowered. Good points! I will definitely spend some time explaining this. Best of luck :) Of course, I have to wonder if someone else has contrasting experiences with stream use-cases. Maybe they really would be frustrated with a range-agnostic design. I don't want to alienate this hypothetical individual either, so if this is you, then please share your experiences. I hope this helps and is worth making a bunch of you read a wall of text ;) Thanks for taking the time. -Steve Thank you for making progress on this problem! - Chad
Re: Another new io library
On Wednesday, 17 February 2016 at 06:45:41 UTC, Steven Schveighoffer wrote: It's no secret that I've been looking to create an updated io library for phobos. In fact, I've been working on one on and off since 2011 (ouch). ... Hi everyone, it's been a while. I wanted to chime in on the streams-as-ranges thing, since I've thought about this quite a bit in the past and discussed it with Wyatt outside of the forum. Steve: My apologies in advance if I a misunderstood any of the functionality of your IO library. I haven't read any of the documentation, just this thread, and I my time is over-committed as usual. Anyhow... I believe that when I am dealing with streams, >90% of the time I am dealing with data that is *structured* and *heterogeneous*. Here are some use-cases: 1. Parsing/writing configuration files (ex: XML, TOML, etc) 2. Parsing/writing messages from some protocol, possibly over a network socket (or sockets). Example: I am writing a PostgreSQL client and need to deserialize messages: http://www.postgresql.org/docs/9.2/static/protocol-message-formats.html 3. Serializing/deserializing some data structures to/from disk. Example: I am writing a game and I need to implement save/load functionality. 4. Serializing/deserializing tabular data to/from disk (ex: .CSV files). 5. Reading/writing binary data, such as images or video, from/to disk. This will probably involve doing a bunch of (3), which is kind of like (2), but followed by large homogenous arrays of some data (ex: pixels). 6. Receiving unstructured user input. This is my <10%. Note that (6) is likely to happen eventually but also likely to be minuscule: why are we receiving user input? Maybe it's just to store it for retrieval later. BUT, maybe we actually want it to DO something. If we want it to do something, then we need to structure it before code will be able to operate on it. (5) is a mix of structured heterogeneous data and structured homogenous data. In aggregate, this is structured heterogeneous data, because you need to do parsing to figure out where the arrays of homogeneous data start and end (and what they *mean*). This is why I think it will be much more important to have at least these two interfaces take front-and-center: A. The presence of a .popAs!(...) operation (mentioned by Wyatt in this thread, IIRC) for simple deserialization, and maybe for other miscellaneous things like structured user interaction. B. The ability to attach parsers to streams easily. This might be as easy as coercing the input stream into the basic encoding that the parser expects (ex: char/wchar/dchar Ranges for compilers, or maybe ubyte Ranges for our PostgreSQL client's network layer), though it might need (A) to help a bit first if the encoding isn't known in advance (text files can be represented in sooo many ways! isn't it fabulous!). I understand that most unsuspecting programmers will arrive at a stream library expecting to immediately see an InputRange interface. This /probably/ is not what they really want at the end of the day. So, I think it will be very important for any such library to concisely and convincingly explain the design methodology and rationale early and aggressively. Neglect to do this, and the library and it's documentation will become a frustration and a violation of expectations (an "astonishment"). Do it right, and the library's documentation will become a teaching tool that leaves visitors feeling enlightened and empowered. Of course, I have to wonder if someone else has contrasting experiences with stream use-cases. Maybe they really would be frustrated with a range-agnostic design. I don't want to alienate this hypothetical individual either, so if this is you, then please share your experiences. I hope this helps and is worth making a bunch of you read a wall of text ;) - Chad
Re: Scriptlike v0.9.4 - Perl-like interpolated strings, full examples and more.
On Wednesday, 23 September 2015 at 14:33:23 UTC, Nick Sabalausky
wrote:
On 09/23/2015 01:44 AM, Sönke Ludwig wrote:
An alternative idea would be to mix in a local "writeln"
function, which
can then be used multiple times without syntax overhead:
mixin template interp()
{
void iwriteln(string str)()
{
// pretend that we actually parse the string ;)
write("This is ");
write(somevar);
writeln(".");
}
}
void main()
{
int somevar = 42;
mixin interp;
iwriteln!("This is ${somevar}.");
}
Hmm, interesting idea. I'd leave it as a string-returning
function, rather than automatically printing, but a writeln
convenience wrapper is a nice idea too.
The one problem I'm seeing with it though, is it wouldn't be
able to see symbols declared between the "mixin interp;" and
any later uses of it. Ie:
void main()
{
int somevar = 42;
mixin interp;
iwriteln!("This is ${somevar}.");
int another = 17;
iwriteln!("This won't work, using ${another}.");
}
Seems like it would be too awkward and confusing to be
worthwhile. :(
This is why I argued for alternative mixin syntax in D some ...
years? ... ago.
It'd be really cool to have a writefln overload that did this:
int somevar = 42;
writefln#("This is ${somevar}");
writefln#("Plus two and you get ${somevar+1}");
Which would just be shorthand for
int somevar = 42;
mixin writefln!("This is ${somevar}");
mixin writefln!("Plus two and you get ${somevar+2}");
I feel like a bit of syntax sugar could go a long way ;)
Re: Scriptlike v0.9.4 - Perl-like interpolated strings, full examples and more.
On Wednesday, 23 September 2015 at 19:28:03 UTC, Nick Sabalausky
wrote:
On 09/23/2015 03:18 PM, Chad Joan wrote:
This is why I argued for alternative mixin syntax in D some
... years?
... ago.
It'd be really cool to have a writefln overload that did this:
int somevar = 42;
writefln#("This is ${somevar}");
writefln#("Plus two and you get ${somevar+1}");
Which would just be shorthand for
int somevar = 42;
mixin writefln!("This is ${somevar}");
mixin writefln!("Plus two and you get ${somevar+2}");
I feel like a bit of syntax sugar could go a long way ;)
Yea, the trouble with string mixins is that they're ugly enough
people don't like to use them.
I'd argued in the past for a way to tag a CTFE-able
string-returning function as being intended for mixing-in, so
you could omit the "mixin(...)" part. But we only ever got it
for template mixins. Allowing it for string mixins was too
controversial. :(
I dunno, maybe even a string mixin sugar as simple as this
would be a big help:
mixin!func(args to func here)
ie:
mixin!interp("Some string here")
But I'm guessing the ship's ling since sailed for anything like
that.
I hope not :(
I remember when Walter originally designed mixins, he stated
something to the effect that he wanted them to be easily
greppable at all times.
I would argue that they are so centrally important that they
should be a symbol rather than a keyword. Still greppable. But
also much more useful.
Since D already has the mixin keyword, I suspect it would be more
practical to just ask people to grep for 'mixin|'
instead of just 'mixin'.
There are similar (but orthogonal) concerns with delegate
(anonymous function) nesting:
// D notation.
foo ( (x,y) {
auto z = doSomething(x+y);
return z*z;
});
vs
// Speculative notation.
foo() : (x,y) {
auto z = doSomething(x+y);
return z*z;
}
Current D notation for nesting functions reminds me of C's
notation for structs...
// C notation.
typedef struct WhatDoIPutHereFoo
{
int x,y;
} Foo;
// D notation. (Yay, consistency!)
struct Foo
{
int x,y;
}
Extra semicolon and syntax noise and such.
I'm still incredibly glad D even has delegates and mixins at this
point ;)
Re: Microsoft now giving away VS 2013
On Friday, 14 November 2014 at 06:21:45 UTC, Joakim wrote: On Thursday, 13 November 2014 at 13:59:32 UTC, Wyatt wrote: On Thursday, 13 November 2014 at 08:50:29 UTC, Ola Fosheim Grøstad wrote: So, how to write a source-to-source compiler from CS to D…? ;-) I think it would be more useful would be to go the other way around for targeting Windows Phone. Or rather, it would be if anyone actually used WP. (Going after Microsoft's also-ran mobile OS isn't particularly compelling when our story for targeting Android is still in such a dire state.) Dire state? All druntime/phobos tests pass on Android/x86, except for std.datetime: http://wiki.dlang.org/Build_DMD_for_Android OK, that's not ARM that everybody cares about, but that just means combining an ARM backend from ldc or gdc and the existing linux/ARM and Android support in druntime/phobos, then hacking around the lack of native TLS on Android. gdc supposedly supports emulated TLS, so all the pieces should be in place there to do it. I've been recently looking into hacking the packed TLS solution I used with dmd for Android/x86 into ldc and llvm. Dan Olson got pretty far with iOS support too, early this year: http://forum.dlang.org/thread/m2txc2kqxv@comcast.net D has some support for mobile, albeit not fully polished. We really should have had a toolchain compiling D into working (release quality) Android/iOS executables about 3-5 years ago. This would have allowed D to scoop up a HUGE share of deployment in a market that was very new and welcoming of experimentation. It seems like there have been a few people enthusiastically working on this, but with little support from the broader community. I really wish this were a higher development priority and more actively encouraged a long time ago.
Where should D programs look for .so files on Linux (by default)?
--- The Problem --- I was using Derelict to play around with SDL/OpenGL in D for a bit, and I was initially unable to do so because DerelictSDL2's search paths for .so files could not find libSDL2_ttf.so on my Linux distribution (Gentoo). This seems quite solvable these days, so I entered an issue on DerelictSDL2's issue tracker. As I typed away, it became apparent that this is probably something that the entire D community should care about: intelligent lookup of a system's .so files (or lack thereof) could have a profound impact on user experience for anyone on Linux using programs written in D. Mike (aldacron) responded with a request to gather input from the D community at large, so here it is ;) The original issue tracker is here: https://github.com/DerelictOrg/DerelictSDL2/issues/28 --- One Solution --- As an initial proposal, how about this: 1. Look in ./ 2. Look in $LD_LIBRARY_PATH 3. Look in all directories listed in /etc/ld.so.conf 4. Look in a baked-in list of last-ditch fallback options. Although I'm sure better could be done, my current suggestion would look like this: 4a. ~/lib 4b. /usr/local/lib 4c. /usr/lib This logic should probably end up in Phobos or somewhere suitably common, and revised as needed, so that all D programs can benefit from the community's research and experience on the subject. As a data point: on my Gentoo box, the truly important step (usually) is to look in '/etc/ld.so.conf'. My machine has a fairly comprehensive list in that file, and LD_LIBRARY_PATH doesn't have squat. It may be desirable to first search folders or specific paths that are explicitly provided by the caller, and possibly skip the default search paths entirely if the caller insists. I'm not sure how that would look, since the explicit path would probably be passed to a (non-Phobos) library-specific loading function (ex: DerelictSDL2.load(...)) before making its way to any community ordained logic. In retrospect, I also have some reservations about looking in './'. I am not sure what the security implications are. If there are no security concerns, then I, as both a developer and a user, would intuitively want D programs to look in ./ first (unless, perhaps, the program has an explicit search path defined, such as a packaged folder full of .so files, then /maybe/ those should go first). As a developer, I might want to drop my program's .so/.dll files into the same directory as the executable, and I'd expect it to just work. This is important whenever package managers can't be relied upon to deliver a correct version of the .so (ex: for obscure libraries), or can't be relied upon to release the program at all. As a user, I sometimes want to replace .so/.dll files with different versions (ex: if the developer's version links to symbols or is compiled with features that don't exist on my system and are not strictly needed by the program). Is this a reasonable approach, or is there a more appropriate way to handle this search?
