I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
I wanted to make a 2D array like structure and support D slice
like operations,
but I had surprisingly bad experience.
I quickly copy pasted the example from the docs:
https://dlang.org/spec/operatoroverloading.html#array-ops
It's something like this:
struct Array2D(E)
{
E[] impl;
int stride;
int width, height;
this(int width, int height, E[] initialData = [])
ref E opIndex(int i, int j)
Array2D opIndex(int[2] r1, int[2] r2)
auto opIndex(int[2] r1, int j)
auto opIndex(int i, int[2] r2)
int[2] opSlice(size_t dim)(int start, int end)
@property int opDollar(size_t dim : 0)()
@property int opDollar(size_t dim : 1)()
}
So basic indexing works fine:
Array2D!int foo(4, 4);
foo[0, 1] = foo[2, 3];
But array copy and setting/clearing doesn't:
int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15 ];
foo[] = bar[];
And I get this very cryptic message:
(6): Error: template `example.Array2D!int.Array2D.opSlice` cannot
deduce function from argument types `!()()`, candidates are:
(51):`example.Array2D!int.Array2D.opSlice(ulong dim)(int
start, int end) if (dim >= 0 && (dim < 2))`
1. WTF `!()()` and I haven't even called anything with opSlice
i.e. `a .. b`?
Anyway, it doesn't overload [] with opIndex(), so fine, I add
that.
T[] opIndex() { return impl; }
Now I get:
foo[] = bar[]; // or foo[] = bar;
Error: `foo[]` is not an lvalue and cannot be modified
Array copying docs say:
When the slice operator appears as the left-hand side of an
assignment expression, it means that the contents of the array
are the target of the assignment rather than a reference to the
array. Array copying happens when the left-hand side is a slice,
and the right-hand side is an array of or pointer to the same
type.
2.WTF I do have slice operator left of assignment.
So I guess [] is just wonky named getter (and not an operator)
for a slice object and that receives the = so it's trying to
overwrite/set the slice object itself.
Next I added a ref to the E[] opIndex():
ref E[] opIndex() { return impl; }
Now foo[] = bar[] works as expected, but then I tried
foo[] = 0;
and that fails:
Error: cannot implicitly convert expression `0` of type `int` to
`int[]`
3. WTF. Didn't I just get reference directly to the slice and
array copy works, why doesn't array setting?
The ugly foo[][] = 0 does work, but it's so ugly/confusing that
I'd rather just use a normal function.
So I added:
ref E[] opIndexAssign(E value) { impl[] = value; return impl; }
And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't.
I get:
Error: function `example.Array2D!int.Array2D.opIndexAssign(int
f)` is not callable using argument types `(int, int, int)`
expected 1 argument(s), not 3
4. WTF. So basically adding opIndexAssign(E value) disabled ref E
opIndex(int i, int j). Shouldn't it consider both?
I'm surprised how convoluted this is. Is this really the way it's
supposed to work or is there a bug or something?
So what is the best/clear/concise/D way to do these for a custom
type?
I was planning for:
foo[] = bar; // Full copy
foo[] = 0; // Full clear
foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy
foo[1, 0 .. 5] = 0; // Row/Col clear
foo[0 .. 5, 2 .. 4] = bar[ 1 .. 6, 0 .. 2 ]; // Box copy
foo[0 .. 5, 2 .. 4] = 0; // Box clear
Anyway, this is not a huge deal breaker for me, I was just
surprised and felt like I'm missing something.
I suppose I can manually define every case one by one and not
return/use any references etc.
or use alias this to forward to impl[] (which I don't want to do
since I don't want to change .length for example)
or just use normal functions and be done with it.
And it's not actually just a regular array I'm making, so that's
why it will be mostly custom code, except the very basics.
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote:
I wanted to make a 2D array like structure and support D slice
like operations,
but I had surprisingly bad experience.
I quickly copy pasted the example from the docs:
https://dlang.org/spec/operatoroverloading.html#array-ops
It's something like this:
struct Array2D(E)
{
E[] impl;
int stride;
int width, height;
this(int width, int height, E[] initialData = [])
ref E opIndex(int i, int j)
Array2D opIndex(int[2] r1, int[2] r2)
auto opIndex(int[2] r1, int j)
auto opIndex(int i, int[2] r2)
int[2] opSlice(size_t dim)(int start, int end)
@property int opDollar(size_t dim : 0)()
@property int opDollar(size_t dim : 1)()
}
So basic indexing works fine:
Array2D!int foo(4, 4);
foo[0, 1] = foo[2, 3];
But array copy and setting/clearing doesn't:
int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15 ];
foo[] = bar[];
And I get this very cryptic message:
(6): Error: template `example.Array2D!int.Array2D.opSlice`
cannot deduce function from argument types `!()()`, candidates
are:
(51):`example.Array2D!int.Array2D.opSlice(ulong
dim)(int start, int end) if (dim >= 0 && (dim < 2))`
1. WTF `!()()` and I haven't even called anything with opSlice
i.e. `a .. b`?
Anyway, it doesn't overload [] with opIndex(), so fine, I add
that.
T[] opIndex() { return impl; }
Now I get:
foo[] = bar[]; // or foo[] = bar;
Error: `foo[]` is not an lvalue and cannot be modified
Array copying docs say:
When the slice operator appears as the left-hand side of an
assignment expression, it means that the contents of the array
are the target of the assignment rather than a reference to the
array. Array copying happens when the left-hand side is a
slice, and the right-hand side is an array of or pointer to the
same type.
2.WTF I do have slice operator left of assignment.
So I guess [] is just wonky named getter (and not an operator)
for a slice object and that receives the = so it's trying to
overwrite/set the slice object itself.
Next I added a ref to the E[] opIndex():
ref E[] opIndex() { return impl; }
Now foo[] = bar[] works as expected, but then I tried
foo[] = 0;
and that fails:
Error: cannot implicitly convert expression `0` of type `int`
to `int[]`
3. WTF. Didn't I just get reference directly to the slice and
array copy works, why doesn't array setting?
The ugly foo[][] = 0 does work, but it's so ugly/confusing that
I'd rather just use a normal function.
So I added:
ref E[] opIndexAssign(E value) { impl[] = value; return impl; }
And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't.
I get:
Error: function `example.Array2D!int.Array2D.opIndexAssign(int
f)` is not callable using argument types `(int, int, int)`
expected 1 argument(s), not 3
4. WTF. So basically adding opIndexAssign(E value) disabled ref
E opIndex(int i, int j). Shouldn't it consider both?
I'm surprised how convoluted this is. Is this really the way
it's supposed to work or is there a bug or something?
So what is the best/clear/concise/D way to do these for a
custom type?
I was planning for:
foo[] = bar; // Full copy
foo[] = 0; // Full clear
foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy
foo[1, 0 .. 5] = 0; // Row/Col clear
foo[0 .. 5, 2 .. 4] = bar[ 1 .. 6, 0 .. 2 ]; // Box copy
foo[0 .. 5, 2 .. 4] = 0; // Box clear
Anyway, this is not a huge deal breaker for me, I was just
surprised and felt like I'm missing something.
I suppose I can manually define every case one by one and not
return/use any references etc.
or use alias this to forward to impl[] (which I don't want to
do since I don't want to change .length for example)
or just use normal functions and be done with it.
And it's not actually just a regular array I'm making, so
that's why it will be mostly custom code, except the very
basics.
The de facto multi dimensional array type in D is mir's ndslice
https://github.com/libmir/mir-algorithm/blob/master/source/mir/ndslice/slice.d#L479
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote:
But array copy and setting/clearing doesn't:
int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15 ];
foo[] = bar[];
And I get this very cryptic message:
(6): Error: template `example.Array2D!int.Array2D.opSlice`
cannot deduce function from argument types `!()()`, candidates
are:
(51):`example.Array2D!int.Array2D.opSlice(ulong
dim)(int start, int end) if (dim >= 0 && (dim < 2))`
1. WTF `!()()` and I haven't even called anything with opSlice
This comes from some old history: arr[] used to call opSlice(),
and now it is preferred to implement opIndex() instead, but the
compiler still supports the old zero-arg opSlice() too.
Since opIndex() didn't work, it moved to trying the older
opSlice(), and that didn't work leading it to give up and issue
the error.
But yeah, the error should probably mention the newer function
name instead of the fallback it is failing on...
Next I added a ref to the E[] opIndex():
ref E[] opIndex() { return impl; }
I would avoid ref as much as you can, for one because it
conflates getting and assigning (as you saw), but also because it
leaks your internal implementation detail to the user; if you
didn't use an array internally, that api would break.
So I added:
ref E[] opIndexAssign(E value) { impl[] = value; return impl; }
And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't.
*snip*
4. WTF. So basically adding opIndexAssign(E value) disabled ref
E opIndex(int i, int j). Shouldn't it consider both?
Once you implement an opIndexAssign - any opIndexAssign - all
uses of `a[...] = c` will go through it instead of normal opIndex.
Generally speaking, opIndex is for getting, opIndexAssign is for
setting. Sometimes setting can be done via a getter function
(like if ref), but they are mostly two different functions and
you should implement them both if you want read/write access.
This is especially important if your underlying data isn't
actually in a literal array.
foo[] = bar; // Full copy
implement:
opIndexAssign(typeof(this) rhs);
In opIndexAssign, generally, the arguments are
(value_on_right_hand_side, indexes...)
Since there was no index given here, you don't want an index
argument.
foo[] = 0; // Full clear
implement:
opIndexAssign(int rhs);
Now it will take any int for the whole thing.
foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy
This is translated to:
this.opIndexAssign(bar.opIndex(bar.opSlice!0(0, 5)),
this.opSlice!0(0, 5), 1)
Three different functions there. On the left, we see x[...] = y,
so we know that is opIndexAssign again. The first argument is
what it is set to, other arguments are the slices given.
Any x .. y is translated to opSlice!dim(low, high). The !0 in
there is because it was given as the zeroth (first) argument in
the slice.
In this example, `foo[1 .. 2, 3 .. 4]`, it would call
opIndex( opSlice!0(1, 2), opSlice!1(3, 4) )
With the !0 and !1 indicating which position the slice was in.
The implementation of these functions would depend on just what
your innards are... but the types might be something like this:
struct SliceHelper {
size_t start;
size_t end;
int stride;
}
SliceHelper opSlice(size_t dimension)(size_t start, size_t end) {
/* return the helper with the appropriate values */
}
now, we can implement opIndex for getting in terms of that and
some regular items:
// get a single item at point x, y
int opIndex(size_t x, size_t y) { }
// well not necessarily an array, maybe a range for lazy
processing, but meh you get the idea
// this gives a slice in the X dimension with a fixed Y coordinate
// e.g. foo[ 0 .. 5, 3]
int[] opIndex(SliceHelper x, size_t y) {}
// foo[0, 4 .. 6]
int[] opIndex(size_t x, SliceHelper y) {}
// and now a 2d section of it
Array2d!int opIndex(SliceHelper x, SliceHelper y) { }
// and the zero-arg version, for foo[]
// here I return this for an example, but by convention,
// this should actually return a range object that is a
// view into this container... which might be `this` but
// might not be, depending on the details of your code like
// if it has internal references or other stuff you don't want
// to leak to the outside.
typeof(this) opIndex() { return this; }
And then a similar combination of arguments for opIndexAssign
(I return void here but it is also common to return this; )
void opIndexAssign(int rhs) // this[] = rhs
void opIndexAssign(int rhs, size_t x, size_t y) // this[x, y] =
rhs
void opIndexAssign(int rhs, size_t x, SliceHelper y) // this[x,
y1 .. y2] = rhs
you get the idea. The overload for other types of rhs.
This might be dozens of functions! You can minimize that a bit by
templating them and using internal static if, loops, etc to
narrow it down.
void opIndexAssign(R, Idx1, Idx2)(R rhs, Idx1 x, Idx2 y) {
// test types of Idx1+Idx2 for figuring out what to change
// test type of R to see what to c
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 16:10:36 UTC, Adam D. Ruppe wrote: On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote: But array copy and setting/clearing doesn't: int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ]; foo[] = bar[]; Generally speaking, opIndex is for getting, opIndexAssign is for setting. Thanks a lot for a very detailed answer. Sorry about the late reply. But for 2d and 3d and more arrays, the number of functions explodes really fast. Yeah, tastes like C++, but I guess I'll bite. I value debuggability and I only have the 2D case, so I think templates are out.
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 15:36:51 UTC, Nicholas Wilson wrote: On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote: I wanted to make a 2D array like structure and support D slice like operations, but I had surprisingly bad experience. The de facto multi dimensional array type in D is mir's ndslice https://github.com/libmir/mir-algorithm/blob/master/source/mir/ndslice/slice.d#L479 Thanks. I'll take a look.
