I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
I wanted to make a 2D array like structure and support D slice like operations, but I had surprisingly bad experience. I quickly copy pasted the example from the docs: https://dlang.org/spec/operatoroverloading.html#array-ops It's something like this: struct Array2D(E) { E[] impl; int stride; int width, height; this(int width, int height, E[] initialData = []) ref E opIndex(int i, int j) Array2D opIndex(int[2] r1, int[2] r2) auto opIndex(int[2] r1, int j) auto opIndex(int i, int[2] r2) int[2] opSlice(size_t dim)(int start, int end) @property int opDollar(size_t dim : 0)() @property int opDollar(size_t dim : 1)() } So basic indexing works fine: Array2D!int foo(4, 4); foo[0, 1] = foo[2, 3]; But array copy and setting/clearing doesn't: int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ]; foo[] = bar[]; And I get this very cryptic message: (6): Error: template `example.Array2D!int.Array2D.opSlice` cannot deduce function from argument types `!()()`, candidates are: (51):`example.Array2D!int.Array2D.opSlice(ulong dim)(int start, int end) if (dim >= 0 && (dim < 2))` 1. WTF `!()()` and I haven't even called anything with opSlice i.e. `a .. b`? Anyway, it doesn't overload [] with opIndex(), so fine, I add that. T[] opIndex() { return impl; } Now I get: foo[] = bar[]; // or foo[] = bar; Error: `foo[]` is not an lvalue and cannot be modified Array copying docs say: When the slice operator appears as the left-hand side of an assignment expression, it means that the contents of the array are the target of the assignment rather than a reference to the array. Array copying happens when the left-hand side is a slice, and the right-hand side is an array of or pointer to the same type. 2.WTF I do have slice operator left of assignment. So I guess [] is just wonky named getter (and not an operator) for a slice object and that receives the = so it's trying to overwrite/set the slice object itself. Next I added a ref to the E[] opIndex(): ref E[] opIndex() { return impl; } Now foo[] = bar[] works as expected, but then I tried foo[] = 0; and that fails: Error: cannot implicitly convert expression `0` of type `int` to `int[]` 3. WTF. Didn't I just get reference directly to the slice and array copy works, why doesn't array setting? The ugly foo[][] = 0 does work, but it's so ugly/confusing that I'd rather just use a normal function. So I added: ref E[] opIndexAssign(E value) { impl[] = value; return impl; } And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't. I get: Error: function `example.Array2D!int.Array2D.opIndexAssign(int f)` is not callable using argument types `(int, int, int)` expected 1 argument(s), not 3 4. WTF. So basically adding opIndexAssign(E value) disabled ref E opIndex(int i, int j). Shouldn't it consider both? I'm surprised how convoluted this is. Is this really the way it's supposed to work or is there a bug or something? So what is the best/clear/concise/D way to do these for a custom type? I was planning for: foo[] = bar; // Full copy foo[] = 0; // Full clear foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy foo[1, 0 .. 5] = 0; // Row/Col clear foo[0 .. 5, 2 .. 4] = bar[ 1 .. 6, 0 .. 2 ]; // Box copy foo[0 .. 5, 2 .. 4] = 0; // Box clear Anyway, this is not a huge deal breaker for me, I was just surprised and felt like I'm missing something. I suppose I can manually define every case one by one and not return/use any references etc. or use alias this to forward to impl[] (which I don't want to do since I don't want to change .length for example) or just use normal functions and be done with it. And it's not actually just a regular array I'm making, so that's why it will be mostly custom code, except the very basics.
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote: I wanted to make a 2D array like structure and support D slice like operations, but I had surprisingly bad experience. I quickly copy pasted the example from the docs: https://dlang.org/spec/operatoroverloading.html#array-ops It's something like this: struct Array2D(E) { E[] impl; int stride; int width, height; this(int width, int height, E[] initialData = []) ref E opIndex(int i, int j) Array2D opIndex(int[2] r1, int[2] r2) auto opIndex(int[2] r1, int j) auto opIndex(int i, int[2] r2) int[2] opSlice(size_t dim)(int start, int end) @property int opDollar(size_t dim : 0)() @property int opDollar(size_t dim : 1)() } So basic indexing works fine: Array2D!int foo(4, 4); foo[0, 1] = foo[2, 3]; But array copy and setting/clearing doesn't: int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ]; foo[] = bar[]; And I get this very cryptic message: (6): Error: template `example.Array2D!int.Array2D.opSlice` cannot deduce function from argument types `!()()`, candidates are: (51):`example.Array2D!int.Array2D.opSlice(ulong dim)(int start, int end) if (dim >= 0 && (dim < 2))` 1. WTF `!()()` and I haven't even called anything with opSlice i.e. `a .. b`? Anyway, it doesn't overload [] with opIndex(), so fine, I add that. T[] opIndex() { return impl; } Now I get: foo[] = bar[]; // or foo[] = bar; Error: `foo[]` is not an lvalue and cannot be modified Array copying docs say: When the slice operator appears as the left-hand side of an assignment expression, it means that the contents of the array are the target of the assignment rather than a reference to the array. Array copying happens when the left-hand side is a slice, and the right-hand side is an array of or pointer to the same type. 2.WTF I do have slice operator left of assignment. So I guess [] is just wonky named getter (and not an operator) for a slice object and that receives the = so it's trying to overwrite/set the slice object itself. Next I added a ref to the E[] opIndex(): ref E[] opIndex() { return impl; } Now foo[] = bar[] works as expected, but then I tried foo[] = 0; and that fails: Error: cannot implicitly convert expression `0` of type `int` to `int[]` 3. WTF. Didn't I just get reference directly to the slice and array copy works, why doesn't array setting? The ugly foo[][] = 0 does work, but it's so ugly/confusing that I'd rather just use a normal function. So I added: ref E[] opIndexAssign(E value) { impl[] = value; return impl; } And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't. I get: Error: function `example.Array2D!int.Array2D.opIndexAssign(int f)` is not callable using argument types `(int, int, int)` expected 1 argument(s), not 3 4. WTF. So basically adding opIndexAssign(E value) disabled ref E opIndex(int i, int j). Shouldn't it consider both? I'm surprised how convoluted this is. Is this really the way it's supposed to work or is there a bug or something? So what is the best/clear/concise/D way to do these for a custom type? I was planning for: foo[] = bar; // Full copy foo[] = 0; // Full clear foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy foo[1, 0 .. 5] = 0; // Row/Col clear foo[0 .. 5, 2 .. 4] = bar[ 1 .. 6, 0 .. 2 ]; // Box copy foo[0 .. 5, 2 .. 4] = 0; // Box clear Anyway, this is not a huge deal breaker for me, I was just surprised and felt like I'm missing something. I suppose I can manually define every case one by one and not return/use any references etc. or use alias this to forward to impl[] (which I don't want to do since I don't want to change .length for example) or just use normal functions and be done with it. And it's not actually just a regular array I'm making, so that's why it will be mostly custom code, except the very basics. The de facto multi dimensional array type in D is mir's ndslice https://github.com/libmir/mir-algorithm/blob/master/source/mir/ndslice/slice.d#L479
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote: But array copy and setting/clearing doesn't: int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ]; foo[] = bar[]; And I get this very cryptic message: (6): Error: template `example.Array2D!int.Array2D.opSlice` cannot deduce function from argument types `!()()`, candidates are: (51):`example.Array2D!int.Array2D.opSlice(ulong dim)(int start, int end) if (dim >= 0 && (dim < 2))` 1. WTF `!()()` and I haven't even called anything with opSlice This comes from some old history: arr[] used to call opSlice(), and now it is preferred to implement opIndex() instead, but the compiler still supports the old zero-arg opSlice() too. Since opIndex() didn't work, it moved to trying the older opSlice(), and that didn't work leading it to give up and issue the error. But yeah, the error should probably mention the newer function name instead of the fallback it is failing on... Next I added a ref to the E[] opIndex(): ref E[] opIndex() { return impl; } I would avoid ref as much as you can, for one because it conflates getting and assigning (as you saw), but also because it leaks your internal implementation detail to the user; if you didn't use an array internally, that api would break. So I added: ref E[] opIndexAssign(E value) { impl[] = value; return impl; } And now foo[] = 0; works, but foo[0, 1] = foo[2, 3] doesn't. *snip* 4. WTF. So basically adding opIndexAssign(E value) disabled ref E opIndex(int i, int j). Shouldn't it consider both? Once you implement an opIndexAssign - any opIndexAssign - all uses of `a[...] = c` will go through it instead of normal opIndex. Generally speaking, opIndex is for getting, opIndexAssign is for setting. Sometimes setting can be done via a getter function (like if ref), but they are mostly two different functions and you should implement them both if you want read/write access. This is especially important if your underlying data isn't actually in a literal array. foo[] = bar; // Full copy implement: opIndexAssign(typeof(this) rhs); In opIndexAssign, generally, the arguments are (value_on_right_hand_side, indexes...) Since there was no index given here, you don't want an index argument. foo[] = 0; // Full clear implement: opIndexAssign(int rhs); Now it will take any int for the whole thing. foo[0 .. 5, 1] = bar[ 0 .. 5]; // Row/Col copy This is translated to: this.opIndexAssign(bar.opIndex(bar.opSlice!0(0, 5)), this.opSlice!0(0, 5), 1) Three different functions there. On the left, we see x[...] = y, so we know that is opIndexAssign again. The first argument is what it is set to, other arguments are the slices given. Any x .. y is translated to opSlice!dim(low, high). The !0 in there is because it was given as the zeroth (first) argument in the slice. In this example, `foo[1 .. 2, 3 .. 4]`, it would call opIndex( opSlice!0(1, 2), opSlice!1(3, 4) ) With the !0 and !1 indicating which position the slice was in. The implementation of these functions would depend on just what your innards are... but the types might be something like this: struct SliceHelper { size_t start; size_t end; int stride; } SliceHelper opSlice(size_t dimension)(size_t start, size_t end) { /* return the helper with the appropriate values */ } now, we can implement opIndex for getting in terms of that and some regular items: // get a single item at point x, y int opIndex(size_t x, size_t y) { } // well not necessarily an array, maybe a range for lazy processing, but meh you get the idea // this gives a slice in the X dimension with a fixed Y coordinate // e.g. foo[ 0 .. 5, 3] int[] opIndex(SliceHelper x, size_t y) {} // foo[0, 4 .. 6] int[] opIndex(size_t x, SliceHelper y) {} // and now a 2d section of it Array2d!int opIndex(SliceHelper x, SliceHelper y) { } // and the zero-arg version, for foo[] // here I return this for an example, but by convention, // this should actually return a range object that is a // view into this container... which might be `this` but // might not be, depending on the details of your code like // if it has internal references or other stuff you don't want // to leak to the outside. typeof(this) opIndex() { return this; } And then a similar combination of arguments for opIndexAssign (I return void here but it is also common to return this; ) void opIndexAssign(int rhs) // this[] = rhs void opIndexAssign(int rhs, size_t x, size_t y) // this[x, y] = rhs void opIndexAssign(int rhs, size_t x, SliceHelper y) // this[x, y1 .. y2] = rhs you get the idea. The overload for other types of rhs. This might be dozens of functions! You can minimize that a bit by templating them and using internal static if, loops, etc to narrow it down. void opIndexAssign(R, Idx1, Idx2)(R rhs, Idx1 x, Idx2 y) { // test types of Idx1+Idx2 for figuring out what to change // test type of R to see what to c
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 16:10:36 UTC, Adam D. Ruppe wrote: On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote: But array copy and setting/clearing doesn't: int[] bar = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ]; foo[] = bar[]; Generally speaking, opIndex is for getting, opIndexAssign is for setting. Thanks a lot for a very detailed answer. Sorry about the late reply. But for 2d and 3d and more arrays, the number of functions explodes really fast. Yeah, tastes like C++, but I guess I'll bite. I value debuggability and I only have the 2D case, so I think templates are out.
Re: I had a bad time with slice-in-struct array operation forwarding/mimicing. What's the best way to do it?
On Saturday, 4 May 2019 at 15:36:51 UTC, Nicholas Wilson wrote: On Saturday, 4 May 2019 at 15:18:58 UTC, Random D user wrote: I wanted to make a 2D array like structure and support D slice like operations, but I had surprisingly bad experience. The de facto multi dimensional array type in D is mir's ndslice https://github.com/libmir/mir-algorithm/blob/master/source/mir/ndslice/slice.d#L479 Thanks. I'll take a look.