Re: Finally full multidimensional arrays support in D
On Tuesday, 10 June 2014 at 19:04:18 UTC, Stefan Frijters wrote:
I've been using the multidimensional arrays for a while now,
but recently I've run into a problem w.r.t. optimization:
import std.stdio;
import unstd.multidimarray;
void main() {
MultidimArray!(double, 3) arr;
arr = multidimArray!double([1,2,42]);
writeln(arr.lengths);
}
If I compile with 'dmd multidimtest.d unstd/multidimarray.d', I
get [1,2,42], as expected, but when I compile in release mode
'dmd multidimtest.d unstd/multidimarray.d -release' I get
[0,0,0]. Any ideas what might cause this?
I've looked into this and it seems there is required code inside
a contract 'in', which is omitted with -release.
I've gone ahead and opened a pull request to fix this:
https://bitbucket.org/denis-sh/unstandard/pull-request/2/bugfixes-for-multidimarrayd
Stefan
Re: Finally full multidimensional arrays support in D
I've been using the multidimensional arrays for a while now, but
recently I've run into a problem w.r.t. optimization:
import std.stdio;
import unstd.multidimarray;
void main() {
MultidimArray!(double, 3) arr;
arr = multidimArray!double([1,2,42]);
writeln(arr.lengths);
}
If I compile with 'dmd multidimtest.d unstd/multidimarray.d', I
get [1,2,42], as expected, but when I compile in release mode
'dmd multidimtest.d unstd/multidimarray.d -release' I get
[0,0,0]. Any ideas what might cause this?
Also, when I compile with -w (which I normally prefer) I get a
list of errors of the type
unstd/multidimarray.d(101): Warning: calling
unstd.multidimarray.MultidimArray!(double,
3LU).MultidimArray.this without side effects discards return
value of type inout(MultidimArray!(double, 3LU)), prepend a
cast(void) if intentional
I've acted on these errors in the version I use for my HPC test
code, and I have not noticed any deleterious effects; would you
accept a pull request to fix these?
Kind regards,
Stefan Frijters
Re: Finally full multidimensional arrays support in D
Am 17.03.2014 18:39, schrieb Denis Shelomovskij: Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html Yes please
Re: Finally full multidimensional arrays support in D
On Monday, 17 March 2014 at 21:25:34 UTC, bearophile wrote: Jared Miller: And yes, I think that a matrix / linear algebra library, as well as NumPy-style ND-Arrays are great candidates for future Phobos modules. I suggest to not put such library in Phobos before few years of usage in the wild. +1 Good matrix support would be awesome. But getting it wrong would be a catastrophe. I don't really support ever putting it in phobos, but if it is, then it should only be added after lots of experience.
Re: Finally full multidimensional arrays support in D
On Thu, Apr 17, 2014 at 03:16:20PM +, CJS via Digitalmars-d wrote: On Monday, 17 March 2014 at 21:25:34 UTC, bearophile wrote: Jared Miller: And yes, I think that a matrix / linear algebra library, as well as NumPy-style ND-Arrays are great candidates for future Phobos modules. I suggest to not put such library in Phobos before few years of usage in the wild. +1 Good matrix support would be awesome. But getting it wrong would be a catastrophe. I don't really support ever putting it in phobos, but if it is, then it should only be added after lots of experience. I've been longing for a high-quality, flexible, generic linear algebra library in D. I don't have the time / resources to implement it myself, otherwise I would. But I agree that any such candidate library needs to be put in real-life use for a while before being considered for Phobos. I think the first step would be to refine Denis' n-dimensional array library until it's Phobos-quality, then linear algebra specific adaptations can be built on top. I think the two should be separated, even if they are still related. Conflating 2D arrays with matrices at a fundamental level is a mistake IMO. 2D arrays are just one of the possible representations of a matrix, and any linear algebra library should be flexible enough to use other representations (e.g., sparse matrices). T -- He who laughs last thinks slowest.
Re: Finally full multidimensional arrays support in D
On Monday, 17 March 2014 at 17:39:41 UTC, Denis Shelomovskij wrote: Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html First of all, thank you very much for making such nice additions to D available for general use. I finally got around to giving this a spin. I'm using it for a proof-of-context HPC simuation code written in D (currently mostly experimenting with D's features), and as such I'm interfacing with the C MPI library to communicate between processes. The basis of the simulation is a 3D lattice, so I was eagerly awaiting a nice solution in D. So far I've run into two things while using your library. The first is that I need to provide void pointers to the data to the MPI functions, so I currently hacked your code to make the _data storage array publicly accessible and that seems to work. To give an idea, I currently have code like this (just a snippet): arr = multidimArray!T(nxH, nyH, nzH); // [...] fill the array with data // Prepare a buffer to receive a slice from another process. rbuffer = multidimArray!T(haloSize, nyH, nzH); // Prepare a buffer to send a slice to another process. sbuffer = arr[$-2*haloSize-1..$ - haloSize-1, 0..$, 0..$].dup; // Here I now use the pointer of the storage arrays to send the buffer around. MPI_Sendrecv(sbuffer._data.ptr, nyH * nzH, MPI_INT, M.nbx[1], 0, rbuffer._data.ptr, nyH * nzH, mpiType, M.nbx[0], 0, M.comm, mpiStatus); // Put the buffer in the correct spot in the main array. arr[0..haloSize, 0..$, 0..$] = rbuffer; Am I missing a nicer way to accomplish this? I like the compactness of the code (compared to what I'm currently used to with our F90 simulation code). Secondly, the property that returns the dimensions of the array is called 'dimentions' (with a t), this should be fixed. Regards, Stefan
Re: Finally full multidimensional arrays support in D
Stefan Frijters: First of all, thank you very much for making such nice additions to D available for general use. I finally got around to giving this a spin. Recently I've shown a possible usage example of the multidimensional arrays indexing and slicing syntax: http://forum.dlang.org/thread/cizugfrkaunlkzyjp...@forum.dlang.org Bye, bearophile
Re: Finally full multidimensional arrays support in D
18.03.2014 1:35, Christof Schardt пишет: I looked to the doku-page and expected to see, how some basic matrix-operations could be performed. Like e.g. (basic syntax) this: Dim a(10,12) a(3,4) = 7 a(i,j) = a(j,i) * 12 Instead I found exotic assignments and calculations. Did I expect the wrong thing? Sorry for that. Do you mead there is a lack of simple indexing examples or lack of some matrix functionality? The latter case is if you meant transpose and multiply by 12 with `a(i,j) = a(j,i) * 12`. -- Денис В. Шеломовский Denis V. Shelomovskij
Finally full multidimensional arrays support in D
Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Finally full multidimensional arrays support in D
+100 On Monday, 17 March 2014 at 17:39:41 UTC, Denis Shelomovskij wrote: Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html
Re: Finally full multidimensional arrays support in D
On Mon, Mar 17, 2014 at 09:39:40PM +0400, Denis Shelomovskij wrote: Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html [...] Let's do it! Put it up for review! I almost gave up hope that this day would come, but this morning I looked at the dmd commit log and saw Kenji's pull was merged. You couldn't believe how happy I was. Finally, sane multidimensional array support!! T -- Almost all proofs have bugs, but almost all theorems are true. -- Paul Pedersen
Re: Finally full multidimensional arrays support in D
Good news and kudos to Kenji. And yes, I think that a matrix / linear algebra library, as well as NumPy-style ND-Arrays are great candidates for future Phobos modules. We've done a good bit of work with matrices at my workplace (EMSI), so I bet someone here could find time to contribute a bit. As a start, we've got some basic BLAS and MKL bindings that we really should clean up and get into Deimos. On Monday, 17 March 2014 at 17:39:41 UTC, Denis Shelomovskij wrote: Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html
Re: Finally full multidimensional arrays support in D
Jared Miller: And yes, I think that a matrix / linear algebra library, as well as NumPy-style ND-Arrays are great candidates for future Phobos modules. I suggest to not put such library in Phobos before few years of usage in the wild. Bye, bearophile
Re: Finally full multidimensional arrays support in D
I looked to the doku-page and expected to see, how some basic matrix-operations could be performed. Like e.g. (basic syntax) this: Dim a(10,12) a(3,4) = 7 a(i,j) = a(j,i) * 12 Instead I found exotic assignments and calculations. Did I expect the wrong thing?
Re: Finally full multidimensional arrays support in D
17.03.2014 21:39, Denis Shelomovskij пишет: Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html An important note: My proposal is just a sliceable multidimensional array implementation, a try to provide the same effort we have for simple arrays for multidimensional ones. It isn't aimed to be a dedicated matrix / linear algebra module. -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Finally full multidimensional arrays support in D
On Tue, Mar 18, 2014 at 01:55:31AM +0400, Denis Shelomovskij wrote: 17.03.2014 21:39, Denis Shelomovskij пишет: Multidimensional arrays indexing and slicing syntax is finally added [1] (thanks to Kenji Hara). So it was a good cause to update my multidimensional arrays library implementation and add support for the new syntax. So here we are: [2]. Also should we add it to the standard library? [1] https://github.com/D-Programming-Language/dmd/pull/443 [2] http://denis-sh.bitbucket.org/unstandard/unstd.multidimarray.html An important note: My proposal is just a sliceable multidimensional array implementation, a try to provide the same effort we have for simple arrays for multidimensional ones. It isn't aimed to be a dedicated matrix / linear algebra module. [...] It will be a good low-level module that will allow a linear algebra module to be built on. I am against the idea of conflating multidimensional arrays with matrices; the operations and interpretations are not the same, and it leads to poor designs (I've tried it before, and the results were not so good). Better have a general multidimensional array with per-element operations, and then build matrix algebra as a separate module by using multidimensional array as underlying storage implementation. T -- Answer: Because it breaks the logical sequence of discussion. / Question: Why is top posting bad?
Re: Finally full multidimensional arrays support in D
It will be a good low-level module that will allow a linear algebra module to be built on. Agreed. In fields like graphics and computer vision, large multidimensional arrays aren't always matrices/tensors in the linear algebra sense. And it's always possible to specialize. NumPy got this right.
