Re: turning an array of structs into a struct of arrays
On Monday, 6 July 2015 at 17:35:22 UTC, Vlad Levenfeld wrote: On Sunday, 5 July 2015 at 00:07:59 UTC, Laeeth Isharc wrote: Posted short write-up here. Please make it better... http://wiki.dlang.org/Transforming_slice_of_structs_into_struct_of_slices In John Colvin's solution, should alias TransformMembers(alias TypeTransform, alias NameTransform, T) = Tuple!( RoundRobin!(Pack!(staticMap!(TypeTransform, FieldTypeTuple!PriceBar)), Pack!(staticMap!(NameTransform, FieldNameTuple!PriceBar; read alias TransformMembers(alias TypeTransform, alias NameTransform, T) = Tuple!( RoundRobin!(Pack!(staticMap!(TypeTransform, FieldTypeTuple!T)), Pack!(staticMap!(NameTransform, FieldNameTuple!T; ??? Yikes, yes, of course!
Re: turning an array of structs into a struct of arrays
On Sunday, 5 July 2015 at 00:07:59 UTC, Laeeth Isharc wrote: Posted short write-up here. Please make it better... http://wiki.dlang.org/Transforming_slice_of_structs_into_struct_of_slices In John Colvin's solution, should alias TransformMembers(alias TypeTransform, alias NameTransform, T) = Tuple!( RoundRobin!(Pack!(staticMap!(TypeTransform, FieldTypeTuple!PriceBar)), Pack!(staticMap!(NameTransform, FieldNameTuple!PriceBar; read alias TransformMembers(alias TypeTransform, alias NameTransform, T) = Tuple!( RoundRobin!(Pack!(staticMap!(TypeTransform, FieldTypeTuple!T)), Pack!(staticMap!(NameTransform, FieldNameTuple!T; ???
Re: turning an array of structs into a struct of arrays
Posted short write-up here. Please make it better... http://wiki.dlang.org/Transforming_slice_of_structs_into_struct_of_slices
Re: turning an array of structs into a struct of arrays
I can use FieldTypeTuple and FieldNameTuple, but I am a bit
lost as to how without static foreach to loop through these in
order to generate a mixin to declare the new type. I can turn
it into a string, but what is the better option?
The simplest solution is something like:
template SOA(Struct, size_t LENGTH) {
struct SOA {
enum MEMBERNAME(size_t N) = __traits(identifier,
Struct.tupleof[N]);
static __gentypes() {
string ret;
foreach (I, TYPE; typeof(Struct.tupleof))
ret ~= "align(16)
typeof(Struct.tupleof["~I.stringof~"])["~LENGTH.stringof~"] "
~ MEMBERNAME!I ~ ";";
return ret;
}
mixin(__gentypes());
}
}
alias PriceBars = SOA!(PriceBar, 8);
which you'll have to adjust to your requirements (eg drop the
'"~LENGTH.stringof~"' part to get a struct-of-slices, which is
what your example above shows).
artur
Thanks to Artur, John Colvin, and Babu for the very helpful
responses. This should be up on the wiki, and ideally someone
should write a little tutorial text talking the reader through
how this works and why choices were made a certain way. I will
stick these up on the wiki with attribution if there are no
objections. Something is better than perfect in this situation.
There is a big gap between understanding what language terms do
and knowing in practice how to use them for metaprogramming if
you don't previously come from a C++ background. I originally
learnt C with parameters in K&R style, and the world has changed
quite a lot since those days - both the relative slowness of
memory, and CTFE/metaprogramming are quite new concepts to me
(Forth doesn't count). It's very easy to be quickly productive
in D for most things (I found it easier than Python, which I
learnt previously) but there is a bigger leap in learning
templates+CTFE - not because it's intrinsically hard, but it's a
different way of thinking that is unfamiliar. Probably I should
read more source code - the harder way is often easier in the
longer term.
In case it's of interest my own reason for wanting to do this
transformation was not cache efficiency, but just to be able to
easily get the data into a form I can access from PydMagic and
PyD, since for some chart applications it's easier to have the
data in this form, and I would rather do all transformations in D
and stick to Python only for pure charting.
I appreciate the thoughtful responses.
Laeeth.
Re: turning an array of structs into a struct of arrays
On 07/03/15 12:52, Laeeth Isharc via Digitalmars-d-learn wrote:
> I have an array of structs eg
>
> struct PriceBar
> {
> DateTime date;
> double open;
> double high;
> double low;
> double close;
> }
>
> (which fields are present in this particular struct will depend on template
> arguments).
>
> what is the best way to turn these at compile time into a struct of arrays? eg
>
> struct PriceBars
> {
> DateTime[] dates;
> double[] opens;
> double[] highs;
> double[] lows;
> double[] closes;
> }
>
> I can use FieldTypeTuple and FieldNameTuple, but I am a bit lost as to how
> without static foreach to loop through these in order to generate a mixin to
> declare the new type. I can turn it into a string, but what is the better
> option?
The simplest solution is something like:
template SOA(Struct, size_t LENGTH) {
struct SOA {
enum MEMBERNAME(size_t N) = __traits(identifier, Struct.tupleof[N]);
static __gentypes() {
string ret;
foreach (I, TYPE; typeof(Struct.tupleof))
ret ~= "align(16)
typeof(Struct.tupleof["~I.stringof~"])["~LENGTH.stringof~"] "
~ MEMBERNAME!I ~ ";";
return ret;
}
mixin(__gentypes());
}
}
alias PriceBars = SOA!(PriceBar, 8);
which you'll have to adjust to your requirements (eg drop the
'"~LENGTH.stringof~"' part to get a struct-of-slices, which
is what your example above shows).
artur
Re: turning an array of structs into a struct of arrays
Not necessarily the best solution to the problem, but I just wanted to show this because I think it's kinda cool: https://gist.github.com/John-Colvin/103d3c064ad6cb4cf435#file-transformmembers-d Very impressive - now I have to try to understand how it works...!
Re: turning an array of structs into a struct of arrays
On Friday, 3 July 2015 at 10:52:03 UTC, Laeeth Isharc wrote:
I have an array of structs eg
struct PriceBar
{
DateTime date;
double open;
double high;
double low;
double close;
}
(which fields are present in this particular struct will depend
on template arguments).
what is the best way to turn these at compile time into a
struct of arrays? eg
struct PriceBars
{
DateTime[] dates;
double[] opens;
double[] highs;
double[] lows;
double[] closes;
}
I can use FieldTypeTuple and FieldNameTuple, but I am a bit
lost as to how without static foreach to loop through these in
order to generate a mixin to declare the new type. I can turn
it into a string, but what is the better option?
Thanks.
Not necessarily the best solution to the problem, but I just
wanted to show this because I think it's kinda cool:
https://gist.github.com/John-Colvin/103d3c064ad6cb4cf435#file-transformmembers-d
Re: turning an array of structs into a struct of arrays
see https://github.com/economicmodeling/soa Thanks v much for this. Figured out in the meantime one solution here (probably quite ugly): https://gist.github.com/Laeeth/9174498f9b79dc90ac18
Re: turning an array of structs into a struct of arrays
On Friday, 3 July 2015 at 10:52:03 UTC, Laeeth Isharc wrote:
I have an array of structs eg
struct PriceBar
{
DateTime date;
double open;
double high;
double low;
double close;
}
(which fields are present in this particular struct will depend
on template arguments).
what is the best way to turn these at compile time into a
struct of arrays? eg
struct PriceBars
{
DateTime[] dates;
double[] opens;
double[] highs;
double[] lows;
double[] closes;
}
I can use FieldTypeTuple and FieldNameTuple, but I am a bit
lost as to how without static foreach to loop through these in
order to generate a mixin to declare the new type. I can turn
it into a string, but what is the better option?
Thanks.
see https://github.com/economicmodeling/soa
