On Thu, 20 Dec 2012, Craig DeForest wrote:
You can specify $PDL::undefval - it will use that instead of 0 for undefs.
I'd figured out you can't set undefval to anything resembling 'BAD'
directly (strings get numified to 0), but that did give me an idea:
$PDL::undefval = 'nan';
...;
my $pdl = pdl($ref)->inplace->setnantobad;
Worked in test, same error in production, though with a wrinkle: Now the
recursion limit is exceeded when I'm using a piddle (e.g., $pdl->avg,
$pdl->stdv, $pdl1->corr($pdl2)) rather than when I'm making it.
On Thu, 20 Dec 2012, David Mertens wrote:
I do not know much about the constructor internals, you will probably need to
wait for Craig or Chris to weigh in. But if there is no way to do it, I know
exactly how to write an efficient implementation, and could crank something out
quickly, tomorrow morning if you need it, or tonight if you're in a real
pickle.
It's not a rush, more something I was hoping to have done before the
holidays, but I'm interested in whatever you come up with.
Thanks.
-- Erich
On Thu, 20 Dec 2012, Erich Greene wrote:
Hi,
I'm fairly new to PDL and this seems like an obvious thing to ask about, but I
haven't found it addressed in any docs or FAQs or in this list's archives.
I'm trying to create piddles from arrays that might include undefined values
and treat those values as bad in further processing. But when pdl() encounters
an input value of undef, it puts a 0 into the piddle. Setting 0 as the bad
value later doesn't work for me because 0 is also a legitimate data value, and
once the piddle is made, there's no distinguishing an undef treated as 0 from a
real 0. (In fact, any float could be legitimate -- that's why I've been using
undef for missing data -- so experimenting with badvalue was also a dead end.)
Eventually, I came up with this:
use 5.014;
use warnings;
use PDL::LiteF;
# make a piddle where undef becomes BAD
sub pdlify {
my @in = map {ref eq 'ARRAY' ? @$_ : $_} @_;
my @out = map {defined($_) ? pdl($_) : pdl(0)->setbadat(0)} @in;
return cat(@out);
}
This works fine in small-scale tests (my $x = pdlify(1..5,undef,7..10); say $x;
spits out [1 2 3 4 5 BAD 7 8 9 10] as it should), but in production, one of two
things eventually happens:
1) segfault
2) cat: unknown error from the internals:
PDL: Problem with assignment: PDL:Internal Error: data structure recursion
limit exceeded (max 1000 levels)
This could mean that you have found an infinite-recursion error in PDL,
or
that you are building data structures with very long dataflow
dependency
chains. You may want to try using sever() to break the dependency.
Each call to pdlify is acting on a fresh list of scalars (all numbers or
undef), so there shouldn't be any dependencies to speak of at this point.
Bottom line, is there an idiom or package already out there that will take a
list and create a piddle where undefined input values are represented as BAD
rather than 0, or is there at least some way to do it that doesn't go down some
mysterious recursive rabbit hole?
Thanks.
-- Erich
_______________________________________________
Perldl mailing list
[email protected]
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
