Hi kmx-
This the sf bug #343, "longlong constructor and display lose digits
due to implicit double precision conversions" And is the main remaining
issue with 64bit index support.
The PDL_Anyval type is a union type for all the basic PDL types with
64bit integers as well. I've gotten a lot of the logic in but the
remaining problem is, I believe, in the typemap conversions. At that
point I ran out of time and we probably need a clean up of the code.
What would really help would be some fresh eyes with XS experience!
I consider working 64bit index support the remaining "missing piece"
for PDL-2.x development. With that in place, we would support current
memory sizes for pdl objects and would be able to use memory mapping
for all sorts of neat things like direct file access.
That would be a PDL-2.infinity that could be stable while some aggressive
refactoring and PDL3 kernel compute development takes place.
--Chris
On 8/21/2015 13:52, kmx wrote:
Patching Core.xs improved situation slightly but I have found another
problem in pdlsections.c / pdl_at()
PDL_Anyval pdl_at( void* x, int datatype, PDL_Indx* pos, PDL_Indx*
dims, PDL_Indx* incs, PDL_Indx offset, int ndims)
{
....
case PDL_LL:
{
PDL_LongLong *xx = (PDL_LongLong *) x;
result = (PDL_Anyval)xx[ioff];
}
....
}
Where PDL_Anyval is in fact double which is not able to accommodate
full 64bit signed integer.
What is also interesting in the same function is:
case PDL_IND:
{
PDL_Indx *xx = (PDL_Indx *) x;
result = (PDL_Anyval)xx[ioff];
}
Which means that not only LongLong accuracy is not what one would
expect but 64bit indexing might be also affected.
I might be wrong but this seems to be not easy to fix.
--
kmx
On 21.8.2015 16:48, kmx wrote:
Yes, any integer that fits to IV should be converted via newSViv
BTW meanwhile I have found in Core.xs SET_RETVAL_NV macro:
#define SET_RETVAL_NV(x) x->datatype<PDL_F ? (RETVAL=newSViv(
(IV)result )) : (RETVAL=newSVnv( result ))
we would need something like
#define SET_RETVAL_NV(rv,val,x) x->datatype<PDL_F ?
(rv=newSViv((IV)(val))) : (rv=newSVnv((val)))
but basically this is it
--
kmx
On 21.8.2015 16:26, Joel Berger wrote:
I imagine it would be any int type, no?
On Fri, Aug 21, 2015 at 9:21 AM kmx <k...@atlas.cz> wrote:
You are right, at least during unpdl() all values are converted
to perl's NV
It happens in Core.xs inside SV *listref_c(x) - there is in all
cases (apart from badval): sv = newSVnv( pdl_val )
Correct me if I'm wrong but the patch is quite straightforward
if ( datatype == PDL_LongLong && sizeof(IV) == 8) {
sv = newSViv( pdl_val );
}
else {
sv = newSVnv( pdl_val );
}
or not?
--
kmx
On 21.8.2015 13:52, Ingo Schmid wrote:
Hi,
as far as I remember, all data gets converted into double
during pdl construction before cast to the desired type.
Apparently, unpdl does the same. It's a known issue for a long
time, AFAIK. The same even happens when you assign to a
longlong pdl. If I remember, nobody dared touch the core hard
enough to fix that one, yet.
Ingo
On 08/21/2015 12:55 PM, kmx wrote:
Hi,
while playing with LongLong piddles representing timestamps I
have run into the following troubles with precision loss when
passing 64bit signed int values between perl and PDL (both
directions).
Let's have 64bit number 253402300799999999 (BTW it is a number
of microseconds since UNIX epoch beginning corresponding to
9999-12-31T23:59:59.999999Z) - it occupies "only" 58bits, so
we are not even approaching 63bit edge which might become
tricky when dealing with 64bit signed integers.
Demonstration code:
use PDL;
use Config;
my $perl_var = 253402300799999999;
my $pdl_var = longlong(253402300799999999);
print "PDL::VERSION: ", $PDL::VERSION, "\n";
print "ivsize : ", $Config{ivsize}, "\n";
print "use64bitint : ", $Config{use64bitint}, "\n";
print "archname : ", $Config{archname}, "\n";
print "perl 1: ", $perl_var, "\n";
print "perl 2: ", ($perl_var % 1000), "\n";
print "pdl : ", $pdl_var->info, "\n";
print "pdl 1: ", $pdl_var, "\n";
print "pdl 2: ", ($pdl_var % 1000), "\n";
print "pdl 3: ", $pdl_var->unpdl->[0], "\n";
Output:
PDL::VERSION: 2.011
ivsize : 8
use64bitint : define
archname : MSWin32-x86-multi-thread-64int
perl 1: 253402300799999999
perl 2: 999
pdl : PDL: LongLong D []
pdl 1: 253402300800000000
pdl 2: 0
pdl 3: 2.534023008e+017
From my point of view there are 2 issues:
1/ A completely legal / not even extremely large / 64bit
integer value is OK on perl side, but when turned into a
piddle the value changes (253402300799999999 >
253402300800000000). This may be a bug.
2/ When exporting a LongLong piddle via unpdl the value (64bit
signed int) is converted into double (perl's NV) even when the
perl is compiled with 64bit integers. Maybe unpdl() is not the
best way for extracting piddle value into a perl array (but I
really like this function :).
Thanks in advance for any hints.
--
kmx
------------------------------------------------------------------------------
_______________________________________________
pdl-devel mailing list
pdl-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-devel
------------------------------------------------------------------------------
_______________________________________________
pdl-devel mailing list
pdl-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-devel
