Squashing the whole branch sounds good to me.
I'm working to set up a large memory machine to write/run the t/bigmem.t
and others against.
On Mon, Aug 31, 2015 at 4:45 PM, kmx <k...@atlas.cz> wrote:
> Ed,
>
> Frankly speaking I would even suggest rebasing + squashing the whole
> longlong branch into one commit before merging to master (of course if
> Chris agrees as most of the commits are from him). This will make it more
> obvious that all these changes came together and make future investigation
> via "git blame" more straightforward.
>
> If you look at the commit history there are many do/undo changes,
> put-debug/remove-debug, try-this/try-that etc. which is absolutely fine for
> devel branch but has IMO not that much value in master.
>
> The more important question is: Is it ready for master?
>
> From my point of view I have currently nothing more to commit unless
> somebody reports a bug.
>
> --
> kmx
>
> On 31.8.2015 16:15, Ed wrote:
>
> I’ll be pleased to write some tests for it that are basically what you
> have below. I’d also like to tidy up the lldf branch as it currently is by
> rebasing it to get rid of the merge commit.
>
> Chris, kmx, any objections?
>
> Ed
>
> *From:* Chris Marshall <devel.chm...@gmail.com>
> *Sent:* Monday, August 31, 2015 10:58 AM
> *To:* pdl-devel@lists.sourceforge.net
> *Subject:* Re: [Pdl-devel] LongLong precision (fixed)
>
> The LongLong to Double conversion problems
> appear to have been resolved. I would invite
> any PDL developers with systems having lots
> of RAM to test large memory operations.
>
> We need to add some tests to check for correct
> operation when the PDLs and index values are
> >=2**31. For instance, here is a problem caused
> by an int where a PDL_Indx should have been:
>
> > It crashes for me as does this:
> >
> > pdl> $PDL::BIGPDL = 1;
> > pdl> $x = zeros(byte,1024,1024,2049);
> > pdl> $y = $x->copy;
> > Segmentation fault (core dumped)
> >
> > but this is ok:
> >
> > pdl> $PDL::BIGPDL = 1;
> > pdl> $x = zeros(byte,1024,1024,2047);
> > pdl> $y = $x->copy;
> > pdl> q
> >
> > so there is a problem for piddles with more
> > than 2**31 elements.
>
> t/bigmem.t anyone?
> t/bigmemmap.t anyone?
>
>
> --Chris
>
>
>
> On 8/26/2015 17:45, kmx wrote:
>
> Hi all,
>
> After some off-list discussion with Chris I have fixed his fix in
> longlong-double-fix branch. It seems to work, passes all tests and is more
> or less ready for merging to master.
>
> Travis-CI is all green see
> <https://travis-ci.org/PDLPorters/pdl/builds/77348194>
> https://travis-ci.org/PDLPorters/pdl/builds/77348194
>
> All changes:
> <https://github.com/PDLPorters/pdl/compare/master...longlong-double-fix#files_bucket>
> https://github.com/PDLPorters/pdl/compare/master...longlong-double-fix#files_bucket
>
> The main idea is a new type PDL_Anyval capable of keeping values of all
> types without loosing any precision. Like this:
>
> typedef enum { PDL_B, PDL_S, PDL_US, PDL_L, PDL_IND, PDL_LL, PDL_F, PDL_D
> } pdl_datatypes;
>
> typedef struct {
> pdl_datatypes type;
> union {
> PDL_Byte B;
> PDL_Short S;
> PDL_Ushort U;
> PDL_Long L;
> PDL_Indx N;
> PDL_LongLong Q;
> PDL_Float F;
> PDL_Double D;
> } value;
> } PDL_Anyval;
>
> There are still some questions that I want to discuss here:
>
> *1/ "broken" C API (not that much, but still)*
>
> Using PDL_Anyval struct (instead of plain double) has some side effects
> concerning C API, namely:
> PDL->put_offs(..)
> PDL->get_offs(..)
> PDL->get(..)
> PDL->get_pdl_badvalue(..)
>
> The relevant declarations in Core structure were not changed:
> void (*put_offs)(pdl *pdl,PDL_Indx offs, PDL_Anyval val);
> PDL_Anyval (*get_offs)(pdl *pdl,PDL_Indx offs);
> PDL_Anyval (*get)(pdl *pdl,PDL_Indx *inds);
> PDL_Anyval (*get_pdl_badvalue)(pdl *it);
>
> however the old code like:
> if( PDL->get_pdl_badvalue(a) == 0 ) { ... }
> does not work anymore (it does not even compile) from obvious reason.
>
> The new style should be
> double bad_a;
> ANYVAL_TO_CTYPE(bad_a, double, PDL->get_pdl_badvalue(a));
> if( bad_a == 0 ) { ... }
>
> BTW: via <http://grep.cpan.me/>http://grep.cpan.me/ I have found only one
> Zaki's module that is affected (+of course possibly a lot of code not on
> CPAN)
>
> There are also four newly introduced ANYVAL_* macros in pdl.h for handling
> PDL_Anyval from C code.
> #define ANYVAL_TO_SV(outsv,inany) ...
> #define ANYVAL_FROM_CTYPE(outany,avtype,inval) ...
> #define ANYVAL_TO_CTYPE(outval,ctype,inany) ...
> #define ANYVAL_EQ_ANYVAL(x,y) ...
>
> Which will be a pain for those who want to write their modules/program so
> that it works with old PDL versions (without these macros) as well as with
> the new version. The only recommendation I have for this scenario is the
> following set of defines that has to be added to the C code for
> compatibility with older PDL.
>
> #if PDL_CORE_VERSION < 12
> #define ANYVAL_TO_SV(outsv,inany) outsv = newSVnv((NV)(inany)
> #define ANYVAL_FROM_CTYPE(outany,avtype,inval) outany =
> (PDL_Double)(inval)
> #define ANYVAL_TO_CTYPE(outval,ctype,inany) outval = (ctype)(inany)
> #define ANYVAL_EQ_ANYVAL(x,y) (x == y)
> #endif
>
> *2/ typemap & undef handling*
>
> I am not quite sure what is the expected behaviour when undef SV argument
> has to be converted to PDL_Anyval. Now I convert it into a invalid
> PDL_Anyval { type=-1, value.B=0 } which might not be the best idea.
>
> The old behaviour was always (even for undef) use:
> $var = (double)SvNV($arg)
>
> The difference can be seen for example here:
> my $p = sequence(4);
> PDL::Core::set_c($p, [1], undef);
> print $p
> The old behaviour is "undefined ... warning" + setting the value to 0
> (zero). The new behaviour is to croak.
>
> Of course it would be easy to change croak() to silently setting 0 value
> but I want to ask: what is correct/expected/best behaviour in this case?
>
> We are talking here about a typemap so we do not have any context (e.g.
> type of a piddle where PDL_Anyval value will be later assigned) therefore
> the obvious suggestion for using badval for undef is not possible as it is
> not clear which badval.
>
> *3/ PDL data type auto-detection*
>
> Auto-detection (we have a SV and want to decide for the best PDL datatype)
> is implemented at couple of places. Based on perl's SV type (int/IV or
> double/NV) we decide for PDL_Long (on perl with 64bit int for PDL_LongLong)
> or PDL_Double.
>
> Simplified code works like this:
> if (SvIOK(sv)) {
> #if IVSIZE == 8
> val = (PDL_LongLong)SvIV(sv)
> #else
> val = (PDL_Long)SvIV(sv)
> #endif
> } else if (SvOK(sv)) {
> val = (PDL_Double)SvNV(sv) /* tries also to convert strings */
> } else {
> // do some magick in undef case
> }
>
> or even simpler:
> if (SvIOK(sv)) {
> #if IVSIZE == 8
> val = (PDL_LongLong)SvIV(sv)
> #else
> val = (PDL_Long)SvIV(sv)
> #endif
> } else {
> val = (PDL_Double)SvNV(sv) /* tries also to convert strings + undef
> */
> }
>
> This kind of works but I am open to any comments/improvements.
>
> BTW: let's have:
> $p = pdl( [ 1, 2, 3 ] ); # perl integers
> print $p;
> Is it correct that type of $p is always Double? although LongLong would be
> more than enough?
>
>
> Thanks in advance for any feedback.
>
> --
> kmx
>
>
>
> ------------------------------------------------------------------------------
>
>
>
> _______________________________________________
> pdl-devel mailing
> listpdl-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/pdl-devel
>
>
> ------------------------------
>
> ------------------------------------------------------------------------------
>
> ------------------------------
> _______________________________________________
> pdl-devel mailing list
> pdl-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pdl-devel
>
>
> ------------------------------------------------------------------------------
>
>
>
> _______________________________________________
> pdl-devel mailing
> listpdl-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/pdl-devel
>
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> pdl-devel mailing list
> pdl-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pdl-devel
>
>
------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
_______________________________________________
pdl-devel mailing list
pdl-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-devel
