Hi Dave, On Feb 16, 2013, at 6:53 PM, Dave Wade-Stein wrote:
> While I agree that two NaNs are technically equal, in the hdf5 sense, a NaN > always indicates a programmatic/numerical error. So an option to h5diff which > alerted the user to the presence of NaNs is not "clearly useless." > > I want to know immediately when a code is producing them. It doesn't seem > unreasonable to assume that NaNs could be detected when h5diff-ing, > especially given that h5diff is doing NaN detection in the first place. > > We've had cases where NaNs crept in to accepted results, and then because the > code continued to produce NaNs, we assumed the code was working because > h5diff showed no differences. I can of course prevent this in the future by > doing an h5dump of each generated hdf5 file, and grep-ing for NaNs, and I > guess that's what I'll do. > I don't think we thought about the HDF5 tools as of data validation tools. Using h5dump to dump GBs or TBs of data to get NaN doesn't make much sense if h5diff already reads each element and reports the differences, i.e., I see your point. My personal opinion is that an application should be checking the validity of the data, but I can see the HDF5 tools as a "second line of defense" :-) If we provide an option to h5diff to choose NaN behavior, will it help? What do other members of this FORUM think? Thank you! Elena > Dave > >> >>> > I agree that h5diff is just comparing bit patterns, but obviously h5diff >>> > knows when a bit pattern equals NaN and when it doesn't. So it could >>> > refuse to says that two NaNs are equal. >>> >>> But then a file containing a NaN would be different from itself, a clearly >>> useless state of affairs. >> This was exactly the reason why The HDF Group decided that h5diff should >> behave this way. BTW, the current h5diff behavior with NaN is documented in >> the tool's RM page http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff. >> >> Dave, >> >> In your opinion, what would be the use case when NaNs should be treated as >> not equal when compared to each other by h5diff? >> >> Thank you! >> >> Elena >> >> > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
