Hi Jarom, Yeah, so I agree that having HDF5 types that match natural types in the languages being used is probably the most important thing. That was my reason for asking about variations in how Fortran handles things -- that is the only language I've used that supports complex out of the box.
At the same time, I always like having an idea of the bigger picture just so I can understand how much I'll wind up shooting myself in the foot for the sake of keeping it simple ;) Didn't know C++ defined a std::polar Mark "Hdf-forum on behalf of Nelson, Jarom" wrote: The reason that I asked about the complex datatype specifically is that complex is a built-in type in Fortran, Python, IDL, and others, and in many other languages is part of the core libraries (C++ std::complex, Perl Math::Complex, etc.). I think in all these cases (correct me if I’m wrong), the type is typically implemented as a Cartesian representation of complex. (though C++ <complex> also gives std::polar). Expanding the scope to include more general complex numbers, polar representation, etc. certainly does make it much more complex, and I can understand if the scope of that problem would be difficult to grapple with and come up with a standard that works. Jarom From: Hdf-forum [mailto:[email protected]] On Behalf Of Werner Benger Sent: Wednesday, February 22, 2017 11:53 AM To: [email protected] Cc: Dietmar Hildenbrand <[email protected]> Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?) The topic becomes indeed more complex once you think about complex numbers as just special cases of Geometric Algebra (also known as Clifford Algebra), where complex number are the special case in 2D and quaternions are the special case in 3D. But if that scheme is implemented and provided systematically, then rather do it n-dimensional, which automatically leads to Geometric Algebra. And furthermore, doing it systematically would also imply supporting different coordinate representations of the same number (like cartesian vs. polar coordinates, in higher dimensions even more variations). Also we may want to distinguish between different kinds of inner products, which lead to different algebraic properties. For instance a complex quaternion is used in spacetime algebra for Lorentz transformations, useful both in relativistic astrophysics and in quantum mechanics for the Dirac equations, a Dirac spinor is a rotor (a multivector with a scalar and bi-vector component) in 4D Geometric Algebra. So, it's yet an open end there. A couple of years ago I had a paper on just that topic, including about storing multivector types (the elementary objects of Geometric Algebra, quaternions are a sub-algebra of 3D GA) in HDF5, interested party may find it here: http://sciviz.cct.lsu.edu/papers/2009/GraVisMa09.pdf That article is far from a complete scheme or worked out in detail, but it raises some issues on just this topic, food for thought. And it would be good if complex numbers would fit into a bigger scheme yet to come when they are implemented once. Werner On 22.02.2017 18:25, Miller, Mark C. wrote: This has been an interesting thread to follow. Thanks, Jarom, for starting it ;) Having said that, in reading here and in the various links contributors have referenced, I didn't necessarily see anything that described the 'issues' with supporting complex types in HDF5 in a first-class (e.g. built in to the library) sort of way. And, now, I am just really curious what those issues are and if/how they are different from any other primitive type HDF5 supports? I mean, I get that complex numbers represent a 'pair' of scalar values (which in cartesian coords are typically represented as real/imaginary and in polor coords, magnitude/phase) and I get that HDF5's underlying primitive type model is for the most part like standard floating point and integer computer representations including such things as offset, base, bias, mantissa, exponent, etc., but is there anything substantially more comlicated about handling them as first class type in the library than, as Francesc described below? Are there Fortran variants or language extensions that have something other than that basic modality of a pair of either 32 bit or 64 bit floating point numbers? Are there any Fortrans that use magnitude/phase (e.g. polar coords) instead of real/imaginary (cartesian coords). Question for Francesc...do you define *both* polar and cartesian forms for complex numbers and if so, do you define a conversion method to go between them? If not, would anything like that ever be useful if it existed? Seems like someone could define a 'complex types' support library extension on github, perhaps refactored from Francesc' code that might have the ability to become the defacto HDF5 standard for complex data in HDF5 files. Finally, what about quarternions ;}? I am not kidding, just curious. I think those might be relevant in astro-physical contexts wouldn't they? Mark "Hdf-forum on behalf of Nelson, Jarom" wrote: Thanks for all the responses. Very helpful. From: Hdf-forum [mailto:[email protected]] On Behalf Of Filipe Maia Sent: Wednesday, February 22, 2017 6:49 AM To: HDF Users Discussion List <[email protected]><mailto:[email protected]> Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?) I think the main issue is the "almost exactly". I'm also using h5py's convention, but having an HDF5 "officially endorsed" way of representing complex numbers would greatly help with compatibility across different user groups, instead of having *almost* compatible representations. And I hope the endorsed way matches the h5py convention, but any convention is better than none. Cheers, Filipe On 22 February 2017 at 15:34, Barbara Jones <[email protected]<mailto:[email protected]>> wrote: Hi Jarom, In case it might help, there are C and Fortran code examples in the User’s Guide that do almost exactly what Francesc mentioned. See Section 6.5.2.2.1 in the “HDF5 Datatypes” chapter of the HDF5 User’s Guide. The link to section 6.5.2 is: https://support.hdfgroup.org/HDF5/doc/UG/HDF5_Users_Guide-Responsive%20HTML5/index.html#t=HDF5_Users_Guide%2FDatatypes%2FHDF5_Datatypes.htm%23TOC_6_5_2_Definition_ofbc-14&rhtocid=6.3.0_2 Scroll down to 6.5.2.2.1, “Compound Datatypes”. Scroll down a little further and you will see “Code Example 6-9” for C and “Code Example 6-10” for Fortran. -Barbara From: Hdf-forum [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Francesc Altet Sent: Wednesday, February 22, 2017 2:02 AM To: HDF Users Discussion List Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?) Hi Jarom, For what is worth, in PyTables and h5py (and possibly in others libraries too) we used the convention of declaring the complex type by using a compound type as follows: """ The H5T_COMPOUND type class contains two members. Both members must have the H5T_FLOAT atomic datatype class. The name of the first member should be "r" and represents the real part. The name of the second member should be "i" and represents the imaginary part. The precision property of both of the H5T_FLOAT members must be either 32 significant bits (e.g. H5T_NATIVE_FLOAT) or 64 significant bits (e.g. H5T_NATIVE_DOUBLE). They represent Complex32 and Complex64 types respectively. """ Perhaps you may want to use this convention until a more formal one is implemented. Francesc Alted ________________________________ From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of Nelson, Jarom <[email protected]<mailto:[email protected]>> Sent: Wednesday, February 22, 2017 2:36:23 AM To: HDF Users Discussion List Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?) Can we get a FAQ entry on the topic? Bonus points for an example user implementation. Thanks for your answer and for a great library! Jarom From: Hdf-forum [mailto:[email protected]] On Behalf Of Barbara Jones Sent: Tuesday, February 21, 2017 1:24 PM To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?) Hi Jarom, We did originally plan to add support for complex types in HDF5-1.10. However, after the issue was examined closely, we decided to wait until we had a well-defined scope of what it means to add support for complex types within the library. The issue is still open but not assigned to be fixed in a specific release. -Barbara From: Hdf-forum [mailto:[email protected]] On Behalf Of Nelson, Jarom Sent: Thursday, February 09, 2017 10:32 AM To: [email protected]<mailto:[email protected]> Subject: [Hdf-forum] Complex numbers (1.10.x update?) At what point (if any?) will HDF5 support a complex floating point datatype (i.e. pair of real and imaginary floating point numbers) “out-of-the-box”? This is probably something that comes up frequently, but I haven’t found the current status of the question. Searching the archives and the documentation, it seems that a complex floating point data type was at one point planned for the 1.10 release, however, I don’t see anything about continuation of that plan in the documentation. Ref: these two threads from 2010 https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2010-December/004011.html https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2010-April/003049.html Perhaps the answer can be implied from this point in that thread: To be fair, because there's no one-size-fits-all complex number solution outside Fortran, a one-size-fits-all cross-language HDF5 helper method to define a complex type would be difficult to get right. Not technically difficult, just socially difficult. - Rhys Also, this seems like a likely topic for a FAQ. (There’s one there for boolean datatype) Jarom Nelson, LLNL _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected]<mailto:[email protected]> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected]<mailto:[email protected]> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5 -- ___________________________________________________________________________ Dr. Werner Benger Visualization Research Center for Computation & Technology at Louisiana State University (CCT/LSU) 2019 Digital Media Center, Baton Rouge, Louisiana 70803 Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
