Hi Mark,
Well, most of the languages that I am familiar with represent complex as
cartesian pairs, so I supposed this representation implicitly. Having said
that, and after further investigation, there seems two main groups of
conventions for representing complex numbers in HDF5:
- Python libraries (h5py, PyTables), that adopted ('r', 'i') compound types.
Also, there are some readers for MatLab that adopt this convention too:
http://www.mathworks.com/matlabcentral/fileexchange/23934-hdf5-data-to-struct-reader
- Octave and some C++ libraries (Armadillo) adopted ('real', 'imag') compound
types (for compatibility, PyTables also supports this convention when reading).
As the Zen of Python saying goes "practicality beats purity", I would say that
adopting one of the above conventions for writing, and supporting both for
reading would provide maximum compatibility with existing files out there. But
I agree that an official blessing would be great.
Francesc
________________________________
From: Hdf-forum <[email protected]> on behalf of Miller,
Mark C. <[email protected]>
Sent: Wednesday, February 22, 2017 6:25 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?)
This has been an interesting thread to follow. Thanks, Jarom, for starting it ;)
Having said that, in reading here and in the various links contributors have
referenced, I didn't necessarily see anything that described the 'issues' with
supporting complex types in HDF5 in a first-class (e.g. built in to the
library) sort of way. And, now, I am just really curious what those issues are
and if/how they are different from any other primitive type HDF5 supports?
I mean, I get that complex numbers represent a 'pair' of scalar values (which
in cartesian coords are typically represented as real/imaginary and in polor
coords, magnitude/phase) and I get that HDF5's underlying primitive type model
is for the most part like standard floating point and integer computer
representations including such things as offset, base, bias, mantissa,
exponent, etc., but is there anything substantially more comlicated about
handling them as first class type in the library than, as Francesc described
below? Are there Fortran variants or language extensions that have something
other than that basic modality of a pair of either 32 bit or 64 bit floating
point numbers? Are there any Fortrans that use magnitude/phase (e.g. polar
coords) instead of real/imaginary (cartesian coords).
Question for Francesc...do you define *both* polar and cartesian forms for
complex numbers and if so, do you define a conversion method to go between
them? If not, would anything like that ever be useful if it existed?
Seems like someone could define a 'complex types' support library extension on
github, perhaps refactored from Francesc' code that might have the ability to
become the defacto HDF5 standard for complex data in HDF5 files.
Finally, what about quarternions ;}? I am not kidding, just curious. I think
those might be relevant in astro-physical contexts wouldn't they?
Mark
"Hdf-forum on behalf of Nelson, Jarom" wrote:
Thanks for all the responses. Very helpful.
From: Hdf-forum [mailto:[email protected]] On Behalf Of
Filipe Maia
Sent: Wednesday, February 22, 2017 6:49 AM
To: HDF Users Discussion List <[email protected]>
Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?)
I think the main issue is the "almost exactly".
I'm also using h5py's convention, but having an HDF5 "officially endorsed" way
of representing complex numbers would greatly help with compatibility across
different user groups, instead of having *almost* compatible representations.
And I hope the endorsed way matches the h5py convention, but any convention is
better than none.
Cheers,
Filipe
On 22 February 2017 at 15:34, Barbara Jones
<[email protected]<mailto:[email protected]>> wrote:
Hi Jarom,
In case it might help, there are C and Fortran code examples in the User’s
Guide that do almost exactly what Francesc mentioned.
See Section 6.5.2.2.1 in the “HDF5 Datatypes” chapter of the HDF5 User’s Guide.
The link to section 6.5.2 is:
https://support.hdfgroup.org/HDF5/doc/UG/HDF5_Users_Guide-Responsive%20HTML5/index.html#t=HDF5_Users_Guide%2FDatatypes%2FHDF5_Datatypes.htm%23TOC_6_5_2_Definition_ofbc-14&rhtocid=6.3.0_2
Scroll down to 6.5.2.2.1, “Compound Datatypes”. Scroll down a little further
and you will see “Code Example 6-9” for C and “Code Example 6-10” for Fortran.
-Barbara
From: Hdf-forum
[mailto:[email protected]<mailto:[email protected]>]
On Behalf Of Francesc Altet
Sent: Wednesday, February 22, 2017 2:02 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?)
Hi Jarom,
For what is worth, in PyTables and h5py (and possibly in others libraries too)
we used the convention of declaring the complex type by using a compound type
as follows:
"""
The H5T_COMPOUND type class contains two members. Both members must have the
H5T_FLOAT atomic datatype class. The name of the first member should be "r" and
represents the real part. The name of the second member should be "i" and
represents the imaginary part. The precision property of both of the H5T_FLOAT
members must be either 32 significant bits (e.g. H5T_NATIVE_FLOAT) or 64
significant bits (e.g. H5T_NATIVE_DOUBLE). They represent Complex32 and
Complex64 types respectively.
"""
Perhaps you may want to use this convention until a more formal one is
implemented.
Francesc Alted
________________________________
From: Hdf-forum
<[email protected]<mailto:[email protected]>>
on behalf of Nelson, Jarom <[email protected]<mailto:[email protected]>>
Sent: Wednesday, February 22, 2017 2:36:23 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?)
Can we get a FAQ entry on the topic? Bonus points for an example user
implementation.
Thanks for your answer and for a great library!
Jarom
From: Hdf-forum [mailto:[email protected]] On Behalf Of
Barbara Jones
Sent: Tuesday, February 21, 2017 1:24 PM
To: HDF Users Discussion List
<[email protected]<mailto:[email protected]>>
Subject: Re: [Hdf-forum] Complex numbers (1.10.x update?)
Hi Jarom,
We did originally plan to add support for complex types in HDF5-1.10. However,
after the issue was examined closely, we decided to wait until we had a
well-defined scope of
what it means to add support for complex types within the library.
The issue is still open but not assigned to be fixed in a specific release.
-Barbara
From: Hdf-forum [mailto:[email protected]] On Behalf Of
Nelson, Jarom
Sent: Thursday, February 09, 2017 10:32 AM
To: [email protected]<mailto:[email protected]>
Subject: [Hdf-forum] Complex numbers (1.10.x update?)
At what point (if any?) will HDF5 support a complex floating point datatype
(i.e. pair of real and imaginary floating point numbers) “out-of-the-box”?
This is probably something that comes up frequently, but I haven’t found the
current status of the question.
Searching the archives and the documentation, it seems that a complex floating
point data type was at one point planned for the 1.10 release, however, I don’t
see anything about continuation of that plan in the documentation.
Ref: these two threads from 2010
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2010-December/004011.html
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2010-April/003049.html
Perhaps the answer can be implied from this point in that thread:
To be fair, because there's no one-size-fits-all complex number
solution outside Fortran, a one-size-fits-all cross-language HDF5
helper method to define a complex type would be difficult to get
right. Not technically difficult, just socially difficult.
- Rhys
Also, this seems like a likely topic for a FAQ. (There’s one there for boolean
datatype)
Jarom Nelson, LLNL
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]<mailto:[email protected]>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
