On 9/16/25 23:15, Praveen C wrote:
**
This limit of 4GB seems to be a hard limit set within hdf5
https://support.hdfgroup.org/documentation/hdf5/latest/
group___d_c_p_l.html#title22 <https://nam10.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fsupport.hdfgroup.org%2Fdocumentation%2Fhdf5%2Flatest%2Fgroup___d_c_p_l.html%23title22&data=05%7C02%7CWolfgang.Bangerth%40colostate.edu%7C36f38e15fd704b1ac1e008ddf5a94778%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C638936829597879160%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=lpQ49VlJ7amraXkdKiU6CqPQeckdP9zuNBNLSNf311k%3D&reserved=0>
In this case, how can we save larger files ? The problem we see is with a data
of the order of 512^3 grid with 5 unknowns at each node.
Should we switch to some other format ?
If you want to output stuff on regular NxNxN grids, you really only have the
choice between HDF5 and NetCDF. If HDF5 works for you, it seems easier to fix
the underlying issue than to switch to something else.
I must admit that I don't know the HDF5 interfaces. In fact, I don't know
whether there is anyone left who does. But it seems like a reasonable choice
to change the deal.II code to just limit the chunk size to a maximum of 4 GB.
In the place where that is currently happening, lines 8314 and 8337, you
define the chunk size via a 2d array with sizes
cell_ds_dim[0] = global_node_cell_count[1];
cell_ds_dim[1] = patches[0].reference_cell.n_vertices();
If the product of these two numbers happen to exceed 4B elements, then call
H5Pset_chunk with a different array that has one of the two dimensions reduced
to a point where the product is no longer bigger than 4B. I *think*, but it
might require some reading, that the chunk size is only relevant for how the
data is stored, but not what is being stored and how it is to be interpreted
by any reader. As a consequence, you might get away with just setting a
smaller cell_ds_dim in these calls, and everything else might still work.
Want to try this out?
Cheers
W.
--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see
https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/dealii/b3fff020-3821-45b3-b769-8175bc54de73%40colostate.edu.
