> Instead, my recommendation would be to use an ext4 filesystem image to hold > the many small files (during create, if from a single client, or aggregated > after they are created). Later, this filesystem image could be mounted > read-only on multiple clients for access. Also, the whole image file can be > archived to tape efficiently (taking all small files with it, instead of > keeping a stub in Lustre for each file). > > The use of loopback mounting image files from Lustre already works today, but > needs userspace help to create and mount/unmount them. There was some > proposal "Client Container Image (CCI)" on how this could be integrated > directly into Lustre. Please see my LUG presentation for details (maybe 2019 > or so?)
Would squashfs files be a good alternative to this? The user space tools already exist. We have a couple of workflows here that create a lot of small files and we are using squashfs to aggregate those files for the purposes of archival and to reduce the metadata burden on our lustre filesystem. _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
