The most likely candidate for effectively using Optane/NVRAM would be via Persistent Client Cache (PCC), which allows client-local storage to be part of the Lustre namespace. Files can be cached on a local NVRAM device (managed by a local filesystem like ext4, or possibly something more experimental like NOVA for better performance) and then migrated into the cache.
Once the file is in PCC, it can be accessed via the local filesystem operations, including DAX, for very low-latency operations. See the presentation from LAD'19 for details: https://www.eofs.eu/_media/events/lad19/07_li_xi-nvram_pcc.pdf It should be noted that in Lustre 2.13, files in PCC are NOT resident in the main filesystem, so if the client node goes offline then the files will not be accessible until the client node is restarted. For some workloads this is OK (e.g. files being generated locally with high IOPS that are occasionally needed on other clients), but not for others. We will be improving PCC to use FLR to mirror a copy into the client cache and still keep a copy in the main filesystem, but that is not available yet. Cheers, Andreas On Jan 13, 2020, at 10:03, Dave Holland <d...@sanger.ac.uk<mailto:d...@sanger.ac.uk>> wrote: I haven't been to LUG or LAD recently, so I'm a bit out of the loop, but how much use is Optane finding in the Lustre world? The main obstacle I see is that it's server-local, so building a resilient/failover-capable system isn't straightforward. Thanks for any observations. Cheers, Dave -- ** Dave Holland ** Systems Support -- Informatics Systems Group ** ** 01223 496923 ** Wellcome Sanger Institute, Hinxton, UK ** Cheers, Andreas -- Andreas Dilger Principal Lustre Architect Whamcloud
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org