Hiin the past we had issues with the mmdf heap memory. Due to special workload it increased and took GB of memory, but after usage it was not freed again. we had long discussions with IBM about it and it ends up in a Development User Story (261213) which was realized in 5.1.2:
--- In this story, the InodeAllocSegment object will be allocated when accessed. For commands that iterates all InodeAllocSegment, we will release the object immediately after use.An undocumented configuration "!maxIAllocSegmentsToCache" is provided to control
the upper limit of the count of InodeAllocSegment objects. When the count approaches the limit, a pre stealing thread will be started to steal and release some InodeAllocSegment objects. Its default value is 1000,000. ---
since than we are fine so far. But this was on plain GPFS clients, no CES node where the service like NFS comes into play.
You can monitor the heap memory usage by using "mmdiag --memory"@IBM colleagues: If there is something wrong in my explanation please correct me.
Stephan On 9/6/23 20:55, Christoph Martin wrote:
Hi all,on a three node GPFS cluster with CES enabled and AFM-DR mirroring to a second cluster we see frequent OOM killer events due to a constantly growing mmfsd.The machines have 256G memory. The pagepool is configured to 16G. The GPFS version is 5.1.6-1.After a restart mmfsd rapidly grows to about 100G usage and grows over some days up to 250G virtual and 220G physical memory usage. OOMkiller tries kill process like pmcollector or others and sometime kills mmfsd.Does anybody see a similar behavior? Any guess what could help with this problem? Regards Christoph Martin _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
-- Stephan Graf Juelich Supercomputing Centre Phone: +49-2461-61-6578 Fax: +49-2461-61-6656 E-mail: st.g...@fz-juelich.de WWW: http://www.fz-juelich.de/jsc/ --------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht, Prof. Dr. Frauke Melchior --------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
