<font size=2 face="sans-serif">Sorry for my fast ( and not well thought)
answer, before. You obviously are correct, there is no relation between
the setting of maxFilesToCache, and the </font><br><br><font size=3>fileCacheMem 38359956 k = 11718554
* 3352 bytes (inode size 512 + 2840)</font><br><br><font size=2 face="sans-serif">usage. it is rather a statement of how
many metadata may fit in the remaining structures outside the pagepool.
this value does not change at all, when you modify your mFtC setting. </font><br><br><font size=2 face="sans-serif">There is a really good presentation
by Tomer Perry on the User Group meetings, referring about memory footprint
of GPFS under various conditions.</font><br><br><font size=2 face="sans-serif">In your case, you may very well hit
the CES nodes memleak you just pointed out. </font><br><br><font size=2 face="sans-serif">Sorry for my hasty reply earlier ;)
</font><br><br><font size=2 face="sans-serif">Achim</font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Stijn De Weirdt <stijn.deweirdt@ugent.be></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug-discuss@spectrumscale.org</font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">09/05/2019 16:48</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
advanced filecache math</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><tt><font size=2>seems like we are suffering from<br></font></tt><a href="http://www-01.ibm.com/support/docview.wss?uid=isg1IJ12737"><tt><font size=2>http://www-01.ibm.com/support/docview.wss?uid=isg1IJ12737</font></tt></a><tt><font size=2><br><br>as these are ces nodes, we susepcted something wrong the caches, but it<br>looks like a memleak instead.<br><br>sorry for the noise (as usual you find the solution right after sending<br>the mail ;)<br><br>stijn<br><br>On 5/9/19 4:38 PM, Stijn De Weirdt wrote:<br>> hi achim,<br>> <br>>> you just misinterpreted the term fileCacheLimit.<br>>> This is not in byte, but specifies the maxFilesToCache setting
:<br>> i understand that, but how does the fileCacheLimit relate to the<br>> fileCacheMem number?<br>> <br>> <br>> <br>> (we have a 32GB pagepool, and mmfsd is using 80GB RES (101 VIRT),
so we<br>> are looking for large numbers that might explain wtf is going on<br>> (pardon my french ;)<br>> <br>> stijn<br>> <br>>><br>>> UMALLOC limits:<br>>> bufferDescLimit 40000
desired 40000<br>>> fileCacheLimit 4000 desired
4000 <=== mFtC<br>>> statCacheLimit 1000 desired
1000 <=== mSC<br>>> diskAddrBuffLimit 200
desired 200<br>>><br>>> # mmfsadm dump config | grep -E "maxFilesToCache|maxStatCache"<br>>> maxFilesToCache 4000<br>>> maxStatCache 1000<br>>><br>>> Mit freundlichen Gren / Kind regards<br>>><br>>> *Achim Rehor*<br>>><br>>> --------------------------------------------------------------------------------<br>>> Software Technical Support Specialist AIX/ Emea HPC Support
<br>>> IBM Certified Advanced Technical Expert - Power Systems with AIX<br>>> TSCC Software Service, Dept. 7922<br>>> Global Technology Services<br>>> --------------------------------------------------------------------------------<br>>> Phone:
+49-7034-274-7862
IBM Deutschland<br>>> E-Mail:
Achim.Rehor@de.ibm.com
Am Weiher 24<br>>>
65451 Kelsterbach<br>>>
Germany<br>>>
<br>>> --------------------------------------------------------------------------------<br>>> IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Martin
Jetter<br>>> Geschftsfhrung: Martin Hartmann (Vorsitzender), Norbert Janzen,
Stefan Lutz, <br>>> Nicole Reimer, Dr. Klaus Seifert, Wolfgang Wendt<br>>> Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht
Stuttgart, HRB <br>>> 14562 WEEE-Reg.-Nr. DE 99369940<br>>><br>>><br>>><br>>><br>>><br>>><br>>> From: Stijn De Weirdt <stijn.deweirdt@ugent.be><br>>> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>>> Date: 09/05/2019 16:21<br>>> Subject: [gpfsug-discuss] advanced filecache math<br>>> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>>><br>>> --------------------------------------------------------------------------------<br>>><br>>><br>>><br>>> hi all,<br>>><br>>> we are looking into some memory issues with gpfs 5.0.2.2, and
found<br>>> following in mmfsadm dump fs:<br>>><br>>> > fileCacheLimit 1000000
desired 1000000<br>>> ...<br>>> > fileCacheMem 38359956 k
= 11718554 * 3352 bytes (inode size 512 + 2840)<br>>><br>>> the limit is 1M (we configured that), however, the fileCacheMem
mentions<br>>> 11.7M?<br>>><br>>> this is also reported right after a mmshutdown/startup.<br>>><br>>> how do these 2 relate (again?)?<br>>><br>>> mnay thanks,<br>>><br>>> stijn<br>>> _______________________________________________<br>>> gpfsug-discuss mailing list<br>>> gpfsug-discuss at spectrumscale.org<br>>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>>><br>>><br>>><br>>><br>>><br>>> _______________________________________________<br>>> gpfsug-discuss mailing list<br>>> gpfsug-discuss at spectrumscale.org<br>>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>>><br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>> <br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br><br></font></tt><br><br><BR>