<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10pt" ><div dir="ltr" >All,</div>
<div dir="ltr" > </div>
<div dir="ltr" >Looks to me that the demands of the workload will dictate how many files we should be cache, that is: maxStatCache + maxFilesToCache .</div>
<div dir="ltr" > </div>
<div dir="ltr" >The "mix" between maxStatCache and maxFilesToCache depends on how much memory can be made available. Accessing files from maxFilesToCache is more efficient, but stat cache entries use much less space.</div>
<div dir="ltr" > </div>
<div dir="ltr" >With the</div>
<div dir="ltr" > </div>
<div dir="ltr" ><font size="2" face="Default Monospace,Courier New,Courier,monospace" > ! maxFilesToCache 3000000<br> maxStatCache 10000</font><br> </div>
<div dir="ltr" >combination, the stat cache is not providing any significant help, since only 0.3% of the files that are cached can fit in the stat cache. If enough memory is available then maxStatCache could be increased to (say) 3000000, at a cost of 1.4GB. But maxFilesToCache = 3000000 uses up to 27GB. The next questions are then</div>
<div dir="ltr" > </div>
<div dir="ltr" >1) Can such memory become available on the node, given the pagepool size ?</div>
<div dir="ltr" > </div>
<div dir="ltr" >2) Does the workload require caching that many files?</div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<div dir="ltr" > Felipe</div>
<div dir="ltr" > </div>
<div dir="ltr" >----<br>Felipe Knop knop@us.ibm.com<br>GPFS Development and Security<br>IBM Systems<br>IBM Building 008<br>2455 South Rd, Poughkeepsie, NY 12601<br>(845) 433-9314 T/L 293-9314<br> </div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" data-history-expanded="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: "Frederick Stock" <stockf@us.ibm.com><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug-discuss@spectrumscale.org<br>Cc: gpfsug-discuss@spectrumscale.org<br>Subject: [EXTERNAL] Re: [gpfsug-discuss] maxStatCache and maxFilesToCache: Tip"gpfs_maxstatcache_low".<br>Date: Fri, Mar 13, 2020 10:01 AM<br>
<div dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:12pt" ><div dir="ltr" >As you have learned there is no simple formula for setting the maxStatToCache, or for that matter the maxFilesToCache, configuration values. Memory is certainly one consideration but another is directory listing operations. The information kept in the stat cache is sufficient for fulfilling directory listings. If your users are doing directory listings regularly then a larger stat cache could be helpful. </div>
<div dir="ltr" ><div dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" ><br><font size="2" face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" ><span style="font-size:1.143em;" >Fred<br>__________________________________________________<br>Fred Stock | IBM Pittsburgh Lab | 720-430-8821<br>stockf@us.ibm.com</span></font></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: Philipp Grau <phgrau@zedat.fu-berlin.de><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: [EXTERNAL] [gpfsug-discuss] maxStatCache and maxFilesToCache: Tip "gpfs_maxstatcache_low".<br>Date: Fri, Mar 13, 2020 8:49 AM<br>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >Hello,<br><br>we have a two node NSD cluster based on a DDN system. Currently we<br>run Spectrum Scale 5.0.4.1 in an HPC environment.<br><br>Mmhealth shows a tip stating "gpfs_maxstatcache_low". Our current settings are:<br><br># mmdiag --config | grep -i cache<br> ! maxFilesToCache 3000000<br> maxStatCache 10000<br><br>maxFilesToCache was tuned during installion and maxStatCache is the<br>according default value.<br><br>After discussing this issue on the german spectumscale meeting, I<br>understand that it is difficult to give a formula on howto calulate<br>this values.<br><br>But I learnt that a FilesToCache entry costs about 10 kbytes of memory<br>and a StatCache entry about 500 bytes. And typically maxStatCache<br>should (obviously) be greater than maxFilesToCache. There is a average<br>100 GB memory usage on our systems (with a total of 265 GB RAM).<br><br>So setting maxStatCache to at least 3000000 should be no problem. But<br>is that correct or to high/low?<br><br>Has anyone some hints or thoughts on this topic? Help is welcome.<br><br>Regards,<br><br>Philipp<br><br>--<br> Philipp Grau | Freie Universitaet Berlin <br> phgrau@ZEDAT.FU-Berlin.DE | Zentraleinrichtung fuer Datenverarbeitung<br> Tel: +49 (30) 838 56583 | Fabeckstr. 32 <br> Fax: +49 (30) 838 56721 | 14195 Berlin <br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </font><br> </div></blockquote>
<div dir="ltr" > </div></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </font></div></blockquote>
<div dir="ltr" > </div></div><BR>