<font size=2 face="sans-serif">Hi Aaron , </font><br><br><font size=2 face="sans-serif">0,0009 s response time for your meta
data IO ... seems to be a very good/fast storage BE.. which is hard to
improve.. </font><br><font size=2 face="sans-serif">you can raise the parallelism a bit
for accessing metadata , but if this will help to improve your "workload"
is not assured </font><br><br><font size=2 face="sans-serif">The worker3threads parameter specifies
the number of threads to use for inode prefetch. Usually , I would suggest,
that you should not touch single parameters any longer. By the great improvements
of the last few releases.. GPFS can calculate / retrieve the right settings
semi-automatically...</font><br><font size=2 face="sans-serif">You only need to set simpler "workerThreads"
.. </font><br><br><font size=2 face="sans-serif">But in your case , you can see, if this
more specific value will help you out . </font><br><br><font size=2 face="sans-serif">depending on your blocksize and average
filesize .. you may see additional improvements when tuning nfsPrefetchStrategy
, which tells GPFS to consider all IOs wihtin <b><i>N</i></b> blockboundaries
as sequential and starts prefetch</font><br><br><font size=2 face="sans-serif">l.b.n.t. set ignoreprefetchLunCount
to yes .. (if not already done) . this helps GPFS to use all available
workerThreads </font><br><br><font size=2 face="sans-serif">cheers</font><br><font size=2 face="sans-serif">olaf</font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Aaron Knister <aaron.s.knister@nasa.gov></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif"><gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">09/11/2017 02:50 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
tuning parameters question</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><tt><font size=2>As an aside, my initial attempt was to use Ganesha
via CES but the <br>performance was significantly worse than CNFS for this workload. The <br>docs seem to suggest that CNFS performs better for metadata intensive <br>workloads which certainly seems to fit the bill here.<br><br>-Aaron<br><br>On 9/10/17 8:43 PM, Aaron Knister wrote:<br>> Hi All (but mostly Sven),<br>> <br>> I stumbled across this great gem:<br>> <br>> files.gpfsug.org/presentations/2014/UG10_GPFS_Performance_Session_v10.pdf<br>> <br>> and I'm wondering which, if any, of those tuning parameters are still
<br>> relevant with the 4.2.3 code. Specifically for a CNFS cluster. I'm
<br>> exporting a gpfs fs as an NFS root to 1k nodes. The boot storm is
<br>> particularly ugly and the storage doesn't appear to be bottlenecked.<br>> <br>> I see a lot of waiters like these:<br>> <br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 2881 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 26231 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 26146 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 18637 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 25013 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 27879 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 26553 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 25334 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> Waiting 0.0009 sec since 20:41:31, monitored, thread 25337 <br>> InodePrefetchWorkerThread: on ThCond 0x1800635A120 (LkObjCondvar),
<br>> reason 'waiting for LX lock'<br>> <br>> and I'm wondering if there's anything immediate one would suggest
to <br>> help with that.<br>> <br>> -Aaron<br>> <br><br>-- <br>Aaron Knister<br>NASA Center for Climate Simulation (Code 606.2)<br>Goddard Space Flight Center<br>(301) 286-2776<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br><br></font></tt><br><br><BR>