<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10pt" ><div dir="ltr" >Venkat,</div>
<div dir="ltr" >for awareness and response.</div>
<div dir="ltr" > </div>
<div dir="ltr" >Thanks,<br>Lyle</div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: "Billich Heinrich Rainer (ID SD)" <heinrich.billich@id.ethz.ch><br>Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>Cc:<br>Subject: [EXTERNAL] [gpfsug-discuss] AFM Recovery of SW cache does a full scan of home - is this to be expected?<br>Date: Wed, Jan 8, 2020 12:04 PM<br> <br><!--Notes ACF
<meta http-equiv="Content-Type" content="text/html; charset=utf8" >-->
<div><p style="margin: 0px;" >Hello,<o:p></o:p></p>
<p style="margin: 0px;" ><o:p> </o:p></p>
<p style="margin: 0px;" ><o:p> </o:p></p>
<p style="margin: 0px;" ><span lang="EN-US" >still new to AFM, so some basic question on how Recovery works for a SW cache:<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >we have an AFM SW cache in recovery mode \u2013 recovery first did run policies on the cache cluster, but now I see a \u2018tcpcachescan\u2019 process on cache slowly scanning home via nfs. Single host, single process, no parallelism as far as I can see, but I may be wrong. This scan of home on a cache afmgateway takes very long while further updates on cache queue up. Home has about 100M files. After 8hours I see about 70M entries in the file /var/mmfs/afm/\u2026/recovery/homelist, i.e. we get about 2500 lines/s. (We may have very many changes on cache due to some recursive ACL operations, but I\u2019m not sure.)<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >So I expect that 12hours pass to buildup filelists before recovery starts to update home. I see some risk: In this time new changes pile up on cache. Memory may become an issue? Cache may fill up and we can\u2019t evict?<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >I wonder<o:p></o:p></span></p>
<ul style="margin-top:0cm" type="disc" > <li style="margin: 0px 0px 0px 36pt;" ><span lang="EN-US" >Is this to be expected and normal behavior? What to do about it?<o:p></o:p></span></li> <li style="margin: 0px 0px 0px 36pt;" ><span lang="EN-US" >Will every reboot of a gateway node trigger a recovery of all afm filesets and a full scan of home? This would make normal rolling updates very unpractical, or is there some better way?<o:p></o:p></span></li></ul>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >Home is a gpfs cluster, hence we easily could produce the needed filelist on home with a policyscan in a few minutes.<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >Thank you, I will welcome and clarification, advice or comments.<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >Kind regards,<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >Heiner<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" >.<o:p></o:p></span></p>
<p style="margin: 0px;" ><span lang="EN-US" ><o:p> </o:p></span></p>
<p style="margin: 0px;" >-- <o:p></o:p></p>
<p style="margin: 0px;" ><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#18376A" lang="DE" >=======================</span><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black" ><o:p></o:p></span></p>
<p style="margin: 0px;" ><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#18376A" lang="DE" >Heinrich Billich</span><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black" ><o:p></o:p></span></p>
<p style="margin: 0px;" ><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#18376A" lang="DE" >ETH Zürich</span><span style="font-size:12.0pt;font-family:"Times New Roman",serif;color:black" ><o:p></o:p></span></p>
<p style="margin: 0px;" ><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#18376A" >Informatikdienste</span><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black" ><o:p></o:p></span></p>
<p style="margin: 0px;" ><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#18376A" >Tel.: +41 44 632 72 56</span><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black" ><o:p></o:p></span></p>
<p style="margin: 0px;" ><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#18376A" >heinrich.billich@id.ethz.ch</span><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black" ><o:p></o:p></span></p>
<p style="margin: 0px;" ><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#18376A" >========================</span><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black" ><o:p></o:p></span></p>
<p style="margin: 0px;" ><o:p> </o:p></p>
<p style="margin: 0px;" ><o:p> </o:p></p>
<p style="margin: 0px;" ><o:p> </o:p></p></div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace" >_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </font></div></blockquote>
<div dir="ltr" > </div></div><BR>