<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=en-CH link=blue vlink=purple style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Hello,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I need to update/replace our metadata disks but I want to keep old and new in parallel for a while before I remove the old storage: as active/active pair with double copies. This will allow an immediate fall-back if we ever need.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Maybe you want to comment on this procedure \u2013 I probably found it some time ago on this mailing list, sorry if I don\u2019t remember the author.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Some concerns I have<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>I\u2019m a bit worried what happens when we remove the vdisks with the second copy in the last step \u2013 will be get millions of error messages? Is the sequence of commands right? <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>I need to change the failure groups of some vdisks while in use \u2013 I wonder if this poses some risk? As I understand this will change the order of block allocation among the nsds (not the allocation map I guess)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>This will double metadata write-io, the systems should be able to handle it. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>We will get better metadata read-io during the transition than what we\u2019ll finally get.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Start \u2013 \u2018old\u2019 ESS/GNR only<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> -m 1 (single metadata copy), <o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> -M 2<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> -K whenpossible<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> Metadata vdisks in failure groups 1 and 2<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Preparation<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Use\u2019 mmvdisk filesystems\u2019 and move all metadata vdisk-sets to a single failuregroup (this has to be done in operation, under load)<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Set -m 2, while we still have one failure group only. -K whenpossible will keep us running<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Add the new vdisk-set with a second failure group on \u2018new\u2019 ESS/GNR systems<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Now all new inodes have two copies, one on old and one on new<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Create copies on \u2018old\u2019 and \u2018new\u2019<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> mmrestripefs -R with \u2013qos maintenance and -N helper-nodes to minimize the load. This may create some locks on the filesystem/cluster and interfere with backup and snapshots??<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Maybe better: use a policy \u2018replicate\u2019 rule to replicate all files, I can run this in small chunks and run mmrestripefs just once to crosscheck.<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Observe for some days, handle remaining filesystems in the same way<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Finally<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Suspend \u2018old\u2019 disks<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Run mmrestripefs -m<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Remove \u2018old\u2019 vdisk sets with mmvidisk \u2013 this will run another mmdeldisk<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Change to -m 1<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Run fix replication setting mmrestripefs -R (if needed?)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Thank you for reading and for any comments or suggestions.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Heiner<o:p></o:p></span></p></div></body></html>