<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
My issues were never specifically the Unbalanced flag, originally I ran into ilm/policy issues with our metadata and illreplicated files. We were working to expand our Metadata storage at the time uner 4.2.3.xx and had added a number of SSDs to the array.
But AFTER a restripe files were still illreplicated. I then discovered that the policy engine had no way to tell me what files were and were not replicated. Just hoping that more ILM info had been added to the <span style="color:rgb(0,0,0); font-family:"Times New Roman",Times,serif; font-size:12pt">policy
engine since I ran into this, and seeing the unbalanced files jogged my memory to look at the docs again, where I didn't see anything in 5.x and ask the list as well. <span style="background-color:rgb(255,255,255); display:inline!important"> This isn't a
big deal, more curiousity on my part.</span></span></div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
I see you have some of the original thread attached, so perhaps take a glance and see if it makes sense? I </div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
Now if you REALLY want to step into a mine field, go find my thread on SKLM usage and GPFS/SS. Every single respondent to my question about filling up their SKLM logs with errors was a positive. And SKLM L2 and L3 support swears GPFS/SS is using SKLM wrong...</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
Ed Wahl</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
OSC</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div id="appendonsend"></div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Frederick Stock <stockf@us.ibm.com><br>
<b>Sent:</b> Tuesday, February 2, 2021 1:09 PM<br>
<b>To:</b> gpfsug-discuss@spectrumscale.org <gpfsug-discuss@spectrumscale.org><br>
<b>Cc:</b> gpfsug-discuss@spectrumscale.org <gpfsug-discuss@spectrumscale.org><br>
<b>Subject:</b> Re: [gpfsug-discuss] policy ilm features?</font>
<div> </div>
</div>
<div>
<div class="x_socmaildefaultfont" dir="ltr" style="font-family:Arial,Helvetica,sans-serif; font-size:12pt">
<div dir="ltr">Hello Ed. Jordan contacted me about the question you are posing so I am responding to your message. Could you please provide clarification as to why the existence of the unbalanced flag is of a concern, or why you would want to know all the
files that have this flag set? The flag would be cleared once the file was rebalanced either through normal access or through the execution of the mmrestripefs/mmrestripefile commands.</div>
<div dir="ltr">
<div class="x_socmaildefaultfont" dir="ltr" style="font-family:Arial,Helvetica,sans-serif; font-size:10.5pt">
<div dir="ltr"><br>
<font size="2" face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif"><span style="font-size:1.143em">Fred<br>
__________________________________________________<br>
Fred Stock | IBM Pittsburgh Lab | 720-430-8821<br>
stockf@us.ibm.com</span></font></div>
</div>
</div>
<div dir="ltr"> </div>
<div dir="ltr"> </div>
<blockquote dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px">
----- Original message -----<br>
From: "Wahl, Edward" <ewahl@osc.edu><br>
Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
Cc:<br>
Subject: [EXTERNAL] Re: [gpfsug-discuss] policy ilm features?<br>
Date: Tue, Feb 2, 2021 11:52 AM<br>
<br>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
Replying to a 3 year old message I sent, hoping that in the last couple of years that Scale has added some ILM extensions into the policy engine that I have missed, or somehow didn't notice?</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
Just ran into a file with an 'unbalanced' flag and I REALLY don't want to have to mmlsattr everything. AGAIN. /facepalm </div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
<span style="color:rgb(0,0,0); font-family:"Times New Roman",Times,serif; font-size:12pt">IBM? Bueller? Bueller? </span></div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
When everyone answers: "No", I'm guessing this needs to be a request for improvement/enhancement? </div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
Ed Wahl</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
Ohio Supercomputer Center</div>
<div style="font-family:"Times New Roman",Times,serif; font-size:12pt; color:rgb(0,0,0)">
</div>
<div id="x_appendonsend"> </div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div dir="ltr" id="x_divRplyFwdMsg"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> on behalf of Edward Wahl <ewahl@osc.edu><br>
<b>Sent:</b> Friday, February 2, 2018 3:23 PM<br>
<b>To:</b> John Hearns <john.hearns@asml.com><br>
<b>Cc:</b> gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Subject:</b> Re: [gpfsug-discuss] policy ilm features?</font>
<div> </div>
</div>
<div>
<div><br>
<font size="2"><span style="font-size:11pt">Thanks John, this was the path I was HOPING to go down as I do similar things<br>
already, but there appears to be no extended attribute in ILM for what I want.<br>
Data block replication flag exists in the ILM, but not MetaData, or balance.<br>
<br>
Yet these states ARE reported by mmlsattr, so there must be a flag somewhere.<br>
<br>
<br>
bad MD replication & balance example:<br>
<br>
mmlsattr -L /fs/scratch/sysp/ed/180days.pol<br>
file name: /fs/scratch/sysp/ed/180days.pol<br>
metadata replication: 1 max 2<br>
data replication: 1 max 2<br>
flags: illreplicated,unbalanced<br>
Encrypted: yes<br>
<br>
File next to it for comparison. note proper MD replication and balance.<br>
<br>
mmlsattr -L /fs/scratch/sysp/ed/120days.pol<br>
file name: /fs/scratch/sysp/ed/120days.pol<br>
metadata replication: 2 max 2<br>
data replication: 1 max 2<br>
flags: <br>
Encrypted: yes<br>
<br>
misc_attributes flags from a policy run showing no difference in status:<br>
FJAEu -- /fs/scratch/sysp/ed/180days.pol<br>
FJAEu -- /fs/scratch/sysp/ed/120days.pol<br>
<br>
<br>
File system has MD replication enabled, but not Data, so ALL files show "J" ilm<br>
flag<br>
<br>
mmlsfs scratch -m<br>
flag value description<br>
------------------- ------------------------ -----------------------------------<br>
-m 2 Default number of metadata replicas<br>
mmlsfs scratch -r<br>
flag value description<br>
------------------- ------------------------ -----------------------------------<br>
-r 1 Default number of data replicas<br>
<br>
<br>
I poked around a little trying to find out if perhaps using GetXattr would<br>
work and show me what I wanted, it does not. All I sem to be able to get is the<br>
File Encryption Key.<br>
<br>
<br>
I was hoping perhaps someone had found a cheaper way for this to work rather<br>
than hundreds of millions of 'mmlsattr' execs. :-(<br>
<br>
On the plus side, I've only run across a few of these and all appear to be<br>
from before we did the MD replication and re-striping. On the minus, I have NO<br>
idea where they are, and they appears to be on both of our filesystems. So<br>
several hundred million files to check.<br>
<br>
Ed<br>
<br>
<br>
On Mon, 22 Jan 2018 08:29:42 +0000<br>
John Hearns <john.hearns@asml.com> wrote:<br>
<br>
> Ed,<br>
> This is not a perfect answer. You need to look at policies for this. I have<br>
> been doing something similar recently.<br>
><br>
> Something like:<br>
><br>
> RULE 'list_file' EXTERNAL LIST 'all-files' EXEC<br>
> '/var/mmfs/etc/mmpolicyExec-list' RULE 'listall' list 'all-files'<br>
> SHOW( varchar(kb_allocated) || ' ' || varchar(file_size) || ' ' ||<br>
> varchar(misc_attributes) || ' ' || name || ' ' || fileset_name ) WHERE<br>
> REGEX(misc_attributes,'[J]')<br>
><br>
><br>
> So this policy shows the kbytes allocates, file size, the miscellaneous<br>
> attributes, name and fileset name For all files with miscellaneous<br>
> attributes of 'J' which means 'Some data blocks might be ill replicated'<br>
><br>
><br>
><br>
><br>
> -----Original Message-----<br>
> From: gpfsug-discuss-bounces@spectrumscale.org<br>
> [<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" target="_blank">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>] On Behalf Of Edward Wahl<br>
> Sent: Friday, January 19, 2018 10:38 PM To: gpfsug-discuss@spectrumscale.org<br>
> Subject: [gpfsug-discuss] policy ilm features?<br>
><br>
><br>
> This one has been on my list a long time so I figured I'd ask here first<br>
> before I open an apar or request an enhancement (most likely).<br>
><br>
> Is there a way using the policy engine to determine the following?<br>
><br>
> -metadata replication total/current<br>
> -unbalanced file<br>
><br>
> Looking to catch things like this that stand out on my filesystem without<br>
> having to run several hundred million 'mmlsattr's.<br>
><br>
> metadata replication: 1 max 2<br>
> flags: unbalanced<br>
><br>
> Ed<br>
><br>
><br>
><br>
> --<br>
><br>
> Ed Wahl<br>
> Ohio Supercomputer Center<br>
> 614-292-9302<br>
> _______________________________________________<br>
> gpfsug-discuss mailing list<br>
> gpfsug-discuss at spectrumscale.org<br>
> <a href="https://urldefense.com/v3/__https://emea01.safelinks.protection.outlook.com/?url=http*3A*2F*2Fgpfsug.org*2Fmailman*2Flistinfo*2Fgpfsug-discuss&data=01*7C01*7Cjohn.hearns*40asml.com*7C056e34c5a8df4d8f10fd08d55f91e73c*7Caf73baa8f5944eb2a39d93e96cad61fc*7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM*2B8pNrZElImSZeZbit8*3D&reserved=0__;JSUlJSUlJSUlJSUlJSU!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foRAVfkHJ$" target="_blank">
https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7C056e34c5a8df4d8f10fd08d55f91e73c%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=dnt7vV4TCd68l7fSJnY35eyNM%2B8pNrZElImSZeZbit8%3D&reserved=0</a><br>
> -- The information contained in this communication and any attachments is<br>
> confidential and may be privileged, and is for the sole use of the intended<br>
> recipient(s). Any unauthorized review, use, disclosure or distribution is<br>
> prohibited. Unless explicitly stated otherwise in the body of this<br>
> communication or the attachment thereto (if any), the information is provided<br>
> on an AS-IS basis without any express or implied warranties or liabilities.<br>
> To the extent you are relying on this information, you are doing so at your<br>
> own risk. If you are not the intended recipient, please notify the sender<br>
> immediately by replying to this message and destroy all copies of this<br>
> message and any attachments. Neither the sender nor the company/group of<br>
> companies he or she represents shall be liable for the proper and complete<br>
> transmission of the information contained in this communication, or for any<br>
> delay in its receipt. _______________________________________________<br>
> gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<br>
> <a href="https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foWJut1Kk$" target="_blank">
http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
<br>
<br>
<br>
--<br>
<br>
Ed Wahl<br>
Ohio Supercomputer Center<br>
614-292-9302<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foWJut1Kk$" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a></span></font></div>
</div>
<div><font size="2" face="Default Monospace,Courier New,Courier,monospace">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at spectrumscale.org<br>
<a href="https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!KGKeukY!m3C_znJaMf7LXsm5R_eRvzdKNoF5AXQHQ2BjkX26CDxhDCwjsl0foWJut1Kk$" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a> </font></div>
</blockquote>
<div dir="ltr"> </div>
</div>
<br>
</div>
</body>
</html>