Hi Please check mmdf output to see that MetaData disks are not full, or you have i-nodes issue. In case you have Independent File-Sets , please run : mmlsfileset <fs> -L -i to get the status of each fileset inodes. Regards Yaron Daniel 94 Em Ha'Moshavot Rd Server, Storage and Data Services - Team Leader Petach Tiqva, 49527 Global Technology Services Israel Phone: +972-3-916-5672 Fax: +972-3-916-5672 Mobile: +972-52-8395593 e-mail: yard at il.ibm.com IBM Israel From: John Hanks <griznog at gmail.com> To: gpfsug <gpfsug-discuss at spectrumscale.org> Date: 11/02/2017 12:54 AM Subject: [gpfsug-discuss] mmrestripefs "No space left on device" Sent by: gpfsug-discuss-bounces at spectrumscale.org Hi all, I'm trying to do a restripe after setting some nsds to metadataOnly and I keep running into this error: Scanning user file metadata ... 0.01 % complete on Wed Nov 1 15:36:01 2017 ( 40960 inodes with total 531689 MB data processed) Error processing user file metadata. Check file '/var/mmfs/tmp/gsfs0.pit.interestingInodes.12888779708' on scg-gs0 for inodes with broken disk addresses or failures. mmrestripefs: Command failed. Examine previous error messages to determine cause. The file it points to says: This inode list was generated in the Parallel Inode Traverse on Wed Nov 1 15:36:06 2017 INODE_NUMBER DUMMY_INFO SNAPSHOT_ID ISGLOBAL_SNAPSHOT INDEPENDENT_FSETID MEMO(INODE_FLAGS FILE_TYPE [ERROR]) 53504 0:0 0 1 0 illreplicated REGULAR_FILE RESERVED Error: 28 No space left on device /var on the node I am running this on has > 128 GB free, all the NSDs have plenty of free space, the filesystem being restriped has plenty of free space and if I watch the node while running this no filesystem on it even starts to get full. Could someone tell me where mmrestripefs is attempting to write and/or how to point it at a different location? Thanks, jbh_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=Bn1XE9uK2a9CZQ8qKnJE3Q&m=WTfQpWOsmp-BdHZ0PWDbaYsxq-5Q1ZH26IyfrBRe3_c&s=SJg8NrUXWEpaxDhqECkwkbJ71jtxjLZz5jX7FxmYMBk&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171102/f872df98/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 1851 bytes Desc: not available URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171102/f872df98/attachment.gif> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 4376 bytes Desc: not available URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171102/f872df98/attachment-0001.gif> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 5093 bytes Desc: not available URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20171102/f872df98/attachment-0002.gif>
