<html><body><p><font size="2">On the subject of using zvols for software Raid/ replication, can ask as a quick poll, how many people are doing this?<br></font><font size="2"><br></font><font size="2">And any feedback on stability, tuning and performance?<br></font><font size="2"><br></font><font size="2">Daniel<br></font><font size="2">IBM Technical Presales<br></font><font size="2"><br></font><font size="2">> On 10 Mar 2017, at 22:44, Aaron Knister <aaron.s.knister@nasa.gov> wrote:<br></font><font size="2">> <br></font><font size="2">> Those look like zvol's. Out of curiosity have you set sync=always on the <br></font><font size="2">> filesystem root or zvols themselves? It's my understanding that without <br></font><font size="2">> that you risk data loss since GPFS won't ever cause a sync to be issued <br></font><font size="2">> to the zvol for zfs to flush acknowledged but uncommitted writes.<br></font><font size="2">> <br></font><font size="2">> -Aaron<br></font><font size="2">> <br></font><font size="2">>> On 3/10/17 4:36 PM, Sanchez, Paul wrote:<br></font><font size="2">>> See:<br></font><font size="2">>> <a href="https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_nsddevices.htm">https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_nsddevices.htm</a><br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> GPFS has a limited set of device search specs it uses to find connected<br></font><font size="2">>> NSDs. When using exotic devices, you need to whitelist the devices<br></font><font size="2">>> yourself using the user exit script at /var/mmfs/etc/nsddevices.<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> *From:*gpfsug-discuss-bounces@spectrumscale.org<br></font><font size="2">>> [<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">mailto:gpfsug-discuss-bounces@spectrumscale.org</a>] *On Behalf Of<br></font><font size="2">>> *Buterbaugh, Kevin L<br></font><font size="2">>> *Sent:* Friday, March 10, 2017 3:44 PM<br></font><font size="2">>> *To:* gpfsug main discussion list<br></font><font size="2">>> *Subject:* [gpfsug-discuss] mmcrfs issue<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> Hi All,<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> We are testing out some flash storage. I created a couple of NSDs<br></font><font size="2">>> successfully (?):<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> root@nec:~/gpfs# mmlsnsd -F<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> File system Disk name NSD servers<br></font><font size="2">>> <br></font><font size="2">>> ---------------------------------------------------------------------------<br></font><font size="2">>> <br></font><font size="2">>> (free disk) nsd1 nec<br></font><font size="2">>> <br></font><font size="2">>> (free disk) nsd2 nec<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> root@nec:~/gpfs#<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> So I tried to create a filesystem:<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> root@nec:~/gpfs# mmcrfs gpfs0 -F ~/gpfs/flash.stanza -A yes -B 1M -j<br></font><font size="2">>> scatter -k all -m 1 -M 3 -Q no -r 1 -R 3 -T /gpfs0<br></font><font size="2">>> <br></font><font size="2">>> GPFS: 6027-441 Unable to open disk 'nsd2' on node nec.<br></font><font size="2">>> <br></font><font size="2">>> GPFS: 6027-441 Unable to open disk 'nsd1' on node nec.<br></font><font size="2">>> <br></font><font size="2">>> No such device<br></font><font size="2">>> <br></font><font size="2">>> No such device<br></font><font size="2">>> <br></font><font size="2">>> GPFS: 6027-538 Error accessing disks.<br></font><font size="2">>> <br></font><font size="2">>> mmcrfs: 6027-1200 tscrfs failed. Cannot create gpfs0<br></font><font size="2">>> <br></font><font size="2">>> mmcrfs: 6027-1639 Command failed. Examine previous error messages to<br></font><font size="2">>> determine cause.<br></font><font size="2">>> <br></font><font size="2">>> root@nec:~/gpfs#<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> Does this output from readdescraw look normal?<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> root@nec:~/gpfs# mmfsadm test readdescraw /dev/zd16<br></font><font size="2">>> <br></font><font size="2">>> NSD descriptor in sector 64 of /dev/zd16<br></font><font size="2">>> <br></font><font size="2">>> NSDid: 0A0023D258C1C02C format version: 1403 Label:<br></font><font size="2">>> <br></font><font size="2">>> Paxos sector: -1931478434 number of sectors: 8192 isPdisk: 0<br></font><font size="2">>> <br></font><font size="2">>> Comment: NSD descriptor for <unknown> Thu Mar 9 14:50:52 2017<br></font><font size="2">>> <br></font><font size="2">>> No Disk descriptor in sector 96 of /dev/zd16<br></font><font size="2">>> <br></font><font size="2">>> No FS descriptor in sector 2048 of /dev/zd16<br></font><font size="2">>> <br></font><font size="2">>> root@nec:~/gpfs# mmfsadm test readdescraw /dev/zd32<br></font><font size="2">>> <br></font><font size="2">>> NSD descriptor in sector 64 of /dev/zd32<br></font><font size="2">>> <br></font><font size="2">>> NSDid: 0A0023D258C1C02B format version: 1403 Label:<br></font><font size="2">>> <br></font><font size="2">>> Paxos sector: -1880562609 number of sectors: 8192 isPdisk: 0<br></font><font size="2">>> <br></font><font size="2">>> Comment: NSD descriptor for <unknown> Thu Mar 9 14:50:51 2017<br></font><font size="2">>> <br></font><font size="2">>> No Disk descriptor in sector 96 of /dev/zd32<br></font><font size="2">>> <br></font><font size="2">>> No FS descriptor in sector 2048 of /dev/zd32<br></font><font size="2">>> <br></font><font size="2">>> root@nec:~/gpfs#<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> Thanks in advance, all\u2026<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> Kevin<br></font><font size="2">>> <br></font><font size="2">>> \u2014<br></font><font size="2">>> <br></font><font size="2">>> Kevin Buterbaugh - Senior System Administrator<br></font><font size="2">>> <br></font><font size="2">>> Vanderbilt University - Advanced Computing Center for Research and Education<br></font><font size="2">>> <br></font><font size="2">>> Kevin.Buterbaugh@vanderbilt.edu<br></font><font size="2">>> <<a href="mailto:Kevin.Buterbaugh@vanderbilt.edu">mailto:Kevin.Buterbaugh@vanderbilt.edu</a>> - (615)875-9633<br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> <br></font><font size="2">>> _______________________________________________<br></font><font size="2">>> gpfsug-discuss mailing list<br></font><font size="2">>> gpfsug-discuss at spectrumscale.org<br></font><font size="2">>> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br></font><font size="2">>> <br></font><font size="2">> <br></font><font size="2">> -- <br></font><font size="2">> Aaron Knister<br></font><font size="2">> NASA Center for Climate Simulation (Code 606.2)<br></font><font size="2">> Goddard Space Flight Center<br></font><font size="2">> (301) 286-2776<br></font><font size="2">> _______________________________________________<br></font><font size="2">> gpfsug-discuss mailing list<br></font><font size="2">> gpfsug-discuss at spectrumscale.org<br></font><font size="2">> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br></font><font size="2">> <br></font>Unless stated otherwise above:<BR>
IBM United Kingdom Limited - Registered in England and Wales with number 741598. <BR>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU<BR>
<BR>
</body></html>