<font size=2 face="sans-serif">How do you set the size of a ZFS file that
is simulating a GPFS disk? How do "tell" GPFS about that?</font><br><br><font size=2 face="sans-serif">How efficient is this layering, compared
to just giving GPFS direct access to the same kind of LUNs that ZFS is
using?<br></font><br><font size=2 face="sans-serif">Hmmm... to partially answer my question,
I do something similar, but strictly for testing non-performance critical
GPFS functions.</font><br><font size=2 face="sans-serif">On any file system one can:</font><br><br><font size=2 face="sans-serif"> dd if=/dev/zero of=/fakedisks/d3
count=1 bs=1M seek=3000 # create a fake 3GB disk for GPFS</font><br><br><font size=2 face="sans-serif">Then use a GPFS nsd configuration record
like this:</font><br><br><font size=2 face="sans-serif">%nsd: nsd=d3 device=/fakedisks/d3
usage=dataOnly pool=xtra servers=bog-xxx</font><br><br><font size=2 face="sans-serif">Which starts out as sparse and the filesystem
will dynamically "grow" as GPFS writes to it...</font><br><br><font size=2 face="sans-serif">But I have no idea how well this will
work for a critical "production" system...</font><br><br><font size=2 face="sans-serif">tx, marc kaplan.</font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">"Allen, Benjamin
S." <bsallen@alcf.anl.gov></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">gpfsug main discussion
list <gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">06/13/2016 12:34 PM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
GPFS on ZFS?</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><tt><font size=2>Jaime,<br><br>See </font></tt><a href=https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_nsddevices.htm><tt><font size=2>https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_nsddevices.htm</font></tt></a><tt><font size=2>.
An example I have for add /dev/nvme* devices:<br><br>* GPFS doesn't know how that /dev/nvme* are valid block devices, use a
user exit script to let it know about it<br><br>cp /usr/lpp/mmfs/samples/nsddevices.sample /var/mmfs/etc/nsddevices<br><br>* Edit /var/mmfs/etc/nsddevices, and add to linux section:<br><br>if [[ $osName = Linux ]]<br>then<br> : # Add function to discover disks in the Linux environment.<br> for dev in $( cat /proc/partitions | grep nvme | awk '{print
$4}' )<br> do<br> echo $dev generic<br> done<br>fi<br><br>* Copy edited nsddevices to the rest of the nodes at the same path<br>for host in n01 n02 n03 n04; do<br> scp /var/mmfs/etc/nsddevices ${host}:/var/mmfs/etc/nsddevices<br>done<br><br><br>Ben<br><br>> On Jun 13, 2016, at 11:26 AM, Jaime Pinto <pinto@scinet.utoronto.ca>
wrote:<br>> <br>> Hi Chris<br>> <br>> As I understand, GPFS likes to 'see' the block devices, even on a
hardware raid solution such at DDN's.<br>> <br>> How is that accomplished when you use ZFS for software raid?<br>> On page 4 I see this info, and I'm trying to interpret it:<br>> <br>> General Configuration<br>> ...<br>> * zvols<br>> * nsddevices<br>> - echo "zdX generic"<br>> <br>> <br>> Thanks<br>> Jaime<br>> <br>> Quoting "Hoffman, Christopher P" <cphoffma@lanl.gov>:<br>> <br>>> Hi Jaime,<br>>> <br>>> What in particular would you like explained more? I'd be more
than happy to discuss things further.<br>>> <br>>> Chris<br>>> ________________________________________<br>>> From: gpfsug-discuss-bounces@spectrumscale.org [gpfsug-discuss-bounces@spectrumscale.org]
on behalf of Jaime Pinto [pinto@scinet.utoronto.ca]<br>>> Sent: Monday, June 13, 2016 10:11<br>>> To: gpfsug main discussion list<br>>> Subject: Re: [gpfsug-discuss] GPFS on ZFS?<br>>> <br>>> I just came across this presentation on "GPFS with underlying
ZFS<br>>> block devices", by Christopher Hoffman, Los Alamos National
Lab,<br>>> although some of the<br>>> implementation remains obscure.<br>>> <br>>> </font></tt><a href="http://files.gpfsug.org/presentations/2016/anl-june/LANL_GPFS_ZFS.pdf"><tt><font size=2>http://files.gpfsug.org/presentations/2016/anl-june/LANL_GPFS_ZFS.pdf</font></tt></a><tt><font size=2><br>>> <br>>> It would be great to have more details, in particular the possibility<br>>> of straight use of GPFS on ZFS, instead of the 'archive' use case
as<br>>> described on the presentation.<br>>> <br>>> Thanks<br>>> Jaime<br>>> <br>>> <br>>> <br>>> <br>>> Quoting "Jaime Pinto" <pinto@scinet.utoronto.ca>:<br>>> <br>>>> Since we can not get GNR outside ESS/GSS appliances, is anybody
using<br>>>> ZFS for software raid on commodity storage?<br>>>> <br>>>> Thanks<br>>>> Jaime<br>>>> <br>>>> <br>>> <br>>> <br>>> <br>>> <br>>> ************************************<br>>> TELL US ABOUT YOUR SUCCESS
STORIES<br>>> </font></tt><a href=http://www.scinethpc.ca/testimonials><tt><font size=2>http://www.scinethpc.ca/testimonials</font></tt></a><tt><font size=2><br>>> ************************************<br>>> ---<br>>> Jaime Pinto<br>>> SciNet HPC Consortium - Compute/Calcul Canada<br>>> </font></tt><a href=www.scinet.utoronto.ca><tt><font size=2>www.scinet.utoronto.ca</font></tt></a><tt><font size=2>- </font></tt><a href=www.computecanada.org><tt><font size=2>www.computecanada.org</font></tt></a><tt><font size=2><br>>> University of Toronto<br>>> 256 McCaul Street, Room 235<br>>> Toronto, ON, M5T1W5<br>>> P: 416-978-2755<br>>> C: 416-505-1477<br>>> <br>>> ----------------------------------------------------------------<br>>> This message was sent using IMP at SciNet Consortium, University
of Toronto.<br>>> <br>>> _______________________________________________<br>>> gpfsug-discuss mailing list<br>>> gpfsug-discuss at spectrumscale.org<br>>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>>> _______________________________________________<br>>> gpfsug-discuss mailing list<br>>> gpfsug-discuss at spectrumscale.org<br>>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>>> <br>> <br>> <br>> <br>> <br>> <br>> <br>> ************************************<br>> TELL US ABOUT YOUR SUCCESS STORIES<br>> </font></tt><a href=http://www.scinethpc.ca/testimonials><tt><font size=2>http://www.scinethpc.ca/testimonials</font></tt></a><tt><font size=2><br>> ************************************<br>> ---<br>> Jaime Pinto<br>> SciNet HPC Consortium - Compute/Calcul Canada<br>> </font></tt><a href=www.scinet.utoronto.ca><tt><font size=2>www.scinet.utoronto.ca</font></tt></a><tt><font size=2>- </font></tt><a href=www.computecanada.org><tt><font size=2>www.computecanada.org</font></tt></a><tt><font size=2><br>> University of Toronto<br>> 256 McCaul Street, Room 235<br>> Toronto, ON, M5T1W5<br>> P: 416-978-2755<br>> C: 416-505-1477<br>> <br>> ----------------------------------------------------------------<br>> This message was sent using IMP at SciNet Consortium, University of
Toronto.<br>> <br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br><br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br><br></font></tt><br><BR>