<div dir="ltr">Just remember that replication protects against data availability, not integrity. GPFS still requires the underlying block device to return good data.<div><br><div>If you're using it on plain disks (SAS or SSD), and the drive returns corrupt data, GPFS won't know any better and just deliver it to the client. Further, if you do a partial read followed by a write, both replicas could be destroyed. There's also no efficient way to force use of a second replica if you realize the first is bad, short of taking the first entirely offline. In that case while migrating data, there's no good way to prevent read-rewrite of other corrupt data on your drive that has the "good copy" while restriping off a faulty drive.</div><div><br></div><div>Ideally RAID would have a goal of only returning data that passed the RAID algorithm, so shouldn't be corrupt, or made good by recreating from parity. However, as we all know RAID controllers are definitely prone to failures as well for many reasons, but at least a drive can go bad in various ways (bad sectors, slow, just dead, poor SSD cell wear, etc) without (hopefully) silent corruption..</div><div><br></div><div><div>Just something to think about while considering replication ..</div></div><div><br></div><div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 30, 2016 at 11:28 AM, Uwe Falke <span dir="ltr"><<a href="mailto:UWEFALKE@de.ibm.com" target="_blank">UWEFALKE@de.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I have once set up a small system with just a few SSDs in two NSD servers,<br>
providin a scratch file system in a computing cluster.<br>
No RAID, two replica.<br>
works, as long the admins do not do silly things (like rebooting servers<br>
in sequence without checking for disks being up in between).<br>
Going for RAIDs without GPFS replication protects you against single disk<br>
failures, but you're lost if just one of your NSD servers goes off.<br>
<br>
FPO makes sense only sense IMHO if your NSD servers are also processing<br>
the data (and then you need to control that somehow).<br>
<br>
Other ideas? what else can you do with GPFS and local disks than what you<br>
considered? I suppose nothing reasonable ...<br>
<br>
<br>
Mit freundlichen Grüßen / Kind regards<br>
<br>
<br>
Dr. Uwe Falke<br>
<br>
IT Specialist<br>
High Performance Computing Services / Integrated Technology Services /<br>
Data Center Services<br>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>-------------------<br>
IBM Deutschland<br>
Rathausstr. 7<br>
09111 Chemnitz<br>
Phone: <a href="tel:%2B49%20371%206978%202165" value="+4937169782165">+49 371 6978 2165</a><br>
Mobile: <a href="tel:%2B49%20175%20575%202877" value="+491755752877">+49 175 575 2877</a><br>
E-Mail: <a href="mailto:uwefalke@de.ibm.com">uwefalke@de.ibm.com</a><br>
------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>-------------------<br>
IBM Deutschland Business & Technology Services GmbH / Geschäftsführung:<br>
Frank Hammer, Thorsten Moehring<br>
Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,<br>
HRB 17122<br>
<br>
<br>
<br>
<br>
From: "Oesterlin, Robert" <<a href="mailto:Robert.Oesterlin@nuance.com">Robert.Oesterlin@nuance.com</a>><br>
To: gpfsug main discussion list <<a href="mailto:gpfsug-discuss@spectrumscale.org">gpfsug-discuss@spectrumscale.<wbr>org</a>><br>
Date: 11/30/2016 03:34 PM<br>
Subject: [gpfsug-discuss] Strategies - servers with local SAS disks<br>
Sent by: <a href="mailto:gpfsug-discuss-bounces@spectrumscale.org">gpfsug-discuss-bounces@<wbr>spectrumscale.org</a><br>
<span class=""><br>
<br>
<br>
Looking for feedback/strategies in setting up several GPFS servers with<br>
local SAS. They would all be part of the same file system. The systems are<br>
all similar in configuration - 70 4TB drives.<br>
<br>
</span>Options I?m considering:<br>
<span class=""><br>
- Create RAID arrays of the disks on each server (worried about the RAID<br>
rebuild time when a drive fails with 4, 6, 8TB drives)<br>
- No RAID with 2 replicas, single drive per NSD. When a drive fails,<br>
</span>recreate the NSD ? but then I need to fix up the data replication via<br>
restripe<br>
- FPO ? with multiple failure groups - letting the system manage replica<br>
<span class="">placement and then have GPFS due the restripe on disk failure<br>
automatically<br>
<br>
Comments or other ideas welcome.<br>
<br>
Bob Oesterlin<br>
Sr Principal Storage Engineer, Nuance<br>
<a href="tel:507-269-0413" value="+15072690413">507-269-0413</a><br>
<br>
</span> ______________________________<wbr>_________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</a><br>
<br>
<br>
<br>
<br>
______________________________<wbr>_________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" target="_blank">spectrumscale.org</a><br>
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" target="_blank">http://gpfsug.org/mailman/<wbr>listinfo/gpfsug-discuss</a><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Zach Giles<br><a href="mailto:zgiles@gmail.com" target="_blank">zgiles@gmail.com</a></div>
</div>