<span style=" font-size:10pt;font-family:sans-serif">It's good to try
to reason and think this out... But there's a good likelihood that we don't
understand ALL the details, some of which may negatively impact performance
- so no matter what scheme you come up with - test, test, and re-test before
deploying and depending on it in production.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">Having said that,
I'm pretty sure that old "spinning" RAID 5 implementations had
horrible performance for GPFS metadata/system pool.</span><br><span style=" font-size:10pt;font-family:sans-serif">Why? Among other
things, the large stripe size vs the almost random small writes directed
to system pool.</span><br><br><span style=" font-size:10pt;font-family:sans-serif">That random-small-writes
pattern won't change when we go to SSD RAID 5 - so you'd have to see if
the SSD implementation is somehow smarter than an old fashioned RAID 5
implementation which I believe requires several physical reads and writes,
for each "small" logical write.</span><br><span style=" font-size:10pt;font-family:sans-serif">(Top decent google
result I found quickly </span><a href="http://rickardnobel.se/raid-5-write-penalty/"><span style=" font-size:10pt;color:blue;font-family:sans-serif">http://rickardnobel.se/raid-5-write-penalty/</span></a><span style=" font-size:10pt;font-family:sans-serif">But you will probably want to do more research!)</span><br><br><span style=" font-size:10pt;font-family:sans-serif">Consider GPFS
small write performance for: inode updates, log writes, small files
(possibly in inode), directory updates, allocation map updates, index of
indirect blocks.</span><br><br><br><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">From:
</span><span style=" font-size:9pt;font-family:sans-serif">"Buterbaugh,
Kevin L" <Kevin.Buterbaugh@Vanderbilt.Edu></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">To:
</span><span style=" font-size:9pt;font-family:sans-serif">gpfsug
main discussion list <gpfsug-discuss@spectrumscale.org></span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Date:
</span><span style=" font-size:9pt;font-family:sans-serif">09/05/2018
11:36 AM</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Subject:
</span><span style=" font-size:9pt;font-family:sans-serif">[gpfsug-discuss]
RAID type for system pool</span><br><span style=" font-size:9pt;color:#5f5f5f;font-family:sans-serif">Sent
by: </span><span style=" font-size:9pt;font-family:sans-serif">gpfsug-discuss-bounces@spectrumscale.org</span><br><hr noshade><br><br><br><span style=" font-size:12pt">Hi All, </span><br><br><span style=" font-size:12pt">We are in the process of finalizing the
purchase of some new storage arrays (so no sales people who might be monitoring
this list need contact me) to life-cycle some older hardware. One
of the things we are considering is the purchase of some new SSD\u2019s for
our \u201c/home\u201d filesystem and I have a question or two related to that.</span><br><br><span style=" font-size:12pt">Currently, the existing home filesystem
has it\u2019s metadata on SSD\u2019s \u2026 two RAID 1 mirrors and metadata replication
set to two. However, the filesystem itself is old enough that it
uses 512 byte inodes. We have analyzed our users files and know that
if we create a new filesystem with 4K inodes that a very significant portion
of the files would now have their _data_ stored in the inode as well due
to the files being 3.5K or smaller (currently all data is on spinning HD
RAID 1 mirrors).</span><br><br><span style=" font-size:12pt">Of course, if we increase the size of
the inodes by a factor of 8 then we also need 8 times as much space to
store those inodes. Given that Enterprise class SSDs are still very
expensive and our budget is not unlimited, we\u2019re trying to get the best
bang for the buck.</span><br><br><span style=" font-size:12pt">We have always - even back in the day
when our metadata was on spinning disk and not SSD - used RAID 1 mirrors
and metadata replication of two. However, we are wondering if it
might be possible to switch to RAID 5? Specifically, what we are
considering doing is buying 8 new SSDs and creating two 3+1P RAID 5 LUNs
(metadata replication would stay at two). That would give us 50%
more usable space than if we configured those same 8 drives as four RAID
1 mirrors.</span><br><br><span style=" font-size:12pt">Unfortunately, unless I\u2019m misunderstanding
something, mean that the RAID stripe size and the GPFS block size could
not match. Therefore, even though we don\u2019t need the space, would
we be much better off to buy 10 SSDs and create two 4+1P RAID 5 LUNs?</span><br><br><span style=" font-size:12pt">I\u2019ve searched the mailing list archives
and scanned the DeveloperWorks wiki and even glanced at the GPFS documentation
and haven\u2019t found anything that says \u201cbad idea, Kevin\u201d\u2026 ;-)</span><br><br><span style=" font-size:12pt">Expanding on this further \u2026 if we just
present those two RAID 5 LUNs to GPFS as NSDs then we can only have two
NSD servers as primary for them. So another thing we\u2019re considering
is to take those RAID 5 LUNs and further sub-divide them into a total of
8 logical volumes, each of which could be a GPFS NSD and therefore would
allow us to have each of our 8 NSD servers be primary for one of them.
Even worse idea?!? Good idea?</span><br><br><span style=" font-size:12pt">Anybody have any better ideas??? ;-)</span><br><br><span style=" font-size:12pt">Oh, and currently we\u2019re on GPFS 4.2.3-10,
but are also planning on moving to GPFS 5.0.1-x before creating the new
filesystem.</span><br><br><span style=" font-size:12pt">Thanks much\u2026</span><br><br><span style=" font-size:12pt">\u2014</span><br><span style=" font-size:12pt">Kevin Buterbaugh - Senior System Administrator</span><br><span style=" font-size:12pt">Vanderbilt University - Advanced Computing
Center for Research and Education</span><br><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu"><span style=" font-size:12pt;color:blue"><u>Kevin.Buterbaugh@vanderbilt.edu</u></span></a><span style=" font-size:12pt">- (615)875-9633</span><br><br><br><tt><span style=" font-size:10pt">_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></span></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><span style=" font-size:10pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span></tt></a><tt><span style=" font-size:10pt"><br></span></tt><br><br><BR>