<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi All,
<div class=""><br class="">
</div>
<div class="">Just wanted to follow up with the results of my survey \u2026 I received a grand total of two responses (Thanks Alex and John). In their case, they\u2019re using SSDs with a 10 DWPD rating.</div>
<div class=""><br class="">
</div>
<div class="">The motivation behind my asking this question was \u2026 money! ;-). Seriously, 10 DWPD drives are still very expensive, while 3 DWPD drives are significantly less expensive and 1 DWPD drives are even cheaper still. While we would NOT feel comfortable
using anything less than 10 DWPD drives for metadata, we\u2019re wondering about using less expensive drives for data.</div>
<div class=""><br class="">
</div>
<div class="">For example, let\u2019s just say that you\u2019re getting ready to set up a brand new GPFS 5 formatted filesystem of 1-2 PB in size. You\u2019re considering having 3 pools:</div>
<div class=""><br class="">
</div>
<div class="">1) a metadata only system pool of 10 DWPD SSDs. 4K inodes, and a ton of small files that\u2019ll fit in the inode.</div>
<div class="">2) a data only \u201chot\u201d pool (i.e. the default pool for writes) of SSDs.</div>
<div class="">3) a data only \u201ccapacity\u201d pool of 12 TB spinning disks.</div>
<div class=""><br class="">
</div>
<div class="">And let\u2019s just say that you have looked back at the historical data you\u2019ve collected and you see that over the last 6 months or so you\u2019ve been averaging 10-12 TB of data being written into your existing filesystem per day. You want to do migrations
between pools only on the weekends if at all possible.</div>
<div class=""><br class="">
</div>
<div class="">12 * 7 = 84 TB. So if you had somewhere between 125 - 150 TB of SSDs ... 1 DWPD SSDs \u2026 then in theory you should easily be able to handle your anticipated workload without coming close to exceeding the 1 DWPD rating of the SSDs.</div>
<div class=""><br class="">
</div>
<div class="">However, as the saying goes, while in theory there\u2019s no difference between theory and practice, in practice there is ... so am I overlooking anything here from a GPFS perspective???</div>
<div class=""><br class="">
</div>
<div class="">If anybody still wants to respond on the DWPD rating of the SSDs they use for data, I\u2019m still listening.</div>
<div class=""><br class="">
</div>
<div class="">Thanks\u2026</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class=""><br class="">
</div>
<div class="">P.S. I still have a couple of \u201coutstanding issues\u201d to respond to that I\u2019ve posted to the list about previously:</div>
<div class=""><br class="">
</div>
<div class="">1) the long I/O\u2019s we see occasionally in the output of \u201cmmdiag \u2014iohist\u201d on our NSD servers. We\u2019re still trying to track that down \u2026 it seems to happen only with a subset of our hardware - most of the time at least - but we\u2019re still working to
track down what triggers it \u2026 i.e. at this point I can\u2019t say whether it\u2019s really the hardware or a user abusing the hardware.</div>
<div class=""><br class="">
</div>
<div class="">2) I promised to post benchmark results of 3 different metadata configs: a) RAID 1 mirrors, b) a RAID 5 stripe, c) no RAID, but GPFS metadata replication of 3. That benchmarking has been put on hold for reasons I can\u2019t really discuss on this
mailing list at this time \u2026 but hopefully soon.</div>
<div class=""><br class="">
</div>
<div class="">I haven\u2019t forgotten the above and will respond back on the list when it\u2019s appropriate. Thanks...<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Mar 8, 2019, at 10:24 AM, Buterbaugh, Kevin L <<a href="mailto:Kevin.Buterbaugh@Vanderbilt.Edu" class="">Kevin.Buterbaugh@Vanderbilt.Edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi All,
<div class=""><br class="">
</div>
<div class="">This is kind of a survey if you will, so for this one it might be best if you responded directly to me and I\u2019ll summarize the results next week.</div>
<div class=""><br class="">
</div>
<div class="">Question 1 - do you use SSDs for data? If not - i.e. if you only use SSDs for metadata (as we currently do) - thanks, that\u2019s all! If, however, you do use SSDs for data, please see Question 2.</div>
<div class=""><br class="">
</div>
<div class="">Question 2 - what is the DWPD (daily writes per day) of the SSDs that you use for data?</div>
<div class=""><br class="">
</div>
<div class="">Question 3 - is that different than the DWPD of the SSDs for metadata?</div>
<div class=""><br class="">
</div>
<div class="">Question 4 - any pertinent information in regards to your answers above (i.e. if you\u2019ve got a filesystem that data is uploaded to only once and never modified after that then that\u2019s useful to know!)?</div>
<div class=""><br class="">
</div>
<div class="">Thanks\u2026</div>
<div class=""><br class="">
</div>
<div class="">Kevin</div>
<div class=""><br class="">
<div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class="">\u2014</div>
<div class="">Kevin Buterbaugh - Senior System Administrator</div>
<div class="">Vanderbilt University - Advanced Computing Center for Research and Education</div>
<div class=""><a href="mailto:Kevin.Buterbaugh@vanderbilt.edu" class="">Kevin.Buterbaugh@vanderbilt.edu</a> - (615)875-9633</div>
</div>
</div>
<br class="">
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>