<span style=" font-size:10pt;font-family:sans-serif">This could be related
to the following flash:</span><br><br><a href="http://www-01.ibm.com/support/docview.wss?uid=ssg1S1012054"><span style=" font-size:10pt;color:blue;font-family:sans-serif">http://www-01.ibm.com/support/docview.wss?uid=ssg1S1012054</span></a><br><br><span style=" font-size:10pt;font-family:sans-serif">You should contact
IBM service to obtain the fix for your release.<br><br>Steve Y. Xiao<br></span><br><tt><span style=" font-size:10pt">gpfsug-discuss-bounces@spectrumscale.org
wrote on 02/14/2018 02:18:02 PM:<br><br>&gt; From: gpfsug-discuss-request@spectrumscale.org</span></tt><br><tt><span style=" font-size:10pt">&gt; To: gpfsug-discuss@spectrumscale.org</span></tt><br><tt><span style=" font-size:10pt">&gt; Date: 02/14/2018 02:18 PM</span></tt><br><tt><span style=" font-size:10pt">&gt; Subject: gpfsug-discuss Digest,
Vol 73, Issue 36</span></tt><br><tt><span style=" font-size:10pt">&gt; Sent by: gpfsug-discuss-bounces@spectrumscale.org</span></tt><br><tt><span style=" font-size:10pt">&gt; <br>&gt; Send gpfsug-discuss mailing list submissions to<br>&gt; &nbsp; &nbsp;gpfsug-discuss@spectrumscale.org<br>&gt; <br>&gt; To subscribe or unsubscribe via the World Wide Web, visit<br>&gt; &nbsp; &nbsp;</span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=aVWMptxcCR3po3ijmRweTyjbs1Pp5D7WEiJTYvSYLUk&amp;e=<br>&gt; or, via email, send a message with subject or body 'help' to<br>&gt; &nbsp; &nbsp;gpfsug-discuss-request@spectrumscale.org<br>&gt; <br>&gt; You can reach the person managing the list at<br>&gt; &nbsp; &nbsp;gpfsug-discuss-owner@spectrumscale.org<br>&gt; <br>&gt; When replying, please edit your Subject line so it is more specific<br>&gt; than &quot;Re: Contents of gpfsug-discuss digest...&quot;<br>&gt; <br>&gt; <br>&gt; Today's Topics:<br>&gt; <br>&gt; &nbsp; &nbsp;1. Re: Odd behavior with cat followed by grep. (John
Hanks)<br>&gt; <br>&gt; <br>&gt; ----------------------------------------------------------------------<br>&gt; <br>&gt; Message: 1<br>&gt; Date: Wed, 14 Feb 2018 11:17:19 -0800<br>&gt; From: John Hanks &lt;griznog@gmail.com&gt;<br>&gt; To: gpfsug main discussion list &lt;gpfsug-discuss@spectrumscale.org&gt;<br>&gt; Subject: Re: [gpfsug-discuss] Odd behavior with cat followed by grep.<br>&gt; Message-ID:<br>&gt; &nbsp; &nbsp;&lt;CAGrHuK7okijzHLysDNCxR8tn1fKeMDwW_iT5898tBvaCR_QJUg@mail.gmail.com&gt;<br>&gt; Content-Type: text/plain; charset=&quot;utf-8&quot;<br>&gt; <br>&gt; Thanks Bryan, mystery solved :)<br>&gt; <br>&gt; We also stumbled across these related items, in case anyone else wanders<br>&gt; into this thread.<br>&gt; <br>&gt; </span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__bug-2Dgrep.gnu.narkive.com_Y8cfvWDt_bug-2D27666-2Dgrep-2Don-2Dgpfs-2Dfilesystem-2Dseek-2Dhole-2Dproblem&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=FgxYBxqHZ0bHdWirEs1U_B3oDpeHJe8iRd-<br>&gt; TYrXh6FI&amp;e=<br>&gt; <br>&gt; </span></tt><a href=https://www.ibm.com/developerworks/community/forums/html/topic?><tt><span style=" font-size:10pt">https://www.ibm.com/developerworks/community/forums/html/topic?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; id=c2a94433-9ec0-4a4b-abfe-d0a1e721d630<br>&gt; <br>&gt; GPFS, the gift that keeps on giving ... me more things to do instead
of<br>&gt; doing the things I want to be doing.<br>&gt; <br>&gt; Thanks all,<br>&gt; <br>&gt; jbh<br>&gt; <br>&gt; On Wed, Feb 14, 2018 at 10:48 AM, Bryan Banister &lt;bbanister@jumptrading.com&gt;<br>&gt; wrote:<br>&gt; <br>&gt; &gt; Hi all,<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; We found this a while back and IBM fixed it. &nbsp;Here?s your
answer:<br>&gt; &gt; </span></tt><a href="http://www-01.ibm.com/support/docview.wss?uid=isg1IV87385"><tt><span style=" font-size:10pt">http://www-01.ibm.com/support/docview.wss?uid=isg1IV87385</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; Cheers,<br>&gt; &gt;<br>&gt; &gt; -Bryan<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; *From:* gpfsug-discuss-bounces@spectrumscale.org [</span></tt><a href="mailto:gpfsug-discuss-"><tt><span style=" font-size:10pt">mailto:gpfsug-discuss-</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; &gt; bounces@spectrumscale.org] *On Behalf Of *John Hanks<br>&gt; &gt; *Sent:* Wednesday, February 14, 2018 12:31 PM<br>&gt; &gt; *To:* gpfsug main discussion list &lt;gpfsug-discuss@spectrumscale.org&gt;<br>&gt; &gt; *Subject:* Re: [gpfsug-discuss] Odd behavior with cat followed
by grep.<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; *Note: External Email*<br>&gt; &gt; ------------------------------<br>&gt; &gt;<br>&gt; &gt; Straces are interesting, but don't immediately open my eyes:<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; strace of grep on NFS (works as expected)<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; openat(AT_FDCWD, &quot;/home/griznog/pipetest.tmp.txt&quot;,
O_RDONLY) = 3<br>&gt; &gt;<br>&gt; &gt; fstat(3, {st_mode=S_IFREG|0644, st_size=530721, ...}) = 0<br>&gt; &gt;<br>&gt; &gt; ioctl(3, TCGETS, 0x7ffe2c26b0b0) &nbsp; &nbsp; &nbsp; &nbsp;=
-1 ENOTTY (Inappropriate ioctl<br>&gt; &gt; for device)<br>&gt; &gt;<br>&gt; &gt; read(3, &quot;chr1\t43452652\t43452652\tL1HS\tlib1&quot;...,
32768) = 32768<br>&gt; &gt;<br>&gt; &gt; lseek(3, 32768, SEEK_HOLE) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp;= 530721<br>&gt; &gt;<br>&gt; &gt; lseek(3, 32768, SEEK_SET) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; = 32768<br>&gt; &gt;<br>&gt; &gt; fstat(1, {st_mode=S_IFREG|0644, st_size=5977, ...}) = 0<br>&gt; &gt;<br>&gt; &gt; mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) =<br>&gt; &gt; 0x7f3bf6c43000<br>&gt; &gt;<br>&gt; &gt; write(1, &quot;chr1\t43452652\t43452652\tL1HS\tlib1&quot;...,
8192chr1<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; strace on GPFS (thinks file is binary)<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; openat(AT_FDCWD, &quot;/srv/gsfs0/projects/pipetest.tmp.txt&quot;,
O_RDONLY) = 3<br>&gt; &gt;<br>&gt; &gt; fstat(3, {st_mode=S_IFREG|0644, st_size=530721, ...}) = 0<br>&gt; &gt;<br>&gt; &gt; ioctl(3, TCGETS, 0x7ffc9b52caa0) &nbsp; &nbsp; &nbsp; &nbsp;=
-1 ENOTTY (Inappropriate ioctl<br>&gt; &gt; for device)<br>&gt; &gt;<br>&gt; &gt; read(3, &quot;chr1\t43452652\t43452652\tL1HS\tlib1&quot;...,
32768) = 32768<br>&gt; &gt;<br>&gt; &gt; lseek(3, 32768, SEEK_HOLE) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp;= 262144<br>&gt; &gt;<br>&gt; &gt; lseek(3, 32768, SEEK_SET) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; = 32768<br>&gt; &gt;<br>&gt; &gt; fstat(1, {st_mode=S_IFREG|0644, st_size=6011, ...}) = 0<br>&gt; &gt;<br>&gt; &gt; mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) =<br>&gt; &gt; 0x7fd45ee88000<br>&gt; &gt;<br>&gt; &gt; close(3) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= 0<br>&gt; &gt;<br>&gt; &gt; write(1, &quot;Binary file /srv/gsfs0/projects/&quot;..., 72Binary
file<br>&gt; &gt; /srv/gsfs0/projects/levinson/xwzhu/pipetest.tmp.txt matches<br>&gt; &gt;<br>&gt; &gt; ) = 72<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; Do the lseek() results indicate that the grep on the GPFS mounted
version<br>&gt; &gt; thinks the file is a sparse file? For comparison I strace'd md5sum
in place<br>&gt; &gt; of the grep and it does not lseek() with SEEK_HOLE, it's access
in both<br>&gt; &gt; cases look identical, like:<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; open(&quot;/home/griznog/pipetest.tmp.txt&quot;, O_RDONLY) =
3<br>&gt; &gt;<br>&gt; &gt; fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0<br>&gt; &gt;<br>&gt; &gt; fstat(3, {st_mode=S_IFREG|0644, st_size=530721, ...}) = 0<br>&gt; &gt;<br>&gt; &gt; mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) =<br>&gt; &gt; 0x7fb7d2c2b000<br>&gt; &gt;<br>&gt; &gt; read(3, &quot;chr1\t43452652\t43452652\tL1HS\tlib1&quot;...,
32768) = 32768<br>&gt; &gt;<br>&gt; &gt; ...[reads clipped]...<br>&gt; &gt;<br>&gt; &gt; read(3, &quot;&quot;, 24576) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= 0<br>&gt; &gt;<br>&gt; &gt; lseek(3, 0, SEEK_CUR) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; = 530721<br>&gt; &gt;<br>&gt; &gt; close(3) &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;= 0<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; jbh<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; On Wed, Feb 14, 2018 at 9:51 AM, Aaron Knister &lt;aaron.s.knister@nasa.gov&gt;<br>&gt; &gt; wrote:<br>&gt; &gt;<br>&gt; &gt; Just speculating here (also known as making things up) but I
wonder if<br>&gt; &gt; grep is somehow using the file's size in its determination of
binary<br>&gt; &gt; status. I also see mmap in the strace so maybe there's some issue
with<br>&gt; &gt; mmap where some internal GPFS buffer is getting truncated<br>&gt; &gt; inappropriately but leaving a bunch of null values which gets
returned<br>&gt; &gt; to grep.<br>&gt; &gt;<br>&gt; &gt; -Aaron<br>&gt; &gt;<br>&gt; &gt; On 2/14/18 10:21 AM, John Hanks wrote:<br>&gt; &gt; &gt; Hi Valdis,<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; I tired with the grep replaced with 'ls -ls' and 'md5sum',
I don't think<br>&gt; &gt; &gt; this is a data integrity issue, thankfully:<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; $ ./pipetestls.sh<br>&gt; &gt; &gt; 256 -rw-r--r-- 1 39073 3001 530721 Feb 14 07:16<br>&gt; &gt; &gt; /srv/gsfs0/projects/pipetest.tmp.txt<br>&gt; &gt; &gt; 0 -rw-r--r-- 1 39073 3953 530721 Feb 14 07:16<br>&gt; &gt; /home/griznog/pipetest.tmp.txt<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; $ ./pipetestmd5.sh<br>&gt; &gt; &gt; 15cb81a85c9e450bdac8230309453a0a &nbsp;/srv/gsfs0/projects/pipetest.tmp.txt<br>&gt; &gt; &gt; 15cb81a85c9e450bdac8230309453a0a &nbsp;/home/griznog/pipetest.tmp.txt<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; And replacing grep with 'file' even properly sees the files
as ASCII:<br>&gt; &gt; &gt; $ ./pipetestfile.sh<br>&gt; &gt; &gt; /srv/gsfs0/projects/pipetest.tmp.txt: ASCII text, with very
long lines<br>&gt; &gt; &gt; /home/griznog/pipetest.tmp.txt: ASCII text, with very long
lines<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; I'll poke a little harder at grep next and see what the
difference in<br>&gt; &gt; &gt; strace of each reveals.<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; Thanks,<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; jbh<br>&gt; &gt; &gt;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; On Wed, Feb 14, 2018 at 7:08 AM, &lt;valdis.kletnieks@vt.edu<br>&gt; &gt;<br>&gt; &gt; &gt; &lt;</span></tt><a href=mailto:valdis.kletnieks@vt.edu><tt><span style=" font-size:10pt">mailto:valdis.kletnieks@vt.edu</span></tt></a><tt><span style=" font-size:10pt">&gt;&gt;
wrote:<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; On Wed, 14 Feb 2018 06:20:32 -0800, John Hanks
said:<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; # &nbsp;ls -aln /srv/gsfs0/projects/pipetest.tmp.txt<br>&gt; &gt; $HOME/pipetest.tmp.txt<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; -rw-r--r-- 1 39073 3953 530721 Feb 14
06:10<br>&gt; &gt; /home/griznog/pipetest.tmp.txt<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; -rw-r--r-- 1 39073 3001 530721 Feb 14
06:10<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; /srv/gsfs0/projects/pipetest.tmp.txt<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; We can &quot;fix&quot; the user case
that exposed this by not using a temp<br>&gt; &gt; file or<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; inserting a sleep, but I'd still like
to know why GPFS is behaving<br>&gt; &gt; this way<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; and make it stop.<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; May be related to replication, or other behind-the-scenes
behavior.<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; Consider this example - 4.2.3.6, data and
metadata replication both<br>&gt; &gt; &gt; &nbsp; &nbsp; set to 2, 2 sites 95 cable miles apart, each
is 3 Dell servers with<br>&gt; &gt; &gt; &nbsp; &nbsp; a full<br>&gt; &gt; &gt; &nbsp; &nbsp; fiberchannel mesh to 3 Dell MD34something
arrays.<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; % dd if=/dev/zero bs=1k count=4096 of=sync.test;
ls -ls sync.test;<br>&gt; &gt; &gt; &nbsp; &nbsp; sleep 5; ls -ls sync.test; sleep 5; ls -ls
sync.test<br>&gt; &gt; &gt; &nbsp; &nbsp; 4096+0 records in<br>&gt; &gt; &gt; &nbsp; &nbsp; 4096+0 records out<br>&gt; &gt; &gt; &nbsp; &nbsp; 4194304 bytes (4.2 MB) copied, 0.0342852 s,
122 MB/s<br>&gt; &gt; &gt; &nbsp; &nbsp; 2048 -rw-r--r-- 1 root root 4194304 Feb 14
09:35 sync.test<br>&gt; &gt; &gt; &nbsp; &nbsp; 8192 -rw-r--r-- 1 root root 4194304 Feb 14
09:35 sync.test<br>&gt; &gt; &gt; &nbsp; &nbsp; 8192 -rw-r--r-- 1 root root 4194304 Feb 14
09:35 sync.test<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; Notice that the first /bin/ls shouldn't be
starting until after the<br>&gt; &gt; &gt; &nbsp; &nbsp; dd has<br>&gt; &gt; &gt; &nbsp; &nbsp; completed - at which point it's only allocated
half the blocks<br>&gt; &gt; &gt; &nbsp; &nbsp; needed to hold<br>&gt; &gt; &gt; &nbsp; &nbsp; the 4M of data at one site. &nbsp;5 seconds
later, it's allocated the<br>&gt; &gt; &gt; &nbsp; &nbsp; blocks at both<br>&gt; &gt; &gt; &nbsp; &nbsp; sites and thus shows the full 8M needed for
2 copies.<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; I've also seen (but haven't replicated it
as I write this) a small<br>&gt; &gt; &gt; &nbsp; &nbsp; file (4-8K<br>&gt; &gt; &gt; &nbsp; &nbsp; or so) showing first one full-sized block,
then a second full-sized<br>&gt; &gt; &gt; &nbsp; &nbsp; block, and<br>&gt; &gt; &gt; &nbsp; &nbsp; then dropping back to what's needed for 2
1/32nd fragments. &nbsp;That<br>&gt; &gt; had me<br>&gt; &gt; &gt; &nbsp; &nbsp; scratching my head<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; Having said that, that's all metadata fun
and games, while your case<br>&gt; &gt; &gt; &nbsp; &nbsp; appears to have some problems with data integrity
(which is a whole<br>&gt; &gt; lot<br>&gt; &gt; &gt; &nbsp; &nbsp; scarier). &nbsp;It would be *really* nice
if we understood the problem<br>&gt; &gt; here.<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; The scariest part is:<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; The first grep | wc -l returns 1, because
grep outputs &nbsp;&quot;Binary<br>&gt; &gt; file /path/to/<br>&gt; &gt; &gt; &nbsp; &nbsp; &gt; gpfs/mount/test matches&quot;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; which seems to be implying that we're failing
on semantic<br>&gt; &gt; consistency.<br>&gt; &gt; &gt; &nbsp; &nbsp; Basically, your 'cat' command is completing
and closing the file,<br>&gt; &gt; &gt; &nbsp; &nbsp; but then a<br>&gt; &gt; &gt; &nbsp; &nbsp; temporally later open of the same find is
reading something other<br>&gt; &gt; &gt; &nbsp; &nbsp; that only the<br>&gt; &gt; &gt; &nbsp; &nbsp; just-written data. &nbsp;My first guess is
that it's a race condition<br>&gt; &gt; &gt; &nbsp; &nbsp; similar to the<br>&gt; &gt; &gt; &nbsp; &nbsp; following: The cat command is causing a write
on one NSD server, and<br>&gt; &gt; &gt; &nbsp; &nbsp; the first<br>&gt; &gt; &gt; &nbsp; &nbsp; grep results in a read from a *different*
NSD server, returning the<br>&gt; &gt; &gt; &nbsp; &nbsp; data that<br>&gt; &gt; &gt; &nbsp; &nbsp; *used* to be in the block because the read
actually happens before<br>&gt; &gt; &gt; &nbsp; &nbsp; the first<br>&gt; &gt; &gt; &nbsp; &nbsp; NSD server actually completes the write.<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; It may be interesting to replace the grep's
with pairs of 'ls -ls /<br>&gt; &gt; &gt; &nbsp; &nbsp; dd' commands to grab the<br>&gt; &gt; &gt; &nbsp; &nbsp; raw data and its size, and check the following:<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; 1) does the size (both blocks allocated and
logical length) reported<br>&gt; &gt; by<br>&gt; &gt; &gt; &nbsp; &nbsp; ls match the amount of data actually read
by the dd?<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; 2) Is the file length as actually read equal
to the written length,<br>&gt; &gt; &gt; &nbsp; &nbsp; or does it<br>&gt; &gt; &gt; &nbsp; &nbsp; overshoot and read all the way to the next
block boundary?<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; 3) If the length is correct, what's wrong
with the data that's<br>&gt; &gt; &gt; &nbsp; &nbsp; telling grep that<br>&gt; &gt; &gt; &nbsp; &nbsp; it's a binary file? &nbsp;( od -cx is your
friend here).<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; 4) If it overshoots, is the remainder all-zeros
(good) or does it<br>&gt; &gt; &gt; &nbsp; &nbsp; return semi-random<br>&gt; &gt; &gt; &nbsp; &nbsp; &quot;what used to be there&quot; data (bad,
due to data exposure issues)?<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; (It's certainly not the most perplexing data
consistency issue I've<br>&gt; &gt; &gt; &nbsp; &nbsp; hit in 4 decades - the<br>&gt; &gt; &gt; &nbsp; &nbsp; winner *has* to be a intermittent data read
corruption on a GPFS 3.5<br>&gt; &gt; &gt; &nbsp; &nbsp; cluster that<br>&gt; &gt; &gt; &nbsp; &nbsp; had us, IBM, SGI, DDN, and at least one vendor
of networking gear<br>&gt; &gt; &gt; &nbsp; &nbsp; all chasing our<br>&gt; &gt; &gt; &nbsp; &nbsp; tails for 18 months before we finally tracked
it down. :)<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; _______________________________________________<br>&gt; &gt; &gt; &nbsp; &nbsp; gpfsug-discuss mailing list<br>&gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; gpfsug-discuss at spectrumscale.org &lt;https://<br>&gt; urldefense.proofpoint.com/v2/url?<br>&gt; u=http-3A__spectrumscale.org&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=jUBFb8C9yai1TUTu1BVnNTNcOnJXGxupWiEKkEjT4pM&amp;e=<br>&gt; &gt;<br>&gt; &gt; &gt; &nbsp; &nbsp; </span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=aVWMptxcCR3po3ijmRweTyjbs1Pp5D7WEiJTYvSYLUk&amp;e=<br>&gt; &gt; &gt; &nbsp; &nbsp; &lt;</span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=aVWMptxcCR3po3ijmRweTyjbs1Pp5D7WEiJTYvSYLUk&amp;e=<br>&gt; &gt;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt;<br>&gt; &gt; &gt; _______________________________________________<br>&gt; &gt; &gt; gpfsug-discuss mailing list<br>&gt; &gt; &gt; gpfsug-discuss at spectrumscale.org<br>&gt; &gt; &gt; </span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=aVWMptxcCR3po3ijmRweTyjbs1Pp5D7WEiJTYvSYLUk&amp;e=<br>&gt; &gt; &gt;<br>&gt; &gt;<br>&gt; &gt; --<br>&gt; &gt; Aaron Knister<br>&gt; &gt; NASA Center for Climate Simulation (Code 606.2)<br>&gt; &gt; Goddard Space Flight Center<br>&gt; &gt; (301) 286-2776<br>&gt; &gt;<br>&gt; &gt; _______________________________________________<br>&gt; &gt; gpfsug-discuss mailing list<br>&gt; &gt; gpfsug-discuss at spectrumscale.org<br>&gt; &gt; </span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=aVWMptxcCR3po3ijmRweTyjbs1Pp5D7WEiJTYvSYLUk&amp;e=<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt;<br>&gt; &gt; ------------------------------<br>&gt; &gt;<br>&gt; &gt; Note: This email is for the confidential use of the named addressee(s)<br>&gt; &gt; only and may contain proprietary, confidential or privileged
information.<br>&gt; &gt; If you are not the intended recipient, you are hereby notified
that any<br>&gt; &gt; review, dissemination or copying of this email is strictly prohibited,
and<br>&gt; &gt; to please notify the sender immediately and destroy this email
and any<br>&gt; &gt; attachments. Email transmission cannot be guaranteed to be secure
or<br>&gt; &gt; error-free. The Company, therefore, does not make any guarantees
as to the<br>&gt; &gt; completeness or accuracy of this email or any attachments. This
email is<br>&gt; &gt; for informational purposes only and does not constitute a recommendation,<br>&gt; &gt; offer, request or solicitation of any kind to buy, sell, subscribe,
redeem<br>&gt; &gt; or perform any type of transaction of a financial product.<br>&gt; &gt;<br>&gt; &gt; _______________________________________________<br>&gt; &gt; gpfsug-discuss mailing list<br>&gt; &gt; gpfsug-discuss at spectrumscale.org<br>&gt; &gt; </span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=aVWMptxcCR3po3ijmRweTyjbs1Pp5D7WEiJTYvSYLUk&amp;e=<br>&gt; &gt;<br>&gt; &gt;<br>&gt; -------------- next part --------------<br>&gt; An HTML attachment was scrubbed...<br>&gt; URL: &lt;</span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20180214_d62fc203_attachment.html&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=nUcKIKr84CRhS0EbxV5vwjSlEr4p3Wf6Is3EDKvOjJg&amp;e=<br>&gt; &gt;<br>&gt; <br>&gt; ------------------------------<br>&gt; <br>&gt; _______________________________________________<br>&gt; gpfsug-discuss mailing list<br>&gt; gpfsug-discuss at spectrumscale.org<br>&gt; </span></tt><a href=https://urldefense.proofpoint.com/v2/url?><tt><span style=" font-size:10pt">https://urldefense.proofpoint.com/v2/url?</span></tt></a><tt><span style=" font-size:10pt"><br>&gt; u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&amp;d=DwICAg&amp;c=jf_iaSHvJObTbx-<br>&gt; siA1ZOg&amp;r=ck4PYlaRFvCcNKlfHMPhoA&amp;m=Jv87Pffe4kSlhiO2NmMbL4HQo_zJ-8s8CkIRy7p92r4&amp;s=aVWMptxcCR3po3ijmRweTyjbs1Pp5D7WEiJTYvSYLUk&amp;e=<br>&gt; <br>&gt; <br>&gt; End of gpfsug-discuss Digest, Vol 73, Issue 36<br>&gt; **********************************************<br>&gt; <br></span></tt><BR>