<font size=2 face="sans-serif">Aaron, brilliant! Your example is close
to the worst case, where every file is 512K+1 bytes and the blocksize is
1024K.<br>Yes, in the worse case 49.99999% of space is "lost" or wasted.
Don't do that!</font><br><br><font size=2 face="sans-serif">One can construct such a worst case
for any system that allocates by blocks or sectors or whatever you want
to call it.</font><br><font size=2 face="sans-serif">Just fill the system with files that
are each 0.5*Block_Size+1 bytes and argue that 1/2 the space is wasted.</font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Aaron Knister <aaron.s.knister@nasa.gov></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif"><gpfsug-discuss@spectrumscale.org></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">11/06/2017 12:10 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
file layout API + file fragmentation</font><br><font size=1 color=#5f5f5f face="sans-serif">Sent by:
</font><font size=1 face="sans-serif">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr noshade><br><br><br><tt><font size=2>Thanks, Frank!<br><br>That's truly fascinating and has some interesting implications that I<br>hadn't thought of before. I just ran a test on an ~8G fs with a block<br>size of 1M:<br><br>for i in `seq 1 100000`; do<br>
dd if=/dev/zero of=foofile${i} bs=520K count=1<br>done<br><br>The fs is "full" according to df/mmdf but there's 3.6G left in
subblocks<br>but yeah, I can't allocate any new files that wouldn't fit into the<br>inode and I can't seem to allocate any new subblocks to existing files<br>(e.g. append).<br><br>What's interesting is if I do the same exercise but with a file size of<br>30K or even 260K I don't seem to run into the same issue. I'm not sure
I<br>understand that yet.<br><br>I was curious about what this meant in the case of appending to a file<br>where the last offset in the file was allocated to a fragment. By<br>looking at "tsdbfs listda" and appending to a file I could see
that the<br>last DA would change (presumably to point to the DA of the start of a<br>contiguous subblock) once the amount of data appended caused the file<br>size to exceed the space available in the trailing subblocks.<br><br>-Aaron<br><br><br>On 11/5/17 7:57 PM, Frank Schmuck wrote:<br>> In GPFS blocks within a file are never fragmented. For example,
if you<br>> have a file of size 7.3 MB and your file system block size is 1MB,
then<br>> this file will be made up of 7 full blocks and one fragment of size
320k<br>> (10 subblocks). Each of the 7 full blocks will be contiguous
on a singe<br>> diks (LUN) behind a single NSD server. The fragment that makes
up the<br>> last part of the file will also be contiguous on a single disk, just<br>> shorter than a full block.<br>> <br>> Frank Schmuck<br>> IBM Almaden Research Center<br>> <br>> <br>> <br>> ----- Original message -----<br>> From: Aaron Knister <aaron.s.knister@nasa.gov><br>> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>> To: <gpfsug-discuss@spectrumscale.org><br>> Cc:<br>> Subject: Re: [gpfsug-discuss] file layout API + file
fragmentation<br>> Date: Sun, Nov 5, 2017 3:39 PM<br>> <br>> Thanks Marc, that helps. I can't easily use tsdbfs for
what I'm working<br>> on since it needs to be run as unprivileged users.<br>> <br>> Perhaps I'm not asking the right question. I'm wondering
how the file<br>> layout api behaves if a given "block"-aligned
offset in a file is made<br>> up of sub-blocks/fragments that are not all on the same
NSD. The<br>> assumption based on how I've seen the API used so far
is that all<br>> sub-blocks within a block at a given offset within a
file are all on the<br>> same NSD.<br>> <br>> -Aaron<br>> <br>> On 11/5/17 6:01 PM, Marc A Kaplan wrote:<br>> > I googled GPFS_FCNTL_GET_DATABLKDISKIDX<br>> ><br>> > and found this discussion:<br>> ><br>> ><br>> </font></tt><a href="https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50"><tt><font size=2>https://www.ibm.com/developerworks/community/forums/html/topic?id=db48b190-4f2f-4e24-a035-25d3e2b06b2d&ps=50</font></tt></a><tt><font size=2><br>> ><br>> > In general, GPFS files ARE deliberately "fragmented"
but we don't say<br>> > that - we say they are "striped" over
many disks -- and that is<br>> > generally a good thing for parallel performance.<br>> ><br>> > Also, in GPFS, if the last would-be block of a
file is less than a<br>> > block, then it is stored in a "fragment"
of a block. <br>> > So you see we use "fragment" to mean
something different than<br>> other file<br>> > systems you may know.<br>> ><br>> > --marc<br>> ><br>> ><br>> ><br>> > From: Aaron Knister
<aaron.s.knister@nasa.gov><br>> > To: gpfsug main
discussion list<br>> <gpfsug-discuss@spectrumscale.org><br>> > Date: 11/04/2017
12:22 PM<br>> > Subject: [gpfsug-discuss]
file layout API + file<br>> fragmentation<br>> > Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>> ><br>> ------------------------------------------------------------------------<br>> ><br>> ><br>> ><br>> > I've got a question about the file layout API and
how it reacts in the<br>> > case of fragmented files.<br>> ><br>> > I'm using the GPFS_FCNTL_GET_DATABLKDISKIDX structure
and have<br>> some code<br>> > based on tsGetDataBlk.C. I'm basing the block size
based off of what's<br>> > returned by filemapOut.blockSize but that only
seems to return a<br>> value ><br>> > 0 when filemapIn.startOffset is 0.<br>> ><br>> > In a case where a file were to be made up of a
significant number of<br>> > non-contiguous fragments (which... would be awful
in of itself) how<br>> > would this be reported by the file layout API?
Does the interface<br>> > technically just report the disk location information
of the first<br>> block<br>> > of the $blockSize range and assume that it's contiguous?<br>> ><br>> > Thanks!<br>> ><br>> > -Aaron<br>> ><br>> > --<br>> > Aaron Knister<br>> > NASA Center for Climate Simulation (Code 606.2)<br>> > Goddard Space Flight Center<br>> > (301) 286-2776<br>> > _______________________________________________<br>> > gpfsug-discuss mailing list<br>> > gpfsug-discuss at spectrumscale.org<br>> ><br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e="><tt><font size=2>https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=wnR7m6d4urZ_8dM4mkHQjMbFD9xJEeesmJyzt1osCnM&s=-dgGO6O5i1EqWj-8MmzjxJ1Iz2I5gT1aRmtyP44Cvdg&e=</font></tt></a><tt><font size=2><br>> ><br>> ><br>> ><br>> ><br>> ><br>> ><br>> > _______________________________________________<br>> > gpfsug-discuss mailing list<br>> > gpfsug-discuss at spectrumscale.org<br>> ><br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e="><tt><font size=2>https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=</font></tt></a><tt><font size=2><br>> ><br>> <br>> --<br>> Aaron Knister<br>> NASA Center for Climate Simulation (Code 606.2)<br>> Goddard Space Flight Center<br>> (301) 286-2776<br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e="><tt><font size=2>https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIF-g&c=jf_iaSHvJObTbx-siA1ZOg&r=ai3ddVzf50ktH78ovGv6NU4O2LZUOWLpiUiggb8lEgA&m=pUdB4fbWLD03ZTAhk9OlpRdIasz628Oa_yG8z8NOjsk&s=kisarJ7IVnyYBx05ZZiGzdwaXnPqNR8UJoywU1OJNRU&e=</font></tt></a><tt><font size=2><br>> <br>> <br>> <br>> <br>> <br>> <br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=_xM9xVsqOuNiCqn3ikx6ZaaIHChTPhz_8iDmEKoteX4&s=uy462L5sxX_3Mm3Dh824ptJIxtah9LVRPMmyKz1lAdg&e="><tt><font size=2>https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=_xM9xVsqOuNiCqn3ikx6ZaaIHChTPhz_8iDmEKoteX4&s=uy462L5sxX_3Mm3Dh824ptJIxtah9LVRPMmyKz1lAdg&e=</font></tt></a><tt><font size=2><br>> <br><br>-- <br>Aaron Knister<br>NASA Center for Climate Simulation (Code 606.2)<br>Goddard Space Flight Center<br>(301) 286-2776<br>_______________________________________________<br>gpfsug-discuss mailing list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=_xM9xVsqOuNiCqn3ikx6ZaaIHChTPhz_8iDmEKoteX4&s=uy462L5sxX_3Mm3Dh824ptJIxtah9LVRPMmyKz1lAdg&e="><tt><font size=2>https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=_xM9xVsqOuNiCqn3ikx6ZaaIHChTPhz_8iDmEKoteX4&s=uy462L5sxX_3Mm3Dh824ptJIxtah9LVRPMmyKz1lAdg&e=</font></tt></a><tt><font size=2><br><br></font></tt><br><br><BR>