<font size=2 face="sans-serif">Jaime,</font><br><br><font size=2 face="sans-serif"> While we're waiting for the mmbackup
expert to weigh in, notice that the mmbackup command does have a -P option
that allows you to provide a customized policy rules file.</font><br><font size=2 face="sans-serif"><br>So... a fairly safe hack is to do a trial mmbackup run, capture the automatically
generated policy file, and then augment it with FOR FILESET('fileset-I-want-to-backup')
clauses.... Then run the mmbackup for real with your customized policy
file.<br></font><br><font size=2 face="sans-serif">mmbackup uses mmapplypolicy which by
itself is happy to limit its directory scan to a particular fileset by
using</font><br><br><font size=2 face="sans-serif">mmapplypolicy /path-to-any-directory-within-a-gpfs-filesystem
--scope fileset ....</font><br><br><font size=2 face="sans-serif">However, mmbackup probably has other
worries and for simpliciity and helping make sure you get complete, sensible
backups, apparently has imposed some restrictions to preserve sanity (yours
and our support team! ;-) ) ... (For example, suppose you
were doing incremental backups, starting at different paths each time?
-- happy to do so, but when disaster strikes and you want to restore --
you'll end up confused and/or unhappy!)</font><br><br><font size=2 face="sans-serif">"converting from one fileset to
another" --- sorry there is no such thing. Filesets are kinda
like little filesystems within filesystems. Moving a file from one
fileset to another requires a copy operation. There is no fast move
nor hardlinking. </font><br><br><font size=2 face="sans-serif">--marc</font><br><br><br><br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">"Jaime Pinto"
<pinto@scinet.utoronto.ca></font><br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">"gpfsug main discussion
list" <gpfsug-discuss@spectrumscale.org>, "Marc
A Kaplan" <makaplan@us.ibm.com></font><br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">05/18/2017 09:58 AM</font><br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [gpfsug-discuss]
What is an independent fileset? was: mmbackup with
fileset : scope errors</font><br><hr noshade><br><br><br><tt><font size=2>Thanks for the explanation Mark and Luis,<br><br>It begs the question: why filesets are created as dependent by <br>default, if the adverse repercussions can be so great afterward? Even <br>in my case, where I manage GPFS and TSM deployments (and I have been <br>around for a while), didn't realize at all that not adding and extra <br>option at fileset creation time would cause me huge trouble with <br>scaling later on as I try to use mmbackup.<br><br>When you have different groups to manage file systems and backups that
<br>don't read each-other's manuals ahead of time then we have a really <br>bad recipe.<br><br>I'm looking forward to your explanation as to why mmbackup cares one <br>way or another.<br><br>I'm also hoping for a hint as to how to configure backup exclusion <br>rules on the TSM side to exclude fileset traversing on the GPFS side. <br>Is mmbackup smart enough (actually smarter than TSM client itself) to <br>read the exclusion rules on the TSM configuration and apply them <br>before traversing?<br><br>Thanks<br>Jaime<br><br>Quoting "Marc A Kaplan" <makaplan@us.ibm.com>:<br><br>> When I see "independent fileset" (in Spectrum/GPFS/Scale)
I always think<br>> and try to read that as "inode space".<br>><br>> An "independent fileset" has all the attributes of an (older-fashioned)<br>> dependent fileset PLUS all of its files are represented by inodes
that are<br>> in a separable range of inode numbers - this allows GPFS to efficiently
do<br>> snapshots of just that inode-space (uh... independent fileset)...<br>><br>> And... of course the files of dependent filesets must also be represented<br>> by inodes -- those inode numbers are within the inode-space of whatever<br>> the containing independent fileset is... as was chosen when you created<br>> the fileset.... If you didn't say otherwise, inodes come from
the<br>> default "root" fileset....<br>><br>> Clear as your bath-water, no?<br>><br>> So why does mmbackup care one way or another ??? Stay tuned....<br>><br>> BTW - if you look at the bits of the inode numbers carefully --- you
may<br>> not immediately discern what I mean by a "separable range of
inode<br>> numbers" -- (very technical hint) you may need to permute the
bit order<br>> before you discern a simple pattern...<br>><br>><br>><br>> From: "Luis Bolinches" <luis.bolinches@fi.ibm.com><br>> To: gpfsug-discuss@spectrumscale.org<br>> Cc: gpfsug-discuss@spectrumscale.org<br>> Date: 05/18/2017 02:10 AM<br>> Subject: Re: [gpfsug-discuss] mmbackup
with fileset : scope errors<br>> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>><br>><br>><br>> Hi<br>><br>> There is no direct way to convert the one fileset that is dependent
to<br>> independent or viceversa.<br>><br>> I would suggest to take a look to chapter 5 of the 2014 redbook, lots
of<br>> definitions about GPFS ILM including filesets<br>> </font></tt><a href=http://www.redbooks.ibm.com/abstracts/sg248254.html?Open><tt><font size=2>http://www.redbooks.ibm.com/abstracts/sg248254.html?Open</font></tt></a><tt><font size=2>Is not the only<br>> place that is explained but I honestly believe is a good single start<br>> point. It also needs an update as does nto have anything on CES nor
ESS,<br>> so anyone in this list feel free to give feedback on that page people
with<br>> funding decisions listen there.<br>><br>> So you are limited to either migrate the data from that fileset to
a new<br>> independent fileset (multiple ways to do that) or use the TSM client<br>> config.<br>><br>> ----- Original message -----<br>> From: "Jaime Pinto" <pinto@scinet.utoronto.ca><br>> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>> To: "gpfsug main discussion list" <gpfsug-discuss@spectrumscale.org>,<br>> "Jaime Pinto" <pinto@scinet.utoronto.ca><br>> Cc:<br>> Subject: Re: [gpfsug-discuss] mmbackup with fileset : scope errors<br>> Date: Thu, May 18, 2017 4:43 AM<br>><br>> There is hope. See reference link below:<br>> </font></tt><a href=https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.1.1/com.ibm.spectrum.scale.v4r11.ins.doc/bl1ins_tsm_fsvsfset.htm><tt><font size=2>https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.1.1/com.ibm.spectrum.scale.v4r11.ins.doc/bl1ins_tsm_fsvsfset.htm</font></tt></a><tt><font size=2><br>><br>><br>> The issue has to do with dependent vs. independent filesets, something<br>> I didn't even realize existed until now. Our filesets are dependent<br>> (for no particular reason), so I have to find a way to turn them into<br>> independent.<br>><br>> The proper option syntax is "--scope inodespace", and the
error<br>> message actually flagged that out, however I didn't know how to<br>> interpret what I saw:<br>><br>><br>> # mmbackup /gpfs/sgfs1/sysadmin3 -N tsm-helper1-ib0 -s /dev/shm<br>> --scope inodespace --tsm-errorlog $logfile -L 2<br>> --------------------------------------------------------<br>> mmbackup: Backup of /gpfs/sgfs1/sysadmin3 begins at Wed May 17<br>> 21:27:43 EDT 2017.<br>> --------------------------------------------------------<br>> Wed May 17 21:27:45 2017 mmbackup:mmbackup: Backing up *dependent*<br>> fileset sysadmin3 is not supported<br>> Wed May 17 21:27:45 2017 mmbackup:This fileset is not suitable for<br>> fileset level backup. exit 1<br>> --------------------------------------------------------<br>><br>> Will post the outcome.<br>> Jaime<br>><br>><br>><br>> Quoting "Jaime Pinto" <pinto@scinet.utoronto.ca>:<br>><br>>> Quoting "Luis Bolinches" <luis.bolinches@fi.ibm.com>:<br>>><br>>>> Hi<br>>>><br>>>> have you tried to add exceptions on the TSM client config
file?<br>>><br>>> Hey Luis,<br>>><br>>> That would work as well (mechanically), however it's not elegant
or<br>>> efficient. When you have over 1PB and 200M files on scratch it
will<br>>> take many hours and several helper nodes to traverse that fileset
just<br>>> to be negated by TSM. In fact exclusion on TSM are just as inefficient.<br>>> Considering that I want to keep project and sysadmin on different<br>>> domains then it's much worst, since we have to traverse and exclude<br>>> scratch & (project|sysadmin) twice, once to capture sysadmin
and again<br>>> to capture project.<br>>><br>>> If I have to use exclusion rules it has to rely sole on gpfs rules,
and<br>>> somehow not traverse scratch at all.<br>>><br>>> I suspect there is a way to do this properly, however the examples
on<br>>> the gpfs guide and other references are not exhaustive. They only
show<br>>> a couple of trivial cases.<br>>><br>>> However my situation is not unique. I suspect there are may facilities<br>>> having to deal with backup of HUGE filesets.<br>>><br>>> So the search is on.<br>>><br>>> Thanks<br>>> Jaime<br>>><br>>><br>>><br>>><br>>>><br>>>> Assuming your GPFS dir is /IBM/GPFS and your fileset to exclude
is<br>> linked<br>>>> on /IBM/GPFS/FSET1<br>>>><br>>>> dsm.sys<br>>>> ...<br>>>><br>>>> DOMAIN /IBM/GPFS<br>>>> EXCLUDE.DIR /IBM/GPFS/FSET1<br>>>><br>>>><br>>>> From: "Jaime Pinto" <pinto@scinet.utoronto.ca><br>>>> To: "gpfsug main discussion list"<br>> <gpfsug-discuss@spectrumscale.org><br>>>> Date: 17-05-17 23:44<br>>>> Subject: [gpfsug-discuss] mmbackup
with fileset : scope errors<br>>>> Sent by: gpfsug-discuss-bounces@spectrumscale.org<br>>>><br>>>><br>>>><br>>>> I have a g200 /gpfs/sgfs1 filesystem with 3 filesets:<br>>>> * project3<br>>>> * scratch3<br>>>> * sysadmin3<br>>>><br>>>> I have no problems mmbacking up /gpfs/sgfs1 (or sgfs1), however
we<br>>>> have no need or space to include *scratch3* on TSM.<br>>>><br>>>> Question: how to craft the mmbackup command to backup<br>>>> /gpfs/sgfs1/project3 and/or /gpfs/sgfs1/sysadmin3 only?<br>>>><br>>>> Below are 3 types of errors:<br>>>><br>>>> 1) mmbackup /gpfs/sgfs1/sysadmin3 -N tsm-helper1-ib0 -s /dev/shm<br>>>> --tsm-errorlog $logfile -L 2<br>>>><br>>>> ERROR: mmbackup: Options /gpfs/sgfs1/sysadmin3 and --scope
filesystem<br>>>> cannot be specified at the same time.<br>>>><br>>>> 2) mmbackup /gpfs/sgfs1/sysadmin3 -N tsm-helper1-ib0 -s /dev/shm<br>>>> --scope inodespace --tsm-errorlog $logfile -L 2<br>>>><br>>>> ERROR: Wed May 17 16:27:11 2017 mmbackup:mmbackup: Backing
up<br>>>> dependent fileset sysadmin3 is not supported<br>>>> Wed May 17 16:27:11 2017 mmbackup:This fileset is not suitable
for<br>>>> fileset level backup. exit 1<br>>>><br>>>> 3) mmbackup /gpfs/sgfs1/sysadmin3 -N tsm-helper1-ib0 -s /dev/shm<br>>>> --scope filesystem --tsm-errorlog $logfile -L 2<br>>>><br>>>> ERROR: mmbackup: Options /gpfs/sgfs1/sysadmin3 and --scope
filesystem<br>>>> cannot be specified at the same time.<br>>>><br>>>> These examples don't really cover my case:<br>>>><br>> </font></tt><a href=https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_mmbackup.htm#mmbackup__mmbackup_examples><tt><font size=2>https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1adm_mmbackup.htm#mmbackup__mmbackup_examples</font></tt></a><tt><font size=2><br>><br>>>><br>>>><br>>>> Thanks<br>>>> Jaime<br>>>><br>>>><br>>>> ************************************<br>>>> TELL US ABOUT YOUR SUCCESS
STORIES<br>>>> </font></tt><a href=http://www.scinethpc.ca/testimonials><tt><font size=2>http://www.scinethpc.ca/testimonials</font></tt></a><tt><font size=2><br>>>> ************************************<br>>>> ---<br>>>> Jaime Pinto<br>>>> SciNet HPC Consortium - Compute/Calcul Canada<br>>>> </font></tt><a href=www.scinet.utoronto.ca><tt><font size=2>www.scinet.utoronto.ca</font></tt></a><tt><font size=2>- </font></tt><a href=www.computecanada.ca><tt><font size=2>www.computecanada.ca</font></tt></a><tt><font size=2><br>>>> University of Toronto<br>>>> 661 University Ave. (MaRS), Suite 1140<br>>>> Toronto, ON, M5G1M1<br>>>> P: 416-978-2755<br>>>> C: 416-505-1477<br>>>><br>>>> ----------------------------------------------------------------<br>>>> This message was sent using IMP at SciNet Consortium, University
of<br>>>> Toronto.<br>>>><br>>>> _______________________________________________<br>>>> gpfsug-discuss mailing list<br>>>> gpfsug-discuss at spectrumscale.org<br>>>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>>>><br>>>><br>>>><br>>>><br>>>><br>>>> Ellei edell ole toisin mainittu: / Unless stated otherwise
above:<br>>>> Oy IBM Finland Ab<br>>>> PL 265, 00101 Helsinki, Finland<br>>>> Business ID, Y-tunnus: 0195876-3<br>>>> Registered in Finland<br>>>><br>>><br>>><br>>><br>>><br>>><br>>><br>>> ************************************<br>>> TELL US ABOUT YOUR SUCCESS
STORIES<br>>> </font></tt><a href=http://www.scinethpc.ca/testimonials><tt><font size=2>http://www.scinethpc.ca/testimonials</font></tt></a><tt><font size=2><br>>> ************************************<br>>> ---<br>>> Jaime Pinto<br>>> SciNet HPC Consortium - Compute/Calcul Canada<br>>> </font></tt><a href=www.scinet.utoronto.ca><tt><font size=2>www.scinet.utoronto.ca</font></tt></a><tt><font size=2>- </font></tt><a href=www.computecanada.ca><tt><font size=2>www.computecanada.ca</font></tt></a><tt><font size=2><br>>> University of Toronto<br>>> 661 University Ave. (MaRS), Suite 1140<br>>> Toronto, ON, M5G1M1<br>>> P: 416-978-2755<br>>> C: 416-505-1477<br>>><br>>> ----------------------------------------------------------------<br>>> This message was sent using IMP at SciNet Consortium, University
of<br>> Toronto.<br>>><br>>> _______________________________________________<br>>> gpfsug-discuss mailing list<br>>> gpfsug-discuss at spectrumscale.org<br>>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>>><br>><br>><br>><br>><br>><br>><br>> ************************************<br>> TELL US ABOUT YOUR SUCCESS
STORIES<br>> </font></tt><a href=http://www.scinethpc.ca/testimonials><tt><font size=2>http://www.scinethpc.ca/testimonials</font></tt></a><tt><font size=2><br>> ************************************<br>> ---<br>> Jaime Pinto<br>> SciNet HPC Consortium - Compute/Calcul Canada<br>> </font></tt><a href=www.scinet.utoronto.ca><tt><font size=2>www.scinet.utoronto.ca</font></tt></a><tt><font size=2>- </font></tt><a href=www.computecanada.ca><tt><font size=2>www.computecanada.ca</font></tt></a><tt><font size=2><br>> University of Toronto<br>> 661 University Ave. (MaRS), Suite 1140<br>> Toronto, ON, M5G1M1<br>> P: 416-978-2755<br>> C: 416-505-1477<br>><br>> ----------------------------------------------------------------<br>> This message was sent using IMP at SciNet Consortium, University of<br>> Toronto.<br>><br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>><br>><br>><br>> Ellei edell ole toisin mainittu: / Unless stated otherwise above:<br>> Oy IBM Finland Ab<br>> PL 265, 00101 Helsinki, Finland<br>> Business ID, Y-tunnus: 0195876-3<br>> Registered in Finland<br>> _______________________________________________<br>> gpfsug-discuss mailing list<br>> gpfsug-discuss at spectrumscale.org<br>> </font></tt><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss"><tt><font size=2>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</font></tt></a><tt><font size=2><br>><br>><br>><br>><br>><br><br><br><br><br><br><br> ************************************<br> TELL US ABOUT YOUR SUCCESS STORIES<br> </font></tt><a href=http://www.scinethpc.ca/testimonials><tt><font size=2>http://www.scinethpc.ca/testimonials</font></tt></a><tt><font size=2><br> ************************************<br>---<br>Jaime Pinto<br>SciNet HPC Consortium - Compute/Calcul Canada<br></font></tt><a href=www.scinet.utoronto.ca><tt><font size=2>www.scinet.utoronto.ca</font></tt></a><tt><font size=2>- </font></tt><a href=www.computecanada.ca><tt><font size=2>www.computecanada.ca</font></tt></a><tt><font size=2><br>University of Toronto<br>661 University Ave. (MaRS), Suite 1140<br>Toronto, ON, M5G1M1<br>P: 416-978-2755<br>C: 416-505-1477<br><br>----------------------------------------------------------------<br>This message was sent using IMP at SciNet Consortium, University of Toronto.<br><br></font></tt><br><br><BR>