[gpfsug-discuss] Antwort: [Newsletter] Re: Combine different rules - tip: use mmfind & co; FOR FILESET; FILESET_NAME
mmfind or rather the convert-script is great! Thanks, Matthias Von:"Marc A Kaplan"An: gpfsug main discussion list Datum: 01.11.2017 15:43 Betreff:[Newsletter] Re: [gpfsug-discuss] Combine different rules - tip: use mmfind & co; FOR FILESET; FILESET_NAME Gesendet von: gpfsug-discuss-boun...@spectrumscale.org Thanks Jonathan B for your comments and tips on experience using mmapplypolicy and policy rules. Good to see that some of the features we put into the product are actually useful. For those not quite as familiar, and have come somewhat later to the game, like Matthias K - I have a few remarks and tips that may be helpful: You can think of and use mmapplypolicy as a fast, parallelized version of the classic `find ... | xargs ... ` pipeline. In fact we've added some "sample" scripts with options that make this easy: samples/ilm/mmfind : "understands" the classic find search arguments as well as all the mmapplypolicy options and the recent versions also support an -xargs option so you can write the classic pipepline as one command: mmfind ... -xargs ... There are debug/diagnostic options so you can see the underlying GPFS commands and policy rules that are generated, so if mmfind doesn't do exactly what you were hoping, you can capture the commands and rules that it does do and tweak/hack those. Two of the most crucial and tricky parts of mmfind are available as separate scripts that can be used separately: tr_findToPol.pl : convert classic options to policy rules. mmxargs : 100% correctly deal with the problem of whitespace and/or "special" characters in the pathnames output as file lists by mmapplypolicy. This is somewhat tricky. EVEN IF you've already worked out your own policy rules and use policy RULE ... EXTERNAL ... EXEC 'myscript' you may want to use mmxargs or "lift" some of the code there-in -- because it is very likely your 'myscript' is not handling the problem of special characters correctly. FILESETs vs POOLs - yes these are "orthogonal" concepts in GPFS (Spectrum Scale!) BUT some customer/admins may choose to direct GPFS to assign to POOL based on FILESET using policy rules clauses like: FOR FILESET('a_fs', 'b_fs') /* handy to restrict a rule to one or a few filesets */ WHERE ... AND (FILESET_NAME LIKE 'xyz_%') AND ... /* restrict to filesets whose name matches a pattern */ -- marc of GPFS___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
[gpfsug-discuss] Antwort: [Newsletter] Re: Combine different rules - tip: use mmfind & co; FOR FILESET; FILESET_NAME
Thanks for this tip. I will try these commands and give feedback in the next week. Matthias Von:"Marc A Kaplan"An: gpfsug main discussion list Datum: 01.11.2017 15:43 Betreff:[Newsletter] Re: [gpfsug-discuss] Combine different rules - tip: use mmfind & co; FOR FILESET; FILESET_NAME Gesendet von: gpfsug-discuss-boun...@spectrumscale.org Thanks Jonathan B for your comments and tips on experience using mmapplypolicy and policy rules. Good to see that some of the features we put into the product are actually useful. For those not quite as familiar, and have come somewhat later to the game, like Matthias K - I have a few remarks and tips that may be helpful: You can think of and use mmapplypolicy as a fast, parallelized version of the classic `find ... | xargs ... ` pipeline. In fact we've added some "sample" scripts with options that make this easy: samples/ilm/mmfind : "understands" the classic find search arguments as well as all the mmapplypolicy options and the recent versions also support an -xargs option so you can write the classic pipepline as one command: mmfind ... -xargs ... There are debug/diagnostic options so you can see the underlying GPFS commands and policy rules that are generated, so if mmfind doesn't do exactly what you were hoping, you can capture the commands and rules that it does do and tweak/hack those. Two of the most crucial and tricky parts of mmfind are available as separate scripts that can be used separately: tr_findToPol.pl : convert classic options to policy rules. mmxargs : 100% correctly deal with the problem of whitespace and/or "special" characters in the pathnames output as file lists by mmapplypolicy. This is somewhat tricky. EVEN IF you've already worked out your own policy rules and use policy RULE ... EXTERNAL ... EXEC 'myscript' you may want to use mmxargs or "lift" some of the code there-in -- because it is very likely your 'myscript' is not handling the problem of special characters correctly. FILESETs vs POOLs - yes these are "orthogonal" concepts in GPFS (Spectrum Scale!) BUT some customer/admins may choose to direct GPFS to assign to POOL based on FILESET using policy rules clauses like: FOR FILESET('a_fs', 'b_fs') /* handy to restrict a rule to one or a few filesets */ WHERE ... AND (FILESET_NAME LIKE 'xyz_%') AND ... /* restrict to filesets whose name matches a pattern */ -- marc of GPFS___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss