Hi Frederick, The ILM active policy (set by mmchpolicy) has an external list rule, the command for the external list runs the mmapplypolicy command. /gpfs1/s3upload/policies/migration.policy has external pool & a migration rule in it. The handler script for the external pool writes the hostname of the server running it out to a file, so that's how I'm trapping which server is running the policy, and that mmapplypolicy is being run.
Hope that explains things, if not let me know and I'll have another try :) Regards, Peter -----Original Message----- From: gpfsug-discuss-boun...@spectrumscale.org [mailto:gpfsug-discuss-boun...@spectrumscale.org] On Behalf Of gpfsug-discuss-requ...@spectrumscale.org Sent: 10 November 2017 16:43 To: gpfsug-discuss@spectrumscale.org Subject: gpfsug-discuss Digest, Vol 70, Issue 32 Send gpfsug-discuss mailing list submissions to gpfsug-discuss@spectrumscale.org To subscribe or unsubscribe via the World Wide Web, visit http://gpfsug.org/mailman/listinfo/gpfsug-discuss or, via email, send a message with subject or body 'help' to gpfsug-discuss-requ...@spectrumscale.org You can reach the person managing the list at gpfsug-discuss-ow...@spectrumscale.org When replying, please edit your Subject line so it is more specific than "Re: Contents of gpfsug-discuss digest..." Today's Topics: 1. Re: Spectrum Scale with NVMe (John Hearns) 2. Specifying nodes in commands (Chase, Peter) 3. Re: Specifying nodes in commands (Frederick Stock) 4. Re: Specifying nodes in commands (Marc A Kaplan) ---------------------------------------------------------------------- Message: 1 Date: Fri, 10 Nov 2017 13:25:40 +0000 From: John Hearns <john.hea...@asml.com> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] Spectrum Scale with NVMe Message-ID: <he1pr02mb1450ce9f60f877d4c010fd8c88...@he1pr02mb1450.eurprd02.prod.outlook.com> Content-Type: text/plain; charset="us-ascii" Chad, Thankyou for the reply. Indded I had that issue - I only noticed because I looked at the utisation of the NSDs and a set of them were not being filled with data... A set which were coincidentally all connected to the same server (me whistles innocently....) -----Original Message----- From: gpfsug-discuss-boun...@spectrumscale.org [mailto:gpfsug-discuss-boun...@spectrumscale.org] On Behalf Of Chad Kerner Sent: Tuesday, November 07, 2017 2:05 PM To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] Spectrum Scale with NVMe Hey John, Once you get /var/mmfs/etc/nsddevices set up, it is all straight forward. We have seen times on reboot where the devices were not ready before gpfs started and the file system started with those disks in an offline state. But, that was just a timing issue with the startup. Chad -- Chad Kerner, Senior Storage Engineer Storage Enabling Technologies National Center for Supercomputing Applications University of Illinois, Urbana-Champaign On 11/7/17, John Hearns <john.hea...@asml.com> wrote: > I am looking for anyone with experience of using Spectrum Scale with > nvme devices. > > I could use an offline brain dump... > > > The specific issue I have is with the nsd device discovery and the naming. > > Before anyone replies, I am gettign excellent support from IBM and > have been directed to the correct documentation. > > I am just looking for any wrinkles or tips that anyone has. > > > Thanks > > -- The information contained in this communication and any attachments > is confidential and may be privileged, and is for the sole use of the > intended recipient(s). Any unauthorized review, use, disclosure or > distribution is prohibited. Unless explicitly stated otherwise in the > body of this communication or the attachment thereto (if any), the > information is provided on an AS-IS basis without any express or > implied warranties or liabilities. To the extent you are relying on > this information, you are doing so at your own risk. If you are not > the intended recipient, please notify the sender immediately by > replying to this message and destroy all copies of this message and > any attachments. Neither the sender nor the company/group of companies > he or she represents shall be liable for the proper and complete > transmission of the information contained in this communication, or for any > delay in its receipt. > -- -- Chad Kerner, Senior Storage Engineer Storage Enabling Technologies National Center for Supercomputing Applications University of Illinois, Urbana-Champaign _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=01%7C01%7Cjohn.hearns%40asml.com%7Ce3875dc1def842e88ee308d525e01e80%7Caf73baa8f5944eb2a39d93e96cad61fc%7C1&sdata=pwti5NtVf7c4SClTUc1PWNz5YW4QHWjM5%2F%2BGLdYHoqQ%3D&reserved=0 -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. Neither the sender nor the company/group of companies he or she represents shall be liable for the proper and complete transmission of the information contained in this communication, or for any delay in its receipt. ------------------------------ Message: 2 Date: Fri, 10 Nov 2017 16:18:36 +0000 From: "Chase, Peter" <peter.ch...@metoffice.gov.uk> To: "'gpfsug-discuss@spectrumscale.org'" <gpfsug-discuss@spectrumscale.org> Subject: [gpfsug-discuss] Specifying nodes in commands Message-ID: <f4c2aa4ab5d84b4a8046e4cedd08253e43804...@exxcmpd1dag1.cmpd1.metoffice.gov.uk> Content-Type: text/plain; charset="iso-8859-1" Hello all, I'm running a script triggered from an ILM external list rule. The script has the following command in, and it isn't work as I'd expect: /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -N cloudNode -P /gpfs1/s3upload/policies/migration.policy --scope fileset I'd expect the mmapplypolicy command to run the policy on all the nodes in the cloudNode class, but it doesn't, it runs on the node that triggered the script. However, the following command does work as I'd expect: /usr/lpp/mmfs/bin/mmdsh -N cloudNode /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -P /gpfs1/s3upload/policies/migration.policy --scope fileset Can any one shed any light on this? Have I just misconstrued how mmapplypolicy works? Regards, Peter Chase GPCS Team Met Office? FitzRoy Road? Exeter? Devon? EX1 3PB? United Kingdom Tel: +44 (0)1392 886921 Email: peter.ch...@metoffice.gov.uk Website: www.metoffice.gov.uk ------------------------------ Message: 3 Date: Fri, 10 Nov 2017 11:41:19 -0500 From: "Frederick Stock" <sto...@us.ibm.com> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] Specifying nodes in commands Message-ID: <of70ba2ef9.4c554f8a-on852581d4.005b8bc3-852581d4.005ba...@notes.na.collabserv.com> Content-Type: text/plain; charset="iso-8859-1" How do you determine if mmapplypolicy is running on a node? Normally mmapplypolicy as a process runs on a single node but its helper processes, policy-help or something similar, run on all the nodes which are referenced by the -N option. Fred __________________________________________________ Fred Stock | IBM Pittsburgh Lab | 720-430-8821 sto...@us.ibm.com From: "Chase, Peter" <peter.ch...@metoffice.gov.uk> To: "'gpfsug-discuss@spectrumscale.org'" <gpfsug-discuss@spectrumscale.org> Date: 11/10/2017 11:18 AM Subject: [gpfsug-discuss] Specifying nodes in commands Sent by: gpfsug-discuss-boun...@spectrumscale.org Hello all, I'm running a script triggered from an ILM external list rule. The script has the following command in, and it isn't work as I'd expect: /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -N cloudNode -P /gpfs1/s3upload/policies/migration.policy --scope fileset I'd expect the mmapplypolicy command to run the policy on all the nodes in the cloudNode class, but it doesn't, it runs on the node that triggered the script. However, the following command does work as I'd expect: /usr/lpp/mmfs/bin/mmdsh -N cloudNode /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -P /gpfs1/s3upload/policies/migration.policy --scope fileset Can any one shed any light on this? Have I just misconstrued how mmapplypolicy works? Regards, Peter Chase GPCS Team Met Office? FitzRoy Road? Exeter? Devon? EX1 3PB? United Kingdom Tel: +44 (0)1392 886921 Email: peter.ch...@metoffice.gov.uk Website: www.metoffice.gov.uk _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=p_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw&m=spXDnba2A_tVauiszV7sXhSkn6GeEljABN4lUEB4f8s&s=1Hd1SNkXtfLRcirmeRfg1JuAERuhbyiVqsLEdYlhFsM&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20171110/3b1dc733/attachment-0001.html> ------------------------------ Message: 4 Date: Fri, 10 Nov 2017 11:42:28 -0500 From: "Marc A Kaplan" <makap...@us.ibm.com> To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] Specifying nodes in commands Message-ID: <of95ae7ceb.8278f465-on852581d4.005a9e64-852581d4.005bc...@notes.na.collabserv.com> Content-Type: text/plain; charset="iso-8859-1" mmapplypolicy ... -N nodeClass ... will use the nodes in nodeClass as helper nodes to get its work done. mmdsh -N nodeClass command ... will run the SAME command on each of the nodes -- probably not what you want to do with mmapplypolicy. To see more about what mmapplypolicy is doing use options -d 1 (debug info) If you are using -N because you have a lot of files to process, you should also use -g /some-gpfs-temp-directory (see doc) If you are running a small test case, it may happen that you don't see the helper nodes doing anything, because there's not enough time and work to get them going... For test purposes you can coax the helper nodes into action with: options -B 1 -m 1 so that each helper node only does one file at a time. From: "Chase, Peter" <peter.ch...@metoffice.gov.uk> To: "'gpfsug-discuss@spectrumscale.org'" <gpfsug-discuss@spectrumscale.org> Date: 11/10/2017 11:18 AM Subject: [gpfsug-discuss] Specifying nodes in commands Sent by: gpfsug-discuss-boun...@spectrumscale.org Hello all, I'm running a script triggered from an ILM external list rule. The script has the following command in, and it isn't work as I'd expect: /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -N cloudNode -P /gpfs1/s3upload/policies/migration.policy --scope fileset I'd expect the mmapplypolicy command to run the policy on all the nodes in the cloudNode class, but it doesn't, it runs on the node that triggered the script. However, the following command does work as I'd expect: /usr/lpp/mmfs/bin/mmdsh -N cloudNode /usr/lpp/mmfs/bin/mmapplypolicy /gpfs1/aws -P /gpfs1/s3upload/policies/migration.policy --scope fileset Can any one shed any light on this? Have I just misconstrued how mmapplypolicy works? Regards, Peter Chase GPCS Team Met Office? FitzRoy Road? Exeter? Devon? EX1 3PB? United Kingdom Tel: +44 (0)1392 886921 Email: peter.ch...@metoffice.gov.uk Website: www.metoffice.gov.uk _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=WjhVVKkS23BlFGP2KHmkndM0AZ4yB2aC81UUHv8iIZs&s=-dPme1SlhBAqo45xVmtvVWNeAjumd7JrtEksW1U8o5w&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20171110/725a922c/attachment.html> ------------------------------ _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss End of gpfsug-discuss Digest, Vol 70, Issue 32 ********************************************** _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
