Re: [gpfsug-discuss] : Re: AFM with Object Storage - fails with invalid skey (secret key)

Hi Pinkesh, You could open a ticket to get the efix. ~Venkat (vpuvv...@in.ibm.com) From: "Pinkesh Valdria" To: "Venkateswara R Puvvada" , "gpfsug main discussion list" Date: 11/12/2021 05:57 PM Subject:Re: [External] : Re: [gpfsug-discuss] AF

Re: [gpfsug-discuss] AFM with Object Storage - fails with invalid skey (secret key)

Hi, AFM does not accept character '=' as part of access and secret keys. It matches the keys with below expression "$KEY" =~ ^[0-9a-zA-Z/+._]+$ We will fix it to accept other allowed characters in future releases including char '=', for now generate secret key without '=' char. ~Venkat

Re: [gpfsug-discuss] SpectrumScale / AFM / Singularity soft lockups

Horton To: "gpfsug-discuss@spectrumscale.org" Date: 03/24/2021 12:33 AM Subject:[EXTERNAL] Re: [gpfsug-discuss] SpectrumScale / AFM / Singularity soft lockups Sent by:gpfsug-discuss-boun...@spectrumscale.org Hi, Sorry for the delay... On Fri, 2021-03-19 at 12:02 +0530

Re: [gpfsug-discuss] SpectrumScale / AFM / Singularity soft lockups

reproduce it in a non-AFM fileset. Rob On Fri, 2021-03-19 at 12:02 +0530, Venkateswara R Puvvada wrote: CAUTION: This email originated from outside of the ICR. Do not click links or open attachments unless you recognize the sender's email address and know the content is safe. Robert, What is

Re: [gpfsug-discuss] SpectrumScale / AFM / Singularity soft lockups

Robert, What is the scale version ? This issue may be related to these alerts. https://www.ibm.com/support/pages/node/6355983 https://www.ibm.com/support/pages/node/6380740 These are the recommended steps to resolve the issue, but need more details on the scale version. 1. Stop all AFM

Re: [gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over NFS?

kat, If ACLs and other EAs migration from non scale is not supported by AFM, is there any 3rd party tool that could complement that when paired with AFM? On Tue, Nov 24, 2020 at 10:41 AM Venkateswara R Puvvada < vpuvv...@in.ibm.com> wrote: AFM provides near zero downtime for migration. As of tod

Re: [gpfsug-discuss] Migrate/syncronize data from Isilon to Scale over NFS?

AFM provides near zero downtime for migration. As of today, AFM migration does not support ACLs or other EAs migration from non scale (GPFS) source. https://www.ibm.com/support/knowledgecenter/STXKQY_5.1.0/com.ibm.spectrum.scale.v5r10.doc/bl1ins_uc_migrationusingafmmigrationenhancements.htm

Re: [gpfsug-discuss] AFM experiences?

Dean, This is one of the corner case which is associated with sparse files at the home cluster. You could try with latest versions of scale, AFM indepedent-writer mode have many performance/functional improvements in newer releases. ~Venkat (vpuvv...@in.ibm.com) From: "Flanders, Dean"

Re: [gpfsug-discuss] AFM experiences?

>- Quota management on the home cluster - we need a way of ensuring >people don't write data to the cache which can't be accomodated on >home. Probably not insurmountable but needs a bit of thought... You could set same quotas between cache and home clusters. AFM does not support replication of

Re: [gpfsug-discuss] Checking if a AFM-managed file is still inflight

tspcacheutil , this command provides information about the file's replication state. You can also run policy to find these files. Example: tspcacheutil /gpfs/gpfs1/sw2/1.txt inode: ino=524290 gen=235142808 uid=1000 gid=0 size=3 mode=0200100777 nlink=1 ctime=1600366912.382081156

Re: [gpfsug-discuss] AFM cache rolling upgrade with minimal impact / no directory scan

Billich, >The cache filesets holds about 500M used inodes. Does a specific procedure exist, or is it good enough to just shutdown scale on the node I want to update? And maybe flush >the queues first as far as possible? It is recommended to stop (mmafmctl device stop) the filesets and

Re: [gpfsug-discuss] Tuning Spectrum Scale AFM for stability?

Hi, What is lock down of AFM fileset ? Are the messages in requeued state and AFM won't replicate any data ? I would recommend opening a ticket by collecting the logs and internaldump from the gateway node when the replication is stuck. You can also try increasing the value of

Re: [gpfsug-discuss] afmHashVersion

afmHashVersion=5 does not cause any performance degradation, this hash version allows assigning a gateway for the fileset using mmchfileset command. This option is not required for AFM home cluster(assuming that home is not a cache for other home). It is needed only at the AFM cache cluster

Re: [gpfsug-discuss] AFM gateway node scaling

somewhere. On 3/25/20 11:50 AM, Venkateswara R Puvvada wrote: Matt, It is recommended to have dedicated AFM gateway nodes. Memory and CPU requirements for AFM gateway node depends on the number of filesets handled by the node and the inode usage of those filesets. Since AFM keeps track of changes

Re: [gpfsug-discuss] AFM gateway node scaling

Matt, It is recommended to have dedicated AFM gateway nodes. Memory and CPU requirements for AFM gateway node depends on the number of filesets handled by the node and the inode usage of those filesets. Since AFM keeps track of changes in the memory, any network disturbance can cause the

Re: [gpfsug-discuss] AFM Alternative? Aspera?

Transferring the small files with AFM + NFS over high latency networks is always a challenge. For example, for each small file replication AFM performs a lookup, create, write and set mtime operation. If the latency is 10ms, replication of each file takes minimum (10 * 4 = 40 ms) amount

Re: [gpfsug-discuss] AFM Recovery of SW cache does a full scan of home - is this to be expected?

l continue there. Kind regards, Heiner From: on behalf of Venkateswara R Puvvada Reply to: gpfsug main discussion list Date: Monday, 13 January 2020 at 08:40 To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] AFM Recovery of SW cache does a full scan of home - is this to be expect

Re: [gpfsug-discuss] AFM Recovery of SW cache does a full scan of home - is this to be expected?

AFM maintains in-memory queue at the gateway node to keep track of changes happening on the fileset. If the in-memory queue is lost (memory pressure, daemon shutdown etc..), AFM runs recovery process which involves creating the snapshot, running the policy scan and finally queueing the

Re: [gpfsug-discuss] Spectrum Scale mirrored filesets across sites.

get deleted from the filesets at C? ~Venkat (vpuvv...@in.ibm.com) From: Lyle Gayne/Poughkeepsie/IBM To: gpfsug-discuss@spectrumscale.org, Venkateswara R Puvvada/India/IBM@IBMIN Date: 12/18/2019 08:03 PM Subject:Re: [EXTERNAL] [gpfsug-discuss] Spectrum Scale mirrored filesets ac

Re: [gpfsug-discuss] afmRefreshAsync questions

Mattsson Systems Engineer MAX IV Laboratory Lund University P.O. Box 118, SE-221 00 Lund, Sweden Visiting address: Fotongatan 2, 224 84 Lund Mobile: +46 706 64 95 44 andreas.matts...@maxiv.lu.se www.maxiv.se Från: gpfsug-discuss-boun...@spectrumscale.org för Venkateswara R Puvvada Skickat: den 27

Re: [gpfsug-discuss] afmRefreshAsync questions

Hi, Both storage and client clusters have to be on 5.0.3.x to get the AFM revalidation performance with afmRefreshAsync. What are the refresh intervals ?, you could also try increasing them. Is this config option set at fileset level or cluster level ? ~Venkat (vpuvv...@in.ibm.com) From:

Re: [gpfsug-discuss] Inode reuse on AFM cache eviction

Hi, AFM does not support inode eviction, only data blocks are evicted and the file's metadata will remain in the fileset. ~Venkat (vpuvv...@in.ibm.com) From: Andreas Mattsson To: GPFS User Group Date: 09/05/2019 03:39 PM Subject:[EXTERNAL] [gpfsug-discuss] Inode reuse on

Re: [gpfsug-discuss] A net new cluster

AFM supports data migration between the two different file systems from the same cluster using NSD protocol. AFM based migration using the NSD protocol is usually performed by remote mounting the old filesystem (if not in the same cluster) at the new cluster's gateway node(s). Only gateway

Re: [gpfsug-discuss] AFM negative file caching

>I wasn’t sure if afmDirLookupRefreshInterval and afmFileLookupRefreshInterval would be the right thing if it’s a file/directory that doesn’t exist? These refresh intervals applies to all the lookups and not just for negative lookups. For working around in AFM itself, you could try setting

Re: [gpfsug-discuss] AFM relation on the fs level

AFM cannot be enabled at root fileset level today. ~Venkat (vpuvv...@in.ibm.com) From: Janusz Malka To: gpfsug main discussion list Date: 05/29/2018 07:06 PM Subject:[gpfsug-discuss] AFM relation on the fs level Sent by:gpfsug-discuss-boun...@spectrumscale.org Dear

Re: [gpfsug-discuss] AFM with clones

>What I’m not clear on is what happens if we update a clone file at home? I know AFM is supposed to only transfer the exact bytes updated, does this work with clones? i.e. at home do we just get the bytes updated in the >copy-on-write clone, or do we accidentally end up shipping the whole file

Re: [gpfsug-discuss] AFM cache re-link

-j NICKTESTFSET --new-target nfs://10.0.0.142/ibm/scalefs2/fsettest Appreciate the confirmation Nick From: gpfsug-discuss-boun...@spectrumscale.org <gpfsug-discuss-boun...@spectrumscale.org> On Behalf Of Venkateswara R Puvvada Sent: Monday, 23 April 2018 8:56 PM To: gpfsug main di

Re: [gpfsug-discuss] afmPrepopEnd Callback

Hi Luke, This issue has been fixed now. You could either request efix or try workaround as suggested by Simon. The following parameters are supported. prepopCompletedReads prepopFailedReads prepopData This one is missing from the mmfsfuncs and is fixed now.

Re: [gpfsug-discuss] AFM cache re-link

What is the fileset mode ? AFM won't attempt to copy the data back to home if file data already exists (checks if file size, mtime with nano seconds granularity and number of data blocks allocated are same). For example rsync version >= 3.1.0 keeps file mtime in sync with nano seconds

Re: [gpfsug-discuss] AFM-DR Questions

Hi, > - Any reason why we changed the Recovery point objective (RPO) snapshots by 15 minutes to 720 minutes in the version 5.0.0 of IBM Spectrum Scale AFM-DR? AFM DR doesn't require RPO snapshots for replication, it is continuous replication. Unless there is a need for crash consistency

Re: [gpfsug-discuss] AFM and RHEL 7.4

This is testing effort, AFM with RHEL 7.4 will be officially supported in next PTF releases 5.0.0.1 and 4.2.3.7. ~Venkat (vpuvv...@in.ibm.com) From: Jan-Frode Myklebust To: gpfsug main discussion list Date: 01/25/2018 01:57 PM

Re: [gpfsug-discuss] Use AFM for migration of many small files

0 Display Home Snapshots no Number of Gateway Flush Threads 8 Prefetch Threshold 0 (default) Eviction Enabledno Thanks in advance. Peter Childs On Tue, 2017-09-05 at 19:57 +0530, Venkateswara R Puvvada wrote: Whic

Re: [gpfsug-discuss] ACLs on AFM Filesets

, 7 Nov 2017 at 08:45 Venkateswara R Puvvada <vpuvv...@in.ibm.com> wrote: Luke, This issue has been fixed. As a workaround you could you also try resetting the same ACLs at home (instead of cache) or change directory ctime at home and verify that ACLs are updated correctly on fileset root

Re: [gpfsug-discuss] ACLs on AFM Filesets

Nov 2017, 10:22 Venkateswara R Puvvada, <vpuvv...@in.ibm.com> wrote: Is this problem happens only for the fileset root directory ? Could you try accessing the fileset as privileged user after the fileset link and verify if ACLs are set properly ? AFM reads the ACLs from home and

Re: [gpfsug-discuss] ACLs on AFM Filesets

Is this problem happens only for the fileset root directory ? Could you try accessing the fileset as privileged user after the fileset link and verify if ACLs are set properly ? AFM reads the ACLs from home and sets in the cache automatically during the file/dir lookup. What is the Spectrum

Re: [gpfsug-discuss] AFM fun (more!)

Simon, >Question 1. >Can we force the gateway node for the other file-sets to our "02" node. >I.e. So that we can get the queue services for the other filesets. AFM automatically maps the fileset to gateway node, and today there is no option available for users to assign fileset to a particular

Re: [gpfsug-discuss] AFM from generic NFS share - mmafmconfig

mmafmconfig command should be run on the target path (path specified in the afmTarget option when fileset is created). If many filesets are sharing the same target (ex independent writer mode) , enable AFM once on target path. Run the command at home cluster. mmafmconifg enable afmTarget

Re: [gpfsug-discuss] Deletion of a pcache snapshot?

Snapshots created by AFM (recovery, resync or peer snapshots) cannot be deleted by user using mmdelsnapshot command directly. After recovery or resync completion they get deleted automatically. For peer snapshots deletion mmpsnasp command is used. Which version of GPFS? Try with -p

Re: [gpfsug-discuss] Use AFM for migration of many small files

Which version of Spectrum Scale ? What is the fileset mode ? >We use AFM prefetch to migrate data between two clusters (using NFS). This works fine with large files, say 1+GB. But we have millions of smaller files, about 1MB each. Here >I see just ~150MB/s – compare this to the 1000+MB/s we

Re: [gpfsug-discuss] AFM - how to update directories with deleted files during prefetch

What is the version of GPFS ? >Mostly I want to update the directories to make AFM cache aware of file deletions on home. On home I can use a policy run to find all directories which changed since the last >update and pass them to prefetch on AFM cache. AFM prefetch has undocumented option

Re: [gpfsug-discuss] AFM gateways

rumscale.org [ mailto:gpfsug-discuss-boun...@spectrumscale.org] On Behalf Of Venkateswara R Puvvada Sent: 10 April 2017 11:56 To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] AFM gateways It is not recommended to make NSD servers as gateway nodes for nat

Re: [gpfsug-discuss] AFM gateways

It is not recommended to make NSD servers as gateway nodes for native GPFS protocol. Unresponsive remote cluster mount might cause gateway node to hang on synchronous operations (ex. Lookup, Read, Open etc..), this will affect NSD server functionality. More information is documented @

Re: [gpfsug-discuss] Quota issues, eviction, AFM won't stop throwing data to a full location - probably a rookie AFM mistake?

1. What is the version of GPFS ? Eviction should not be blocking the applications. Was partial file caching enabled ? Eviction cannot evict partially cached files in recent releases. Eviction does not use space inside .afm directory, and its logs are stored under /var/mmfs/tmp by default. 2.

Re: [gpfsug-discuss] AFM Migration Issue

AFM cannot keep directory mtime in sync. Directory mtime changes during readdir when files are created inside it after initial lookup. This is a known limitation today. ~Venkat (vpuvv...@in.ibm.com) From: To: Date:

Re: [gpfsug-discuss] Using AFM to migrate files. (Peter Childs) (Peter Childs) - URL encoding for pathnames

g-discuss@spectrumscale.org> Date: 10/24/2016 10:20 PM Subject:Re: [gpfsug-discuss] Using AFM to migrate files. (Peter Childs) (Peter Childs) - URL encoding for pathnames Sent by:gpfsug-discuss-boun...@spectrumscale.org On 10/24/2016 11:44 AM, Venkateswara R Puvvada wrote: > >

Re: [gpfsug-discuss] Extended attributes and ACLs with AFM-based "NFS migration"

Hi, Can you check if AFM was enabled at home cluster using "mmafmconfig enable" command? What is the fileset mode are you using ? Regards, Venkat --- Venkateswara R Puvvada/India/IBM@IBMIN vpuvv...@in.ibm.com Fr