R: [ADSM-L] R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts
The BA Spectrum clients get the list of files to backup from the NetApp; you can read it from the IBM page: The second snapshot is called the diffsnapshot, or differences snapshot. The client then incrementally backs up the files that are reported as changed, by NetApp, to the IBM Spectrum Protect server. You can find the same concept also at https://www.lascon.co.uk/tsm-backups.php#bsnetapp, where you'll find: The second snapshot is called the diffsnapshot. TSM then incrementally backs up the files reported as changed by NetApp to the TSM server. I am pretty sure it's not necessary to create exclusion rule of .snapshot (NFS) or ~ snapshot (CIFS) it is the "entry point" from which the BA Spectrum agent get the access of the files who needs backups. I say that because web documentation never require the adoption of this rule. These are NetApp construct that is automatically recognized by the BA Spectrum agents. Cheers. -Messaggio originale- Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di Efim Inviato: giovedì 20 settembre 2018 10:52 A: ADSM-L@VM.MARIST.EDU Oggetto: Re: [ADSM-L] R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts SP client use NetApp API to detect file changes between 2(two) snapshots. After that client have the list of changed files and backup it from mounted volume. I believe that it is unnecessary to copy the snapshot itself because it's pointless. Efim > 20 сент. 2018 г., в 11:29, Tommaso Bollini написал(а): > > @Efim > the .snapshot (or ~snapshot) folders are NetApp construct recognized by the > BA Spectrum agent api: > https://www.ibm.com/support/knowledgecenter/en/SSEQVQ_8.1.2/client/r_o > pt_snapdiff.html The BA Spectrum agent will copy all files listed in > the NetApp snapshot image. > The log error simply reports the point at which this backup job has in > exception echoing the full path of what BA API read from the snapshot image. > > I hope I was helpful. > Greetings. > > -Messaggio originale- > Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di > Efim > Inviato: mercoledì 19 settembre 2018 18:39 > A: ADSM-L@VM.MARIST.EDU > Oggetto: Re: [ADSM-L] Netapp snapdiff issues having TSM client on a > Linux machine, making use of NFS mounts > > Arnaud, > > Can you explain why you are copying the .snapshot folder during incremental > filesystem backup with snapdiff? > It contains snapshots created by netapp and SP for this FS. > I believe that it should be excluded from copying in your backup job. > > Efim > > >> 19 сент. 2018 г., в 18:28, PAC Brion Arnaud >> написал(а): >> >> Hi Robert, >> >> Thanks a lot for appreciated feedback, unfortunately even when using TSM own >> snapshots the problem still exists : >> >> root@xx:~ # dsmc i /Airwarder -snapdiff -diffsnapshot=create >> -snapdiffhttps -optfile=$OPTPATH >/tmp/snapshot_Airwarder.log >> >> root@ xx:~ # more /tmp/snapshot_Airwarder.log IBM Spectrum >> Protect Command Line Backup-Archive Client Interface Client Version >> 8, Release 1, Level 4.0 Client date/time: 09/19/2018 17:17:41 >> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights >> Reserved. >> >> Node Name: NETAPP_CH1_NFS >> Session established with server xx: Linux/ppc64le Server Version >> 8, Release 1, Level 4.000 Server date/time: 09/19/2018 17:17:42 >> Last >> access: 09/18/2018 12:12:17 >> >> >> Incremental by snapshot difference of volume '/Airwarder' >> >> Connected to NetApp Storage Virtual Machine >> Management Filer Host/IP Address : xx >> Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 >> 13:07:06 >> UTC 2018 >> Storage VM Name : ch1ns100 >> Storage VM Host/IP Address : xx >> Storage VM Volume Style : Flex >> Login User : tsm-backup-user >> Transport: HTTPS >> >> Performing a Full Incremental Backup of volume '/Airwarder' >> Creating Base Snapshot. >> Using Base Snapshot 'TSM_CH135BA2689781857_AIRWARDER' with timestamp >> 09/19/2018 >> 17:17:43 >> ANS4007E Error processing >> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/ >> acrimport_ref/ch': access to the object is denied ANS4007E Error >> processing '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/ >> acrimport_test/ch': access to the object is denied >> ANS1898I * Processed 500 files * >> ANS4007E Err
Re: R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts
client has been very efficient in its management of the snapshots, and in >> those rare cases when it doesn’t, the snapshots are easy to identify and >> clean up. >> >> FWIW, >> Bob >> >> >> Robert Talda >> EZ-Backup Systems Engineer >> Cornell University >> +1 607-255-8280 >> r...@cornell.edu >> >> >>> On Sep 18, 2018, at 8:35 AM, PAC Brion Arnaud >>> wrote: >>> >>> Hi Tommaso, >>> >>> The backup command I'm using (dsmc i /Airwarder -snapdiff -snapdiffhttps >>> -diffsnapshotname="daily.*" -diffsnapshot=latest -useexistingbase >>> -optfile=dsm_netapp.opt) includes the "useexistingbase" statement. >>> Based on my understanding of the documentation, this means that the S.P. >>> client will not trigger the creation of a new snapshot on the Netapp >>> device, but instead, make use of a snapshot that had been created >>> previously by some internal mechanism of the Netapp. >>> Therefore, it seems very unlikely that any process still has a grip on any >>> of the files being part of this snapshot ... >>> My humble opinion only of course, I'm waiting on others fellow members of >>> this list feedback ! >>> >>> In addition to this, I would like to avoid the use of any exclude statement >>> during the backup : I doubt that any of my customers would be very happy to >>> hear that I'm not able to restore some of his precious files because the >>> Spectrum Protect client had limitations ... Such issues never aroused with >>> NDMP-based backups ! >>> >>> Cheers. >>> >>> Arnaud >>> >>> >>> -Original Message- >>> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf >>> Of Tommaso Bollini >>> Sent: Tuesday, September 18, 2018 12:29 PM >>> To: ADSM-L@VM.MARIST.EDU >>> Subject: R: [ADSM-L] Netapp snapdiff issues having TSM client on a >>> Linux machine, making use of NFS mounts >>> >>> Hello Arnaud, maybe it would be: The NetApp snapshot indicates which files >>> are modified on the LUN (for which their backup is triggered); but the >>> Backup-Archive client cannot "take" them because they are locked by the >>> application. >>> Perhaps you can solve by inserting the appropiate rules in incl.excl file >>> to skip them. >>> >>> >>> -Messaggio originale- >>> Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto >>> di PAC Brion Arnaud >>> Inviato: martedì 18 settembre 2018 12:21 >>> A: ADSM-L@VM.MARIST.EDU >>> Oggetto: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux >>> machine, making use of NFS mounts >>> >>> Hi All, >>> >>> Following to IBM's decision to withdraw the web GUI in the latest versions >>> of their clients (thus making NDMP hardly usable), as well as to the >>> renewal of our NAS infrastructure (now using Netapp), I'm trying to >>> implement netapp snapdiffs in our shop. >>> >>> So far we succeeded in defining necessary user and roles on the Netapp, as >>> well as to setup the TSM client in order to interact with our filer, but >>> I'm now facing an issue where some files cannot be backed up, apparently >>> due to access rights. >>> >>> Here an extract of our backup log : >>> >>> IBM Spectrum Protect >>> Command Line Backup-Archive Client Interface Client Version 8, >>> Release 1, Level 4.0 Client date/time: 09/18/2018 10:26:28 >>> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights >>> Reserved. >>> >>> Node Name: NETAPP_CH1_NFS >>> Session established with server XXX: Linux/ppc64le Server Version >>> 8, Release 1, Level 4.000 Server date/time: 09/18/2018 10:26:28 Last >>> access: 09/18/2018 10:00:37 >>> >>> >>> Incremental by snapshot difference of volume '/Airwarder' >>> >>> Connected to NetApp Storage Virtual Machine >>> Management Filer Host/IP Address : >>> Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 >>> 13:07:06 UTC 2018 >>> Storage VM Name : xxx >>> Storage VM Host/IP Address : XX >>> Storage VM Volume Style : Flex >>> Login User : tsm-
R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts
ht I had > upgraded to SP 8.1.2.0 months ago*) Running on CentOS Linux release > 7.5.1804 (kernel 3.10.0-862.2.3.el7.x86_64) Accessing TSM Server for > Linux Version 7, Release 1, Level 7.200 > - For NetApp > Filer Version Information: NetApp Release 9.3P6 > > We aren’t seeing your issue, and I suspect I know why. We are telling the > TSM client to create its own snapshots - and not use the snapshots being > created hourly (or daily or weekly even) by the team managing the Netapp - > said snapshots being equivalent to the ones you are trying to take advantage > of in your backups. > > That is, the relevant part of the daily backup schedule looks like: > Schedule Name: DAILY.INCR.NFS.SNAPDIFF > Description: Daily SnapDiff Incremental >Action: Incremental > Subaction: > Options: -snapdiff -diffsnapshot=create > -snapdiffhttps > > > During a backup, then, the following type of information is written to the > dsmsched.log: > 09/13/2018 00:38:08 Incremental by snapshot difference of volume > '/nas-backup/vol/cit-pea1-test' > 09/13/2018 00:38:10 ANS2328I Using Snapshot Differential Change Log. > 09/13/2018 00:38:10 > Connected to NetApp Storage Virtual Machine >Management Filer Host/IP Address : ccc-cdot01.cit.cornell.edu >Filer Version Information: NetApp Release 9.3P6: Thu Jun 14 > 20:21:25 UTC 2018 >Storage VM Name : cit-sfshared05 >Storage VM Host/IP Address : cit-sfshared05-backup.files.cornell.edu >Storage VM Volume Style : Flex >Login User : tsmbackup >Transport: HTTPS > > 09/13/2018 00:38:10 Performing a Snapshot Differential Backup of volume > '/nas-backup/vol/cit-pea1-test' > 09/13/2018 00:38:10 Creating Diff Snapshot. > 09/13/2018 00:38:10 Using Base Snapshot > 'TSM_SANT5B9759FC1AB6_CIT_PEA1_TEST_VOL' with timestamp 09/11/2018 > 06:00:28 > 09/13/2018 00:38:10 Using Diff Snapshot > 'TSM_SANT5B99E9B24F814_CIT_PEA1_TEST_VOL' with timestamp 09/13/2018 > 04:38:10 > > Then, on the next backup (48 hours later thanks to a large amount of data to > backup on 9/13), the Diff Snapshot becomes the base: > 09/15/2018 01:29:22 ANS2328I Using Snapshot Differential Change Log. > 09/15/2018 01:29:22 > Connected to NetApp Storage Virtual Machine >Management Filer Host/IP Address : ccc-cdot01.cit.cornell.edu >Filer Version Information: NetApp Release 9.3P6: Thu Jun 14 > 20:21:25 UTC 2018 >Storage VM Name : cit-sfshared05 >Storage VM Host/IP Address : cit-sfshared05-backup.files.cornell.edu >Storage VM Volume Style : Flex >Login User : tsmbackup >Transport: HTTPS > > 09/15/2018 01:29:22 Performing a Snapshot Differential Backup of volume > '/nas-backup/vol/cit-pea1-test' > 09/15/2018 01:29:22 Creating Diff Snapshot. > 09/15/2018 01:29:22 Using Base Snapshot > 'TSM_SANT5B99E9B24F814_CIT_PEA1_TEST_VOL' with timestamp 09/13/2018 > 04:38:10 > 09/15/2018 01:29:22 Using Diff Snapshot > 'TSM_SANT5B9C98B1A15B2_CIT_PEA1_TEST_VOL' with timestamp 09/15/2018 > 05:29:21 > > I think that consistency is what is missing in your approach - I suspect that > there are intervening snapshots with information about files that have been > deleted or changed that are not part of what the TSM client is seeing. > > I will add that our NetApp team is very happy with this approach. The TSM > client has been very efficient in its management of the snapshots, and in > those rare cases when it doesn’t, the snapshots are easy to identify and > clean up. > > FWIW, > Bob > > > Robert Talda > EZ-Backup Systems Engineer > Cornell University > +1 607-255-8280 > r...@cornell.edu > > >> On Sep 18, 2018, at 8:35 AM, PAC Brion Arnaud >> wrote: >> >> Hi Tommaso, >> >> The backup command I'm using (dsmc i /Airwarder -snapdiff -snapdiffhttps >> -diffsnapshotname="daily.*" -diffsnapshot=latest -useexistingbase >> -optfile=dsm_netapp.opt) includes the "useexistingbase" statement. >> Based on my understanding of the documentation, this means that the S.P. >> client will not trigger the creation of a new snapshot on the Netapp device, >> but instead, make use of a snapshot that had been created previously by some >> internal mechanism of the Netapp. >> Therefore, it seems very unlikely that any process still has a grip on any >> of the
R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts
Hello Arnaud, maybe it would be: The NetApp snapshot indicates which files are modified on the LUN (for which their backup is triggered); but the Backup-Archive client cannot "take" them because they are locked by the application. Perhaps you can solve by inserting the appropiate rules in incl.excl file to skip them. -Messaggio originale- Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di PAC Brion Arnaud Inviato: martedì 18 settembre 2018 12:21 A: ADSM-L@VM.MARIST.EDU Oggetto: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts Hi All, Following to IBM's decision to withdraw the web GUI in the latest versions of their clients (thus making NDMP hardly usable), as well as to the renewal of our NAS infrastructure (now using Netapp), I'm trying to implement netapp snapdiffs in our shop. So far we succeeded in defining necessary user and roles on the Netapp, as well as to setup the TSM client in order to interact with our filer, but I'm now facing an issue where some files cannot be backed up, apparently due to access rights. Here an extract of our backup log : IBM Spectrum Protect Command Line Backup-Archive Client Interface Client Version 8, Release 1, Level 4.0 Client date/time: 09/18/2018 10:26:28 (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights Reserved. Node Name: NETAPP_CH1_NFS Session established with server XXX: Linux/ppc64le Server Version 8, Release 1, Level 4.000 Server date/time: 09/18/2018 10:26:28 Last access: 09/18/2018 10:00:37 Incremental by snapshot difference of volume '/Airwarder' Connected to NetApp Storage Virtual Machine Management Filer Host/IP Address : Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 13:07:06 UTC 2018 Storage VM Name : xxx Storage VM Host/IP Address : XX Storage VM Volume Style : Flex Login User : tsm-backup-user Transport: HTTPS Performing a Full Incremental Backup of volume '/Airwarder' Using Base Snapshot 'vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500' with timestamp 09/18/2018 10:25:02 ANS4007E Error processing '/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_ref/ch': access to the object is denied ANS4007E Error processing '/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_test/ch': access to the object is denied ANS1898I * Processed 500 files * Directory--> 4,096 /Airwarder/ [Sent] Directory--> 4,096 /Airwarder/.etc [Sent] (skipped some lines) ANS1228E Sending of object '/Airwarder/acrimport_ref/config/log4j.conf' failed. ANS4007E Error processing '/Airwarder/acrimport_ref/config/log4j.conf': access to the object is denied Normal File--> 309 /Airwarder/acrimport_ref/config/sapConnector.cfg ** Unsuccessful ** ANS1228E Sending of object '/Airwarder/acrimport_ref/config/sapConnector.cfg' failed. ANS4007E Error processing '/Airwarder/acrimport_ref/config/sapConnector.cfg': access to the object is denied Normal File--> 330 /Airwarder/acrimport_ref/config/sapConnector.cfg.bak ** Unsuccessful ** ANS1228E Sending of object '/Airwarder/acrimport_ref/config/sapConnector.cfg.bak' failed. ANS4007E Error processing '/Airwarder/acrimport_ref/config/sapConnector.cfg.bak': access to the object is denied (skipped some lines) ANS1802E Incremental backup of '/Airwarder' finished with 90 failure(s) Total number of objects inspected:5,881 Total number of objects backed up:5,791 Total number of objects updated: 0 Total number of objects rebound: 0 Total number of objects deleted: 0 Total number of objects expired: 0 Total number of objects failed: 90 Total number of objects encrypted:0 Total number of objects grew: 0 Total number of retries: 15 Total number of bytes inspected: 1.27 GB Total number of bytes transferred: 1.37 GB Data transfer time: 18.61 sec Network data transfer rate: 77,367.92 KB/sec Aggregate data transfer rate: 36,807.03 KB/sec Objects compressed by:0% Total data reduction ratio:0.00% Elapsed processing time: 00:00:39 Here, a closer look at some of the files making problems : root@xx:~ # ls -al /Airwarder/acrimport_ref/config/sapConnector.cfg.bak -rw-r- 1 root root 330 May 6 2009 /Airwarder/acrimport_ref/config/sapConnector.cfg.bak root@xx:~ # ls -la /Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_121000/acrimport_ref/ch ls: cannot open directory /Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb
