Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[David L.A. De Leeuw]

Good morning, TSM'ers.

My explanation of the problem, as I wrote yesterday, was wrong.
It appears some of our disks on the backup system were defined as "thin 
provisioned".
When these fill up, and cannot expand anymore, Windows will still show empty 
space on them.
I focused my attention on the disks with zero space left. But these were 
working fine.

Once I declared the "thin provisioned" disks as read-only in "update stgpooldir 
XXX access=readonly" on TSM everything works fine.
I will remove them ASAP.

Conclusion:

1. TSM works fine, but could be more tolerant if one of the stgpooldirs 
encounters a problem.
2. Windows Server works fine
3. VMWare works fine, but blocks the service without timely information to the 
end user. Only analysis of the logs point to the solution.
4. System manager (me) is too multifunctional and misses deeper understanding 
of some systems he works with.

Have a nice day

David

-Original Message-
From: דוד דה ליאו 
Sent: Wednesday, August 23, 2023 2:35 PM
To: ADSM: Dist Stor Manager 
Cc: סער קליין - Saar Klein 
Subject: RE: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client

Dear Chavdar, Michall and others,

Just an update on the issue I raised a few days ago concerning the failed 
backups.

Our ISP server uses a separate windows server with about 100 TB of containers, 
divided over 28 disks.
Backups started crashing about a week ago.
At some point the system would get stuck for five minutes or so.

Of course, a lot of different issues happened at the same time:

-The problem started after windows server updates, which forced us to reboot 
most of our systems.
-One user dumped about a terabyte of mostly small files on our system.
- Our Spectrum Protect system manager was on holidays.
-As always there are other usual suspects: antivirus etc.

Our container server runs on Vmware ESXI infrastructure. We opened a call to 
VMWARE, sent them the logs of the ESXI server.
They found a very simple cause to the problem: disks were filled up, and the 
system froze.

When checking the logs, I found that the backup containers opened in write mode 
were on disks without any space left, while other disks were less than half 
full.
So here is my solution: set the containerdirs that are full on read-only, move 
containers, wait till the containers are deleted. 

My question is: why is this process not managed automatically by ISP? Why are 
disks with a lot of space not prioritized for writing?

Thanks for your help !

David de Leeuw

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Chavdar Cholev
Sent: Monday, August 21, 2023 6:11 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client

Hi David,
Just make sure that containers are excluded from anti-virus scan.

On Sunday, August 20, 2023, David L.A. De Leeuw  wrote:

> Hi all,
>
> Apparently, this has nothing to do with SP at all !
>
> The (Windows server 2019 on ESXI) system holding the containers just 
> disconnects for 5 minutes !
>
> No pings to the server.
>
> When access is restored, later on, a message appears in the events:
> "The system time has changed to 2023-08-20T19:05:05 from
> 2023-08-20T19:01:04  "
> This is no warning even, just "information".
>
> I have no idea why this should happen, but we will find it.
> Thanks for your support !
>
> David
>
>
>
> -Original Message-
> From: דוד דה ליאו
> Sent: Sunday, August 20, 2023 9:37 PM
> To: ADSM: Dist Stor Manager 
> Subject: RE: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server 
> and client
>
> Hi Michael,
>
> Thanks a lot.
>
> The SP Server is not on VM, just the storage. I am not the manager to 
> the server.
> Just got a lot of backup storage if we provide the space for the 
> containers.
>
> Sure we run a lot of sessions in parallel as you said. I will try a 
> run according to your recommendations.
> One other thought I am testing, is that over a year ago we also had 
> crashes. The 10 Gb optical network had hickups. Our 1 Gb line worked fine.
> I just switched back to the 1 Gb and see what happens.
>
> Will keep you posted !
>
> David
>
>
> -Original Message-
> From: ADSM: Dist Stor Manager  On Behalf Of 
> Michael Prix
> Sent: Sunday, August 20, 2023 9:04 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server 
> and client
>
> Hello David,
>
>   an *SP-Server in a VM is not the best setup, but nevertheless it 
> should work - and has proven so for the past.
>
> For the client: Please show the dsm.opt. I suspect you are trunning 
> several sessions from this client in parallel during a backup-> stop 
> it for the moment.
> Start with a basic dsm.opt, disable 

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[David L.A. De Leeuw]

Dear Chavdar, Michall and others,

Just an update on the issue I raised a few days ago concerning the failed 
backups.

Our ISP server uses a separate windows server with about 100 TB of containers, 
divided over 28 disks.
Backups started crashing about a week ago.
At some point the system would get stuck for five minutes or so.

Of course, a lot of different issues happened at the same time:

-The problem started after windows server updates, which forced us to reboot 
most of our systems.
-One user dumped about a terabyte of mostly small files on our system.
- Our Spectrum Protect system manager was on holidays.
-As always there are other usual suspects: antivirus etc.

Our container server runs on Vmware ESXI infrastructure. We opened a call to 
VMWARE, sent them the logs of the ESXI server.
They found a very simple cause to the problem: disks were filled up, and the 
system froze.

When checking the logs, I found that the backup containers opened in write mode 
were on disks without any space left, while other disks were less than half 
full.
So here is my solution: set the containerdirs that are full on read-only, move 
containers, wait till the containers are deleted. 

My question is: why is this process not managed automatically by ISP? Why are 
disks with a lot of space not prioritized for writing?

Thanks for your help !

David de Leeuw

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Chavdar Cholev
Sent: Monday, August 21, 2023 6:11 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client

Hi David,
Just make sure that containers are excluded from anti-virus scan.

On Sunday, August 20, 2023, David L.A. De Leeuw  wrote:

> Hi all,
>
> Apparently, this has nothing to do with SP at all !
>
> The (Windows server 2019 on ESXI) system holding the containers just 
> disconnects for 5 minutes !
>
> No pings to the server.
>
> When access is restored, later on, a message appears in the events:
> "The system time has changed to 2023-08-20T19:05:05 from
> 2023-08-20T19:01:04  "
> This is no warning even, just "information".
>
> I have no idea why this should happen, but we will find it.
> Thanks for your support !
>
> David
>
>
>
> -Original Message-
> From: דוד דה ליאו
> Sent: Sunday, August 20, 2023 9:37 PM
> To: ADSM: Dist Stor Manager 
> Subject: RE: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server 
> and client
>
> Hi Michael,
>
> Thanks a lot.
>
> The SP Server is not on VM, just the storage. I am not the manager to 
> the server.
> Just got a lot of backup storage if we provide the space for the 
> containers.
>
> Sure we run a lot of sessions in parallel as you said. I will try a 
> run according to your recommendations.
> One other thought I am testing, is that over a year ago we also had 
> crashes. The 10 Gb optical network had hickups. Our 1 Gb line worked fine.
> I just switched back to the 1 Gb and see what happens.
>
> Will keep you posted !
>
> David
>
>
> -Original Message-
> From: ADSM: Dist Stor Manager  On Behalf Of 
> Michael Prix
> Sent: Sunday, August 20, 2023 9:04 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server 
> and client
>
> Hello David,
>
>   an *SP-Server in a VM is not the best setup, but nevertheless it 
> should work - and has proven so for the past.
>
> For the client: Please show the dsm.opt. I suspect you are trunning 
> several sessions from this client in parallel during a backup-> stop 
> it for the moment.
> Start with a basic dsm.opt, disable the option "resourceutilization", 
> if set,  and set "memoryefficient yes" (or "diskcachem" if you like). 
> I'f it still crashes with a plain dsm.opt, you should open a ticket with IBM.
>
> --
> Michael Prix
>
>
>
>
> August 20, 2023 at 7:25 PM, "David L.A. De Leeuw"  wrote:
>
>
> >
> > Hi Chavdar and Michael,
> >
> > Thanks for your thoughts and help.
> >
> > I added "memoryefficientbackup".
> >
> > But still the sessions keep crashing. Once the session crashes, I 
> > get a
> whole bit of errors for storage pool directories, and in fact the 
> whole pool becomes unavailable.
> > I run "update stgpooldir ... access=readwrite" and all is accessible
> again.
> > Some of the containers are in unavailable state and need audit.
> >
> > Our container storage is on a Dell PowerEdge R730xd, has 24 CPU's
> allocated, 64 GB memory, 110 TB disk. The disks are declared as VMDKs.
> Network is on a 10Gb Intel 82588 card.
> > Nothing I can see points to a lack of resources

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[Chavdar Cholev]

Hi David,
Just make sure that containers are excluded from anti-virus scan.

On Sunday, August 20, 2023, David L.A. De Leeuw  wrote:

> Hi all,
>
> Apparently, this has nothing to do with SP at all !
>
> The (Windows server 2019 on ESXI) system holding the containers just
> disconnects for 5 minutes !
>
> No pings to the server.
>
> When access is restored, later on, a message appears in the events:
> "The system time has changed to 2023-08-20T19:05:05 from
> 2023-08-20T19:01:04  "
> This is no warning even, just "information".
>
> I have no idea why this should happen, but we will find it.
> Thanks for your support !
>
> David
>
>
>
> -Original Message-
> From: דוד דה ליאו
> Sent: Sunday, August 20, 2023 9:37 PM
> To: ADSM: Dist Stor Manager 
> Subject: RE: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and
> client
>
> Hi Michael,
>
> Thanks a lot.
>
> The SP Server is not on VM, just the storage. I am not the manager to the
> server.
> Just got a lot of backup storage if we provide the space for the
> containers.
>
> Sure we run a lot of sessions in parallel as you said. I will try a run
> according to your recommendations.
> One other thought I am testing, is that over a year ago we also had
> crashes. The 10 Gb optical network had hickups. Our 1 Gb line worked fine.
> I just switched back to the 1 Gb and see what happens.
>
> Will keep you posted !
>
> David
>
>
> -Original Message-
> From: ADSM: Dist Stor Manager  On Behalf Of Michael
> Prix
> Sent: Sunday, August 20, 2023 9:04 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and
> client
>
> Hello David,
>
>   an *SP-Server in a VM is not the best setup, but nevertheless it should
> work - and has proven so for the past.
>
> For the client: Please show the dsm.opt. I suspect you are trunning
> several sessions from this client in parallel during a backup-> stop it for
> the moment.
> Start with a basic dsm.opt, disable the option "resourceutilization", if
> set,  and set "memoryefficient yes" (or "diskcachem" if you like). I'f it
> still crashes with a plain dsm.opt, you should open a ticket with IBM.
>
> --
> Michael Prix
>
>
>
>
> August 20, 2023 at 7:25 PM, "David L.A. De Leeuw"  wrote:
>
>
> >
> > Hi Chavdar and Michael,
> >
> > Thanks for your thoughts and help.
> >
> > I added "memoryefficientbackup".
> >
> > But still the sessions keep crashing. Once the session crashes, I get a
> whole bit of errors for storage pool directories, and in fact the whole
> pool becomes unavailable.
> > I run "update stgpooldir ... access=readwrite" and all is accessible
> again.
> > Some of the containers are in unavailable state and need audit.
> >
> > Our container storage is on a Dell PowerEdge R730xd, has 24 CPU's
> allocated, 64 GB memory, 110 TB disk. The disks are declared as VMDKs.
> Network is on a 10Gb Intel 82588 card.
> > Nothing I can see points to a lack of resources.
> >
> > Everything worked fine till 4 days ago. That is why I thought of a
> problem with Windows updates, but as I rolled them back, that does not make
> sense.
> >
> > I am quite at a loss where to look next ...
> >
> > Thanks
> >
> > David
> >
> > [Server Side] .
> > 20-08-2023 19:47:22 ANR0839I Session 197902 started for node MEDFS2
> (WinNT)
> >  (SSL medspice.bgu.ac.il[132.72.73.246]:53184) on
> >  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197902)
> > 20-08-2023 19:47:26 ANR8592I Session 197903 connection is using protocol
> >  TLSV13, cipher specification TLS_AES_256_GCM_SHA384,
> >  certificate TSM Self-Signed Certificate. (SESSION:
> >  197903)
> > 20-08-2023 19:47:26 ANR0839I Session 197903 started for node MEDFS2
> (WinNT)
> >  (SSL medspice.bgu.ac.il[132.72.73.246]:53185) on
> >  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197903)
> > 20-08-2023 19:47:55 ANR2012W Error encountered for storage pool
> directory:
> >  \\medbackup.med.ad.bgu.ac.il\tsmc20 in storage pool:
> >  CPOOL. (SESSION: 197881)
> > 20-08-2023 19:47:55 ANR1181E sdtxn.c(1404): Data storage transaction
> >  0:83236375 was aborted. (SESSION: 197881)
> > 20-08-2023 19:47:55 ANR0204I The container state for
> >  \\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
> >  ncf is updated from AVAILABLE to UNAVAILABLE. (SESSION:
> >  197883)
> > 20-08-2023 19:47:55 ANR3660E An unexpected error occurred while opening
> or
>

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[David L.A. De Leeuw]

Hi all,

Apparently, this has nothing to do with SP at all !

The (Windows server 2019 on ESXI) system holding the containers just 
disconnects for 5 minutes ! 

No pings to the server. 

When access is restored, later on, a message appears in the events:
"The system time has changed to 2023-08-20T19:05:05 from 2023-08-20T19:01:04  "
This is no warning even, just "information". 

I have no idea why this should happen, but we will find it.
Thanks for your support !

David



-Original Message-
From: דוד דה ליאו 
Sent: Sunday, August 20, 2023 9:37 PM
To: ADSM: Dist Stor Manager 
Subject: RE: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client

Hi Michael,

Thanks a lot.

The SP Server is not on VM, just the storage. I am not the manager to the 
server. 
Just got a lot of backup storage if we provide the space for the containers.

Sure we run a lot of sessions in parallel as you said. I will try a run 
according to your recommendations.
One other thought I am testing, is that over a year ago we also had crashes. 
The 10 Gb optical network had hickups. Our 1 Gb line worked fine.
I just switched back to the 1 Gb and see what happens. 

Will keep you posted !

David


-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Michael Prix
Sent: Sunday, August 20, 2023 9:04 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client

Hello David,

  an *SP-Server in a VM is not the best setup, but nevertheless it should work 
- and has proven so for the past.

For the client: Please show the dsm.opt. I suspect you are trunning several 
sessions from this client in parallel during a backup-> stop it for the moment.
Start with a basic dsm.opt, disable the option "resourceutilization", if set,  
and set "memoryefficient yes" (or "diskcachem" if you like). I'f it still 
crashes with a plain dsm.opt, you should open a ticket with IBM.

-- 
Michael Prix




August 20, 2023 at 7:25 PM, "David L.A. De Leeuw"  wrote:


> 
> Hi Chavdar and Michael,
> 
> Thanks for your thoughts and help.
> 
> I added "memoryefficientbackup". 
> 
> But still the sessions keep crashing. Once the session crashes, I get a whole 
> bit of errors for storage pool directories, and in fact the whole pool 
> becomes unavailable. 
> I run "update stgpooldir ... access=readwrite" and all is accessible again.
> Some of the containers are in unavailable state and need audit. 
> 
> Our container storage is on a Dell PowerEdge R730xd, has 24 CPU's allocated, 
> 64 GB memory, 110 TB disk. The disks are declared as VMDKs. Network is on a 
> 10Gb Intel 82588 card.
> Nothing I can see points to a lack of resources.
> 
> Everything worked fine till 4 days ago. That is why I thought of a problem 
> with Windows updates, but as I rolled them back, that does not make sense.
> 
> I am quite at a loss where to look next ...
> 
> Thanks
> 
> David
> 
> [Server Side] .
> 20-08-2023 19:47:22 ANR0839I Session 197902 started for node MEDFS2 (WinNT)
>  (SSL medspice.bgu.ac.il[132.72.73.246]:53184) on
>  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197902)
> 20-08-2023 19:47:26 ANR8592I Session 197903 connection is using protocol
>  TLSV13, cipher specification TLS_AES_256_GCM_SHA384,
>  certificate TSM Self-Signed Certificate. (SESSION:
>  197903)
> 20-08-2023 19:47:26 ANR0839I Session 197903 started for node MEDFS2 (WinNT)
>  (SSL medspice.bgu.ac.il[132.72.73.246]:53185) on
>  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197903)
> 20-08-2023 19:47:55 ANR2012W Error encountered for storage pool directory:
>  \\medbackup.med.ad.bgu.ac.il\tsmc20 in storage pool:
>  CPOOL. (SESSION: 197881)
> 20-08-2023 19:47:55 ANR1181E sdtxn.c(1404): Data storage transaction
>  0:83236375 was aborted. (SESSION: 197881)
> 20-08-2023 19:47:55 ANR0204I The container state for
>  \\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
>  ncf is updated from AVAILABLE to UNAVAILABLE. (SESSION:
>  197883)
> 20-08-2023 19:47:55 ANR3660E An unexpected error occurred while opening or
>  writing to the container. Container
>  \\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
>  ncf in stgpool CPOOL has been marked as UNAVAILABLE and
>  should be audited to validate accessibility and content.
>  (SESSION: 197883)
> 
> [From the client side:]
> 
> During the incr of a large filespace:
> 
> Normal File--> 7.132.827 \\medfs2\e$\medusers14\angel\17.8.23 BU - 
> E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
> 313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
> general MRI data\For-Crop-T2W - coronal Copy.pptx ** Unsuccessful **
> ANS1228E Sending of object '\\medfs2\e$

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[David L.A. De Leeuw]

Hi Michael,

Thanks a lot.

The SP Server is not on VM, just the storage. I am not the manager to the 
server. 
Just got a lot of backup storage if we provide the space for the containers.

Sure we run a lot of sessions in parallel as you said. I will try a run 
according to your recommendations.
One other thought I am testing, is that over a year ago we also had crashes. 
The 10 Gb optical network had hickups. Our 1 Gb line worked fine.
I just switched back to the 1 Gb and see what happens. 

Will keep you posted !

David


-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Michael Prix
Sent: Sunday, August 20, 2023 9:04 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client

Hello David,

  an *SP-Server in a VM is not the best setup, but nevertheless it should work 
- and has proven so for the past.

For the client: Please show the dsm.opt. I suspect you are trunning several 
sessions from this client in parallel during a backup-> stop it for the moment.
Start with a basic dsm.opt, disable the option "resourceutilization", if set,  
and set "memoryefficient yes" (or "diskcachem" if you like). I'f it still 
crashes with a plain dsm.opt, you should open a ticket with IBM.

-- 
Michael Prix




August 20, 2023 at 7:25 PM, "David L.A. De Leeuw"  wrote:


> 
> Hi Chavdar and Michael,
> 
> Thanks for your thoughts and help.
> 
> I added "memoryefficientbackup". 
> 
> But still the sessions keep crashing. Once the session crashes, I get a whole 
> bit of errors for storage pool directories, and in fact the whole pool 
> becomes unavailable. 
> I run "update stgpooldir ... access=readwrite" and all is accessible again.
> Some of the containers are in unavailable state and need audit. 
> 
> Our container storage is on a Dell PowerEdge R730xd, has 24 CPU's allocated, 
> 64 GB memory, 110 TB disk. The disks are declared as VMDKs. Network is on a 
> 10Gb Intel 82588 card.
> Nothing I can see points to a lack of resources.
> 
> Everything worked fine till 4 days ago. That is why I thought of a problem 
> with Windows updates, but as I rolled them back, that does not make sense.
> 
> I am quite at a loss where to look next ...
> 
> Thanks
> 
> David
> 
> [Server Side] .
> 20-08-2023 19:47:22 ANR0839I Session 197902 started for node MEDFS2 (WinNT)
>  (SSL medspice.bgu.ac.il[132.72.73.246]:53184) on
>  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197902)
> 20-08-2023 19:47:26 ANR8592I Session 197903 connection is using protocol
>  TLSV13, cipher specification TLS_AES_256_GCM_SHA384,
>  certificate TSM Self-Signed Certificate. (SESSION:
>  197903)
> 20-08-2023 19:47:26 ANR0839I Session 197903 started for node MEDFS2 (WinNT)
>  (SSL medspice.bgu.ac.il[132.72.73.246]:53185) on
>  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197903)
> 20-08-2023 19:47:55 ANR2012W Error encountered for storage pool directory:
>  \\medbackup.med.ad.bgu.ac.il\tsmc20 in storage pool:
>  CPOOL. (SESSION: 197881)
> 20-08-2023 19:47:55 ANR1181E sdtxn.c(1404): Data storage transaction
>  0:83236375 was aborted. (SESSION: 197881)
> 20-08-2023 19:47:55 ANR0204I The container state for
>  \\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
>  ncf is updated from AVAILABLE to UNAVAILABLE. (SESSION:
>  197883)
> 20-08-2023 19:47:55 ANR3660E An unexpected error occurred while opening or
>  writing to the container. Container
>  \\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
>  ncf in stgpool CPOOL has been marked as UNAVAILABLE and
>  should be audited to validate accessibility and content.
>  (SESSION: 197883)
> 
> [From the client side:]
> 
> During the incr of a large filespace:
> 
> Normal File--> 7.132.827 \\medfs2\e$\medusers14\angel\17.8.23 BU - 
> E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
> 313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
> general MRI data\For-Crop-T2W - coronal Copy.pptx ** Unsuccessful **
> ANS1228E Sending of object '\\medfs2\e$\medusers14\angel\17.8.23 BU - 
> E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
> 313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
> general MRI data\For-Crop-T2W - coronal Copy.pptx' failed.
> ANS1311E Server out of data storage space
> 
> [I ran sel of the latest file. It failed because all containerdirs were 
> unavailable.]
> 
> ANS1804E Selective Backup processing of '\\medfs2\e$\medusers14\angel\17.8.23 
> BU - E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
> 313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
> general 

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[Michael Prix]

Hello David,

  an *SP-Server in a VM is not the best setup, but nevertheless it should work 
- and has proven so for the past.

For the client: Please show the dsm.opt. I suspect you are trunning several 
sessions from this client in parallel during a backup-> stop it for the moment.
Start with a basic dsm.opt, disable the option "resourceutilization", if set,  
and set "memoryefficient yes" (or "diskcachem" if you like). I'f it still 
crashes with a plain dsm.opt, you should open a ticket with IBM.

-- 
Michael Prix




August 20, 2023 at 7:25 PM, "David L.A. De Leeuw"  wrote:


> 
> Hi Chavdar and Michael,
> 
> Thanks for your thoughts and help.
> 
> I added "memoryefficientbackup". 
> 
> But still the sessions keep crashing. Once the session crashes, I get a whole 
> bit of errors for storage pool directories, and in fact the whole pool 
> becomes unavailable. 
> I run "update stgpooldir ... access=readwrite" and all is accessible again.
> Some of the containers are in unavailable state and need audit. 
> 
> Our container storage is on a Dell PowerEdge R730xd, has 24 CPU's allocated, 
> 64 GB memory, 110 TB disk. The disks are declared as VMDKs. Network is on a 
> 10Gb Intel 82588 card.
> Nothing I can see points to a lack of resources.
> 
> Everything worked fine till 4 days ago. That is why I thought of a problem 
> with Windows updates, but as I rolled them back, that does not make sense.
> 
> I am quite at a loss where to look next ...
> 
> Thanks
> 
> David
> 
> [Server Side] .
> 20-08-2023 19:47:22 ANR0839I Session 197902 started for node MEDFS2 (WinNT)
>  (SSL medspice.bgu.ac.il[132.72.73.246]:53184) on
>  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197902)
> 20-08-2023 19:47:26 ANR8592I Session 197903 connection is using protocol
>  TLSV13, cipher specification TLS_AES_256_GCM_SHA384,
>  certificate TSM Self-Signed Certificate. (SESSION:
>  197903)
> 20-08-2023 19:47:26 ANR0839I Session 197903 started for node MEDFS2 (WinNT)
>  (SSL medspice.bgu.ac.il[132.72.73.246]:53185) on
>  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197903)
> 20-08-2023 19:47:55 ANR2012W Error encountered for storage pool directory:
>  \\medbackup.med.ad.bgu.ac.il\tsmc20 in storage pool:
>  CPOOL. (SESSION: 197881)
> 20-08-2023 19:47:55 ANR1181E sdtxn.c(1404): Data storage transaction
>  0:83236375 was aborted. (SESSION: 197881)
> 20-08-2023 19:47:55 ANR0204I The container state for
>  \\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
>  ncf is updated from AVAILABLE to UNAVAILABLE. (SESSION:
>  197883)
> 20-08-2023 19:47:55 ANR3660E An unexpected error occurred while opening or
>  writing to the container. Container
>  \\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
>  ncf in stgpool CPOOL has been marked as UNAVAILABLE and
>  should be audited to validate accessibility and content.
>  (SESSION: 197883)
> 
> [From the client side:]
> 
> During the incr of a large filespace:
> 
> Normal File--> 7.132.827 \\medfs2\e$\medusers14\angel\17.8.23 BU - 
> E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
> 313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
> general MRI data\For-Crop-T2W - coronal Copy.pptx ** Unsuccessful **
> ANS1228E Sending of object '\\medfs2\e$\medusers14\angel\17.8.23 BU - 
> E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
> 313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
> general MRI data\For-Crop-T2W - coronal Copy.pptx' failed.
> ANS1311E Server out of data storage space
> 
> [I ran sel of the latest file. It failed because all containerdirs were 
> unavailable.]
> 
> ANS1804E Selective Backup processing of '\\medfs2\e$\medusers14\angel\17.8.23 
> BU - E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
> 313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
> general MRI data\For-Crop-T2W - coronal Copy.pptx' finished with failures.
> 
> Total number of objects inspected: 1
> Total number of objects backed up: 0
> Total number of objects updated: 0
> Total number of objects rebound: 0
> Total number of objects deleted: 0
> Total number of objects expired: 0
> Total number of objects failed: 1
>  ...
> Network data transfer rate: 148.306,35 KB/sec
> Aggregate data transfer rate: 211,50 KB/sec
> Objects compressed by: 0%
> Total data reduction ratio: 0.23%
> Subfile objects reduced by: 0%
> Elapsed processing time: 00:00:32
> ANS1311E Server out of data storage space
> 
> [Then I updated the containerdirs to readwrite and ran the selective backup. 
> No 

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[David L.A. De Leeuw]

Hi Chavdar and Michael,

Thanks for your thoughts and help.

I added "memoryefficientbackup". 

But still the sessions keep crashing. Once the session crashes, I get a whole 
bit of errors   for storage pool directories, and in fact the whole pool 
becomes unavailable. 
I run "update stgpooldir ... access=readwrite" and all is accessible again.
Some of the containers are in unavailable state and need audit. 

Our container storage is on a Dell PowerEdge R730xd, has 24 CPU's allocated, 64 
GB memory, 110 TB disk.  The disks are declared as VMDKs.  Network is on a 10Gb 
Intel 82588 card.
Nothing I can see points to a lack of resources.

Everything worked fine till 4 days ago. That is why I thought of a problem with 
Windows updates, but as I rolled them back, that does not make sense.

I am quite at a loss where to look next ...

Thanks

David

[Server Side] .
20-08-2023 19:47:22  ANR0839I Session 197902 started for node MEDFS2 (WinNT)
  (SSL medspice.bgu.ac.il[132.72.73.246]:53184) on
  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197902)
20-08-2023 19:47:26  ANR8592I Session 197903 connection is using protocol
  TLSV13, cipher specification TLS_AES_256_GCM_SHA384,
      certificate TSM Self-Signed Certificate. (SESSION:
  197903)
20-08-2023 19:47:26  ANR0839I Session 197903 started for node MEDFS2 (WinNT)
  (SSL medspice.bgu.ac.il[132.72.73.246]:53185) on
  STOREWARE13.auth.ad.bgu.ac.il:1502. (SESSION: 197903)
20-08-2023 19:47:55  ANR2012W Error encountered for storage pool directory:
  \\medbackup.med.ad.bgu.ac.il\tsmc20 in storage pool:
  CPOOL. (SESSION: 197881)
20-08-2023 19:47:55  ANR1181E sdtxn.c(1404): Data storage transaction
  0:83236375 was aborted. (SESSION: 197881)
20-08-2023 19:47:55  ANR0204I The container state for
  
\\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
  ncf is updated from AVAILABLE to UNAVAILABLE. 
(SESSION:
  197883)
20-08-2023 19:47:55  ANR3660E An unexpected error occurred while opening or
  writing to the container. Container
  
\\medbackup.med.ad.bgu.ac.il\tsmc17\18\1853.-
  ncf in stgpool CPOOL has been marked as UNAVAILABLE 
and
  should be audited to validate accessibility and 
content.
   (SESSION: 197883)

[From the client side:]

During the incr of a large filespace:

Normal File--> 7.132.827 \\medfs2\e$\medusers14\angel\17.8.23 BU - 
E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
general MRI data\For-Crop-T2W - coronal Copy.pptx  ** Unsuccessful **
ANS1228E Sending of object '\\medfs2\e$\medusers14\angel\17.8.23 BU - 
E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
general MRI data\For-Crop-T2W - coronal Copy.pptx' failed.
ANS1311E Server out of data storage space

[I ran sel of the latest file. It failed because all containerdirs were 
unavailable.]

ANS1804E Selective Backup processing of '\\medfs2\e$\medusers14\angel\17.8.23 
BU - E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
general MRI data\For-Crop-T2W - coronal Copy.pptx' finished with failures.


Total number of objects inspected:1
Total number of objects backed up:0
Total number of objects updated:  0
Total number of objects rebound:  0
Total number of objects deleted:  0
Total number of objects expired:  0
Total number of objects failed:   1
 ...
Network data transfer rate:  148.306,35 KB/sec
Aggregate data transfer rate:211,50 KB/sec
Objects compressed by:0%
Total data reduction ratio:0.23%
Subfile objects reduced by:   0%
Elapsed processing time:   00:00:32
ANS1311E Server out of data storage space

[Then I updated the containerdirs to readwrite and ran the selective backup. No 
problem]
---
Protect> sel '\\medfs2\e$\medusers14\angel\17.8.23 BU - 
E\MyDocs(E)-PrevOLD-D\MyDocs (D)\PERSON-CRITER\FAMILY\OMRI's folder 
313843070\OMRI 1-16 medical issue\MRIs - CTs - OMRI\MY PROCESSING of MRI and 
general MRI data\For-Crop-T2W - coronal Copy.pptx'
Selective Backup function invoked.

Nor

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[Chavdar Cholev]

Just to make sure that we are on the same page...
You have TSM installed on VM running on VMware. This VM has few LUNs
presented and those LUN are used for containers?

Short in the dark:
1. Check VM resources if they are as IBM TSM blue print.
2. Check LUNs/HDDs response time in perf. monitor. The response time should
around 20-30 Ms during the backup operating.
3. Do you know if those HDDd for LUNs are .vmdk or RDM (raw device map)?

Thank you!
Chavdar

On Saturday, August 19, 2023, David L.A. De Leeuw  wrote:

> Hi TSM experts,
>
> Our incr backup fails consistently in the last few days. It starts alright
> but after a few gigabyte on the client we get the error:
>
> ANS1301E This operation cannot continue due to an error on the IBM
> Spectrum Protect server. See your IBM Spectrum Protect server administrator
> for assistance.
>
> On the server side we see:
>
> 18-08-2023 22:57:25 ANR2012W Error encountered for storage pool directory:
> \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> CPOOL. (SESSION: 194578)
> 18-08-2023 22:57:25 ANR0530W Transaction failed for session 194578 for node
> MEDFS2 (WinNT) - internal server error detected.
> (SESSION: 194578)
> 18-08-2023 22:57:26 ANR2012W Error encountered for storage pool directory:
> \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> CPOOL. (SESSION: 194578)
>
>
> Then we find one or more containers unavailable. We fix the containers
> with "audit container ... action=scanall"
> No errors are found. But the next backup will fail again.
>
> The server is on 8.1.17, the client as well.
> The containers are on a number of disks on a shared windows server 2019.
> There have been some updates on the windows server recently.
> (KB5029247,KB5029647)
>
> The audits are fine, data is accessible, but backups fail.
> Any ideas ?
>
> David de Leeuw
> Ben-Gurion University of the Negev
> Beer Sheva Israel
>
>

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[Michael Prix]

Hello David,

  if partial incremental backups work, but full incrementals fail, this points 
to a problem with the client.
Have you tried the clientoption "memoryefficientbackup" already for this 
client? It might be that the number of files grew in the past and just hit the 
memory boundaries of the client.

Always remember: IBM error messages aren't there to help YOU :-)

-- 
Michael Prix




August 20, 2023 at 10:46 AM, "David L.A. De Leeuw"  wrote:


> 
> Hi Chavdar,
> 
> For the containers we use a dedicated Windows Server, (on VMWare EsxI), with 
> just a bunch of disks on NTFS. Each disk about 4 TB.
> INCR of one specific client crashes since a few days every time. There are 
> about 25 million files there (40 TB).
> If I try incremental backup on a part of the disk, it works fine.
> 
> On the server with the containers, I checked events, antivirus, uninstalled 
> Windows updates of August. Did not find any problems.
> The only similar case I found was here:
> https://adsm.org/forum/index.php?threads/storagepool-containers-going-unavailable.32858/
> 
> That points to a problem of memory on the Spectrum Protect server.
> Our system manager is on holidays this week, so cannot check that further.
> 
> Thanks
> 
> David
> 
> -Original Message-
> From: ADSM: Dist Stor Manager  On Behalf Of Chavdar 
> Cholev
> Sent: Sunday, August 20, 2023 11:16 AM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client
> 
> Hi David,
> What kind of storage do you use for containers (SAN, NAS...)?
> 
> On Saturday, August 19, 2023, David L.A. De Leeuw  wrote:
> 
> > 
> > Hi TSM experts,
> > 
> >  Our incr backup fails consistently in the last few days. It starts 
> >  alright but after a few gigabyte on the client we get the error:
> > 
> >  ANS1301E This operation cannot continue due to an error on the IBM 
> >  Spectrum Protect server. See your IBM Spectrum Protect server 
> >  administrator for assistance.
> > 
> >  On the server side we see:
> > 
> >  18-08-2023 22:57:25 ANR2012W Error encountered for storage pool directory:
> >  \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> >  CPOOL. (SESSION: 194578)
> >  18-08-2023 22:57:25 ANR0530W Transaction failed for session 194578 for 
> >  node
> >  MEDFS2 (WinNT) - internal server error detected.
> >  (SESSION: 194578)
> >  18-08-2023 22:57:26 ANR2012W Error encountered for storage pool directory:
> >  \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> >  CPOOL. (SESSION: 194578)
> > 
> >  Then we find one or more containers unavailable. We fix the containers 
> >  with "audit container ... action=scanall"
> >  No errors are found. But the next backup will fail again.
> > 
> >  The server is on 8.1.17, the client as well.
> >  The containers are on a number of disks on a shared windows server 2019.
> >  There have been some updates on the windows server recently.
> >  (KB5029247,KB5029647)
> > 
> >  The audits are fine, data is accessible, but backups fail.
> >  Any ideas ?
> > 
> >  David de Leeuw
> >  Ben-Gurion University of the Negev
> >  Beer Sheva Israel
> >
>

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[David L.A. De Leeuw]

Hi Chavdar,

For the containers we use a dedicated Windows Server, (on VMWare EsxI), with 
just a bunch of disks on NTFS. Each disk about 4 TB.
INCR of one specific client crashes since a few days every time. There are 
about 25 million files there (40 TB).
If I try incremental backup on a part of the disk, it works fine.

On the server with the containers, I checked events, antivirus, uninstalled 
Windows updates of August. Did not find any problems.
The only similar case I found was here:
https://adsm.org/forum/index.php?threads/storagepool-containers-going-unavailable.32858/

That points to a problem of memory on the Spectrum Protect server.
Our system manager is on holidays this week, so cannot check that further.

Thanks

David


-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Chavdar Cholev
Sent: Sunday, August 20, 2023 11:16 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] INCR backups fail ! TSM 8.1.17 Windows Server and client

Hi David,
What kind of storage do you use for containers (SAN, NAS...)?

On Saturday, August 19, 2023, David L.A. De Leeuw  wrote:

> Hi TSM experts,
>
> Our incr backup fails consistently in the last few days. It starts 
> alright but after a few gigabyte on the client we get the error:
>
> ANS1301E This operation cannot continue due to an error on the IBM 
> Spectrum Protect server. See your IBM Spectrum Protect server 
> administrator for assistance.
>
> On the server side we see:
>
> 18-08-2023 22:57:25 ANR2012W Error encountered for storage pool directory:
> \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> CPOOL. (SESSION: 194578)
> 18-08-2023 22:57:25 ANR0530W Transaction failed for session 194578 for 
> node
> MEDFS2 (WinNT) - internal server error detected.
> (SESSION: 194578)
> 18-08-2023 22:57:26 ANR2012W Error encountered for storage pool directory:
> \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> CPOOL. (SESSION: 194578)
>
>
> Then we find one or more containers unavailable. We fix the containers 
> with "audit container ... action=scanall"
> No errors are found. But the next backup will fail again.
>
> The server is on 8.1.17, the client as well.
> The containers are on a number of disks on a shared windows server 2019.
> There have been some updates on the windows server recently.
> (KB5029247,KB5029647)
>
> The audits are fine, data is accessible, but backups fail.
> Any ideas ?
>
> David de Leeuw
> Ben-Gurion University of the Negev
> Beer Sheva Israel
>
>

Re: INCR backups fail ! TSM 8.1.17 Windows Server and client

[Chavdar Cholev]

Hi David,
What kind of storage do you use for containers (SAN, NAS...)?

On Saturday, August 19, 2023, David L.A. De Leeuw  wrote:

> Hi TSM experts,
>
> Our incr backup fails consistently in the last few days. It starts alright
> but after a few gigabyte on the client we get the error:
>
> ANS1301E This operation cannot continue due to an error on the IBM
> Spectrum Protect server. See your IBM Spectrum Protect server administrator
> for assistance.
>
> On the server side we see:
>
> 18-08-2023 22:57:25 ANR2012W Error encountered for storage pool directory:
> \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> CPOOL. (SESSION: 194578)
> 18-08-2023 22:57:25 ANR0530W Transaction failed for session 194578 for node
> MEDFS2 (WinNT) - internal server error detected.
> (SESSION: 194578)
> 18-08-2023 22:57:26 ANR2012W Error encountered for storage pool directory:
> \\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
> CPOOL. (SESSION: 194578)
>
>
> Then we find one or more containers unavailable. We fix the containers
> with "audit container ... action=scanall"
> No errors are found. But the next backup will fail again.
>
> The server is on 8.1.17, the client as well.
> The containers are on a number of disks on a shared windows server 2019.
> There have been some updates on the windows server recently.
> (KB5029247,KB5029647)
>
> The audits are fine, data is accessible, but backups fail.
> Any ideas ?
>
> David de Leeuw
> Ben-Gurion University of the Negev
> Beer Sheva Israel
>
>

INCR backups fail ! TSM 8.1.17 Windows Server and client

[David L.A. De Leeuw]

Hi TSM experts,
 
Our incr backup fails consistently in the last few days. It starts alright but 
after a few gigabyte on the client we get the error:

ANS1301E This operation cannot continue due to an error on the IBM Spectrum 
Protect server. See your IBM Spectrum Protect server administrator for 
assistance.

On the server side we see:

18-08-2023 22:57:25 ANR2012W Error encountered for storage pool directory:
\\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
CPOOL. (SESSION: 194578)
18-08-2023 22:57:25 ANR0530W Transaction failed for session 194578 for node
MEDFS2 (WinNT) - internal server error detected.
(SESSION: 194578)
18-08-2023 22:57:26 ANR2012W Error encountered for storage pool directory:
\\medbackup.med.ad.bgu.ac.il\tsmc1 in storage pool:
CPOOL. (SESSION: 194578)
 

Then we find one or more containers unavailable. We fix the containers with 
"audit container ... action=scanall"
No errors are found. But the next backup will fail again.

The server is on 8.1.17, the client as well.
The containers are on a number of disks on a shared windows server 2019. There 
have been some updates on the windows server recently. (KB5029247,KB5029647)

The audits are fine, data is accessible, but backups fail.
Any ideas ?

David de Leeuw
Ben-Gurion University of the Negev
Beer Sheva Israel

Re: TSM db backup delayed or hung for more than few hours

[Skylar Thompson]

Ah, OK, I was thinking in relation to tape devices, I wouldn't expect SAN
discovery to be in play for disk, or at least not in a way that's visible
to TSM.

Definitely let the list know what the results of the PMR are, I suspect
many of us will be curious...

On Tue, May 30, 2023 at 11:52:14PM +0800, Saravanan Palanisamy wrote:
> Hi Skylar
>
> We use disk for db backup and it will finish within 30 minutes once db
> backup started at db2 level for 2 tb database.
>
> There is no errors or warnings reported during db backup and only concern
> it takes more than 30 min to see db backup message in db2diag log though
> tsm side backup db command issued.
>
> We just need to ensure db backup get complete two times to clear all the
> logs.
>
> Sometime we noticed there was no db backup message in db2 diag log and db
> backup process hang with 0 bytes even after  12 hours.
>
> There is more stress for us to monitor db backup process very closely as it
> may get hung anytime.
>
> Our data fully resides on all flash disk ( db , active , archive and
> directory container )
>
>
>
>
>
> On Tue, 30 May 2023 at 11:29 PM, Skylar Thompson  wrote:
>
> > What's the device type for the DB backup? We'll see this when SAN discovery
> > hasn't caught up with new device paths after a replacement. The telltale
> > will be ANR8975I messages on the library manager and ANR8974I messages on
> > the clients.
> >
> > As others noted, though, it can also be an API package problem. It might be
> > fruitful looking in the activity log around the time backups start, along
> > with the db2diag.log and tsmdbmgr.log files.
> >
> > On Mon, May 29, 2023 at 05:16:50AM +0800, Saravanan Palanisamy wrote:
> > > !---|
> > >   This Message Is From an Untrusted Sender
> > >   You have not previously corresponded with this sender.
> > >   See https://itconnect.uw.edu/email-tags for additional
> > >   information.Please contact the UW-IT Service Center,
> > >   h...@uw.edu 206.221.5000, for assistance.
> > > |---!
> > >
> > > V8.1.18
> > >
> > > We have noticed tsm backup started at tsm level but  didn???t issue any
> > message to db2 and tsm level db backup appears hung
> > >
> > > db2 list utilities show details - No process
> > > db2diag.log - didn???t see any message like ( Starting an online db
> > backup )
> > >
> > > Tried to restart tsm server to clear db2 hung process but no luck and db
> > backup again hung.
> > >
> > > Main issue here :
> > >
> > > When we start tsm db backup it???s not able start and archive log became
> > full. Tried to increase archive log few times.
> > >
> > > We tried almost 3 times ( we waited more than 1 hour ) but no luck.
> > >
> > > we always run db backup to clear archive log so that server will up and
> > running.
> > >
> > > Same issue observed on last week in different server and today one more
> > server.
> > >
> > > We have raised PMR to check this issue.
> > >
> > > Did anyone face this issue in v8.1.18 ?
> > >
> > > We have never seen this issue earlier but noticed after upgrade to
> > v8.1.18
> > >
> > >
> > > Usually when we issue db backup command it will immediately start and we
> > will see tsmdbmgr sessions in tsm server.  It looks like strange issue and
> > causing outage for production jobs.
> > >
> > >
> > > Regards
> > > Sarav
> > > +65 9857 8665
> >
> > --
> > -- Skylar Thompson (skyl...@u.washington.edu)
> > -- Genome Sciences Department (UW Medicine), System Administrator
> > -- Foege Building S046, (206)-685-7354
> > -- Pronouns: He/Him/His
> >
> --
> Thanks & Regards,
> Saravanan
> Mobile: +65-8228 4384

--
-- Skylar Thompson (skyl...@u.washington.edu)
-- Genome Sciences Department (UW Medicine), System Administrator
-- Foege Building S046, (206)-685-7354
-- Pronouns: He/Him/His

Re: TSM db backup delayed or hung for more than few hours

[Saravanan Palanisamy]

Hi Skylar

We use disk for db backup and it will finish within 30 minutes once db
backup started at db2 level for 2 tb database.

There is no errors or warnings reported during db backup and only concern
it takes more than 30 min to see db backup message in db2diag log though
tsm side backup db command issued.

We just need to ensure db backup get complete two times to clear all the
logs.

Sometime we noticed there was no db backup message in db2 diag log and db
backup process hang with 0 bytes even after  12 hours.

There is more stress for us to monitor db backup process very closely as it
may get hung anytime.

Our data fully resides on all flash disk ( db , active , archive and
directory container )





On Tue, 30 May 2023 at 11:29 PM, Skylar Thompson  wrote:

> What's the device type for the DB backup? We'll see this when SAN discovery
> hasn't caught up with new device paths after a replacement. The telltale
> will be ANR8975I messages on the library manager and ANR8974I messages on
> the clients.
>
> As others noted, though, it can also be an API package problem. It might be
> fruitful looking in the activity log around the time backups start, along
> with the db2diag.log and tsmdbmgr.log files.
>
> On Mon, May 29, 2023 at 05:16:50AM +0800, Saravanan Palanisamy wrote:
> > !---|
> >   This Message Is From an Untrusted Sender
> >   You have not previously corresponded with this sender.
> >   See https://itconnect.uw.edu/email-tags for additional
> >   information.Please contact the UW-IT Service Center,
> >   h...@uw.edu 206.221.5000, for assistance.
> > |-----------!
> >
> > V8.1.18
> >
> > We have noticed tsm backup started at tsm level but  didn???t issue any
> message to db2 and tsm level db backup appears hung
> >
> > db2 list utilities show details - No process
> > db2diag.log - didn???t see any message like ( Starting an online db
> backup )
> >
> > Tried to restart tsm server to clear db2 hung process but no luck and db
> backup again hung.
> >
> > Main issue here :
> >
> > When we start tsm db backup it???s not able start and archive log became
> full. Tried to increase archive log few times.
> >
> > We tried almost 3 times ( we waited more than 1 hour ) but no luck.
> >
> > we always run db backup to clear archive log so that server will up and
> running.
> >
> > Same issue observed on last week in different server and today one more
> server.
> >
> > We have raised PMR to check this issue.
> >
> > Did anyone face this issue in v8.1.18 ?
> >
> > We have never seen this issue earlier but noticed after upgrade to
> v8.1.18
> >
> >
> > Usually when we issue db backup command it will immediately start and we
> will see tsmdbmgr sessions in tsm server.  It looks like strange issue and
> causing outage for production jobs.
> >
> >
> > Regards
> > Sarav
> > +65 9857 8665
>
> --
> -- Skylar Thompson (skyl...@u.washington.edu)
> -- Genome Sciences Department (UW Medicine), System Administrator
> -- Foege Building S046, (206)-685-7354
> -- Pronouns: He/Him/His
>
--
Thanks & Regards,
Saravanan
Mobile: +65-8228 4384

Re: TSM db backup delayed or hung for more than few hours

[Skylar Thompson]

What's the device type for the DB backup? We'll see this when SAN discovery
hasn't caught up with new device paths after a replacement. The telltale
will be ANR8975I messages on the library manager and ANR8974I messages on
the clients.

As others noted, though, it can also be an API package problem. It might be
fruitful looking in the activity log around the time backups start, along
with the db2diag.log and tsmdbmgr.log files.

On Mon, May 29, 2023 at 05:16:50AM +0800, Saravanan Palanisamy wrote:
> !---|
>   This Message Is From an Untrusted Sender
>   You have not previously corresponded with this sender.
>   See https://itconnect.uw.edu/email-tags for additional
>   information.Please contact the UW-IT Service Center,
>   h...@uw.edu 206.221.5000, for assistance.
> |---!
>
> V8.1.18
>
> We have noticed tsm backup started at tsm level but  didn???t issue any 
> message to db2 and tsm level db backup appears hung
>
> db2 list utilities show details - No process
> db2diag.log - didn???t see any message like ( Starting an online db backup )
>
> Tried to restart tsm server to clear db2 hung process but no luck and db 
> backup again hung.
>
> Main issue here :
>
> When we start tsm db backup it???s not able start and archive log became 
> full. Tried to increase archive log few times.
>
> We tried almost 3 times ( we waited more than 1 hour ) but no luck.
>
> we always run db backup to clear archive log so that server will up and 
> running.
>
> Same issue observed on last week in different server and today one more 
> server.
>
> We have raised PMR to check this issue.
>
> Did anyone face this issue in v8.1.18 ?
>
> We have never seen this issue earlier but noticed after upgrade to v8.1.18
>
>
> Usually when we issue db backup command it will immediately start and we will 
> see tsmdbmgr sessions in tsm server.  It looks like strange issue and causing 
> outage for production jobs.
>
>
> Regards
> Sarav
> +65 9857 8665

--
-- Skylar Thompson (skyl...@u.washington.edu)
-- Genome Sciences Department (UW Medicine), System Administrator
-- Foege Building S046, (206)-685-7354
-- Pronouns: He/Him/His

Re: TSM db backup delayed or hung for more than few hours

[Saravanan Palanisamy]

Hi Vikas 

We are using latest tsm client (8.1.17.2) and server version (8.1.18) 

Regards 
Sarav 
+65 9857 8665


> On 30 May 2023, at 2:27 AM, Vikas Arora  wrote:
> 
> Hi Sarav,
> 
> May  I ask which tsm client version (dsmadmc) you are using? Can you please
> try to update it with the latest version and then re-try? I had a similar
> experience a few days ago and that helped!
> 
> thanks
> 
> 
> 
> Grüße
> Vikas Arora
> 
> *e Mail* : vikar...@gmail.com
> 
> 
>> On Mon, 29 May 2023 at 13:38, Saravanan Palanisamy <
>> evergreen.sa...@gmail.com> wrote:
>> 
>> Today it took more than 90 min to initiate db backup at db2 level where as
>> admin schedule issues backup db command 90 min before and backup got
>> completed. We will monitor closely.
>> 
>> Regards
>> Sarav
>> +65 9857 8665
>> 
>> 
>>>> On 29 May 2023, at 3:29 PM, David L.A. De Leeuw  wrote:
>>> 
>>> Hi Sarav
>>> 
>>> We had a similar case, just yesterday. I noticed the daily full database
>> backup schedule was missed a number of days.
>>> Then I found that five days ago the daily full backup just hanged.
>>> Canceling the process hung as well on "pending".
>>> 
>>> I contacted our system administrator and she reported that db2 got
>> "stuck".
>>> She rebooted the server, and the next backup ran as usual.
>>> If this repeats, we will open a PMR as well.
>>> 
>>> Our version: 8.1.17
>>> 
>>> Best wishes
>>> 
>>> David de Leeuw
>>> 
>>> -Original Message-----
>>> From: ADSM: Dist Stor Manager  On Behalf Of
>> Saravanan Palanisamy
>>> Sent: Monday, May 29, 2023 12:17 AM
>>> To: ADSM-L@VM.MARIST.EDU
>>> Subject: [ADSM-L] TSM db backup delayed or hung for more than few hours
>>> 
>>> V8.1.18
>>> 
>>> We have noticed tsm backup started at tsm level but  didn’t issue any
>> message to db2 and tsm level db backup appears hung
>>> 
>>> db2 list utilities show details - No process db2diag.log - didn’t see
>> any message like ( Starting an online db backup )
>>> 
>>> Tried to restart tsm server to clear db2 hung process but no luck and db
>> backup again hung.
>>> 
>>> Main issue here :
>>> 
>>> When we start tsm db backup it’s not able start and archive log became
>> full. Tried to increase archive log few times.
>>> 
>>> We tried almost 3 times ( we waited more than 1 hour ) but no luck.
>>> 
>>> we always run db backup to clear archive log so that server will up and
>> running.
>>> 
>>> Same issue observed on last week in different server and today one more
>> server.
>>> 
>>> We have raised PMR to check this issue.
>>> 
>>> Did anyone face this issue in v8.1.18 ?
>>> 
>>> We have never seen this issue earlier but noticed after upgrade to
>> v8.1.18
>>> 
>>> 
>>> Usually when we issue db backup command it will immediately start and we
>> will see tsmdbmgr sessions in tsm server.  It looks like strange issue and
>> causing outage for production jobs.
>>> 
>>> 
>>> Regards
>>> Sarav
>>> +65 9857 8665
>> 

Re: TSM db backup delayed or hung for more than few hours

[Vikas Arora]

Hi Sarav,

May  I ask which tsm client version (dsmadmc) you are using? Can you please
try to update it with the latest version and then re-try? I had a similar
experience a few days ago and that helped!

thanks



Grüße
Vikas Arora

*e Mail* : vikar...@gmail.com


On Mon, 29 May 2023 at 13:38, Saravanan Palanisamy <
evergreen.sa...@gmail.com> wrote:

> Today it took more than 90 min to initiate db backup at db2 level where as
> admin schedule issues backup db command 90 min before and backup got
> completed. We will monitor closely.
>
> Regards
> Sarav
> +65 9857 8665
>
>
> > On 29 May 2023, at 3:29 PM, David L.A. De Leeuw  wrote:
> >
> > Hi Sarav
> >
> > We had a similar case, just yesterday. I noticed the daily full database
> backup schedule was missed a number of days.
> > Then I found that five days ago the daily full backup just hanged.
> > Canceling the process hung as well on "pending".
> >
> > I contacted our system administrator and she reported that db2 got
> "stuck".
> > She rebooted the server, and the next backup ran as usual.
> > If this repeats, we will open a PMR as well.
> >
> > Our version: 8.1.17
> >
> > Best wishes
> >
> > David de Leeuw
> >
> > -Original Message-
> > From: ADSM: Dist Stor Manager  On Behalf Of
> Saravanan Palanisamy
> > Sent: Monday, May 29, 2023 12:17 AM
> > To: ADSM-L@VM.MARIST.EDU
> > Subject: [ADSM-L] TSM db backup delayed or hung for more than few hours
> >
> > V8.1.18
> >
> > We have noticed tsm backup started at tsm level but  didn’t issue any
> message to db2 and tsm level db backup appears hung
> >
> > db2 list utilities show details - No process db2diag.log - didn’t see
> any message like ( Starting an online db backup )
> >
> > Tried to restart tsm server to clear db2 hung process but no luck and db
> backup again hung.
> >
> > Main issue here :
> >
> > When we start tsm db backup it’s not able start and archive log became
> full. Tried to increase archive log few times.
> >
> > We tried almost 3 times ( we waited more than 1 hour ) but no luck.
> >
> > we always run db backup to clear archive log so that server will up and
> running.
> >
> > Same issue observed on last week in different server and today one more
> server.
> >
> > We have raised PMR to check this issue.
> >
> > Did anyone face this issue in v8.1.18 ?
> >
> > We have never seen this issue earlier but noticed after upgrade to
> v8.1.18
> >
> >
> > Usually when we issue db backup command it will immediately start and we
> will see tsmdbmgr sessions in tsm server.  It looks like strange issue and
> causing outage for production jobs.
> >
> >
> > Regards
> > Sarav
> > +65 9857 8665
>

Re: TSM db backup delayed or hung for more than few hours

[Saravanan Palanisamy]

Today it took more than 90 min to initiate db backup at db2 level where as 
admin schedule issues backup db command 90 min before and backup got completed. 
We will monitor closely. 

Regards 
Sarav 
+65 9857 8665


> On 29 May 2023, at 3:29 PM, David L.A. De Leeuw  wrote:
> 
> Hi Sarav
> 
> We had a similar case, just yesterday. I noticed the daily full database 
> backup schedule was missed a number of days. 
> Then I found that five days ago the daily full backup just hanged.
> Canceling the process hung as well on "pending".
> 
> I contacted our system administrator and she reported that db2 got "stuck".
> She rebooted the server, and the next backup ran as usual.
> If this repeats, we will open a PMR as well. 
> 
> Our version: 8.1.17
> 
> Best wishes
> 
> David de Leeuw
> 
> -Original Message-
> From: ADSM: Dist Stor Manager  On Behalf Of Saravanan 
> Palanisamy
> Sent: Monday, May 29, 2023 12:17 AM
> To: ADSM-L@VM.MARIST.EDU
> Subject: [ADSM-L] TSM db backup delayed or hung for more than few hours
> 
> V8.1.18 
> 
> We have noticed tsm backup started at tsm level but  didn’t issue any message 
> to db2 and tsm level db backup appears hung 
> 
> db2 list utilities show details - No process db2diag.log - didn’t see any 
> message like ( Starting an online db backup ) 
> 
> Tried to restart tsm server to clear db2 hung process but no luck and db 
> backup again hung. 
> 
> Main issue here : 
> 
> When we start tsm db backup it’s not able start and archive log became full. 
> Tried to increase archive log few times. 
> 
> We tried almost 3 times ( we waited more than 1 hour ) but no luck. 
> 
> we always run db backup to clear archive log so that server will up and 
> running. 
> 
> Same issue observed on last week in different server and today one more 
> server. 
> 
> We have raised PMR to check this issue. 
> 
> Did anyone face this issue in v8.1.18 ? 
> 
> We have never seen this issue earlier but noticed after upgrade to v8.1.18 
> 
> 
> Usually when we issue db backup command it will immediately start and we will 
> see tsmdbmgr sessions in tsm server.  It looks like strange issue and causing 
> outage for production jobs. 
> 
> 
> Regards
> Sarav 
> +65 9857 8665

Re: TSM db backup delayed or hung for more than few hours

[David L.A. De Leeuw]

Hi Sarav

We had a similar case, just yesterday. I noticed the daily full database backup 
schedule was missed a number of days. 
Then I found that five days ago the daily full backup just hanged.
Canceling the process hung as well on "pending".

I contacted our system administrator and she reported that db2 got "stuck".
She rebooted the server, and the next backup ran as usual.
If this repeats, we will open a PMR as well. 

Our version: 8.1.17

Best wishes

David de Leeuw

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Saravanan 
Palanisamy
Sent: Monday, May 29, 2023 12:17 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] TSM db backup delayed or hung for more than few hours

V8.1.18 

We have noticed tsm backup started at tsm level but  didn’t issue any message 
to db2 and tsm level db backup appears hung 

db2 list utilities show details - No process db2diag.log - didn’t see any 
message like ( Starting an online db backup ) 

Tried to restart tsm server to clear db2 hung process but no luck and db backup 
again hung. 

Main issue here : 

When we start tsm db backup it’s not able start and archive log became full. 
Tried to increase archive log few times. 

We tried almost 3 times ( we waited more than 1 hour ) but no luck. 

we always run db backup to clear archive log so that server will up and 
running. 

Same issue observed on last week in different server and today one more server. 

We have raised PMR to check this issue. 

Did anyone face this issue in v8.1.18 ? 

We have never seen this issue earlier but noticed after upgrade to v8.1.18 


Usually when we issue db backup command it will immediately start and we will 
see tsmdbmgr sessions in tsm server.  It looks like strange issue and causing 
outage for production jobs. 


Regards
Sarav 
+65 9857 8665

TSM db backup delayed or hung for more than few hours

[Saravanan Palanisamy]

V8.1.18 

We have noticed tsm backup started at tsm level but  didn’t issue any message 
to db2 and tsm level db backup appears hung 

db2 list utilities show details - No process 
db2diag.log - didn’t see any message like ( Starting an online db backup ) 

Tried to restart tsm server to clear db2 hung process but no luck and db backup 
again hung. 

Main issue here : 

When we start tsm db backup it’s not able start and archive log became full. 
Tried to increase archive log few times. 

We tried almost 3 times ( we waited more than 1 hour ) but no luck. 

we always run db backup to clear archive log so that server will up and 
running. 

Same issue observed on last week in different server and today one more server. 

We have raised PMR to check this issue. 

Did anyone face this issue in v8.1.18 ? 

We have never seen this issue earlier but noticed after upgrade to v8.1.18 


Usually when we issue db backup command it will immediately start and we will 
see tsmdbmgr sessions in tsm server.  It looks like strange issue and causing 
outage for production jobs. 


Regards 
Sarav 
+65 9857 8665

Re: Restoring virus infected file halts TSM client

[Andrew Raibeck]

Hi Bent,

It sounds like a good idea to open a support case. Suggestion for doc you can 
collect in advance:

1. Add these options to the dsm.opt file:

  TRACEFLAGS SERVICE
  TRACEFILE C:\somedir\client_trace.txt

You can choose any valid file path for "C:\somedir\client_trace.txt".

2. Run the "dsmc restore ..." command and redirect stdout and stderr to a file. 
If you can reproduce it with a small directory, that will help keep the trace 
file from growing too large. For example:

  dsmc restore C:\mydata\ -subdir=yes > dsmc_restore.txt 2>&1

3. Collect the trace file, dsmc_restore.txt, and dsmerror.log, and submit them 
when you open the case.

I am not sure what causes the ANS0128S in this case. That message is issued 
several seconds after the ANSE, so I do not know if the messages are 
related to each other. If you can capture that in a trace, that might be 
insightful.

To collect documentation for several concurrent dsmc processes, you can do 
something like this:

  dsmc restore C:\mydata1\ -subdir=yes -traceflags=service 
-tracefile=C:\somedir\client_trace_1.txt > dsmc_restore_1.txt 2>&1
  dsmc restore C:\mydata2\ -subdir=yes -traceflags=service 
-tracefile=C:\somedir\client_trace_2.txt > dsmc_restore_2.txt 2>&1
  dsmc restore C:\mydata3\ -subdir=yes -traceflags=service 
-tracefile=C:\somedir\client_trace_3.txt > dsmc_restore_3.txt 2>&1
  dsmc restore C:\mydata4\ -subdir=yes -traceflags=service 
-tracefile=C:\somedir\client_trace_4.txt > dsmc_restore_4.txt 2>&1

Use your normal "dsmc restore" syntax, but add the -traceflags options, the 
-tracefile option, and output redirection as in the preceding examples.

Finally, based on the info you shared, I suggest you try the restore with this 
option:

  TESTFLAGS CONTINUEWITHUNKNOWNRC

See if that allows the restore to run to completion, skipping past the invalid 
files.

Best regards,

Andy

Andrew Raibeck
IBM Spectrum Protect Level 3
IBM Storage
stor...@us.ibm.com

IBM

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Bent 
Christensen (BVC)
Sent: Tuesday, 24 January, 2023 09:32
To: ADSM-L@VM.MARIST.EDU
Subject: [EXTERNAL] Re: Restoring virus infected file halts TSM client

Hi Andrew,

Thanks for your response and suggestion.

I have been using the weekend to dig a little deeper into the issue, and it 
turns out that if I just restore the folder containing the infected file, TSM 
restores all other files and just responds with a:

01/19/2023 15:58:57 ANSE ..\..\common\winnt\ntrc.cpp(784): Received Win32 
RC 225 (0x00e1) from HlClose(): CreateFile. Error description: Operation 
did not complete successfully because the file contains a virus or potentially 
unwanted software.

But if I run 3-4 or more DSMC RESTORE sessions simultaneously the session which 
has the infected file terminates with this in DSMERROR.LOG:
01/18/2023 17:27:31 ANSE ..\..\common\winnt\ntrc.cpp(784): Received Win32 
RC 225 (0x00e1) from HlClose(): CreateFile. Error description: Operation 
did not complete successfully because the file contains a virus or potentially 
unwanted software.
01/18/2023 17:27:42 ANS1028S An internal program error occurred.

In the last scenario the server receiving the restore is pretty heavily loaded 
on CPU usage with Windows Defender using the major part of the CPUs.

So I will open a case with IBM Support and report this.

 - Bent

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Andrew Raibeck
Sent: Thursday, January 19, 2023 2:19 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Restoring virus infected file halts TSM client

Hello Brent,

Without knowing the specific details of the errors you see, one thing you can 
try is to add this line to the dsm.opt file:

TESTFLAGS CONTINUERESTORE

Restart the client, and see if that causes the operation to continue with the 
next file after an error is reported.

If that does not work, then what error message(s) do you see? What messages, 
coincident with the failed restore, are logged to dsmerror.log? Be sure to 
include the full text, though you can redact user names and file names, as 
appropriate.

Based on that, I might have some other ideas, or else I will suggest opening a 
case with IBM Support.

Unsolicited thought that might be redundant, but I mention it anyway :-) please 
use appropriate care when restoring the files, even if the AV software is 
guarding against suspicious files.

Regards,

Andy

Andrew Raibeck
IBM Spectrum Protect Level 3
IBM Storage
stor...@us.ibm.com

IBM

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Bent 
Christensen (BVC)
Sent: Thursday, 19 January, 2023 06:02
To: ADSM-L@VM.MARIST.EDU
Subject: [EXTERNAL] Restoring virus infected file halts TSM client

Hello,

Just wondered if anyone have had the same issue and maybe found a solution for 
it:

Now and then we are tasked with restoring data that we

Re: Restoring virus infected file halts TSM client

[Bent Christensen (BVC)]

Hi Andrew,

Thanks for your response and suggestion.

I have been using the weekend to dig a little deeper into the issue, and it 
turns out that if I just restore the folder containing the infected file, TSM 
restores all other files and just responds with a:

01/19/2023 15:58:57 ANSE ..\..\common\winnt\ntrc.cpp(784): Received Win32 
RC 225 (0x00e1) from HlClose(): CreateFile. Error description: Operation 
did not complete successfully because the file contains a virus or potentially 
unwanted software.

But if I run 3-4 or more DSMC RESTORE sessions simultaneously the session which 
has the infected file terminates with this in DSMERROR.LOG:
01/18/2023 17:27:31 ANSE ..\..\common\winnt\ntrc.cpp(784): Received Win32 
RC 225 (0x00e1) from HlClose(): CreateFile. Error description: Operation 
did not complete successfully because the file contains a virus or potentially 
unwanted software.
01/18/2023 17:27:42 ANS1028S An internal program error occurred.

In the last scenario the server receiving the restore is pretty heavily loaded 
on CPU usage with Windows Defender using the major part of the CPUs.

So I will open a case with IBM Support and report this.

 - Bent

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Andrew Raibeck
Sent: Thursday, January 19, 2023 2:19 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Restoring virus infected file halts TSM client

Hello Brent,

Without knowing the specific details of the errors you see, one thing you can 
try is to add this line to the dsm.opt file:

TESTFLAGS CONTINUERESTORE

Restart the client, and see if that causes the operation to continue with the 
next file after an error is reported.

If that does not work, then what error message(s) do you see? What messages, 
coincident with the failed restore, are logged to dsmerror.log? Be sure to 
include the full text, though you can redact user names and file names, as 
appropriate.

Based on that, I might have some other ideas, or else I will suggest opening a 
case with IBM Support.

Unsolicited thought that might be redundant, but I mention it anyway :-) please 
use appropriate care when restoring the files, even if the AV software is 
guarding against suspicious files.

Regards,

Andy

Andrew Raibeck
IBM Spectrum Protect Level 3
IBM Storage
stor...@us.ibm.com

IBM

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Bent 
Christensen (BVC)
Sent: Thursday, 19 January, 2023 06:02
To: ADSM-L@VM.MARIST.EDU
Subject: [EXTERNAL] Restoring virus infected file halts TSM client

Hello,

Just wondered if anyone have had the same issue and maybe found a solution for 
it:

Now and then we are tasked with restoring data that were backed up very long 
ago back to Windows file shares. In a few cases it turns out that some of these 
old files are infected by virus/malware which was not detected by the AV 
application at the time when the malicious file was written.

When the TSM client tries to restore an infected file back to a Windows server, 
the AV application on the Windows server will of course prevent the file from 
being written. However, the TSM client interprets this as an disk error (or 
something) and terminates the restore processes so any subsequent non-infected 
files are not restored, making it almost impossible to do un-monitored restores 
of these data sets.

Would really appreciate it if anyone got some ideas to circumvent this (except 
for disabling the AV application while restoring)?

Regards

Bent


COWI handles personal data as stated in our Privacy 
Notice<https://www.cowi.com/privacy >.
COWI handles personal data as stated in our Privacy 
Notice<https://www.cowi.com/privacy>.

Re: Restoring virus infected file halts TSM client

[Francisco J]

Hello,

A possibility could be deleting filespace for that file, example:

https://adsm.org/forum/index.php?threads/delete-filespace-of-a-specific-filespace.11954/

Regards




El jue, 19 ene 2023 a las 5:04, Bent Christensen (BVC) ()
escribió:

> Hello,
>
> Just wondered if anyone have had the same issue and maybe found a solution
> for it:
>
> Now and then we are tasked with restoring data that were backed up very
> long ago back to Windows file shares. In a few cases it turns out that some
> of these old files are infected by virus/malware which was not detected by
> the AV application at the time when the malicious file was written.
>
> When the TSM client tries to restore an infected file back to a Windows
> server, the AV application on the Windows server will of course prevent the
> file from being written. However, the TSM client interprets this as an disk
> error (or something) and terminates the restore processes so any subsequent
> non-infected files are not restored, making it almost impossible to do
> un-monitored restores of these data sets.
>
> Would really appreciate it if anyone got some ideas to circumvent this
> (except for disabling the AV application while restoring)?
>
> Regards
>
> Bent
>
>
> COWI handles personal data as stated in our Privacy Notice<
> https://www.cowi.com/privacy>.
>

Re: Restoring virus infected file halts TSM client

[Andrew Raibeck]

Hello Brent,

Without knowing the specific details of the errors you see, one thing you can 
try is to add this line to the dsm.opt file:

TESTFLAGS CONTINUERESTORE

Restart the client, and see if that causes the operation to continue with the 
next file after an error is reported.

If that does not work, then what error message(s) do you see? What messages, 
coincident with the failed restore, are logged to dsmerror.log? Be sure to 
include the full text, though you can redact user names and file names, as 
appropriate.

Based on that, I might have some other ideas, or else I will suggest opening a 
case with IBM Support.

Unsolicited thought that might be redundant, but I mention it anyway :-) please 
use appropriate care when restoring the files, even if the AV software is 
guarding against suspicious files.

Regards,

Andy

Andrew Raibeck
IBM Spectrum Protect Level 3
IBM Storage
stor...@us.ibm.com

IBM

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Bent 
Christensen (BVC)
Sent: Thursday, 19 January, 2023 06:02
To: ADSM-L@VM.MARIST.EDU
Subject: [EXTERNAL] Restoring virus infected file halts TSM client

Hello,

Just wondered if anyone have had the same issue and maybe found a solution for 
it:

Now and then we are tasked with restoring data that were backed up very long 
ago back to Windows file shares. In a few cases it turns out that some of these 
old files are infected by virus/malware which was not detected by the AV 
application at the time when the malicious file was written.

When the TSM client tries to restore an infected file back to a Windows server, 
the AV application on the Windows server will of course prevent the file from 
being written. However, the TSM client interprets this as an disk error (or 
something) and terminates the restore processes so any subsequent non-infected 
files are not restored, making it almost impossible to do un-monitored restores 
of these data sets.

Would really appreciate it if anyone got some ideas to circumvent this (except 
for disabling the AV application while restoring)?

Regards

Bent


COWI handles personal data as stated in our Privacy 
Notice<https://www.cowi.com/privacy >.

Restoring virus infected file halts TSM client

[Bent Christensen (BVC)]

Hello,

Just wondered if anyone have had the same issue and maybe found a solution for 
it:

Now and then we are tasked with restoring data that were backed up very long 
ago back to Windows file shares. In a few cases it turns out that some of these 
old files are infected by virus/malware which was not detected by the AV 
application at the time when the malicious file was written.

When the TSM client tries to restore an infected file back to a Windows server, 
the AV application on the Windows server will of course prevent the file from 
being written. However, the TSM client interprets this as an disk error (or 
something) and terminates the restore processes so any subsequent non-infected 
files are not restored, making it almost impossible to do un-monitored restores 
of these data sets.

Would really appreciate it if anyone got some ideas to circumvent this (except 
for disabling the AV application while restoring)?

Regards

Bent


COWI handles personal data as stated in our Privacy 
Notice<https://www.cowi.com/privacy>.

Re: TSM 6.2.3 data migration

[Julien Sauvanet]

Can you check whether you have the NOMIGRRECLset in your dsmserv.opt ?
You may have followed this procedure but forgot to remove the entries  ?
https://www.ibm.com/support/pages/recommended-dsmserv-restore-db-point-time-procedure


Cordialement / Kind regards

Julien Sauvanet 
He/Him/His
Technical Advisor Spectrum Protect/Spectrum Protect Plus EMEA


Smarter-Essential-Trusted
Technical Advisor Information Website

Phone: +33 6 76 75 43 95
Email: sauva...@fr.ibm.com
Webex: https://ibm.webex.com/meet/sauvanet

Compagnie IBM France
Siège Social : 17, avenue de l'Europe, 92275 Bois-Colombes Cedex
RCS Nanterre 552 118 465
Forme Sociale : S.A.S.
Capital Social : 663.779.730,90 €
SIRET : 552 118 465 03644 - Code NAF 6203Z

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of D'Antonio, 
Vincent E. III [US-US]
Sent: Friday, September 9, 2022 2:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: [EXTERNAL] [ADSM-L] TSM 6.2.3 data migration

Good Morning,

Had an issue with TSM a few weeks ago and did a DB restore.  I had to drop the 
TSM DB before I could do the restore and now that the restore was done migrate 
and reclamations do not run.

I have had several looking but we can not figure out what is going on.  Anyone 
know what I can check or how to correct?

Vincent D'Antonio

Unless otherwise stated above:

Compagnie IBM France
Siège Social : 17, avenue de l'Europe, 92275 Bois-Colombes Cedex
RCS Nanterre 552 118 465
Forme Sociale : S.A.S.
Capital Social : 664 069 390,60 €
SIRET : 552 118 465 03644 - Code NAF 6203Z

Re: [EXTERNE] [ADSM-L] CentOS alternative on TSM V8.x => RockyLinux :+1:

[Bjørn Nachtwey]

Hi,

I'm running my first servers using RockyLinux 8.5 -- it works well, 
LinTapes 3.0.59 also without any problems using kernel 
"4.18.0-348.12.2.el8_5.x86_64".
Some packages differ from RHEL8.4 so I could not just copy the list of 
installed packages.


As far as I remember, you'll need to use the >>-vmargs 
"-DBYPASS_TSM_REQ_CHECKS=true"<< option during install as RockyLinux is 
not officially supported ;-)


best
Bjørn

Am 11.04.2022 um 15:08 schrieb .Maurizio TERUZZI:

Hello Zoltan,

thanks for the answer...

Yes for sure lin_tape drivers is has to be installed on supported 
kernel, I had a long experience installing them when I worked for IBM, 
also tried to recompile some free versions.


I didn't ear about this distribution but I'll test it... many thanks

I'm personally unlucky with the Ubuntu is so different from Redhat 
fork and have no time yet to learn about it (home in the future).



Have a Nice Day.


Grazie
Maurizio

On 07.04.2022 15:18, Zoltan Forray wrote:
Once we heard CentOS was being retired, we started migrating to Rocky 
Linux
(https://rockylinux.org/) for all Linux servers whose 
applications/vendors
would support it.  We have not tried moving the ISP servers to Rocky 
Linux
since our management will not run enterprise applications on an 
unsupported

OS. There is also the question of kernel level support for things like
lin_tape drivers.

Rocky Linux is an open-source enterprise operating system designed to be
100% bug-for-bug compatible with Red Hat Enterprise Linux®. It is under
intensive development by the community.

On Thu, Apr 7, 2022 at 8:53 AM Michaud, Luc [Analyste principal -
environnement AIX]  wrote:


Hello Maurizio !

Our shop is very happy with the migration from AIX to RHEL 8.5 (on
bare-metal x86 servers) for our TSM 8.1.12 ...

Costs a bit of money, but quite a bit less than TSM !

Coming from AIX, I expected Red Hat to be lesser value with regards to
"how-to" support.  Amazingly enough they are quite good at anything 
we're
throwing at them...  They even assisted on a ReAR issue (which is a 
great

tool for restoring Linux systems from TSM)...

We already have an installed base of >400 RHEL servers so the curve is
nonexisting.   Using on-premises Red Hat Satellite for subscription and
patch management.

Regards,

Luc 

Re: [EXTERNE] [ADSM-L] CentOS alternative on TSM V8.x?

[.Maurizio TERUZZI]

Hello Zoltan,

thanks for the answer...

Yes for sure lin_tape drivers is has to be installed on supported 
kernel, I had a long experience installing them when I worked for IBM, 
also tried to recompile some free versions.


I didn't ear about this distribution but I'll test it... many thanks

I'm personally unlucky with the Ubuntu is so different from Redhat fork 
and have no time yet to learn about it (home in the future).



Have a Nice Day.


Grazie
Maurizio

On 07.04.2022 15:18, Zoltan Forray wrote:

Once we heard CentOS was being retired, we started migrating to Rocky Linux
(https://rockylinux.org/) for all Linux servers whose applications/vendors
would support it.  We have not tried moving the ISP servers to Rocky Linux
since our management will not run enterprise applications on an unsupported
OS. There is also the question of kernel level support for things like
lin_tape drivers.

Rocky Linux is an open-source enterprise operating system designed to be
100% bug-for-bug compatible with Red Hat Enterprise Linux®. It is under
intensive development by the community.

On Thu, Apr 7, 2022 at 8:53 AM Michaud, Luc [Analyste principal -
environnement AIX]  wrote:


Hello Maurizio !

Our shop is very happy with the migration from AIX to RHEL 8.5 (on
bare-metal x86 servers) for our TSM 8.1.12 ...

Costs a bit of money, but quite a bit less than TSM !

Coming from AIX, I expected Red Hat to be lesser value with regards to
"how-to" support.  Amazingly enough they are quite good at anything we're
throwing at them...  They even assisted on a ReAR issue (which is a great
tool for restoring Linux systems from TSM)...

We already have an installed base of >400 RHEL servers so the curve is
nonexisting.   Using on-premises Red Hat Satellite for subscription and
patch management.

Regards,

Luc




--


Cordiali saluti / Meilleures salutations
M.Teruzzi

https://www.linkedin.com/in/teruzzi

Formation de plongeur et baptême, dans l’arc Lémanique
CMAS.CH D1 / NITROX 1
Autres didactiques ou niveaux sur demande

Caution: Please do not click links or open attachments unless you recognize the 
sender and know the content is safe.
The Internet Privacy is Yours!

Re: [EXTERNE] [ADSM-L] CentOS alternative on TSM V8.x?

[.Maurizio TERUZZI]

Hello Luc,

thanks for you message :) .
I also come form the AIX World and I known how this systems powerful are.
Actually I don't have the "privilege" to be on those systems, the covid 
period require some cost effort to restart the business as before, the 
question about the OS was about this situation.
RedHat is good but for the actual test environment goals to expensive, 
also without support subscription.



Have a nice day.

Maurizio


Cordiali saluti / Meilleures salutations
M.Teruzzi

https://www.linkedin.com/in/teruzzi

Formation de plongeur et baptême, dans l’arc Lémanique
CMAS.CH D1 / NITROX 1
Autres didactiques ou niveaux sur demande

Caution: Please do not click links or open attachments unless you recognize the 
sender and know the content is safe.
The Internet Privacy is Yours!

On 07.04.2022 14:52, Michaud, Luc [Analyste principal - environnement 
AIX] wrote:

Hello Maurizio !

Our shop is very happy with the migration from AIX to RHEL 8.5 (on bare-metal 
x86 servers) for our TSM 8.1.12 ...

Costs a bit of money, but quite a bit less than TSM !

Coming from AIX, I expected Red Hat to be lesser value with regards to "how-to" 
support.  Amazingly enough they are quite good at anything we're throwing at them...  
They even assisted on a ReAR issue (which is a great tool for restoring Linux systems 
from TSM)...

We already have an installed base of >400 RHEL servers so the curve is 
nonexisting.   Using on-premises Red Hat Satellite for subscription and patch 
management.

Regards,

Luc


--

Re: [EXTERNE] [ADSM-L] CentOS alternative on TSM V8.x?

[Zoltan Forray]

Once we heard CentOS was being retired, we started migrating to Rocky Linux
(https://rockylinux.org/) for all Linux servers whose applications/vendors
would support it.  We have not tried moving the ISP servers to Rocky Linux
since our management will not run enterprise applications on an unsupported
OS. There is also the question of kernel level support for things like
lin_tape drivers.

Rocky Linux is an open-source enterprise operating system designed to be
100% bug-for-bug compatible with Red Hat Enterprise Linux®. It is under
intensive development by the community.

On Thu, Apr 7, 2022 at 8:53 AM Michaud, Luc [Analyste principal -
environnement AIX]  wrote:

> Hello Maurizio !
>
> Our shop is very happy with the migration from AIX to RHEL 8.5 (on
> bare-metal x86 servers) for our TSM 8.1.12 ...
>
> Costs a bit of money, but quite a bit less than TSM !
>
> Coming from AIX, I expected Red Hat to be lesser value with regards to
> "how-to" support.  Amazingly enough they are quite good at anything we're
> throwing at them...  They even assisted on a ReAR issue (which is a great
> tool for restoring Linux systems from TSM)...
>
> We already have an installed base of >400 RHEL servers so the curve is
> nonexisting.   Using on-premises Red Hat Satellite for subscription and
> patch management.
>
> Regards,
>
> Luc
>


-- 
*Zoltan Forray*
Backup & VMware Systems Administrator
Enterprise Compute & Storage Platforms
VCU Infrastructure Services
www.ucc.vcu.edu
zfor...@vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://phishing.vcu.edu/
<https://adminmicro2.questionpro.com>

CentOS alternative on TSM V8.x?

[.Maurizio TERUZZI]

Hello,

hope you still enjoy this good backup tool :) .


Has anyone try other "unsupported" Linux distribution with TSM 8.1 (more
for test environments) than CentOS?

I had my system working on CentOS 8 but it's was EOL on last 31 Dec and
it's time now to move on.

I try several options but I'm no so fun:

- return back CentOS 7.1 is still an open option (where the previous
test system was implemented support until 2025);
- Ubuntu server (officially supported by ISP but "difficult management"
on raid SW & HW) and;

- open-suse (lot of work-around installing).


Hope the forum will bring some ideas.


Buona Giornata
Maurizio

IBM TSM / SP Health check possibilities of your complete environment --- 4 FREE

[Oscar Kolsteren]

Hi guys,

we have developed a TSM / SP healthcheck report which compares your
environment with the IBM Best Practices and IBM Blueprint recommendations.
The generated report is HTML based, full of useful graphs, vital
information and can be viewed offline.

The only thing you need to do is run our CSA collector (collects only TSM
/ SP metadata via select statements) and upload your collected data into
our Self Service Portal. We'll do the rest.

Big bonus It's FREE

See our flyer for more info:
https://ibm.box.com/v/IBM-CSA-FLYER--SP-Health-Check

Please contact your IBM contact person or BP to get going.

Any questions please feel free to reach out.



Best Regards,

Oscar Kolsteren
Lead Storage Consultant, IBM Client Storage Assessment Team (UKI)


Mobile: 44-7884789761 | Home: 44-7783310170
E-mail: oscar.kolste...@uk.ibm.com
Find me on:
Hursley Park
Hursley, SO212JN
United Kingdom

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Dump registry from tsm backup

[.Maurizio TERUZZI]

Hello Guys/Girls


I'm faced to a windows 10 PC Crash and I have bought a new hardware.

I was try to dump the registry part (do be able to extract some parts 
with regedit) from the latest backup but the client v7 (or v8) not 
display anymore the selection to restore it on a specific folder.



Anyone has an idea?


Grazie/Thanks


--


Cordiali saluti / Meilleures salutations
Maurizio Teruzzi

https://www.linkedin.com/in/teruzzi


Formation de plongeur et bapteme dans l’arc Lémanique
CMAS D1 / NITROX 1
Autres sur demande

Caution: Please do not click links or open attachments unless you recognize the 
sender and know the content is safe.

Re: [E] Re: [ADSM-L] TSM alternatives

[Del Hoobler]

Yes, the API is still supported since Spectrum Protect for Databases - 
Data Protection for Oracle is supported with version 8.1. I believe the 
installation only installs the API parts.


Del




"ADSM: Dist Stor Manager"  wrote on 04/28/2021 
03:48:51 AM:

> From: Saravanan Palanisamy 
> To: ADSM-L@VM.MARIST.EDU
> Date: 04/28/2021 03:49 AM
> Subject: [EXTERNAL] Re: [E] Re: [ADSM-L] TSM alternatives
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi Del 
> 
> I could see oracle SPARC support API v8.1 but BA client not released
> for v8.1. What is the purpose of releasing API support for v8.1 
> without BA client ?
> 
> Means no BA client backup supported but still support TDP for oracle
> ? TDP also require BA client to install. 
> 
> Please advise if you have any background. 
> 
> 
> Regards 
> Sarav 
> +65 9857 8665
> 
> 
> > On 26 Apr 2021, at 7:03 PM, Del Hoobler  wrote:
> > 
> > Hi Michael,
> > 
> > You are correct. At the end of this year, the Backup/Archive client 
for 
> > Solaris SPARC will not be supported. As you mention, you can buy 
extended 
> > support, but it will only be for "Usage or Known Defect" issues, not 
for 
> > new currency updates or defects.
> > 
> > I made the previous comment because I wanted other people reading this 
to 
> > make sure they understood it was only Solaris SPARC that this applied 
to, 
> > not Solaris x86_64.
> > 
> > Thank you.
> > 
> > 
> > Del
> > 
> > 
> > 
> > 
> > "ADSM: Dist Stor Manager"  wrote on 04/26/2021 
> > 06:25:34 AM:
> > 
> >> From: "Stahlberg, Michael" 
> >> To: ADSM-L@VM.MARIST.EDU
> >> Date: 04/26/2021 06:26 AM
> >> Subject: [EXTERNAL] Re: [E] Re: [ADSM-L] TSM alternatives
> >> Sent by: "ADSM: Dist Stor Manager" 
> >> 
> >> Hi Del,
> >> 
> >> We use Solaris Sparc, and it was clear for me that this is fully 
> > supported
> >> until the end of this year, but what happens if a newer OS release 
will
> >> come next year and the client will not work anymore, please.
> >> 
> >> I got the information from the support team that we can buy extended
> >> support, but this means more in case of a failure they will try to 
help,
> >> but there will be no development on this client anymore
> >> 
> >> It might be wrong, but with this statement, I cannot backup 
production 
> > data
> >> anymore.
> >> 
> >> thanks
> >> 
> >> regards
> >> 
> >>  >> u=http-3A__www.verizon.com&d=DwIFaQ&c=jf_iaSHvJObTbx-
> >> 
> > 
> 
siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=xbwRG2jwrVEOfJbwF7XUE6ogWFzsumbFCZ0SPJpEH-
> >> Y&s=FuEfe5SeT2yOWzFApUbjJivuOqOY9QCLmW-e8ZqJV2g&e= >
> >> 
> >> *Michael Stahlberg*
> >> Engineer IV Consultant - Network Eng & Ops
> >> Server & Services Management International -* Global Operations | 
> > Verizon*
> >> Wireline Network
> >> Tel: +49 231 972 0 | Cell: +49 231 972 0
> >> 
> >> 
> >> 
> >> 
> >>> Am Fr., 23. Apr. 2021 um 17:23 Uhr schrieb Del Hoobler 
> >> :
> >> 
> >>> Thanks Michael.
> >>> 
> >>> Just to be sure it's clear to all.
> >>> 
> >>> IBM Spectrum Protect still supports Solaris for x86_64. It's the 
> > Solaris
> >>> SPARC platform that is API only support:
> >>> 
> >>> 
> >>> INVALID URI REMOVED
> >> 
> > 
> 
u=https-3A__www.ibm.com_support_pages_node_84861-23Solaris-2520x86-5F64&d=DwIFAw&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=-0-
> >> r_o4RMF4NYke7vSCYCBjM7xJlyEdabThqplN58jk&m=YtQc_i2zHXvNtA0l-
> >> 
> > 
FkYOG8VOWKBcrHneIIlI92WUq0&s=qAQ_ww-bqRak-N9RxpC9z9bBpXBz00LoQNil0T465iE&e=
> >>> 
> >>> Some other notes:
> >>> - You can still use the Spectrum Protect V7 Solaris SPARC B/A client 

> > until
> >>> the end of this year with full support... and longer with extended
> >>> support.
> >>> - You can also mount the Solaris SPARC filesystems and back it up 
from 
> > a
> >>> proxy server.
> >>> 
> >>> 
> >>> Del
> >>> 
> >>> 
> >>> 
> >>> 
> >>&g

Re: [E] Re: [ADSM-L] TSM alternatives

[Saravanan Palanisamy]

Hi Del 

I could see oracle SPARC support API v8.1 but BA client not released for v8.1. 
What is the purpose of releasing API support for v8.1 without BA client ?

Means no BA client backup supported but still support TDP for oracle ? TDP also 
require BA client to install. 

Please advise if you have any background. 
 

Regards 
Sarav 
+65 9857 8665


> On 26 Apr 2021, at 7:03 PM, Del Hoobler  wrote:
> 
> Hi Michael,
> 
> You are correct. At the end of this year, the Backup/Archive client for 
> Solaris SPARC will not be supported. As you mention, you can buy extended 
> support, but it will only be for "Usage or Known Defect" issues, not for 
> new currency updates or defects.
> 
> I made the previous comment because I wanted other people reading this to 
> make sure they understood it was only Solaris SPARC that this applied to, 
> not Solaris x86_64.
> 
> Thank you.
> 
> 
> Del
> 
> 
> 
> 
> "ADSM: Dist Stor Manager"  wrote on 04/26/2021 
> 06:25:34 AM:
> 
>> From: "Stahlberg, Michael" 
>> To: ADSM-L@VM.MARIST.EDU
>> Date: 04/26/2021 06:26 AM
>> Subject: [EXTERNAL] Re: [E] Re: [ADSM-L] TSM alternatives
>> Sent by: "ADSM: Dist Stor Manager" 
>> 
>> Hi Del,
>> 
>> We use Solaris Sparc, and it was clear for me that this is fully 
> supported
>> until the end of this year, but what happens if a newer OS release will
>> come next year and the client will not work anymore, please.
>> 
>> I got the information from the support team that we can buy extended
>> support, but this means more in case of a failure they will try to help,
>> but there will be no development on this client anymore
>> 
>> It might be wrong, but with this statement, I cannot backup production 
> data
>> anymore.
>> 
>> thanks
>> 
>> regards
>> 
>> > u=http-3A__www.verizon.com&d=DwIFaQ&c=jf_iaSHvJObTbx-
>> 
> siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=xbwRG2jwrVEOfJbwF7XUE6ogWFzsumbFCZ0SPJpEH-
>> Y&s=FuEfe5SeT2yOWzFApUbjJivuOqOY9QCLmW-e8ZqJV2g&e= >
>> 
>> *Michael Stahlberg*
>> Engineer IV Consultant - Network Eng & Ops
>> Server & Services Management International -* Global Operations | 
> Verizon*
>> Wireline Network
>> Tel: +49 231 972 0 | Cell: +49 231 972 0
>> 
>> 
>> 
>> 
>>> Am Fr., 23. Apr. 2021 um 17:23 Uhr schrieb Del Hoobler 
>> :
>> 
>>> Thanks Michael.
>>> 
>>> Just to be sure it's clear to all.
>>> 
>>> IBM Spectrum Protect still supports Solaris for x86_64. It's the 
> Solaris
>>> SPARC platform that is API only support:
>>> 
>>> 
>>> INVALID URI REMOVED
>> 
> u=https-3A__www.ibm.com_support_pages_node_84861-23Solaris-2520x86-5F64&d=DwIFAw&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=-0-
>> r_o4RMF4NYke7vSCYCBjM7xJlyEdabThqplN58jk&m=YtQc_i2zHXvNtA0l-
>> 
> FkYOG8VOWKBcrHneIIlI92WUq0&s=qAQ_ww-bqRak-N9RxpC9z9bBpXBz00LoQNil0T465iE&e=
>>> 
>>> Some other notes:
>>> - You can still use the Spectrum Protect V7 Solaris SPARC B/A client 
> until
>>> the end of this year with full support... and longer with extended
>>> support.
>>> - You can also mount the Solaris SPARC filesystems and back it up from 
> a
>>> proxy server.
>>> 
>>> 
>>> Del
>>> 
>>> 
>>> 
>>> 
>>> "ADSM: Dist Stor Manager"  wrote on 04/23/2021
>>> 09:32:19 AM:
>>> 
>>>> From: "Stahlberg, Michael" 
>>>> To: ADSM-L@VM.MARIST.EDU
>>>> Date: 04/23/2021 09:33 AM
>>>> Subject: [EXTERNAL] TSM alternatives
>>>> Sent by: "ADSM: Dist Stor Manager" 
>>>> 
>>>> HI all,
>>>> 
>>>> unfortunately IBM decided to stop the support for Solaris Clients. 
> This
>>> is
>>>> a pain for us and we have decided to move to a different product. 
> I'm
>>>> checking for alternatives and would like to know if somebody has the
>>> same
>>>> issue and/or has already decided to move to a different product. I'm 
> not
>>>> happy about this, but our main clients are all Solaris Clients
>>> still..
>>>> 
>>>> regards
>>>> 
>>>> >>> u=http-3A__www.verizon.com&d=DwIFaQ&c=jf_iaSHvJObTbx-
>>>> 
>>> 
>>> 
>> 
> siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=6A_Oax46rHV7OFpRD535E9xEkLeChb5REQjB-
>>>> ceSTLc&s=HAwiCutcUMO04kWTHODK3NR79sbupV9T8nQJfPSIaZE&e= >
>>>> 
>>>> *Michael Stahlberg*
>>>> Engineer IV Consultant - Network Eng & Ops
>>>> Server & Services Management International -* Global Operations |
>>> Verizon*
>>>> Wireline Network
>>>> Tel: +49 231 972 0 | Cell: +49 231 972 0
>>>> 
>>>> 
> ==
>>>> 
>>>> Verizon Deutschland GmbH - Sebrathweg 20, 44149 Dortmund, Germany -
>>>> Amtsgericht Dortmund, HRB 14952 - Geschäftsführer: Detlef Eppig -
>>>> Vorsitzender des Aufsichtsrats: Francesco de Maio
>>>> 
>>> 
>> 

Re: [E] Re: [ADSM-L] TSM alternatives

[Stahlberg, Michael]

Hi Del,

We use Solaris Sparc, and it was clear for me that this is fully supported
until the end of this year, but what happens if a newer OS release will
come next year and the client will not work anymore, please.

I got the information from the support team that we can buy extended
support, but this means more in case of a failure they will try to help,
but there will be no development on this client anymore

It might be wrong, but with this statement, I cannot backup production data
anymore.

thanks

regards

<http://www.verizon.com>

*Michael Stahlberg*
Engineer IV Consultant - Network Eng & Ops
Server & Services Management International -* Global Operations | Verizon*
Wireline Network
Tel: +49 231 972 0 | Cell: +49 231 972 0




Am Fr., 23. Apr. 2021 um 17:23 Uhr schrieb Del Hoobler :

> Thanks Michael.
>
> Just to be sure it's clear to all.
>
> IBM Spectrum Protect still supports Solaris for x86_64. It's the Solaris
> SPARC platform that is API only support:
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ibm.com_support_pages_node_84861-23Solaris-2520x86-5F64&d=DwIFAw&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=-0-r_o4RMF4NYke7vSCYCBjM7xJlyEdabThqplN58jk&m=YtQc_i2zHXvNtA0l-FkYOG8VOWKBcrHneIIlI92WUq0&s=qAQ_ww-bqRak-N9RxpC9z9bBpXBz00LoQNil0T465iE&e=
>
> Some other notes:
> - You can still use the Spectrum Protect V7 Solaris SPARC B/A client until
> the end of this year with full support... and longer with extended
> support.
> - You can also mount the Solaris SPARC filesystems and back it up from a
> proxy server.
>
>
> Del
>
>
>
>
> "ADSM: Dist Stor Manager"  wrote on 04/23/2021
> 09:32:19 AM:
>
> > From: "Stahlberg, Michael" 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 04/23/2021 09:33 AM
> > Subject: [EXTERNAL] TSM alternatives
> > Sent by: "ADSM: Dist Stor Manager" 
> >
> > HI all,
> >
> > unfortunately IBM decided to stop the support for Solaris Clients. This
> is
> > a pain for us and we have decided to move to a different product. I'm
> > checking for alternatives and would like to know if somebody has the
> same
> > issue and/or has already decided to move to a different product. I'm not
> > happy about this, but our main clients are all Solaris Clients
> still..
> >
> > regards
> >
> >  > u=http-3A__www.verizon.com&d=DwIFaQ&c=jf_iaSHvJObTbx-
> >
>
> siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=6A_Oax46rHV7OFpRD535E9xEkLeChb5REQjB-
> > ceSTLc&s=HAwiCutcUMO04kWTHODK3NR79sbupV9T8nQJfPSIaZE&e= >
> >
> > *Michael Stahlberg*
> > Engineer IV Consultant - Network Eng & Ops
> > Server & Services Management International -* Global Operations |
> Verizon*
> > Wireline Network
> > Tel: +49 231 972 0 | Cell: +49 231 972 0
> >
> > ==
> >
> > Verizon Deutschland GmbH - Sebrathweg 20, 44149 Dortmund, Germany -
> > Amtsgericht Dortmund, HRB 14952 - Geschäftsführer: Detlef Eppig -
> > Vorsitzender des Aufsichtsrats: Francesco de Maio
> >
>

Re: [E] Re: [ADSM-L] TSM alternatives

[Del Hoobler]

Hi Michael,

You are correct. At the end of this year, the Backup/Archive client for 
Solaris SPARC will not be supported. As you mention, you can buy extended 
support, but it will only be for "Usage or Known Defect" issues, not for 
new currency updates or defects.

I made the previous comment because I wanted other people reading this to 
make sure they understood it was only Solaris SPARC that this applied to, 
not Solaris x86_64.

Thank you.


Del




"ADSM: Dist Stor Manager"  wrote on 04/26/2021 
06:25:34 AM:

> From: "Stahlberg, Michael" 
> To: ADSM-L@VM.MARIST.EDU
> Date: 04/26/2021 06:26 AM
> Subject: [EXTERNAL] Re: [E] Re: [ADSM-L] TSM alternatives
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi Del,
> 
> We use Solaris Sparc, and it was clear for me that this is fully 
supported
> until the end of this year, but what happens if a newer OS release will
> come next year and the client will not work anymore, please.
> 
> I got the information from the support team that we can buy extended
> support, but this means more in case of a failure they will try to help,
> but there will be no development on this client anymore
> 
> It might be wrong, but with this statement, I cannot backup production 
data
> anymore.
> 
> thanks
> 
> regards
> 
>  u=http-3A__www.verizon.com&d=DwIFaQ&c=jf_iaSHvJObTbx-
> 
siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=xbwRG2jwrVEOfJbwF7XUE6ogWFzsumbFCZ0SPJpEH-
> Y&s=FuEfe5SeT2yOWzFApUbjJivuOqOY9QCLmW-e8ZqJV2g&e= >
> 
> *Michael Stahlberg*
> Engineer IV Consultant - Network Eng & Ops
> Server & Services Management International -* Global Operations | 
Verizon*
> Wireline Network
> Tel: +49 231 972 0 | Cell: +49 231 972 0
> 
> 
> 
> 
> Am Fr., 23. Apr. 2021 um 17:23 Uhr schrieb Del Hoobler 
:
> 
> > Thanks Michael.
> >
> > Just to be sure it's clear to all.
> >
> > IBM Spectrum Protect still supports Solaris for x86_64. It's the 
Solaris
> > SPARC platform that is API only support:
> >
> >
> > INVALID URI REMOVED
> 
u=https-3A__www.ibm.com_support_pages_node_84861-23Solaris-2520x86-5F64&d=DwIFAw&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=-0-
> r_o4RMF4NYke7vSCYCBjM7xJlyEdabThqplN58jk&m=YtQc_i2zHXvNtA0l-
> 
FkYOG8VOWKBcrHneIIlI92WUq0&s=qAQ_ww-bqRak-N9RxpC9z9bBpXBz00LoQNil0T465iE&e=
> >
> > Some other notes:
> > - You can still use the Spectrum Protect V7 Solaris SPARC B/A client 
until
> > the end of this year with full support... and longer with extended
> > support.
> > - You can also mount the Solaris SPARC filesystems and back it up from 
a
> > proxy server.
> >
> >
> > Del
> >
> >
> >
> >
> > "ADSM: Dist Stor Manager"  wrote on 04/23/2021
> > 09:32:19 AM:
> >
> > > From: "Stahlberg, Michael" 
> > > To: ADSM-L@VM.MARIST.EDU
> > > Date: 04/23/2021 09:33 AM
> > > Subject: [EXTERNAL] TSM alternatives
> > > Sent by: "ADSM: Dist Stor Manager" 
> > >
> > > HI all,
> > >
> > > unfortunately IBM decided to stop the support for Solaris Clients. 
This
> > is
> > > a pain for us and we have decided to move to a different product. 
I'm
> > > checking for alternatives and would like to know if somebody has the
> > same
> > > issue and/or has already decided to move to a different product. I'm 
not
> > > happy about this, but our main clients are all Solaris Clients
> > still..
> > >
> > > regards
> > >
> > >  > > u=http-3A__www.verizon.com&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > >
> >
> > 
> 
siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=6A_Oax46rHV7OFpRD535E9xEkLeChb5REQjB-
> > > ceSTLc&s=HAwiCutcUMO04kWTHODK3NR79sbupV9T8nQJfPSIaZE&e= >
> > >
> > > *Michael Stahlberg*
> > > Engineer IV Consultant - Network Eng & Ops
> > > Server & Services Management International -* Global Operations |
> > Verizon*
> > > Wireline Network
> > > Tel: +49 231 972 0 | Cell: +49 231 972 0
> > >
> > > 
==
> > >
> > > Verizon Deutschland GmbH - Sebrathweg 20, 44149 Dortmund, Germany -
> > > Amtsgericht Dortmund, HRB 14952 - Geschäftsführer: Detlef Eppig -
> > > Vorsitzender des Aufsichtsrats: Francesco de Maio
> > >
> >
> 

Re: TSM API for Cassandra database

[Mark Yakushev]

This could also work - www.redbooks.ibm.com/redpapers/pdfs/redp3980.pdf

- Mark

"ADSM: Dist Stor Manager"  wrote on 04/23/2021
01:47:39 PM:

> From: Del Hoobler 
> To: ADSM-L@VM.MARIST.EDU
> Date: 04/23/2021 01:48 PM
> Subject: [EXTERNAL] Re: [ADSM-L] TSM API for Cassandra database
> Sent by: "ADSM: Dist Stor Manager" 
>
> I do not personally have details on how this works, but I have heard it
> mentioned by a few people.
>
> I was mainly reminding the community that Spectrum Protect introduced an
> S3 interface last year:
>
> https://www.ibm.com/docs/en/spectrum-protect/8.1.12?topic=cs-
> sending-data-spectrum-protect-by-using-s3-client-applications
>
>
>
> Del
>
> 
>
>
> "ADSM: Dist Stor Manager"  wrote on 04/23/2021
> 03:26:11 PM:
>
> > From: "Bommasani, Venu" 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 04/23/2021 03:26 PM
> > Subject: [EXTERNAL] Re: TSM API for Cassandra database
> > Sent by: "ADSM: Dist Stor Manager" 
> >
> > Hi Del,
> >
> > Thanks for your response.
> >
> > The "dump and sweep" method may not good solution here as taking
> > Backup and dumping onto local/remote filesystem may take some time
> > and then taking that flat file Backup onto TSM using BA client would
> > more time & resource consuming process as the Database size is
> > already more than 3 TB.
> >
> > I was really not sure about that option "1" - Leverage the new "S3"
> > interface to the Spectrum Protect server (instead of the TSM API).
> > Appreciate if you could able to share any reference technote for the
> > same. Thanks.
> >
> >
> > Best Regards,
> > _
> >
> > Venu Bommasani
> > Storage & Data Protection
> > Capgemini India | Bangalore
> >  Mobile: +91 7795213309 / venu.bommas...@capgemini.com
> >
> > -Original Message-
> > From: ADSM: Dist Stor Manager  On Behalf Of Del
> Hoobler
> > Sent: Saturday, April 24, 2021 12:08 AM
> > To: ADSM-L@VM.MARIST.EDU
> > Subject: Re: [ADSM-L] TSM API for Cassandra database
> >
> > There is no "TDP for Cassandra".
> >
> > Some ideas to consider:
> >
> > 1. Leverage the new "S3" interface to the Spectrum Protect server
> > (instead of the TSM API).
> >
> > 2. Use the "dump and sweep" method, i.e. dump the backup to a file
> > and then back up the file with the Backup/Archive client.
> >
> >
> > Del
> >
> > 
> >
> > "ADSM: Dist Stor Manager"  wrote on 04/23/2021
> > 02:24:59 PM:
> >
> > > From: "Bommasani, Venu" 
> > > To: ADSM-L@VM.MARIST.EDU
> > > Date: 04/23/2021 02:25 PM
> > > Subject: [EXTERNAL] TSM API for Cassandra database
> > > Sent by: "ADSM: Dist Stor Manager" 
> > >
> > > Hello Team,
> > >
> > > Does anyone taking Cassandra database Backup using TSM API ?
> > >
> > > I have a request to perform a Backup for Cassandra database , But
> > > could not able to find any TDP agent for the same..!
> > > Does it possible to Backup Cassandra database Backup using a
> > > customize API and perform an online Backups to TSM ? Thanks in
> Advance.
> > >
> > >
> > > Best Regards,
> > > _
> > >
> > > Venu Bommasani
> > > Storage & Data Protection
> > > Capgemini India | Bangalore
> > > Mobile: +91 7795213309 / venu.bommas...@capgemini.com
> > >
> > > This message contains information that may be privileged or
> > > confidential and is the property of the Capgemini Group. It is
> > > intended only for the person to whom it is addressed. If you are not
> > > the intended recipient, you are not authorized to read, print,
> > > retain, copy, disseminate, distribute, or use this message or any
> > > part thereof. If you receive this message in error, please notify
> > > the sender immediately and delete all copies of this message.
> > >
> > This message contains information that may be privileged or
> > confidential and is the property of the Capgemini Group. It is
> > intended only for the person to whom it is addressed. If you are not
> > the intended recipient, you are not authorized to read, print,
> > retain, copy, disseminate, distribute, or use this message or any
> > part thereof. If you receive this message in error, please notify
> > the sender immediately and delete all copies of this message.
> >
>

Re: TSM API for Cassandra database

[Bommasani, Venu]

Hi Del,

Thanks for your response. 

The "dump and sweep" method may not good solution here as taking Backup and 
dumping onto local/remote filesystem may take some time and then taking that 
flat file Backup onto TSM using BA client would more time & resource consuming 
process as the Database size is already more than 3 TB.

I was really not sure about that option "1" - Leverage the new "S3" interface 
to the Spectrum Protect server (instead of the TSM API).
Appreciate if you could able to share any reference technote for the same. 
Thanks.


Best Regards,
_

Venu Bommasani
Storage & Data Protection
Capgemini India | Bangalore
 Mobile: +91 7795213309 / venu.bommas...@capgemini.com

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Del Hoobler
Sent: Saturday, April 24, 2021 12:08 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM API for Cassandra database

There is no "TDP for Cassandra".

Some ideas to consider:

1. Leverage the new "S3" interface to the Spectrum Protect server (instead of 
the TSM API).

2. Use the "dump and sweep" method, i.e. dump the backup to a file and then 
back up the file with the Backup/Archive client.


Del



"ADSM: Dist Stor Manager"  wrote on 04/23/2021 
02:24:59 PM:

> From: "Bommasani, Venu" 
> To: ADSM-L@VM.MARIST.EDU
> Date: 04/23/2021 02:25 PM
> Subject: [EXTERNAL] TSM API for Cassandra database
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hello Team,
> 
> Does anyone taking Cassandra database Backup using TSM API ?
> 
> I have a request to perform a Backup for Cassandra database , But 
> could not able to find any TDP agent for the same..!
> Does it possible to Backup Cassandra database Backup using a 
> customize API and perform an online Backups to TSM ? Thanks in Advance.
> 
> 
> Best Regards,
> _
> 
> Venu Bommasani
> Storage & Data Protection
> Capgemini India | Bangalore
> Mobile: +91 7795213309 / venu.bommas...@capgemini.com
> 
> This message contains information that may be privileged or 
> confidential and is the property of the Capgemini Group. It is 
> intended only for the person to whom it is addressed. If you are not
> the intended recipient, you are not authorized to read, print, 
> retain, copy, disseminate, distribute, or use this message or any 
> part thereof. If you receive this message in error, please notify 
> the sender immediately and delete all copies of this message.
> 
This message contains information that may be privileged or confidential and is 
the property of the Capgemini Group. It is intended only for the person to whom 
it is addressed. If you are not the intended recipient, you are not authorized 
to read, print, retain, copy, disseminate, distribute, or use this message or 
any part thereof. If you receive this message in error, please notify the 
sender immediately and delete all copies of this message.

TSM API for Cassandra database

[Bommasani, Venu]

Hello Team,

Does anyone taking Cassandra database Backup using TSM API ?

I have a request to perform a Backup for Cassandra database , But could not 
able to find any TDP agent for the same..!
Does it possible to Backup Cassandra database Backup using a customize API and 
perform an online Backups to TSM ? Thanks in Advance.


Best Regards,
_

Venu Bommasani
Storage & Data Protection
Capgemini India | Bangalore
Mobile: +91 7795213309 / venu.bommas...@capgemini.com

This message contains information that may be privileged or confidential and is 
the property of the Capgemini Group. It is intended only for the person to whom 
it is addressed. If you are not the intended recipient, you are not authorized 
to read, print, retain, copy, disseminate, distribute, or use this message or 
any part thereof. If you receive this message in error, please notify the 
sender immediately and delete all copies of this message.

Re: TSM API for Cassandra database

[Del Hoobler]

I do not personally have details on how this works, but I have heard it 
mentioned by a few people.

I was mainly reminding the community that Spectrum Protect introduced an 
S3 interface last year:

https://www.ibm.com/docs/en/spectrum-protect/8.1.12?topic=cs-sending-data-spectrum-protect-by-using-s3-client-applications



Del




"ADSM: Dist Stor Manager"  wrote on 04/23/2021 
03:26:11 PM:

> From: "Bommasani, Venu" 
> To: ADSM-L@VM.MARIST.EDU
> Date: 04/23/2021 03:26 PM
> Subject: [EXTERNAL] Re: TSM API for Cassandra database
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi Del,
> 
> Thanks for your response. 
> 
> The "dump and sweep" method may not good solution here as taking 
> Backup and dumping onto local/remote filesystem may take some time 
> and then taking that flat file Backup onto TSM using BA client would
> more time & resource consuming process as the Database size is 
> already more than 3 TB.
> 
> I was really not sure about that option "1" - Leverage the new "S3" 
> interface to the Spectrum Protect server (instead of the TSM API).
> Appreciate if you could able to share any reference technote for the
> same. Thanks.
> 
> 
> Best Regards,
> _
> 
> Venu Bommasani
> Storage & Data Protection
> Capgemini India | Bangalore
>  Mobile: +91 7795213309 / venu.bommas...@capgemini.com
> 
> -Original Message-
> From: ADSM: Dist Stor Manager  On Behalf Of Del 
Hoobler
> Sent: Saturday, April 24, 2021 12:08 AM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] TSM API for Cassandra database
> 
> There is no "TDP for Cassandra".
> 
> Some ideas to consider:
> 
> 1. Leverage the new "S3" interface to the Spectrum Protect server 
> (instead of the TSM API).
> 
> 2. Use the "dump and sweep" method, i.e. dump the backup to a file 
> and then back up the file with the Backup/Archive client.
> 
> 
> Del
> 
> 
> 
> "ADSM: Dist Stor Manager"  wrote on 04/23/2021 
> 02:24:59 PM:
> 
> > From: "Bommasani, Venu" 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 04/23/2021 02:25 PM
> > Subject: [EXTERNAL] TSM API for Cassandra database
> > Sent by: "ADSM: Dist Stor Manager" 
> > 
> > Hello Team,
> > 
> > Does anyone taking Cassandra database Backup using TSM API ?
> > 
> > I have a request to perform a Backup for Cassandra database , But 
> > could not able to find any TDP agent for the same..!
> > Does it possible to Backup Cassandra database Backup using a 
> > customize API and perform an online Backups to TSM ? Thanks in 
Advance.
> > 
> > 
> > Best Regards,
> > _
> > 
> > Venu Bommasani
> > Storage & Data Protection
> > Capgemini India | Bangalore
> > Mobile: +91 7795213309 / venu.bommas...@capgemini.com
> > 
> > This message contains information that may be privileged or 
> > confidential and is the property of the Capgemini Group. It is 
> > intended only for the person to whom it is addressed. If you are not
> > the intended recipient, you are not authorized to read, print, 
> > retain, copy, disseminate, distribute, or use this message or any 
> > part thereof. If you receive this message in error, please notify 
> > the sender immediately and delete all copies of this message.
> > 
> This message contains information that may be privileged or 
> confidential and is the property of the Capgemini Group. It is 
> intended only for the person to whom it is addressed. If you are not
> the intended recipient, you are not authorized to read, print, 
> retain, copy, disseminate, distribute, or use this message or any 
> part thereof. If you receive this message in error, please notify 
> the sender immediately and delete all copies of this message.
> 

Re: TSM API for Cassandra database

[Del Hoobler]

There is no "TDP for Cassandra".

Some ideas to consider:

1. Leverage the new "S3" interface to the Spectrum Protect server (instead 
of the TSM API).

2. Use the "dump and sweep" method, i.e. dump the backup to a file and 
then back up the file with the Backup/Archive client.


Del



"ADSM: Dist Stor Manager"  wrote on 04/23/2021 
02:24:59 PM:

> From: "Bommasani, Venu" 
> To: ADSM-L@VM.MARIST.EDU
> Date: 04/23/2021 02:25 PM
> Subject: [EXTERNAL] TSM API for Cassandra database
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hello Team,
> 
> Does anyone taking Cassandra database Backup using TSM API ?
> 
> I have a request to perform a Backup for Cassandra database , But 
> could not able to find any TDP agent for the same..!
> Does it possible to Backup Cassandra database Backup using a 
> customize API and perform an online Backups to TSM ? Thanks in Advance.
> 
> 
> Best Regards,
> _
> 
> Venu Bommasani
> Storage & Data Protection
> Capgemini India | Bangalore
> Mobile: +91 7795213309 / venu.bommas...@capgemini.com
> 
> This message contains information that may be privileged or 
> confidential and is the property of the Capgemini Group. It is 
> intended only for the person to whom it is addressed. If you are not
> the intended recipient, you are not authorized to read, print, 
> retain, copy, disseminate, distribute, or use this message or any 
> part thereof. If you receive this message in error, please notify 
> the sender immediately and delete all copies of this message.
> 

TSM alternatives

[Stahlberg, Michael]

HI all,

unfortunately IBM decided to stop the support for Solaris Clients. This is
a pain for us and we have decided to move to a different product. I'm
checking for alternatives and would like to know if somebody has the same
issue and/or has already decided to move to a different product. I'm not
happy about this, but our main clients are all Solaris Clients still..

regards



*Michael Stahlberg*
Engineer IV Consultant - Network Eng & Ops
Server & Services Management International -* Global Operations | Verizon*
Wireline Network
Tel: +49 231 972 0 | Cell: +49 231 972 0

==

Verizon Deutschland GmbH - Sebrathweg 20, 44149 Dortmund, Germany - Amtsgericht 
Dortmund, HRB 14952 - Geschäftsführer: Detlef Eppig - Vorsitzender des 
Aufsichtsrats: Francesco de Maio

Re: TSM alternatives

[Del Hoobler]

Thanks Michael.

Just to be sure it's clear to all.

IBM Spectrum Protect still supports Solaris for x86_64. It's the Solaris 
SPARC platform that is API only support:

https://www.ibm.com/support/pages/node/84861#Solaris%20x86_64

Some other notes:
- You can still use the Spectrum Protect V7 Solaris SPARC B/A client until 
the end of this year with full support... and longer with extended 
support.
- You can also mount the Solaris SPARC filesystems and back it up from a 
proxy server.


Del




"ADSM: Dist Stor Manager"  wrote on 04/23/2021 
09:32:19 AM:

> From: "Stahlberg, Michael" 
> To: ADSM-L@VM.MARIST.EDU
> Date: 04/23/2021 09:33 AM
> Subject: [EXTERNAL] TSM alternatives
> Sent by: "ADSM: Dist Stor Manager" 
> 
> HI all,
> 
> unfortunately IBM decided to stop the support for Solaris Clients. This 
is
> a pain for us and we have decided to move to a different product. I'm
> checking for alternatives and would like to know if somebody has the 
same
> issue and/or has already decided to move to a different product. I'm not
> happy about this, but our main clients are all Solaris Clients 
still..
> 
> regards
> 
>  u=http-3A__www.verizon.com&d=DwIFaQ&c=jf_iaSHvJObTbx-
> 
siA1ZOg&r=0hq2JX5c3TEZNriHEs7Zf7HrkY2fNtONOrEOM8Txvk8&m=6A_Oax46rHV7OFpRD535E9xEkLeChb5REQjB-
> ceSTLc&s=HAwiCutcUMO04kWTHODK3NR79sbupV9T8nQJfPSIaZE&e= >
> 
> *Michael Stahlberg*
> Engineer IV Consultant - Network Eng & Ops
> Server & Services Management International -* Global Operations | 
Verizon*
> Wireline Network
> Tel: +49 231 972 0 | Cell: +49 231 972 0
> 
> ==
> 
> Verizon Deutschland GmbH - Sebrathweg 20, 44149 Dortmund, Germany - 
> Amtsgericht Dortmund, HRB 14952 - Geschäftsführer: Detlef Eppig - 
> Vorsitzender des Aufsichtsrats: Francesco de Maio
> 

Re: TSM for VE installation on TSM server

[Mikhail Tolkonyuk]

Not recommend as it uses different versions of the same components (baclient, 
etc.), and updating TSMVE may break the TSM server itself. In my case, DBBackup 
were broken after VE update due version mismatch.

It is much safer to run VE in a separate virtual machine. Or physical, because 
you need a LANfree backup.

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Martin 
Panggabean
Sent: Thursday, March 4, 2021 10:53 PM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] TSM for VE installation on TSM server

Hi everyone,

I have no idea if anyone here has ever tried running TSM for VE on the TSM 
server. To make it short, I was asked by a customer to move their vBackup 
Storage that is now running on a separate physical server than TSM Server 
(physical). I have tried to look at documentation but failed to discover if 
this is possible or not. Beside, they also expect the VM backup will be a 
LANfree backup similar to the existing setup.

Anyone can give an idea if this will work or not?

Thanks in advance,
Martin
Backup Freelance

Re: TSM for VE installation on TSM server

[Andrew Raibeck]

Although you can run the components on the same host, and some users do so
(maybe thinking of such a configuration as an "appliance"), in general, we
do not advise this type of configuration. As Mikhail points out, the
components must all be at the same fix pack level, e.g., if server is at
8.1.10, then DP for VMware must also be at 8.1.10. When you upgrade one
component to a newer fix pack level, you must upgrade the other components
as well.

It is advised that the server be configured in accordance with our
blueprints https://www.ibm.com/support/pages/node/1146352, so any "all in
one box" solution should take into consideration the additional workload
that DP for VMware and the Storage Agent place on the host that also runs
IBM Spectrum Protect server.

Regards,

Andy

Andrew Raibeck
IBM Spectrum Protect Level 3
IBM Systems, Storage
stor...@us.ibm.com

IBM

"ADSM: Dist Stor Manager"  wrote on 2021-03-09
02:30:47:

> From: Mikhail Tolkonyuk 
> To: ADSM-L@VM.MARIST.EDU
> Date: 2021-03-09 02:31
> Subject: [EXTERNAL] Re: TSM for VE installation on TSM server
> Sent by: "ADSM: Dist Stor Manager" 
>
> Not recommend as it uses different versions of the same components
> (baclient, etc.), and updating TSMVE may break the TSM server
> itself. In my case, DBBackup were broken after VE update due
versionmismatch.
>
> It is much safer to run VE in a separate virtual machine. Or
> physical, because you need a LANfree backup.
>
> -Original Message-
> From: ADSM: Dist Stor Manager  On Behalf Of
> Martin Panggabean
> Sent: Thursday, March 4, 2021 10:53 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: [ADSM-L] TSM for VE installation on TSM server
>
> Hi everyone,
>
> I have no idea if anyone here has ever tried running TSM for VE on
> the TSM server. To make it short, I was asked by a customer to move
> their vBackup Storage that is now running on a separate physical
> server than TSM Server (physical). I have tried to look at
> documentation but failed to discover if this is possible or not.
> Beside, they also expect the VM backup will be a LANfree backup
> similar to the existing setup.
>
> Anyone can give an idea if this will work or not?
>
> Thanks in advance,
> Martin
> Backup Freelance

TSM for VE installation on TSM server

[Martin Panggabean]

Hi everyone,

I have no idea if anyone here has ever tried running TSM for VE on the TSM
server. To make it short, I was asked by a customer to move their vBackup
Storage that is now running on a separate physical server than TSM Server
(physical). I have tried to look at documentation but failed to discover if
this is possible or not. Beside, they also expect the VM backup will be a
LANfree backup similar to the existing setup.

Anyone can give an idea if this will work or not?

Thanks in advance,
Martin
Backup Freelance

Very annoying TSM v8 issue

[Tom Alverson]

We are switching all of our storage servers to version 8 and at the same
time trying to upgrade all the clients to 8.1.9.10.  We have had some
issues with the RedHat TCP stack running out of memory and datadomains
crashing that have caused the clients to create very large dsierror.log
files in the TDPSQL folder, filling up the hard disk.  To add injury to
insult, once the disk is full, the client wants to update DSM.OPT (where it
adds the DR server info at the beginning) but it must delete the old
DSM.OPT and the log files fill the drive before it can write the new
DSM.OPT so I am getting 0 length TSM.OPT files in the BACLIENT folder and
the TDPSQL folder.  I have another server that filled up the C: drive for a
reason other than TSM logs and the same zero length DSM.OPT was created.
IBM should create the new DSM.OPT file with a different name, and if that
is successful, then delete (or rename) the old DSM.OPT and rename the
updated one to DSM.OPT.

Re: Change SMTP server - TSM v6.3

[Rick Adamson]

Paul,
TSM 6.3 was a challenging time for TSM reporting. Most likely TSM Operational 
Reporting v5.5 was used.
Version 6.3 had the Tivoli Monitoring component but was not well received by 
most admins.
Instead there was a documented process to modify the TSM Operational reporting 
from version 5.5 to work under version 6.3, it basically consisted on 
replacement of certain XML files.
Unfortunately I do not have the exact process, or xml files, perhaps someone 
else can assist if you need it.

However if you were already receiving the reports it should be installed and 
configured somewhere, you just need to find where.
You should have a server, most likely windows based that has the operational 
reporting installed, it was managed using the Microsoft Management Console 
(MMC).
Open the MMC, you should see the Operational Reporting in the management tree, 
if not you will need to add the snap-in using the menu option and configure it.

Assuming the MMC already has it configured, expand the tree to locate the 
backup server, bring up the server properties, and alter the configuration as 
needed.
It's been a long time so some of this is getting a little fuzzy, if it was me I 
would of course plan for an upgrade soon.
Hope this is helpful.

Thank you,
-Rick Adamson

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of DUDLEY Paul
Sent: Wednesday, July 15, 2020 2:43 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Change SMTP server - TSM v6.3

* This email originated outside of the organization. Use caution when opening 
attachments or clicking links. *

--
We have a legacy TSM backup system - TSM v6.3. I need to change the SMTP server 
it uses to send emails such as the TSM Admin daily operational report. How do I 
do this?

Regards
Paul Dudley
pdud...@anl.com.au<mailto:pdud...@anl.com.au>

**CONFIDENTIALITY NOTICE** This electronic message contains information from 
Southeastern Grocers, Inc and is intended only for the use of the addressee. 
This message may contain information that is privileged, confidential and/or 
exempt from disclosure under applicable Law. This message may not be read, 
used, distributed, forwarded, reproduced or stored by any other than the 
intended recipient. If you are not the intended recipient, please delete and 
notify the sender.

Change SMTP server - TSM v6.3

[DUDLEY Paul]

We have a legacy TSM backup system - TSM v6.3. I need to change the SMTP server 
it uses to send emails such as the TSM Admin daily operational report. How do I 
do this?

Regards
Paul Dudley
pdud...@anl.com.au<mailto:pdud...@anl.com.au>

Re: TSM server performance continuing

[Loon, Eric van (ITOP NS) - KLM]

Hi Stefan,

At the time the servers were installed Redhat 6.4 was our standard. The servers 
need to be upgraded to the latest 6.x release first before we can upgrade TSM 
to 8.1.x. But before making any changes to the environment while we have a case 
open at IBM, I need to be sure there is a clear reason to do so. If there is 
any indication that the issue is related to the OS level or the TSM level, I 
will plan an upgrade of course.
Another reason is that the server which is having the most issues is running a 
7.1 release prior to new security 'enhancements'. I would like to prevent extra 
issues caused by certificates until after we found and fixed the performance 
issues.
Today we have a meeting with IBM and all departments and managers to plan an 
onsite workshop on how we/IBM will tackle the issue. Finally...

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan 
Folkerts
Sent: dinsdag 3 september 2019 09:41
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM server performance continuing

Very strange issue indeed Eric, I have never heard of a specific client causing 
slowdowns, I would understand if it where a massive amount of oracle sessions 
starting but if it's really platform and not session count related it would 
have to be something that IBM needs to look into I guess.
May I ask, is there a specific reason you are running fairly old server code in 
the environment?


On Tue, Aug 27, 2019 at 1:01 PM Loon, Eric van (ITOP NS) - KLM < 
eric-van.l...@klm.com> wrote:

> Hi Stefan,
>
> There is no noticable impact on the TSM performance when we use the 
> tsmdiskperf tool on the database volumes.
> We recently discovered the issue seems to be related to the TDP for 
> Oracle clients. When none are running the server response is OK, even 
> during a reasonable load. But as soon as the TDP for Oracle clients 
> start kicking in, the server becomes slow. We brought it to the 
> attention of the developers, but no response yet...
> Thanks for your help!
>
> Kind regards,
> Eric van Loon
> Air France/KLM Storage & Backup
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
> Of Stefan Folkerts
> Sent: maandag 26 augustus 2019 10:39
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: TSM server performance continuing
>
> Eric,
>
> What happens when you benchmark the DB volumes using the tool provided 
> with the blueprints on an idle system, does the system also slow down 
> with commands such as q stgpool or does it stay fast when the 
> benchmark is running on all volumes?
> Also, what kind of a result does the benchmark give you and what do 
> you use for your database storage?
>
> Regards,
>Stefan
>
>
>
> On Thu, Aug 22, 2019 at 3:36 PM PAC Brion Arnaud < 
> arnaud.br...@panalpina.com>
> wrote:
>
> > Hi Eric,
> >
> > These seem to be default values when installing Red Hat Enterprise 
> > Linux Server 7.3.
> > I checked several other machines, all are having the same value.
> >
> > And following page
> > https://stackoverflow.com/questions/55428812/how-the-values-of-kerne
> > l-
> > parameters-are-defined
> > seems to confirm my statement.
> >
> > Cheers.
> >
> > Arnaud
> >
> >
> >
> > 
> > **
> > **
> > Backup and Recovery Systems Administrator Panalpina Management Ltd., 
> > Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 
> > Basel/CH
> > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > Direct: +41 (61) 226 19 78
> > e-mail: arnaud.br...@panalpina.com
> > This electronic message transmission contains information from 
> > Panalpina and is confidential or privileged.
> > This information is intended only for the person (s) named above. If 
> > you are not the intended recipient, any disclosure, copying, 
> > distribution or use or any other action based on the contents of 
> > this information is strictly prohibited.
> >
> > If you receive this electronic transmission in error, please notify 
> > the sender by e-mail, telephone or fax at the numbers listed above.
> Thank you.
> >
> > ****
> > **
> > **
> > www.panalpina.com
> >
> >
> >
> > -Original Message-
> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
> > Behalf Of Lo

Re: TSM server performance continuing

[Stefan Folkerts]

Very strange issue indeed Eric, I have never heard of a specific client
causing slowdowns, I would understand if it where a massive amount of
oracle sessions starting but if it's really platform and not session count
related it would have to be something that IBM needs to look into I guess.
May I ask, is there a specific reason you are running fairly old server
code in the environment?


On Tue, Aug 27, 2019 at 1:01 PM Loon, Eric van (ITOP NS) - KLM <
eric-van.l...@klm.com> wrote:

> Hi Stefan,
>
> There is no noticable impact on the TSM performance when we use the
> tsmdiskperf tool on the database volumes.
> We recently discovered the issue seems to be related to the TDP for Oracle
> clients. When none are running the server response is OK, even during a
> reasonable load. But as soon as the TDP for Oracle clients start kicking
> in, the server becomes slow. We brought it to the attention of the
> developers, but no response yet...
> Thanks for your help!
>
> Kind regards,
> Eric van Loon
> Air France/KLM Storage & Backup
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
> Stefan Folkerts
> Sent: maandag 26 augustus 2019 10:39
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: TSM server performance continuing
>
> Eric,
>
> What happens when you benchmark the DB volumes using the tool provided
> with the blueprints on an idle system, does the system also slow down with
> commands such as q stgpool or does it stay fast when the benchmark is
> running on all volumes?
> Also, what kind of a result does the benchmark give you and what do you
> use for your database storage?
>
> Regards,
>Stefan
>
>
>
> On Thu, Aug 22, 2019 at 3:36 PM PAC Brion Arnaud <
> arnaud.br...@panalpina.com>
> wrote:
>
> > Hi Eric,
> >
> > These seem to be default values when installing Red Hat Enterprise
> > Linux Server 7.3.
> > I checked several other machines, all are having the same value.
> >
> > And following page
> > https://stackoverflow.com/questions/55428812/how-the-values-of-kernel-
> > parameters-are-defined
> > seems to confirm my statement.
> >
> > Cheers.
> >
> > Arnaud
> >
> >
> >
> > **
> > **
> > Backup and Recovery Systems Administrator Panalpina Management Ltd.,
> > Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002
> > Basel/CH
> > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > Direct: +41 (61) 226 19 78
> > e-mail: arnaud.br...@panalpina.com
> > This electronic message transmission contains information from
> > Panalpina and is confidential or privileged.
> > This information is intended only for the person (s) named above. If
> > you are not the intended recipient, any disclosure, copying,
> > distribution or use or any other action based on the contents of this
> > information is strictly prohibited.
> >
> > If you receive this electronic transmission in error, please notify
> > the sender by e-mail, telephone or fax at the numbers listed above.
> Thank you.
> >
> > ******
> > **
> > www.panalpina.com
> >
> >
> >
> > -Original Message-
> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
> > Of Loon, Eric van (ITOP NS) - KLM
> > Sent: Thursday, August 22, 2019 2:31 PM
> > To: ADSM-L@VM.MARIST.EDU
> > Subject: Re: TSM server performance continuing
> >
> > Hi Arnoud,
> >
> > I do not understand why your max seg size (kbytes) = 18014398509465599
> > and max total shared memory (kbytes) = 18014397435740096 are so huge.
> > If you have 256 GB RAM, like I do, I would expect max seg size
> > (kbytes) =
> > 268435456 and max total shared memory (kbytes) = 268435456.
> > Thanks for your help!
> >
> > Kind regards,
> > Eric van Loon
> > Air France/KLM Storage & Backup
> >
> >
> > -Original Message-
> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
> > Of PAC Brion Arnaud
> > Sent: donderdag 22 augustus 2019 09:02
> > To: ADSM-L@VM.MARIST.EDU
> > Subject: Re: TSM server performance continuing
> >
> > Hi Eric,
> >
> > Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM),
> > and making use of directory-container storage pools only.
> > No server per

Re: TSM server performance continuing

[Loon, Eric van (ITOP NS) - KLM]

Hi Stefan,

There is no noticable impact on the TSM performance when we use the tsmdiskperf 
tool on the database volumes.
We recently discovered the issue seems to be related to the TDP for Oracle 
clients. When none are running the server response is OK, even during a 
reasonable load. But as soon as the TDP for Oracle clients start kicking in, 
the server becomes slow. We brought it to the attention of the developers, but 
no response yet...
Thanks for your help!

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan 
Folkerts
Sent: maandag 26 augustus 2019 10:39
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM server performance continuing

Eric,

What happens when you benchmark the DB volumes using the tool provided with the 
blueprints on an idle system, does the system also slow down with commands such 
as q stgpool or does it stay fast when the benchmark is running on all volumes?
Also, what kind of a result does the benchmark give you and what do you use for 
your database storage?

Regards,
   Stefan



On Thu, Aug 22, 2019 at 3:36 PM PAC Brion Arnaud 
wrote:

> Hi Eric,
>
> These seem to be default values when installing Red Hat Enterprise 
> Linux Server 7.3.
> I checked several other machines, all are having the same value.
>
> And following page
> https://stackoverflow.com/questions/55428812/how-the-values-of-kernel-
> parameters-are-defined
> seems to confirm my statement.
>
> Cheers.
>
> Arnaud
>
>
>
> **
> **
> Backup and Recovery Systems Administrator Panalpina Management Ltd., 
> Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 
> Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from 
> Panalpina and is confidential or privileged.
> This information is intended only for the person (s) named above. If 
> you are not the intended recipient, any disclosure, copying, 
> distribution or use or any other action based on the contents of this  
> information is strictly prohibited.
>
> If you receive this electronic transmission in error, please notify 
> the sender by e-mail, telephone or fax at the numbers listed above. Thank you.
>
> **
> **
> www.panalpina.com
>
>
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
> Of Loon, Eric van (ITOP NS) - KLM
> Sent: Thursday, August 22, 2019 2:31 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: TSM server performance continuing
>
> Hi Arnoud,
>
> I do not understand why your max seg size (kbytes) = 18014398509465599 
> and max total shared memory (kbytes) = 18014397435740096 are so huge. 
> If you have 256 GB RAM, like I do, I would expect max seg size 
> (kbytes) =
> 268435456 and max total shared memory (kbytes) = 268435456.
> Thanks for your help!
>
> Kind regards,
> Eric van Loon
> Air France/KLM Storage & Backup
>
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
> Of PAC Brion Arnaud
> Sent: donderdag 22 augustus 2019 09:02
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: TSM server performance continuing
>
> Hi Eric,
>
> Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), 
> and making use of directory-container storage pools only.
> No server performance or responsiveness  issues here anymore, since we 
> changed our storage to make use of IBM Isilon.
>
> Here my values :
>
> vmstat 1
> procs ---memory-- ---swap-- -io -system--
> --cpu-
>  r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id
> wa st
> 13  0 1420032 1152064 1429248 24384806400  7500  376600
> 24  3 61 11  0
> 13  1 1420032 1046016 1428288 24394348800 61425680 22427 42342
> 50  4 39  7  0
> 12  0 1420032 1144640 1426688 24386195200 542840 0 19109 33340
> 49  2 43  6  0
> 12  4 1420032 883904 1427136 24413382400 577108 0 19777 34954
> 49  2 44  6  0
> 12  1 1420032 936576 1425344 24407276800 613152   372 21110 37559
> 49  2 43  6  0
> 11  5 1420032 1028096 1425600 24397734400 549024 0 24599 47296
> 48  2 42  8  0
> 12  0 1420032 1028032 1424832 24398342400 56618016 21693 39976
> 49  2 43  6  0
>
>
> ipcs -l
>
> -- 

Re: TSM server performance continuing

[Stefan Folkerts]

Eric,

What happens when you benchmark the DB volumes using the tool provided with
the blueprints on an idle system, does the system also slow down with
commands such as q stgpool or does it stay fast when the benchmark is
running on all volumes?
Also, what kind of a result does the benchmark give you and what do you use
for your database storage?

Regards,
   Stefan



On Thu, Aug 22, 2019 at 3:36 PM PAC Brion Arnaud 
wrote:

> Hi Eric,
>
> These seem to be default values when installing Red Hat Enterprise Linux
> Server 7.3.
> I checked several other machines, all are having the same value.
>
> And following page
> https://stackoverflow.com/questions/55428812/how-the-values-of-kernel-parameters-are-defined
> seems to confirm my statement.
>
> Cheers.
>
> Arnaud
>
>
>
> 
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from Panalpina
> and is confidential or privileged.
> This information is intended only for the person (s) named above. If you
> are not the intended recipient, any disclosure, copying, distribution or
> use or any other action based on the contents of this
>  information is strictly prohibited.
>
> If you receive this electronic transmission in error, please notify the
> sender by e-mail, telephone or fax at the numbers listed above. Thank you.
>
> 
> www.panalpina.com
>
>
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
> Loon, Eric van (ITOP NS) - KLM
> Sent: Thursday, August 22, 2019 2:31 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: TSM server performance continuing
>
> Hi Arnoud,
>
> I do not understand why your max seg size (kbytes) = 18014398509465599 and
> max total shared memory (kbytes) = 18014397435740096 are so huge. If you
> have 256 GB RAM, like I do, I would expect max seg size (kbytes) =
> 268435456 and max total shared memory (kbytes) = 268435456.
> Thanks for your help!
>
> Kind regards,
> Eric van Loon
> Air France/KLM Storage & Backup
>
>
> -Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
> PAC Brion Arnaud
> Sent: donderdag 22 augustus 2019 09:02
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: TSM server performance continuing
>
> Hi Eric,
>
> Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), and
> making use of directory-container storage pools only.
> No server performance or responsiveness  issues here anymore, since we
> changed our storage to make use of IBM Isilon.
>
> Here my values :
>
> vmstat 1
> procs ---memory-- ---swap-- -io -system--
> --cpu-
>  r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id
> wa st
> 13  0 1420032 1152064 1429248 24384806400  7500  376600
> 24  3 61 11  0
> 13  1 1420032 1046016 1428288 24394348800 61425680 22427 42342
> 50  4 39  7  0
> 12  0 1420032 1144640 1426688 24386195200 542840 0 19109 33340
> 49  2 43  6  0
> 12  4 1420032 883904 1427136 24413382400 577108 0 19777 34954
> 49  2 44  6  0
> 12  1 1420032 936576 1425344 24407276800 613152   372 21110 37559
> 49  2 43  6  0
> 11  5 1420032 1028096 1425600 24397734400 549024 0 24599 47296
> 48  2 42  8  0
> 12  0 1420032 1028032 1424832 24398342400 56618016 21693 39976
> 49  2 43  6  0
>
>
> ipcs -l
>
> -- Messages Limits 
> max queues system wide = 250880
> max size of message (bytes) = 65536
> default max size of queue (bytes) = 65536
>
> -- Shared Memory Limits 
> max number of segments = 62720
> max seg size (kbytes) = 18014398509465599 max total shared memory (kbytes)
> = 18014397435740096 min seg size (bytes) = 1
>
> -- Semaphore Limits 
> max number of arrays = 62720
> max semaphores per array = 250
> max semaphores system wide = 256000
> max ops per semop call = 32
> semaphore max value = 32767
>
>
> Specific tuning done : disabling CPU virtualization (led to sever hangs
> when active)
>
> ppc64_cpu --smt=off
>
>
> Cheers.
>
> Arnaud
>
>
> *

Re: TSM server performance continuing

[PAC Brion Arnaud]

Hi Eric,

These seem to be default values when installing Red Hat Enterprise Linux Server 
7.3. 
I checked several other machines, all are having the same value.

And following page 
https://stackoverflow.com/questions/55428812/how-the-values-of-kernel-parameters-are-defined
 seems to confirm my statement.

Cheers.

Arnaud



Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. 
This information is intended only for the person (s) named above. If you are 
not the intended recipient, any disclosure, copying, distribution or use or any 
other action based on the contents of this
 information is strictly prohibited. 

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.

www.panalpina.com



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
Eric van (ITOP NS) - KLM
Sent: Thursday, August 22, 2019 2:31 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM server performance continuing

Hi Arnoud,

I do not understand why your max seg size (kbytes) = 18014398509465599 and max 
total shared memory (kbytes) = 18014397435740096 are so huge. If you have 256 
GB RAM, like I do, I would expect max seg size (kbytes) = 268435456 and max 
total shared memory (kbytes) = 268435456.
Thanks for your help!

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC 
Brion Arnaud
Sent: donderdag 22 augustus 2019 09:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM server performance continuing

Hi Eric,

Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), and 
making use of directory-container storage pools only.
No server performance or responsiveness  issues here anymore, since we changed 
our storage to make use of IBM Isilon. 

Here my values :

vmstat 1
procs ---memory-- ---swap-- -io -system-- --cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id wa st
13  0 1420032 1152064 1429248 24384806400  7500  376600 24  3 
61 11  0
13  1 1420032 1046016 1428288 24394348800 61425680 22427 42342 50  
4 39  7  0
12  0 1420032 1144640 1426688 24386195200 542840 0 19109 33340 49  
2 43  6  0
12  4 1420032 883904 1427136 24413382400 577108 0 19777 34954 49  2 
44  6  0
12  1 1420032 936576 1425344 24407276800 613152   372 21110 37559 49  2 
43  6  0
11  5 1420032 1028096 1425600 24397734400 549024 0 24599 47296 48  
2 42  8  0
12  0 1420032 1028032 1424832 24398342400 56618016 21693 39976 49  
2 43  6  0


ipcs -l

-- Messages Limits 
max queues system wide = 250880
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 62720
max seg size (kbytes) = 18014398509465599 max total shared memory (kbytes) = 
18014397435740096 min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 62720
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767


Specific tuning done : disabling CPU virtualization (led to sever hangs when 
active)

ppc64_cpu --smt=off


Cheers.

Arnaud


Backup and Recovery Systems Administrator Panalpina Management Ltd., Basle, 
Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. 
This information is intended only for the person (s) named above. If you are 
not the intended recipient, any disclosure, copying, distribution or use or any 
other action based on the contents of this  information is strictly prohibited. 

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.

www.panalpina

Re: TSM server performance continuing

[Loon, Eric van (ITOP NS) - KLM]

Hi Arnoud,

I do not understand why your max seg size (kbytes) = 18014398509465599 and max 
total shared memory (kbytes) = 18014397435740096 are so huge. If you have 256 
GB RAM, like I do, I would expect max seg size (kbytes) = 268435456 and max 
total shared memory (kbytes) = 268435456.
Thanks for your help!

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of PAC 
Brion Arnaud
Sent: donderdag 22 augustus 2019 09:02
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM server performance continuing

Hi Eric,

Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), and 
making use of directory-container storage pools only.
No server performance or responsiveness  issues here anymore, since we changed 
our storage to make use of IBM Isilon. 

Here my values :

vmstat 1
procs ---memory-- ---swap-- -io -system-- --cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id wa st
13  0 1420032 1152064 1429248 24384806400  7500  376600 24  3 
61 11  0
13  1 1420032 1046016 1428288 24394348800 61425680 22427 42342 50  
4 39  7  0
12  0 1420032 1144640 1426688 24386195200 542840 0 19109 33340 49  
2 43  6  0
12  4 1420032 883904 1427136 24413382400 577108 0 19777 34954 49  2 
44  6  0
12  1 1420032 936576 1425344 24407276800 613152   372 21110 37559 49  2 
43  6  0
11  5 1420032 1028096 1425600 24397734400 549024 0 24599 47296 48  
2 42  8  0
12  0 1420032 1028032 1424832 24398342400 56618016 21693 39976 49  
2 43  6  0


ipcs -l

-- Messages Limits 
max queues system wide = 250880
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 62720
max seg size (kbytes) = 18014398509465599 max total shared memory (kbytes) = 
18014397435740096 min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 62720
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767


Specific tuning done : disabling CPU virtualization (led to sever hangs when 
active)

ppc64_cpu --smt=off


Cheers.

Arnaud


Backup and Recovery Systems Administrator Panalpina Management Ltd., Basle, 
Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. 
This information is intended only for the person (s) named above. If you are 
not the intended recipient, any disclosure, copying, distribution or use or any 
other action based on the contents of this  information is strictly prohibited. 

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.

www.panalpina.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
Eric van (ITOP NS) - KLM
Sent: Wednesday, August 21, 2019 5:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: TSM server performance continuing

Hi guys,

A few weeks ago I already wrote about the severe performance issues we have 
with our TSM 7.1 servers. In the 'old days' we used to back up our clients to 
TSM 6.3 servers with Data Domains attached. Smaller clients backed up through 
the LAN, large ones through the SAN.
Our newer servers use LAN-only with directory containers and the performance of 
these servers really sucks. Setting up a session takes sometimes almost one 
minute and a q stg also takes 30 to 50 seconds. I noticed that performance is 
OK when there are no TDP for Oracle sessions running, but as soon as they are 
started the performance starts to drop drastically.
We are really lost on where to look for the cause. I sent numerous logs and 
traces to IBM, but I guess they are out of ideas too since I don't hear 
anything back from them lately. The only thing is that supports notices delays 
in DB2, but they don't know why...
What I noticed on my TSM server is that as soon as there is a load on the 
server, the blocked queue starts to rise. I would like to know if that's 
something to focus on or not.
Can some of you please run the "vmstat 1" command on their Linux server 
(preferably one with directory containers too) and let me know if you too see 
values other than 0 in the B column?
Thank you very much for your help in advance!

Kin

Re: TSM server performance continuing

[PAC Brion Arnaud]

Hi Rick,

Following to your question, I  realized that I erroneously stated "Isilon" 
instead of "Storwize".

Probably due to the fact that I've been stressed quite a lot with these 
devices, when we initially started setting up a new Spectrum environment, 
making use of Isilon devices as a destination for our directory-container pools.
To make a very long story (more than one year) short,  we experienced huge 
performance issues with Isilon, opened cases with IBM as well as EMC, to 
finally learn that, due to the block size used by container pools we would 
never be able to get sufficient performance with Isilons. 
Final result : we do have a brand new pair of Isilons for sale, which has 
advantageously been replaced by a pair of Storwizes. Since then, no issues any 
longer.

I finally did not answer your question : we used 5 X NL140 nodes in each of our 
Isilons.

Cheers.

Arnaud


Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. 
This information is intended only for the person (s) named above. If you are 
not the intended recipient, any disclosure, copying, distribution or use or any 
other action based on the contents of this
 information is strictly prohibited. 

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.

www.panalpina.com



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Rick 
Adamson
Sent: Thursday, August 22, 2019 1:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM server performance continuing

Arnaud,
Apologies for off topic question but what series nodes are you using in the 
Isilon?

Thank you,
-Rick Adamson

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of PAC Brion 
Arnaud
Sent: Thursday, August 22, 2019 3:02 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM server performance continuing

* This email originated outside of the organization. Use caution when opening 
attachments or clicking links. *

------
Hi Eric,

Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), and 
making use of directory-container storage pools only.
No server performance or responsiveness  issues here anymore, since we changed 
our storage to make use of IBM Isilon.

Here my values :

vmstat 1
procs ---memory-- ---swap-- -io -system-- --cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id wa st
13  0 1420032 1152064 1429248 24384806400  7500  376600 24  3 
61 11  0
13  1 1420032 1046016 1428288 24394348800 61425680 22427 42342 50  
4 39  7  0
12  0 1420032 1144640 1426688 24386195200 542840 0 19109 33340 49  
2 43  6  0
12  4 1420032 883904 1427136 24413382400 577108 0 19777 34954 49  2 
44  6  0
12  1 1420032 936576 1425344 24407276800 613152   372 21110 37559 49  2 
43  6  0
11  5 1420032 1028096 1425600 24397734400 549024 0 24599 47296 48  
2 42  8  0
12  0 1420032 1028032 1424832 24398342400 56618016 21693 39976 49  
2 43  6  0


ipcs -l

-- Messages Limits 
max queues system wide = 250880
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 62720
max seg size (kbytes) = 18014398509465599 max total shared memory (kbytes) = 
18014397435740096 min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 62720
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767


Specific tuning done : disabling CPU virtualization (led to sever hangs when 
active)

ppc64_cpu --smt=off


Cheers.

Arnaud


Backup and Recovery Systems Administrator Panalpina Management Ltd., Basle, 
Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged.
This information is intended only for the person (s) named a

Re: TSM server performance continuing

[Rick Adamson]

Arnaud,
Apologies for off topic question but what series nodes are you using in the 
Isilon?

Thank you,
-Rick Adamson

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of PAC Brion 
Arnaud
Sent: Thursday, August 22, 2019 3:02 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM server performance continuing

* This email originated outside of the organization. Use caution when opening 
attachments or clicking links. *

--
Hi Eric,

Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), and 
making use of directory-container storage pools only.
No server performance or responsiveness  issues here anymore, since we changed 
our storage to make use of IBM Isilon.

Here my values :

vmstat 1
procs ---memory-- ---swap-- -io -system-- --cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id wa st
13  0 1420032 1152064 1429248 24384806400  7500  376600 24  3 
61 11  0
13  1 1420032 1046016 1428288 24394348800 61425680 22427 42342 50  
4 39  7  0
12  0 1420032 1144640 1426688 24386195200 542840 0 19109 33340 49  
2 43  6  0
12  4 1420032 883904 1427136 24413382400 577108 0 19777 34954 49  2 
44  6  0
12  1 1420032 936576 1425344 24407276800 613152   372 21110 37559 49  2 
43  6  0
11  5 1420032 1028096 1425600 24397734400 549024 0 24599 47296 48  
2 42  8  0
12  0 1420032 1028032 1424832 24398342400 56618016 21693 39976 49  
2 43  6  0


ipcs -l

-- Messages Limits 
max queues system wide = 250880
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 62720
max seg size (kbytes) = 18014398509465599 max total shared memory (kbytes) = 
18014397435740096 min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 62720
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767


Specific tuning done : disabling CPU virtualization (led to sever hangs when 
active)

ppc64_cpu --smt=off


Cheers.

Arnaud


Backup and Recovery Systems Administrator Panalpina Management Ltd., Basle, 
Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged.
This information is intended only for the person (s) named above. If you are 
not the intended recipient, any disclosure, copying, distribution or use or any 
other action based on the contents of this  information is strictly prohibited.

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.

https://urldefense.proofpoint.com/v2/url?u=http-3A__www.panalpina.com&d=DwIFAg&c=AzgFQeXLLKhxSQaoFCm29A&r=x0AiM0UTlHOcoXM_eMvEL8EwIXpa5iV2N-3bEgpYmSc&m=_ScFjeOmwDRocHyDnLD8YdQGPCGWXExk2dQSGAvP0DA&s=RKD9HI_FertRCOmtp4vxEb5f6R6GHAlboIoIdMUBzjc&e=


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
Eric van (ITOP NS) - KLM
Sent: Wednesday, August 21, 2019 5:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: TSM server performance continuing

Hi guys,

A few weeks ago I already wrote about the severe performance issues we have 
with our TSM 7.1 servers. In the 'old days' we used to back up our clients to 
TSM 6.3 servers with Data Domains attached. Smaller clients backed up through 
the LAN, large ones through the SAN.
Our newer servers use LAN-only with directory containers and the performance of 
these servers really sucks. Setting up a session takes sometimes almost one 
minute and a q stg also takes 30 to 50 seconds. I noticed that performance is 
OK when there are no TDP for Oracle sessions running, but as soon as they are 
started the performance starts to drop drastically.
We are really lost on where to look for the cause. I sent numerous logs and 
traces to IBM, but I guess they are out of ideas too since I don't hear 
anything back from them lately. The only thing is that supports notices delays 
in DB2, but they don't know why...
What I noticed on my TSM server is that as soon as there is a load on the 
server, the blocked queue starts to rise. I would like to know if that's 
something to focus on or not.
Can some of you please run the "vmstat 1" command on their Linux server 
(preferably on

Re: TSM server performance continuing

[Loon, Eric van (ITOP NS) - KLM]

Hi Eric,

Any reason why your kernel.msgmni = 196608 from the /etc/sysyctl.conf differs 
from the max queues system wide = 516096 from a ipcs -l?
Just curious...
Thanks for your help!

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of J. 
Eric Wonderley
Sent: woensdag 21 augustus 2019 18:18
To: ADSM-L@VM.MARIST.EDU
Subject: Re: TSM server performance continuing

Get all zeros...

We have these settings on a dell730:
ipcs -l

-- Messages Limits 
max queues system wide = 516096
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 129024
max seg size (kbytes) = 528077736
max total shared memory (kbytes) = 17179869184 min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 129024
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767


# Put changes in /etc/sysctl.conf
kernel.shmmni =  49152
kernel.shmmax = 206158430208
kernel.shmsem = 250 256000 32 49152
kernel.msgmni = 196608
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.randomize_va_space = 0
vm.swappiness = 0
vm.overcommit_memory = 0




ERic



540-392-1742 (Cell)


On Wed, Aug 21, 2019 at 11:44 AM Loon, Eric van (ITOP NS) - KLM < 
eric-van.l...@klm.com> wrote:

> Hi guys,
>
> A few weeks ago I already wrote about the severe performance issues we 
> have with our TSM 7.1 servers. In the 'old days' we used to back up 
> our clients to TSM 6.3 servers with Data Domains attached. Smaller 
> clients backed up through the LAN, large ones through the SAN.
> Our newer servers use LAN-only with directory containers and the 
> performance of these servers really sucks. Setting up a session takes 
> sometimes almost one minute and a q stg also takes 30 to 50 seconds. I 
> noticed that performance is OK when there are no TDP for Oracle 
> sessions running, but as soon as they are started the performance 
> starts to drop drastically.
> We are really lost on where to look for the cause. I sent numerous 
> logs and traces to IBM, but I guess they are out of ideas too since I 
> don't hear anything back from them lately. The only thing is that 
> supports notices delays in DB2, but they don't know why...
> What I noticed on my TSM server is that as soon as there is a load on 
> the server, the blocked queue starts to rise. I would like to know if 
> that's something to focus on or not.
> Can some of you please run the "vmstat 1" command on their Linux 
> server (preferably one with directory containers too) and let me know 
> if you too see values other than 0 in the B column?
> Thank you very much for your help in advance!
>
> Kind regards,
> Eric van Loon
> Air France/KLM Storage & Backup
> 
> For information, services and offers, please visit our web site:
> http://www.klm.com. This e-mail and any attachment may contain 
> confidential and privileged material intended for the addressee only. 
> If you are not the addressee, you are notified that no part of the 
> e-mail or any attachment may be disclosed, copied or distributed, and 
> that any other action related to this e-mail or attachment is strictly 
> prohibited, and may be unlawful. If you have received this e-mail by 
> error, please notify the sender immediately by return e-mail, and delete this 
> message.
>
> Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or 
> its employees shall not be liable for the incorrect or incomplete 
> transmission of this e-mail or any attachments, nor responsible for any delay 
> in receipt.
> Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal 
> Dutch
> Airlines) is registered in Amstelveen, The Netherlands, with 
> registered number 33014286
> 
>

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or

Re: TSM server performance continuing

[PAC Brion Arnaud]

Hi Eric,

Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), and 
making use of directory-container storage pools only.
No server performance or responsiveness  issues here anymore, since we changed 
our storage to make use of IBM Isilon. 

Here my values :

vmstat 1
procs ---memory-- ---swap-- -io -system-- --cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id wa st
13  0 1420032 1152064 1429248 24384806400  7500  376600 24  3 
61 11  0
13  1 1420032 1046016 1428288 24394348800 61425680 22427 42342 50  
4 39  7  0
12  0 1420032 1144640 1426688 24386195200 542840 0 19109 33340 49  
2 43  6  0
12  4 1420032 883904 1427136 24413382400 577108 0 19777 34954 49  2 
44  6  0
12  1 1420032 936576 1425344 24407276800 613152   372 21110 37559 49  2 
43  6  0
11  5 1420032 1028096 1425600 24397734400 549024 0 24599 47296 48  
2 42  8  0
12  0 1420032 1028032 1424832 24398342400 56618016 21693 39976 49  
2 43  6  0


ipcs -l

-- Messages Limits 
max queues system wide = 250880
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 62720
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18014397435740096
min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 62720
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767


Specific tuning done : disabling CPU virtualization (led to sever hangs when 
active)

ppc64_cpu --smt=off


Cheers.

Arnaud


Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. 
This information is intended only for the person (s) named above. If you are 
not the intended recipient, any disclosure, copying, distribution or use or any 
other action based on the contents of this
 information is strictly prohibited. 

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.

www.panalpina.com


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
Eric van (ITOP NS) - KLM
Sent: Wednesday, August 21, 2019 5:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: TSM server performance continuing

Hi guys,

A few weeks ago I already wrote about the severe performance issues we have 
with our TSM 7.1 servers. In the 'old days' we used to back up our clients to 
TSM 6.3 servers with Data Domains attached. Smaller clients backed up through 
the LAN, large ones through the SAN.
Our newer servers use LAN-only with directory containers and the performance of 
these servers really sucks. Setting up a session takes sometimes almost one 
minute and a q stg also takes 30 to 50 seconds. I noticed that performance is 
OK when there are no TDP for Oracle sessions running, but as soon as they are 
started the performance starts to drop drastically.
We are really lost on where to look for the cause. I sent numerous logs and 
traces to IBM, but I guess they are out of ideas too since I don't hear 
anything back from them lately. The only thing is that supports notices delays 
in DB2, but they don't know why...
What I noticed on my TSM server is that as soon as there is a load on the 
server, the blocked queue starts to rise. I would like to know if that's 
something to focus on or not.
Can some of you please run the "vmstat 1" command on their Linux server 
(preferably one with directory containers too) and let me know if you too see 
values other than 0 in the B column?
Thank you very much for your help in advance!

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please 

Re: TSM server performance continuing

[Sasa Drnjevic]

Hi - FYI.

Got 3 TSM v8.1.1.0 servers. Srry, no directory containers.

Two are mostly idle ATM, and vmstat 1 B column shows all zeroes on these.

But, the 3rd which is the primary backup server is currently serving
main backup window with 4 data migration to tape processes active ATM so
it is under load ATM and shows versatile values under B column as follows:


procs ---memory-- ---swap-- -io -system--
--cpu-
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy
id wa st
 2  6  0 2194596 978956 11943265600  2064   93300  1
 0 96  3  0
 1  4  0 2195356 978956 11943171200 404128 179057 18786
205808  5  1 85  9  0
 5  6  0 1132332 978960 12049406400 857920 120500 15741
113912  4  2 84  9  0
 3  5  0 1131556 978968 12049353600 490176 476828 14546
111938  4  1 87  8  0
 4  6  0 1130816 978968 12049331200 701384 46616 27406
208660  7  2 74 17  0
 4  6  0 1130840 978968 12049328000 808156 35801 25640
172369  6  2 73 19  0
 2  5  0 1129272 979000 12049321600 719460 37232 27778
189991  7  2 72 19  0
 3 22  0 1130660 979000 12049321600 794196 29051 23424
168768  6  1 74 18  0
 3 28  0 1129624 979000 12049321600 806280 35076 27397
183748  7  2 73 19  0
 2 23  0 1130648 979000 12049321600 747764 36656 26441
172916  6  2 71 21  0
 3  7  0 1131300 979000 12049321600 845076 38928 31620
199619  6  2 68 23  0
 2  5  0 1131672 979000 12049321600 859640 47688 30922
191109  5  2 71 21  0
 2  4  0 1132028 979000 12049321600 749464 40088 27370
179763  4  2 74 20  0
 2  4  0 1132596 979000 12049321600 473644 89892 16032
174401  4  1 85  9  0
12  4  0 1131600 979000 12049313600 430004 38064 21265
179632  6  1 77 16  0
 2 14  0 1132076 979000 12049313600 325864 38388 19710
173502  6  1 77 15  0
 4  4  0 1132432 979000 12049313600 527528 36916 24634
128603  6  1 75 18  0
 7  3  0 1088084 979000 12049850400 562960 24604 24020
118291  6  1 74 18  0
 3 18  0 905236 979000 12049852800 394620 30736 22646 101360
 7  1 72 20  0
 3  4  0 1120396 979032 12049854400 455848 40864 29153
137940  7  2 73 18  0
 3 21  0 1118804 979032 12049843200 381536 38436 20954
111904  6  1 77 16  0
 4 12  0 1118812 979032 12049843200 493040 20504 20089
127127  6  1 74 19  0
 1  3  0 1118448 979032 12049843200 473619 33028 21283
121339  6  1 77 17  0
 1  4  0 1119776 979032 12049843200 71884 39700 8165 45166
3  0 89  7  0
 5 21  0 1120064 979032 12049843200 269032 48024 17034 79659
 5  1 81 14  0
 1  4  0 1120788 979032 12049843200 360992 93243 18190 99504
 4  1 81 14  0
 4 21  0 1120180 979032 12049843200 589440 65658 23841
128754  3  1 74 21  0
 2 22  0 1121212 979032 12049843200 500856 53118 21264
105391  2  1 75 21  0
 2 12  0 1121520 979032 12049843200 644036 36889 23167
124724  3  1 78 18  0
 1  4  0 1120616 979032 12049843200 835200 85787 26382
149143  3  1 76 20  0
 1  4  0 1121308 979032 12049848000 563944 29029 19323
126179  3  1 78 18  0
 4 28  0 1121256 979064 12049840000 375960 39008 19655 91139
 3  1 77 19  0
ctrl-c


All, Dell servers with 24 CPU cores, and 128 GB RAM.

Never had performance problems.

Rgds,

--
Sasa Drnjevic
www.srce.unizg.hr





On 2019-08-21 17:39, Loon, Eric van (ITOP NS) - KLM wrote:
> Hi guys,
>
> A few weeks ago I already wrote about the severe performance issues we have 
> with our TSM 7.1 servers. In the 'old days' we used to back up our clients to 
> TSM 6.3 servers with Data Domains attached. Smaller clients backed up through 
> the LAN, large ones through the SAN.
> Our newer servers use LAN-only with directory containers and the performance 
> of these servers really sucks. Setting up a session takes sometimes almost 
> one minute and a q stg also takes 30 to 50 seconds. I noticed that 
> performance is OK when there are no TDP for Oracle sessions running, but as 
> soon as they are started the performance starts to drop drastically.
> We are really lost on where to look for the cause. I sent numerous logs and 
> traces to IBM, but I guess they are out of ideas too since I don't hear 
> anything back from them lately. The only thing is that supports notices 
> delays in DB2, but they don't know why...
> What I noticed on my TSM server is that as soon as there is a load on the 
> server, the blocked queue starts to rise. I would like to know if that's 
> something to focus on or not.
> Can some of you please run the "vmstat 1" command on their Linux server 
> (preferably one with directory containers too) and let me know if you too see 
> values other than 0 in the B column?
> Tha

Re: TSM server performance continuing

[Kizzire, Chris]

We were having performance issues. I still think there are some. I could tell 
it was having issues when Q PR took way too long to show- if at all. But much 
improved now after RAM increase.

Specs for our environment:
SP 8.1.4.0
IBM 750- AIX 7.2 
3.1 TB DB on SSD
128GB RAM increased to 180GB

DR site:
SP 8.1.4.0
IBM 770- AIX 7.2
3.1TB DB on HDD
104GB RAM increased to 250GB

WOW at the difference.
We did increase Quedepth. The big difference was after RAM increase. DeDup & 
compression are memory hogs.
We still plan to open a PMR for AIX performance. We are now maxed out on RAM.

Chris Kizzire
Backup Administrator (Network Engineer II)

BROOKWOOD BAPTIST HEALTH
Information Systems
O:   205.820.5973

chris.kizz...@bhsala.com
BROOKWOODBAPTISTHEALTH.COM

-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Loon, Eric 
van (ITOP NS) - KLM
Sent: Wednesday, August 21, 2019 10:39 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] TSM server performance continuing


CAUTION: ***EXTERNAL EMAIL*** Do NOT click links or open attachments unless you 
recognize the sender and know the content is safe. If you are unsure, please 
use PhishAlarm to report suspicious emails.

Hi guys,

A few weeks ago I already wrote about the severe performance issues we have 
with our TSM 7.1 servers. In the 'old days' we used to back up our clients to 
TSM 6.3 servers with Data Domains attached. Smaller clients backed up through 
the LAN, large ones through the SAN.
Our newer servers use LAN-only with directory containers and the performance of 
these servers really sucks. Setting up a session takes sometimes almost one 
minute and a q stg also takes 30 to 50 seconds. I noticed that performance is 
OK when there are no TDP for Oracle sessions running, but as soon as they are 
started the performance starts to drop drastically.
We are really lost on where to look for the cause. I sent numerous logs and 
traces to IBM, but I guess they are out of ideas too since I don't hear 
anything back from them lately. The only thing is that supports notices delays 
in DB2, but they don't know why...
What I noticed on my TSM server is that as soon as there is a load on the 
server, the blocked queue starts to rise. I would like to know if that's 
something to focus on or not.
Can some of you please run the "vmstat 1" command on their Linux server 
(preferably one with directory containers too) and let me know if you too see 
values other than 0 in the B column?
Thank you very much for your help in advance!

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286


--- Confidentiality Notice: The 
information contained in this email message is privileged and confidential 
information and intended only for the use of the individual or entity named in 
the address. If you are not the intended recipient, you are hereby notified 
that any dissemination, distribution, or copying of this information is 
strictly prohibited. If you received this information in error, please notify 
the sender and delete this information from your computer and retain no copies 
of any of this information.

Re: TSM server performance continuing

[J. Eric Wonderley]

Get all zeros...

We have these settings on a dell730:
ipcs -l

-- Messages Limits 
max queues system wide = 516096
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 129024
max seg size (kbytes) = 528077736
max total shared memory (kbytes) = 17179869184
min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 129024
max semaphores per array = 250
max semaphores system wide = 256000
max ops per semop call = 32
semaphore max value = 32767


# Put changes in /etc/sysctl.conf
kernel.shmmni =  49152
kernel.shmmax = 206158430208
kernel.shmsem = 250 256000 32 49152
kernel.msgmni = 196608
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.randomize_va_space = 0
vm.swappiness = 0
vm.overcommit_memory = 0




ERic



540-392-1742 (Cell)


On Wed, Aug 21, 2019 at 11:44 AM Loon, Eric van (ITOP NS) - KLM <
eric-van.l...@klm.com> wrote:

> Hi guys,
>
> A few weeks ago I already wrote about the severe performance issues we
> have with our TSM 7.1 servers. In the 'old days' we used to back up our
> clients to TSM 6.3 servers with Data Domains attached. Smaller clients
> backed up through the LAN, large ones through the SAN.
> Our newer servers use LAN-only with directory containers and the
> performance of these servers really sucks. Setting up a session takes
> sometimes almost one minute and a q stg also takes 30 to 50 seconds. I
> noticed that performance is OK when there are no TDP for Oracle sessions
> running, but as soon as they are started the performance starts to drop
> drastically.
> We are really lost on where to look for the cause. I sent numerous logs
> and traces to IBM, but I guess they are out of ideas too since I don't hear
> anything back from them lately. The only thing is that supports notices
> delays in DB2, but they don't know why...
> What I noticed on my TSM server is that as soon as there is a load on the
> server, the blocked queue starts to rise. I would like to know if that's
> something to focus on or not.
> Can some of you please run the "vmstat 1" command on their Linux server
> (preferably one with directory containers too) and let me know if you too
> see values other than 0 in the B column?
> Thank you very much for your help in advance!
>
> Kind regards,
> Eric van Loon
> Air France/KLM Storage & Backup
> 
> For information, services and offers, please visit our web site:
> http://www.klm.com. This e-mail and any attachment may contain
> confidential and privileged material intended for the addressee only. If
> you are not the addressee, you are notified that no part of the e-mail or
> any attachment may be disclosed, copied or distributed, and that any other
> action related to this e-mail or attachment is strictly prohibited, and may
> be unlawful. If you have received this e-mail by error, please notify the
> sender immediately by return e-mail, and delete this message.
>
> Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its
> employees shall not be liable for the incorrect or incomplete transmission
> of this e-mail or any attachments, nor responsible for any delay in receipt.
> Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
> Airlines) is registered in Amstelveen, The Netherlands, with registered
> number 33014286
> 
>

TSM server performance continuing

[Loon, Eric van (ITOP NS) - KLM]

Hi guys,

A few weeks ago I already wrote about the severe performance issues we have 
with our TSM 7.1 servers. In the 'old days' we used to back up our clients to 
TSM 6.3 servers with Data Domains attached. Smaller clients backed up through 
the LAN, large ones through the SAN.
Our newer servers use LAN-only with directory containers and the performance of 
these servers really sucks. Setting up a session takes sometimes almost one 
minute and a q stg also takes 30 to 50 seconds. I noticed that performance is 
OK when there are no TDP for Oracle sessions running, but as soon as they are 
started the performance starts to drop drastically.
We are really lost on where to look for the cause. I sent numerous logs and 
traces to IBM, but I guess they are out of ideas too since I don't hear 
anything back from them lately. The only thing is that supports notices delays 
in DB2, but they don't know why...
What I noticed on my TSM server is that as soon as there is a load on the 
server, the blocked queue starts to rise. I would like to know if that's 
something to focus on or not.
Can some of you please run the "vmstat 1" command on their Linux server 
(preferably one with directory containers too) and let me know if you too see 
values other than 0 in the B column?
Thank you very much for your help in advance!

Kind regards,
Eric van Loon
Air France/KLM Storage & Backup

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286

Running TSM/SP in a cluster?

[Loon, Eric van (ITOP NS) - KLM]

Hi guys,
I'm wondering if more people are running TSM in a cluster configuration. We 
always used Linux clusters based on Veritas Cluster Service (VCS). Ever since 
we were switched to IBM hardware we also had to switch to a different multipath 
driver and that is where the problems started.
We always used Powerpath together with EMC hardware which worked OK. VCS only 
supports Powerpath and Veritas DMP and because we switched to an IBM Storwize 
v5010 we switched to DMP. However, DMP doesn't seem to be able to properly 
handle 4 an 8K I/O's so write performance on the database volumes is awful. 
Veritas doesn't seem to be able to fix this issue, so I'm looking at 
alternatives.
What clustering software are you guys using and what is your experience with it?
Thanks for any reply in advance!
Kind regards,
Eric van Loon
Air France/KLM Storage & Backup

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286

Re: large TSM server anomolies

[Ashish Sharma]

I have a 11 TB TSM server backing up data in container pools with 50+ TB
daily ingest.

On Wed 3 Apr, 2019, 4:44 PM Dwight Cook,  wrote:

> So who all out there has TSM servers with DB's in the 3+ TB range that
> ingest 50+-ish TB's per night into a container storage pool?
> Doesn't matter if it is 7.1 or 8.1 or AIX or Linux
> I'm seeing behavior that I would classify as ~odd~ and wondering if others
> might be too.
>
>
> Dwight E. Cook
> Senior Architect
> TSM Delivery, IBM Services for Managed Applications
> (918) 493-4678
>

large TSM server anomolies

[Dwight Cook]

So who all out there has TSM servers with DB's in the 3+ TB range that 
ingest 50+-ish TB's per night into a container storage pool?
Doesn't matter if it is 7.1 or 8.1 or AIX or Linux
I'm seeing behavior that I would classify as ~odd~ and wondering if others 
might be too.


Dwight E. Cook
Senior Architect
TSM Delivery, IBM Services for Managed Applications
(918) 493-4678

Re: VERY SLOW DB BACKUP TO TAPE | TS3310 with LTO5 | LIN_TAPE 3.033 | TSM SERVER 7.1.9.000

[Schneck Dennis]

Hello Sasa,

had this parameters


tsminst1  soft   nofile 8096
tsminst1  hard   nofile 21832

will change and retest



Am 06.12.18 um 09:50 schrieb Sasa Drnjevic:
> On 6.12.2018. 6:20, Schneck Dennis wrote:
>> Hello Sasa,
>>
>>> did you tune kernel parameters for your Linux TSM server?
>> yes, here is my /etc/sysctl.conf>
>> kernel.randomize_va_space=0
>> vm.swappiness=0
>> vm.overcommit_memory=0
>>
> My ipcs is somewhat different, and I doubt it has anything to do with
> your problem - here it is:
>
>
> # ipcs -l
>
> -- Messages Limits 
> max queues system wide = 131072
> max size of message (bytes) = 65536
> default max size of queue (bytes) = 65536
>
> -- Shared Memory Limits 
> max number of segments = 32768
> max seg size (kbytes) = 134217728
> max total shared memory (kbytes) = 18014398442373116
> min seg size (bytes) = 1
>
> -- Semaphore Limits 
> max number of arrays = 32768
> max semaphores per array = 300
> max semaphores system wide = 9830400
> max ops per semop call = 35
> semaphore max value = 32767
> --
>
>
> And your TSMDB1 looks just fine...
>
>
> Also, I doubt the following would be connected with your issue, but here
> it is... Did you tune TSM instance user in /etc/security/limits.conf ?
>
> This is mine:
>
> tsminst01  softnofile  26400
> tsminst01  hardnofile  66000
>
>
> --
> Sasa
>
>
>
>> tsm01:~ # ipcs -l
>>
>> -- Messages Limits 
>>
>> max queues system wide = 64512
>>
>> max size of message (bytes) = 65536
>>
>> default max size of queue (bytes) = 65536
>>
>> -- Shared Memory Limits 
>>
>> max number of segments = 16128
>>
>> max seg size (kbytes) = 18014398509481983
>>
>> max total shared memory (kbytes) = 18014398509480960
>>
>> min seg size (bytes) = 1
>>
>> -- Semaphore Limits 
>>
>> max number of arrays = 32000
>>
>> max semaphores per array = 32000
>>
>> max semaphores system wide = 102400
>>
>> max ops per semop call = 500
>>
>> semaphore max value = 32767
>>
>>
>>> And what is the size of you TSM DB and on what type of storage and
>>> filesystem is it located?
>> tsm: TSM01>q db
>>
>> Database Name   Total Pages Usable Pages   Used Pages   Free 
>> Pages
>> --    
>> 
>> TSMDB1  102,416   99,312   43,100   
>> 56,212
>>
>> tsm: TSM01>q db f=d
>>
>>     Database Name: TSMDB1
>>   Total Space of File System (MB): 197,939
>>     Space Used on File System(MB): 11,444
>>    Space Used by Database(MB): 1,280
>>     Free Space Available (MB): 186,495
>>   Total Pages: 102,416
>>  Usable Pages: 99,312
>>    Used Pages: 43,100
>>    Free Pages: 56,212
>>     Buffer Pool Hit Ratio: 99.9
>>     Total Buffer Requests: 4,971,182
>>    Sort Overflows: 0
>>   Package Cache Hit Ratio: 87.1
>>  Last Database Reorganization: 12/04/18   16:44:07
>>    Full Device Class Name: 3310L1DEV
>> Number of Database Backup Streams: 1
>>  Incrementals Since Last Full: 0
>>    Last Complete Backup Date/Time: 12/05/18   15:06:39
>>     Compress Database Backups: No
>>     Protect Master Encryption Key: No
>>
>> Filesystem   Size  Used Avail Use% Mounted on
>>
>> /dev/mapper/SLESVG-lvtsmdb01  49G  373M   46G   1% /tsminst1/db01
>>
>> /dev/mapper/SLESVG-lvtsmdb02  49G  373M   46G   1% /tsminst1/db02
>>
>> /dev/mapper/SLESVG-lvtsmdb03  49G  373M   46G   1% /tsminst1/db03
>>
>> /dev/mapper/SLESVG-lvtsmdb04  49G  373M   46G   1% /tsminst1/db04
>>
>>
>>
>>
>> Am 05.12.18 um 16:45 schrieb Sasa Drnjevic:
>>> On 2018-12-05 15:06, Schneck Dennis wrote:
>>>> Hello Sasa
>>>>
>>>>> Check for errors on FC switch ports...
>>>> Ok will inform the colleage in the branch
>>>>
>>>>
>>>>> When have the problems started?
>>>> Its a new server - did not work before
>>> Hi Dennis,
>>>
>>> did you tune kernel 

Re: VERY SLOW DB BACKUP TO TAPE | TS3310 with LTO5 | LIN_TAPE 3.033 | TSM SERVER 7.1.9.000

[Sasa Drnjevic]

On 6.12.2018. 6:20, Schneck Dennis wrote:
> Hello Sasa,
> 
>> did you tune kernel parameters for your Linux TSM server?
> 
> yes, here is my /etc/sysctl.conf>
> kernel.randomize_va_space=0
> vm.swappiness=0
> vm.overcommit_memory=0
> 

My ipcs is somewhat different, and I doubt it has anything to do with
your problem - here it is:


# ipcs -l

-- Messages Limits 
max queues system wide = 131072
max size of message (bytes) = 65536
default max size of queue (bytes) = 65536

-- Shared Memory Limits 
max number of segments = 32768
max seg size (kbytes) = 134217728
max total shared memory (kbytes) = 18014398442373116
min seg size (bytes) = 1

-- Semaphore Limits 
max number of arrays = 32768
max semaphores per array = 300
max semaphores system wide = 9830400
max ops per semop call = 35
semaphore max value = 32767
--


And your TSMDB1 looks just fine...


Also, I doubt the following would be connected with your issue, but here
it is... Did you tune TSM instance user in /etc/security/limits.conf ?

This is mine:

tsminst01  softnofile  26400
tsminst01  hardnofile  66000


--
Sasa



> tsm01:~ # ipcs -l
> 
> -- Messages Limits 
> 
> max queues system wide = 64512
> 
> max size of message (bytes) = 65536
> 
> default max size of queue (bytes) = 65536
> 
> -- Shared Memory Limits 
> 
> max number of segments = 16128
> 
> max seg size (kbytes) = 18014398509481983
> 
> max total shared memory (kbytes) = 18014398509480960
> 
> min seg size (bytes) = 1
> 
> -- Semaphore Limits 
> 
> max number of arrays = 32000
> 
> max semaphores per array = 32000
> 
> max semaphores system wide = 102400
> 
> max ops per semop call = 500
> 
> semaphore max value = 32767
> 
> 
>> And what is the size of you TSM DB and on what type of storage and
>> filesystem is it located?
> 
> tsm: TSM01>q db
> 
> Database Name   Total Pages Usable Pages   Used Pages   Free 
> Pages
> --        
> 
> TSMDB1  102,416   99,312   43,100   
> 56,212
> 
> tsm: TSM01>q db f=d
> 
>     Database Name: TSMDB1
>   Total Space of File System (MB): 197,939
>     Space Used on File System(MB): 11,444
>    Space Used by Database(MB): 1,280
>     Free Space Available (MB): 186,495
>   Total Pages: 102,416
>  Usable Pages: 99,312
>    Used Pages: 43,100
>    Free Pages: 56,212
>     Buffer Pool Hit Ratio: 99.9
>     Total Buffer Requests: 4,971,182
>    Sort Overflows: 0
>   Package Cache Hit Ratio: 87.1
>  Last Database Reorganization: 12/04/18   16:44:07
>    Full Device Class Name: 3310L1DEV
> Number of Database Backup Streams: 1
>  Incrementals Since Last Full: 0
>    Last Complete Backup Date/Time: 12/05/18   15:06:39
>     Compress Database Backups: No
>     Protect Master Encryption Key: No
> 
> Filesystem   Size  Used Avail Use% Mounted on
> 
> /dev/mapper/SLESVG-lvtsmdb01  49G  373M   46G   1% /tsminst1/db01
> 
> /dev/mapper/SLESVG-lvtsmdb02  49G  373M   46G   1% /tsminst1/db02
> 
> /dev/mapper/SLESVG-lvtsmdb03  49G  373M   46G   1% /tsminst1/db03
> 
> /dev/mapper/SLESVG-lvtsmdb04  49G  373M   46G   1% /tsminst1/db04
> 
> 
> 
> 
> Am 05.12.18 um 16:45 schrieb Sasa Drnjevic:
>> On 2018-12-05 15:06, Schneck Dennis wrote:
>>> Hello Sasa
>>>
>>>> Check for errors on FC switch ports...
>>> Ok will inform the colleage in the branch
>>>
>>>
>>>> When have the problems started?
>>> Its a new server - did not work before
>>
>> Hi Dennis,
>>
>> did you tune kernel parameters for your Linux TSM server?
>>
>> https://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.8/srv.install/t_srv_krnlparms_lnx-linux.html
>>
>>
>> And what is the size of you TSM DB and on what type of storage and
>> filesystem is it located?
>>
>> --
>> Sasa Drnjevic
>> www.srce.unizg.hr
>>
>>
>>
>>> Am 05.12.18 um 14:59 schrieb Sasa Drnjevic:
>>>> Check for errors on FC switch ports...
>>>>
>>>> When have the problems started?
>>>>
>>>> --
>>>> Sasa Drnjevic
>>>> www.srce.unizg.hr
>>>>
>>>>
>>>>
>>>>
>>>&g

Re: VERY SLOW DB BACKUP TO TAPE | TS3310 with LTO5 | LIN_TAPE 3.033 | TSM SERVER 7.1.9.000

[Schneck Dennis]

Hello Sasa,

>did you tune kernel parameters for your Linux TSM server?

yes, here is my /etc/sysctl.conf

kernel.randomize_va_space=0
vm.swappiness=0
vm.overcommit_memory=0


tsm01:~ # ipcs -l

-- Messages Limits 

max queues system wide = 64512

max size of message (bytes) = 65536

default max size of queue (bytes) = 65536

-- Shared Memory Limits 

max number of segments = 16128

max seg size (kbytes) = 18014398509481983

max total shared memory (kbytes) = 18014398509480960

min seg size (bytes) = 1

-- Semaphore Limits 

max number of arrays = 32000

max semaphores per array = 32000

max semaphores system wide = 102400

max ops per semop call = 500

semaphore max value = 32767


>And what is the size of you TSM DB and on what type of storage and
>filesystem is it located?

tsm: TSM01>q db

Database Name   Total Pages Usable Pages   Used Pages   Free 
Pages
--    

TSMDB1  102,416   99,312   43,100   
56,212

tsm: TSM01>q db f=d

    Database Name: TSMDB1
  Total Space of File System (MB): 197,939
    Space Used on File System(MB): 11,444
   Space Used by Database(MB): 1,280
    Free Space Available (MB): 186,495
  Total Pages: 102,416
 Usable Pages: 99,312
   Used Pages: 43,100
   Free Pages: 56,212
    Buffer Pool Hit Ratio: 99.9
    Total Buffer Requests: 4,971,182
   Sort Overflows: 0
  Package Cache Hit Ratio: 87.1
 Last Database Reorganization: 12/04/18   16:44:07
   Full Device Class Name: 3310L1DEV
Number of Database Backup Streams: 1
 Incrementals Since Last Full: 0
   Last Complete Backup Date/Time: 12/05/18   15:06:39
    Compress Database Backups: No
    Protect Master Encryption Key: No

Filesystem   Size  Used Avail Use% Mounted on

/dev/mapper/SLESVG-lvtsmdb01  49G  373M   46G   1% /tsminst1/db01

/dev/mapper/SLESVG-lvtsmdb02  49G  373M   46G   1% /tsminst1/db02

/dev/mapper/SLESVG-lvtsmdb03  49G  373M   46G   1% /tsminst1/db03

/dev/mapper/SLESVG-lvtsmdb04  49G  373M   46G   1% /tsminst1/db04




Am 05.12.18 um 16:45 schrieb Sasa Drnjevic:
> On 2018-12-05 15:06, Schneck Dennis wrote:
>> Hello Sasa
>>
>>> Check for errors on FC switch ports...
>> Ok will inform the colleage in the branch
>>
>>
>>> When have the problems started?
>> Its a new server - did not work before
>
> Hi Dennis,
>
> did you tune kernel parameters for your Linux TSM server?
>
> https://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.8/srv.install/t_srv_krnlparms_lnx-linux.html
>
>
> And what is the size of you TSM DB and on what type of storage and
> filesystem is it located?
>
> --
> Sasa Drnjevic
> www.srce.unizg.hr
>
>
>
>> Am 05.12.18 um 14:59 schrieb Sasa Drnjevic:
>>> Check for errors on FC switch ports...
>>>
>>> When have the problems started?
>>>
>>> --
>>> Sasa Drnjevic
>>> www.srce.unizg.hr
>>>
>>>
>>>
>>>
>>> On 5.12.2018. 14:15, Schneck Dennis wrote:
>>>> Hello,
>>>>
>>>> starting with new TSM-Server 7.1.9.000 on a SLES12SP3 x86_64 with LIN_TAPE 
>>>> 3.033
>>>>
>>>> TAPE.Lib is a TS3310 with LTO5 Drives.
>>>>
>>>> Server has: Emulex AJ762B/AH402A FC Adapter an the run both with: 4 Gbit  
>>>> (lspci | grep -i fibre)
>>>>
>>>>
>>>> tsm: TSM01>*SET DBRECOVERY 3310L1DEV PROTECTKeys=no*
>>>> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys 
>>>> to be backed up manually.
>>>>
>>>> Do you wish to proceed? (Yes (Y)/No (N)) *y*
>>>> ANR2782I SET DBRECOVERY completed successfully and device class for 
>>>> automatic DB backup is set to 3310L1DEV.
>>>>
>>>> tsm: TSM01>*backup db devclass=3310L1DEV type=full PROTECTKeys=no*
>>>> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys 
>>>> to be backed up manually.
>>>>
>>>> Do you wish to proceed? (Yes (Y)/No (N))*y*
>>>> ANR2280I Full database backup started as process 3.
>>>> ANS8003I Process number 3 started.
>>>>
>>>> tsm: TSM01>*q act*
>>>>
>>>> 12/05/18   15:07:08  ANR8337I LTO volume 000260L5 mounted in drive 
>>>> 3310L1D2    
>>>>   (/dev/IBMtape2). (SESSION: 1, PROCESS: 3)
>>

Re: VERY SLOW DB BACKUP TO TAPE | TS3310 with LTO5 | LIN_TAPE 3.033 | TSM SERVER 7.1.9.000

[Sasa Drnjevic]

On 2018-12-05 15:06, Schneck Dennis wrote:
> Hello Sasa
> 
>> Check for errors on FC switch ports...
> 
> Ok will inform the colleage in the branch
> 
> 
>> When have the problems started?
> 
> Its a new server - did not work before


Hi Dennis,

did you tune kernel parameters for your Linux TSM server?

https://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.8/srv.install/t_srv_krnlparms_lnx-linux.html


And what is the size of you TSM DB and on what type of storage and
filesystem is it located?

--
Sasa Drnjevic
www.srce.unizg.hr



> 
> Am 05.12.18 um 14:59 schrieb Sasa Drnjevic:
>> Check for errors on FC switch ports...
>>
>> When have the problems started?
>>
>> --
>> Sasa Drnjevic
>> www.srce.unizg.hr
>>
>>
>>
>>
>> On 5.12.2018. 14:15, Schneck Dennis wrote:
>>> Hello,
>>>
>>> starting with new TSM-Server 7.1.9.000 on a SLES12SP3 x86_64 with LIN_TAPE 
>>> 3.033
>>>
>>> TAPE.Lib is a TS3310 with LTO5 Drives.
>>>
>>> Server has: Emulex AJ762B/AH402A FC Adapter an the run both with: 4 Gbit  
>>> (lspci | grep -i fibre)
>>>
>>>
>>> tsm: TSM01>*SET DBRECOVERY 3310L1DEV PROTECTKeys=no*
>>> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to 
>>> be backed up manually.
>>>
>>> Do you wish to proceed? (Yes (Y)/No (N)) *y*
>>> ANR2782I SET DBRECOVERY completed successfully and device class for 
>>> automatic DB backup is set to 3310L1DEV.
>>>
>>> tsm: TSM01>*backup db devclass=3310L1DEV type=full PROTECTKeys=no*
>>> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to 
>>> be backed up manually.
>>>
>>> Do you wish to proceed? (Yes (Y)/No (N))*y*
>>> ANR2280I Full database backup started as process 3.
>>> ANS8003I Process number 3 started.
>>>
>>> tsm: TSM01>*q act*
>>>
>>> 12/05/18   15:07:08  ANR8337I LTO volume 000260L5 mounted in drive 
>>> 3310L1D2    
>>>   (/dev/IBMtape2). (SESSION: 1, PROCESS: 3) 
>>>    
>>> 12/05/18   15:07:08  ANR0513I Process 3 opened output volume 000260L5.  
>>>    
>>>
>>> after ~30 minutes only 34 MB
>>>
>>> tsm: TSM01>*q pro*
>>>
>>> Process  Process Description  Process Status
>>>    
>>>   Number  
>>>   
>>> -
>>>    3 Database Backup  TYPE=FULL in progress. Bytes backed 
>>> up: 34,048   
>>>    KB. Current output volume(s): 
>>> 000260L5. 
>>>
>>> **If I look with nmon, see only one CPU is 100% in use all other 0%
>>>
>>> *How to find the bottleneck ? *
>>>

Re: VERY SLOW DB BACKUP TO TAPE | TS3310 with LTO5 | LIN_TAPE 3.033 | TSM SERVER 7.1.9.000

[Schneck Dennis]

Hello Sasa

>Check for errors on FC switch ports...

Ok will inform the colleage in the branch


>When have the problems started?

Its a new server - did not work before




Am 05.12.18 um 14:59 schrieb Sasa Drnjevic:
> Check for errors on FC switch ports...
>
> When have the problems started?
>
> --
> Sasa Drnjevic
> www.srce.unizg.hr
>
>
>
>
> On 5.12.2018. 14:15, Schneck Dennis wrote:
>> Hello,
>>
>> starting with new TSM-Server 7.1.9.000 on a SLES12SP3 x86_64 with LIN_TAPE 
>> 3.033
>>
>> TAPE.Lib is a TS3310 with LTO5 Drives.
>>
>> Server has: Emulex AJ762B/AH402A FC Adapter an the run both with: 4 Gbit  
>> (lspci | grep -i fibre)
>>
>>
>> tsm: TSM01>*SET DBRECOVERY 3310L1DEV PROTECTKeys=no*
>> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to 
>> be backed up manually.
>>
>> Do you wish to proceed? (Yes (Y)/No (N)) *y*
>> ANR2782I SET DBRECOVERY completed successfully and device class for 
>> automatic DB backup is set to 3310L1DEV.
>>
>> tsm: TSM01>*backup db devclass=3310L1DEV type=full PROTECTKeys=no*
>> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to 
>> be backed up manually.
>>
>> Do you wish to proceed? (Yes (Y)/No (N))*y*
>> ANR2280I Full database backup started as process 3.
>> ANS8003I Process number 3 started.
>>
>> tsm: TSM01>*q act*
>>
>> 12/05/18   15:07:08  ANR8337I LTO volume 000260L5 mounted in drive 
>> 3310L1D2    
>>   (/dev/IBMtape2). (SESSION: 1, PROCESS: 3)  
>>   
>> 12/05/18   15:07:08  ANR0513I Process 3 opened output volume 000260L5.   
>>   
>>
>> after ~30 minutes only 34 MB
>>
>> tsm: TSM01>*q pro*
>>
>> Process  Process Description  Process Status 
>>   
>>   Number  
>>   
>> -
>>    3 Database Backup  TYPE=FULL in progress. Bytes backed 
>> up: 34,048   
>>    KB. Current output volume(s): 
>> 000260L5. 
>>
>> **If I look with nmon, see only one CPU is 100% in use all other 0%
>>
>> *How to find the bottleneck ? *
>>
-- 

Mit freundlichen Grüßen / Best Regards / 诚挚的问候

i.A. Dennis Schneck

 

Schuler Pressen GmbH | Schuler-Platz 1 | 73033 Göppingen | Germany
Tel. +49 7161 66-2933 | Fax +49 7161 66-8440
dennis.schn...@schulergroup.com | http://www.schulergroup.com

Member of the ANDRITZ Group

E-Mail-Pflichtangaben (gem. EHUG): http://pflichtangaben.schulergroup.com

Re: VERY SLOW DB BACKUP TO TAPE | TS3310 with LTO5 | LIN_TAPE 3.033 | TSM SERVER 7.1.9.000

[Sasa Drnjevic]

Check for errors on FC switch ports...

When have the problems started?

--
Sasa Drnjevic
www.srce.unizg.hr




On 5.12.2018. 14:15, Schneck Dennis wrote:
> Hello,
> 
> starting with new TSM-Server 7.1.9.000 on a SLES12SP3 x86_64 with LIN_TAPE 
> 3.033
> 
> TAPE.Lib is a TS3310 with LTO5 Drives.
> 
> Server has: Emulex AJ762B/AH402A FC Adapter an the run both with: 4 Gbit  
> (lspci | grep -i fibre)
> 
> 
> tsm: TSM01>*SET DBRECOVERY 3310L1DEV PROTECTKeys=no*
> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to 
> be backed up manually.
> 
> Do you wish to proceed? (Yes (Y)/No (N)) *y*
> ANR2782I SET DBRECOVERY completed successfully and device class for automatic 
> DB backup is set to 3310L1DEV.
> 
> tsm: TSM01>*backup db devclass=3310L1DEV type=full PROTECTKeys=no*
> ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to 
> be backed up manually.
> 
> Do you wish to proceed? (Yes (Y)/No (N))*y*
> ANR2280I Full database backup started as process 3.
> ANS8003I Process number 3 started.
> 
> tsm: TSM01>*q act*
> 
> 12/05/18   15:07:08  ANR8337I LTO volume 000260L5 mounted in drive 
> 3310L1D2    
>   (/dev/IBMtape2). (SESSION: 1, PROCESS: 3)   
>  
> 12/05/18   15:07:08  ANR0513I Process 3 opened output volume 000260L5.
>  
> 
> after ~30 minutes only 34 MB
> 
> tsm: TSM01>*q pro*
> 
> Process  Process Description  Process Status  
>  
>   Number  
>   
> -
>    3 Database Backup  TYPE=FULL in progress. Bytes backed up: 
> 34,048   
>    KB. Current output volume(s): 
> 000260L5. 
> 
> **If I look with nmon, see only one CPU is 100% in use all other 0%
> 
> *How to find the bottleneck ? *
> 

VERY SLOW DB BACKUP TO TAPE | TS3310 with LTO5 | LIN_TAPE 3.033 | TSM SERVER 7.1.9.000

[Schneck Dennis]

Hello,

starting with new TSM-Server 7.1.9.000 on a SLES12SP3 x86_64 with LIN_TAPE 3.033

TAPE.Lib is a TS3310 with LTO5 Drives.

Server has: Emulex AJ762B/AH402A FC Adapter an the run both with: 4 Gbit  
(lspci | grep -i fibre)


tsm: TSM01>*SET DBRECOVERY 3310L1DEV PROTECTKeys=no*
ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to be 
backed up manually.

Do you wish to proceed? (Yes (Y)/No (N)) *y*
ANR2782I SET DBRECOVERY completed successfully and device class for automatic 
DB backup is set to 3310L1DEV.

tsm: TSM01>*backup db devclass=3310L1DEV type=full PROTECTKeys=no*
ANR2784W Specifying PROTECTKEYS=NO requires the server's encryption keys to be 
backed up manually.

Do you wish to proceed? (Yes (Y)/No (N))*y*
ANR2280I Full database backup started as process 3.
ANS8003I Process number 3 started.

tsm: TSM01>*q act*

12/05/18   15:07:08  ANR8337I LTO volume 000260L5 mounted in drive 3310L1D2 
   
  (/dev/IBMtape2). (SESSION: 1, PROCESS: 3) 
   
12/05/18   15:07:08  ANR0513I Process 3 opened output volume 000260L5.  
   

after ~30 minutes only 34 MB

tsm: TSM01>*q pro*

Process  Process Description  Process Status
   
  Number  
  
-
   3 Database Backup  TYPE=FULL in progress. Bytes backed up: 
34,048   
   KB. Current output volume(s): 000260L5.  
   

**If I look with nmon, see only one CPU is 100% in use all other 0%

*How to find the bottleneck ? *

Re: TSM server upgrade from 7.1.7.000 to 8.1

[Sasa Drnjevic]

On 2018-11-01 18:19, Michaud, Luc [Analyste principal - environnement
AIX] wrote:
> We want to upgrade our AIX-based TSM servers from 7.1.7.000 to
> 8.1.x.yyy
> 
> The official IBM server upgrade process does not worry us by itself.
> 
> However, on this list, over the last year, I did notice issues with
> SESSIONSECURITY, Web Client, as well as some regressions.
> 
> So, here are my questions :
> 
> 1.  Will our TSM 7.1 clients work correctly with 8.1 servers ?

I can help you with the question no.1 and confirm that after we upgraded
our Linux TSM servers from 7.1.7.300 to 8.1.1.000 all our 7.1 and even
6.3 clients work fine. We have about 300 clients (about 250 v7, 35 v8
and the rest at v6.3).

We plan to upgrade all clients to v8.1.2+ soon and then the servers to
the latest v8.1.x version available.

> 2.  Do we need to first upgrade our "Library Manager Server" to
> 8.1 ?  Any caveat with SESSIONSECURITY ?
> 
> 3.  Is 8.1.5.100 a good server level to look at ?
> 
> 4.  Any caveat for TDP clients ?  (such as SQLS, TDP for Oracle
> and TDP for Exchange) ? 


Our six TDP for MSSQL clients work fine in the above scenario, but the
MS Exch v8.1.4.0 suddenly stopped working two weeks ago, after
successful 8-9 months at that level... We then tried the latest MS Exch
v8.1.6.0, but the errors remained the same as v8.1.4.0. and it still
fails every night...

The case for MS Exch is opened with IBM support, but no solution yet...

Regards,

--
Sasa Drnjevic
www.srce.unizg.hr/en/




>Your advice is welcome. LUC MICHAUD, B.Ing 
> Analyste Principal D.E. Technologies de l'information et innovation 
> Sociéte de Transport de Montréal (514) 973-0726 
> luc.micha...@stm.info<mailto:luc.micha...@stm.info>
> 

TSM server upgrade from 7.1.7.000 to 8.1

[Michaud, Luc [Analyste principal - environnement AIX]]

We want to upgrade our AIX-based TSM servers from 7.1.7.000 to 8.1.x.yyy

The official IBM server upgrade process does not worry us by itself.

However, on this list, over the last year, I did notice issues with 
SESSIONSECURITY, Web Client, as well as some regressions.

So, here are my questions :

1.  Will our TSM 7.1 clients work correctly with 8.1 servers ?

2.  Do we need to first upgrade our "Library Manager Server" to 8.1 ?  Any 
caveat with SESSIONSECURITY ?

3.  Is 8.1.5.100 a good server level to look at ?

4.  Any caveat for TDP clients ?  (such as SQLS, TDP for Oracle and TDP for 
Exchange) ?
Your advice is welcome.
LUC MICHAUD, B.Ing
Analyste Principal
D.E. Technologies de l'information et innovation
Sociéte de Transport de Montréal
(514) 973-0726
luc.micha...@stm.info<mailto:luc.micha...@stm.info>

Re: Removing TSM for Databases

[Andrew Raibeck]

Hello Thomas,

This technote includes instructions for removal of the software:

https://www.ibm.com/support/docview.wss?uid=swg21644479

See the "Removal / Uninstall of Data Protection for Oracle" section in that
document.

Regards,

Andy



Andrew Raibeck | IBM Spectrum Protect Level 3 | stor...@us.ibm.com

"ADSM: Dist Stor Manager"  wrote on 2018-10-31
13:21:43:

> From: Thomas Denier 
> To: ADSM-L@VM.MARIST.EDU
> Date: 2018-10-31 13:22
> Subject: Removing TSM for Databases
> Sent by: "ADSM: Dist Stor Manager" 
>
> We have a 64 bit x86 Linux server on which TSM for Databases (for
> Oracle) 7.1 was installed but never used. We would like to remove
> this TSM component. I have so far been completely unsuccessful in
> finding instructions for doing this. Where is the uninstall
> procedure documented?
>
> Thomas Denier,
> Thomas Jefferson University
>
>
>
> The information contained in this transmission contains privileged
> and confidential information. It is intended only for the use of the
> person named above. If you are not the intended recipient, you are
> hereby notified that any review, dissemination, distribution or
> duplication of this communication is strictly prohibited. If you are
> not the intended recipient, please contact the sender by reply email
> and destroy all copies of the original message.
>
> CAUTION: Intended recipients should NOT use email communication for
> emergent or urgent health care matters.
>

Removing TSM for Databases

[Thomas Denier]

We have a 64 bit x86 Linux server on which TSM for Databases (for Oracle) 7.1 
was installed but never used. We would like to remove this TSM component. I 
have so far been completely unsuccessful in finding instructions for doing 
this. Where is the uninstall procedure documented?

Thomas Denier,
Thomas Jefferson University



The information contained in this transmission contains privileged and 
confidential information. It is intended only for the use of the person named 
above. If you are not the intended recipient, you are hereby notified that any 
review, dissemination, distribution or duplication of this communication is 
strictly prohibited. If you are not the intended recipient, please contact the 
sender by reply email and destroy all copies of the original message.

CAUTION: Intended recipients should NOT use email communication for emergent or 
urgent health care matters.

Re: TS4300 / EMC ML3 usage with TSM 7.1.7 ?

[Erwann SIMON]

Hi Luc,

According to this web page below, Dell EMC ML3 requires Spectrum Protect 8.1.2 
(and IBM Device Driver, aka Atape.driver for the library itself). It's the same 
requirement for IBM TS4300.
https://www-01.ibm.com/support/docview.wss?uid=ibm10716993#LTO%20Tape%20Libraries

To be honest, I've configured once a TS4300 whith TSM 7.1.7 on Windows 2012R2 
and it worked. The gap between the guy who sells and the other one who does the 
job... I don't know with AIX, but it should be the same.

PS : Please say hello to David G. and Olivier S. for me.

-- 
Best regards / Cordialement / مع تحياتي
Erwann SIMON

- Mail original -
De: "Luc Michaud [Analyste principal - environnement AIX]" 

À: ADSM-L@VM.MARIST.EDU
Envoyé: Lundi 29 Octobre 2018 14:14:29
Objet: [ADSM-L] TS4300 / EMC ML3 usage with TSM 7.1.7 ?

We looking into replacing our old IBM TS3310 FC LTO4 library with a DELL/EMC 
ML3 (OEM of IBM TS4300) with FC LTO-7 drives.

We are at TSM 7.1.7 on AIX 7.1.

Do we need to upgrade TSM to at least v8.1.2 or are we ok with TSM 7.1.7 ?

Luc

TS4300 / EMC ML3 usage with TSM 7.1.7 ?

[Michaud, Luc [Analyste principal - environnement AIX]]

We looking into replacing our old IBM TS3310 FC LTO4 library with a DELL/EMC 
ML3 (OEM of IBM TS4300) with FC LTO-7 drives.

We are at TSM 7.1.7 on AIX 7.1.

Do we need to upgrade TSM to at least v8.1.2 or are we ok with TSM 7.1.7 ?

Luc

R: [ADSM-L] R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Tommaso Bollini]

The BA Spectrum clients get the list of files to backup from the NetApp; you 
can read it from the IBM page:


The second snapshot is called the diffsnapshot, or differences snapshot. The 
client then incrementally backs up the files that are reported as changed, by 
NetApp, to the IBM Spectrum Protect server. 

You can find the same concept also at 
https://www.lascon.co.uk/tsm-backups.php#bsnetapp, where you'll find:


The second snapshot is called the diffsnapshot. TSM then incrementally backs up 
the files reported as changed by NetApp to the TSM server. 

I am pretty sure it's not necessary to create exclusion rule of .snapshot (NFS) 
or ~ snapshot (CIFS) it is the "entry point" from which the BA Spectrum agent 
get the access of the files who needs backups.
I say that because web documentation never require the adoption of this rule. 
These are NetApp construct that is automatically recognized by the BA Spectrum 
agents.

Cheers.


-Messaggio originale-
Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di Efim
Inviato: giovedì 20 settembre 2018 10:52
A: ADSM-L@VM.MARIST.EDU
Oggetto: Re: [ADSM-L] R: [ADSM-L] Netapp snapdiff issues having TSM client on a 
Linux machine, making use of NFS mounts

SP client use NetApp API to detect file changes between 2(two) snapshots.
After that client have the list of changed files and backup it from mounted 
volume.
I believe that it is unnecessary to copy the snapshot itself because it's 
pointless.
Efim


> 20 сент. 2018 г., в 11:29, Tommaso Bollini  написал(а):
> 
> @Efim
> the .snapshot (or ~snapshot) folders are NetApp construct recognized by the 
> BA Spectrum agent api:
> https://www.ibm.com/support/knowledgecenter/en/SSEQVQ_8.1.2/client/r_o
> pt_snapdiff.html The BA Spectrum agent will copy all files listed in 
> the NetApp snapshot image.
> The log error simply reports the point at which this backup job has in 
> exception echoing the full path of what BA API read from the snapshot image.
> 
> I hope I was helpful.
> Greetings.
> 
> -Messaggio originale-
> Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di 
> Efim
> Inviato: mercoledì 19 settembre 2018 18:39
> A: ADSM-L@VM.MARIST.EDU
> Oggetto: Re: [ADSM-L] Netapp snapdiff issues having TSM client on a 
> Linux machine, making use of NFS mounts
> 
> Arnaud,
> 
> Can you explain why you are copying the .snapshot folder during incremental 
> filesystem backup with snapdiff?
> It contains snapshots created by netapp and SP for this FS. 
> I believe that it should be excluded from copying in your backup job.
> 
> Efim
> 
> 
>> 19 сент. 2018 г., в 18:28, PAC Brion Arnaud  
>> написал(а):
>> 
>> Hi Robert,
>> 
>> Thanks a lot for appreciated feedback, unfortunately even when using TSM own 
>> snapshots the problem still exists :
>> 
>> root@xx:~ # dsmc i /Airwarder -snapdiff -diffsnapshot=create 
>> -snapdiffhttps -optfile=$OPTPATH >/tmp/snapshot_Airwarder.log
>> 
>> root@ xx:~ # more /tmp/snapshot_Airwarder.log IBM Spectrum 
>> Protect Command Line Backup-Archive Client Interface  Client Version 
>> 8, Release 1, Level 4.0  Client date/time: 09/19/2018 17:17:41
>> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights 
>> Reserved. 
>> 
>> Node Name: NETAPP_CH1_NFS
>> Session established with server xx: Linux/ppc64le  Server Version 
>> 8, Release 1, Level 4.000  Server date/time: 09/19/2018 17:17:42  
>> Last
>> access: 09/18/2018 12:12:17
>> 
>> 
>> Incremental by snapshot difference of volume '/Airwarder'
>> 
>> Connected to NetApp Storage Virtual Machine
>>   Management Filer Host/IP Address : xx
>>   Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
>> 13:07:06
>> UTC 2018
>>   Storage VM Name  : ch1ns100
>>   Storage VM Host/IP Address   : xx
>>   Storage VM Volume Style  : Flex
>>   Login User   : tsm-backup-user
>>   Transport: HTTPS
>> 
>> Performing a Full Incremental Backup of volume '/Airwarder'
>> Creating Base Snapshot.
>> Using Base Snapshot 'TSM_CH135BA2689781857_AIRWARDER' with timestamp
>> 09/19/2018
>> 17:17:43
>> ANS4007E Error processing
>> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
>> acrimport_ref/ch': access to the object is denied ANS4007E Error 
>> processing '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
>> acrimport_test/ch': access to the object is denied
>> ANS1898I * Processed   500 files *
>> ANS4007E Err

Re: R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Efim]

SP client use NetApp API to detect file changes between 2(two) snapshots.
After that client have the list of changed files and backup it from mounted 
volume.
I believe that it is unnecessary to copy the snapshot itself because it's 
pointless.
Efim


> 20 сент. 2018 г., в 11:29, Tommaso Bollini  написал(а):
> 
> @Efim
> the .snapshot (or ~snapshot) folders are NetApp construct recognized by the 
> BA Spectrum agent api:
> https://www.ibm.com/support/knowledgecenter/en/SSEQVQ_8.1.2/client/r_opt_snapdiff.html
> The BA Spectrum agent will copy all files listed in the NetApp snapshot image.
> The log error simply reports the point at which this backup job has in 
> exception echoing the full path of what BA API read from the snapshot image.
> 
> I hope I was helpful.
> Greetings.
> 
> -Messaggio originale-
> Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di Efim
> Inviato: mercoledì 19 settembre 2018 18:39
> A: ADSM-L@VM.MARIST.EDU
> Oggetto: Re: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of NFS mounts
> 
> Arnaud,
> 
> Can you explain why you are copying the .snapshot folder during incremental 
> filesystem backup with snapdiff?
> It contains snapshots created by netapp and SP for this FS. 
> I believe that it should be excluded from copying in your backup job.
> 
> Efim
> 
> 
>> 19 сент. 2018 г., в 18:28, PAC Brion Arnaud  
>> написал(а):
>> 
>> Hi Robert,
>> 
>> Thanks a lot for appreciated feedback, unfortunately even when using TSM own 
>> snapshots the problem still exists :
>> 
>> root@xx:~ # dsmc i /Airwarder -snapdiff -diffsnapshot=create 
>> -snapdiffhttps -optfile=$OPTPATH >/tmp/snapshot_Airwarder.log
>> 
>> root@ xx:~ # more /tmp/snapshot_Airwarder.log IBM Spectrum Protect 
>> Command Line Backup-Archive Client Interface  Client Version 8, 
>> Release 1, Level 4.0  Client date/time: 09/19/2018 17:17:41
>> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights 
>> Reserved. 
>> 
>> Node Name: NETAPP_CH1_NFS
>> Session established with server xx: Linux/ppc64le  Server Version 
>> 8, Release 1, Level 4.000  Server date/time: 09/19/2018 17:17:42  Last 
>> access: 09/18/2018 12:12:17
>> 
>> 
>> Incremental by snapshot difference of volume '/Airwarder'
>> 
>> Connected to NetApp Storage Virtual Machine
>>   Management Filer Host/IP Address : xx
>>   Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
>> 13:07:06
>> UTC 2018
>>   Storage VM Name  : ch1ns100
>>   Storage VM Host/IP Address   : xx
>>   Storage VM Volume Style  : Flex
>>   Login User   : tsm-backup-user
>>   Transport: HTTPS
>> 
>> Performing a Full Incremental Backup of volume '/Airwarder'
>> Creating Base Snapshot.
>> Using Base Snapshot 'TSM_CH135BA2689781857_AIRWARDER' with timestamp 
>> 09/19/2018
>> 17:17:43
>> ANS4007E Error processing 
>> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
>> acrimport_ref/ch': access to the object is denied ANS4007E Error 
>> processing '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
>> acrimport_test/ch': access to the object is denied
>> ANS1898I * Processed   500 files *
>> ANS4007E Error processing 
>> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
>> 
>> I'm lost here ...
>> 
>> Cheers.
>> 
>> Arnaud
>> 
>> **
>> **
>> Backup and Recovery Systems Administrator Panalpina Management Ltd., 
>> Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 
>> Basel/CH
>> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
>> Direct: +41 (61) 226 19 78
>> e-mail: arnaud.br...@panalpina.com
>> This electronic message transmission contains information from Panalpina and 
>> is confidential or privileged. 
>> This information is intended only for the person (s) named above. If 
>> you are not the intended recipient, any disclosure, copying, distribution or 
>> use or any other action based on the contents of this information is 
>> strictly prohibited.
>> 
>> If you receive this electronic transmission in error, please notify the 
>> sender by e-mail, telephone or fax at the numbers listed above. Thank you.
>> **

R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Tommaso Bollini]

@Efim
the .snapshot (or ~snapshot) folders are NetApp construct recognized by the BA 
Spectrum agent api:
https://www.ibm.com/support/knowledgecenter/en/SSEQVQ_8.1.2/client/r_opt_snapdiff.html
The BA Spectrum agent will copy all files listed in the NetApp snapshot image.
The log error simply reports the point at which this backup job has in 
exception echoing the full path of what BA API read from the snapshot image.

I hope I was helpful.
Greetings.

-Messaggio originale-
Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di Efim
Inviato: mercoledì 19 settembre 2018 18:39
A: ADSM-L@VM.MARIST.EDU
Oggetto: Re: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux 
machine, making use of NFS mounts

Arnaud,

Can you explain why you are copying the .snapshot folder during incremental 
filesystem backup with snapdiff?
It contains snapshots created by netapp and SP for this FS. 
I believe that it should be excluded from copying in your backup job.

Efim


> 19 сент. 2018 г., в 18:28, PAC Brion Arnaud  
> написал(а):
> 
> Hi Robert,
> 
> Thanks a lot for appreciated feedback, unfortunately even when using TSM own 
> snapshots the problem still exists :
> 
> root@xx:~ # dsmc i /Airwarder -snapdiff -diffsnapshot=create 
> -snapdiffhttps -optfile=$OPTPATH >/tmp/snapshot_Airwarder.log
> 
> root@ xx:~ # more /tmp/snapshot_Airwarder.log IBM Spectrum Protect 
> Command Line Backup-Archive Client Interface  Client Version 8, 
> Release 1, Level 4.0  Client date/time: 09/19/2018 17:17:41
> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights 
> Reserved. 
> 
> Node Name: NETAPP_CH1_NFS
> Session established with server xx: Linux/ppc64le  Server Version 
> 8, Release 1, Level 4.000  Server date/time: 09/19/2018 17:17:42  Last 
> access: 09/18/2018 12:12:17
> 
> 
> Incremental by snapshot difference of volume '/Airwarder'
> 
> Connected to NetApp Storage Virtual Machine
>Management Filer Host/IP Address : xx
>Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
> 13:07:06
> UTC 2018
>Storage VM Name  : ch1ns100
>Storage VM Host/IP Address   : xx
>Storage VM Volume Style  : Flex
>Login User   : tsm-backup-user
>Transport: HTTPS
> 
> Performing a Full Incremental Backup of volume '/Airwarder'
> Creating Base Snapshot.
> Using Base Snapshot 'TSM_CH135BA2689781857_AIRWARDER' with timestamp 
> 09/19/2018
> 17:17:43
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> acrimport_ref/ch': access to the object is denied ANS4007E Error 
> processing '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> acrimport_test/ch': access to the object is denied
> ANS1898I * Processed   500 files *
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> 
> I'm lost here ...
> 
> Cheers.
> 
> Arnaud
> 
> **
> **
> Backup and Recovery Systems Administrator Panalpina Management Ltd., 
> Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 
> Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from Panalpina and 
> is confidential or privileged. 
> This information is intended only for the person (s) named above. If 
> you are not the intended recipient, any disclosure, copying, distribution or 
> use or any other action based on the contents of this information is strictly 
> prohibited.
> 
> If you receive this electronic transmission in error, please notify the 
> sender by e-mail, telephone or fax at the numbers listed above. Thank you.
> **
> ******
> www.panalpina.com
> 
> 
> 
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf 
> Of Robert Talda
> Sent: Tuesday, September 18, 2018 10:48 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of NFS mounts
> 
> Brion:
>  Here at Cornell, we have a TSM client on a Linux system backing up NFS 
> shares via NFS mounts and snapdiff differentials - and have been doing so for 
> years.
> 
>  First, the gory details:
> - For TSM
> TSM Client Version 7, Release 1, Level 6.0 (*red faced, thoug

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[PAC Brion Arnaud]

Hi Efim and All who helped 

Quick update on this : it appeared that some export policy rule (i.e." 
-superuser sys" ) had not been properly set on our Netapp, thus refraining TSM 
client to reach some files. Once this corrected, I was able to get successful 
snapshot backups done.
 
@Efim : thanks for your advice, I'll take it in consideration ! Just have to 
discuss with our storage team what exactly they are expecting from me ;-)

Cheers.

Arnaud

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Efim
Sent: Wednesday, September 19, 2018 6:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Netapp snapdiff issues having TSM client on a Linux machine, 
making use of NFS mounts

Arnaud,

Can you explain why you are copying the .snapshot folder during incremental 
filesystem backup with snapdiff?
It contains snapshots created by netapp and SP for this FS. 
I believe that it should be excluded from copying in your backup job.

Efim


> 19 сент. 2018 г., в 18:28, PAC Brion Arnaud  
> написал(а):
> 
> Hi Robert,
> 
> Thanks a lot for appreciated feedback, unfortunately even when using TSM own 
> snapshots the problem still exists :
> 
> root@xx:~ # dsmc i /Airwarder -snapdiff -diffsnapshot=create 
> -snapdiffhttps -optfile=$OPTPATH >/tmp/snapshot_Airwarder.log
> 
> root@ xx:~ # more /tmp/snapshot_Airwarder.log
> IBM Spectrum Protect
> Command Line Backup-Archive Client Interface
>  Client Version 8, Release 1, Level 4.0 
>  Client date/time: 09/19/2018 17:17:41
> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights 
> Reserved. 
> 
> Node Name: NETAPP_CH1_NFS
> Session established with server xx: Linux/ppc64le
>  Server Version 8, Release 1, Level 4.000
>  Server date/time: 09/19/2018 17:17:42  Last access: 09/18/2018 12:12:17
> 
> 
> Incremental by snapshot difference of volume '/Airwarder'
> 
> Connected to NetApp Storage Virtual Machine
>Management Filer Host/IP Address : xx
>Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
> 13:07:06
> UTC 2018
>Storage VM Name  : ch1ns100
>    Storage VM Host/IP Address   : xx
>Storage VM Volume Style  : Flex
>Login User   : tsm-backup-user
>Transport: HTTPS
> 
> Performing a Full Incremental Backup of volume '/Airwarder'
> Creating Base Snapshot.
> Using Base Snapshot 'TSM_CH135BA2689781857_AIRWARDER' with timestamp 
> 09/19/2018 
> 17:17:43
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> acrimport_ref/ch': access to the object is denied
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> acrimport_test/ch': access to the object is denied
> ANS1898I * Processed   500 files *
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> 
> I'm lost here ...
> 
> Cheers.
> 
> Arnaud
> 
> 
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from Panalpina and 
> is confidential or privileged. 
> This information is intended only for the person (s) named above. If you are 
> not the intended recipient, any disclosure, copying, distribution or use or 
> any other action based on the contents of this
> information is strictly prohibited. 
> 
> If you receive this electronic transmission in error, please notify the 
> sender by e-mail, telephone or fax at the numbers listed above. Thank you.
> ****
> www.panalpina.com
> 
> 
> 
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
> Robert Talda
> Sent: Tuesday, September 18, 2018 10:48 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Netapp snapdiff issues having TSM client on a Linux machine, 
> making use of NFS mounts
> 
> Brion:
>  Here at Cornell, we have a TSM client on a Linux system backing up NFS 
> shares via NFS mounts and snapdiff differentials - and have been doing so for 
> years.
> 
>  First, the gory details:
> - For TSM
> TSM Client Version 7, Release 1, Level 6.0 (*red faced, th

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Efim]

Arnaud,

Can you explain why you are copying the .snapshot folder during incremental 
filesystem backup with snapdiff?
It contains snapshots created by netapp and SP for this FS. 
I believe that it should be excluded from copying in your backup job.

Efim


> 19 сент. 2018 г., в 18:28, PAC Brion Arnaud  
> написал(а):
> 
> Hi Robert,
> 
> Thanks a lot for appreciated feedback, unfortunately even when using TSM own 
> snapshots the problem still exists :
> 
> root@xx:~ # dsmc i /Airwarder -snapdiff -diffsnapshot=create 
> -snapdiffhttps -optfile=$OPTPATH >/tmp/snapshot_Airwarder.log
> 
> root@ xx:~ # more /tmp/snapshot_Airwarder.log
> IBM Spectrum Protect
> Command Line Backup-Archive Client Interface
>  Client Version 8, Release 1, Level 4.0 
>  Client date/time: 09/19/2018 17:17:41
> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights 
> Reserved. 
> 
> Node Name: NETAPP_CH1_NFS
> Session established with server xx: Linux/ppc64le
>  Server Version 8, Release 1, Level 4.000
>  Server date/time: 09/19/2018 17:17:42  Last access: 09/18/2018 12:12:17
> 
> 
> Incremental by snapshot difference of volume '/Airwarder'
> 
> Connected to NetApp Storage Virtual Machine
>Management Filer Host/IP Address : xx
>Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
> 13:07:06
> UTC 2018
>Storage VM Name  : ch1ns100
>Storage VM Host/IP Address   : xx
>Storage VM Volume Style  : Flex
>Login User   : tsm-backup-user
>Transport: HTTPS
> 
> Performing a Full Incremental Backup of volume '/Airwarder'
> Creating Base Snapshot.
> Using Base Snapshot 'TSM_CH135BA2689781857_AIRWARDER' with timestamp 
> 09/19/2018 
> 17:17:43
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> acrimport_ref/ch': access to the object is denied
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> acrimport_test/ch': access to the object is denied
> ANS1898I * Processed   500 files *
> ANS4007E Error processing 
> '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
> 
> I'm lost here ...
> 
> Cheers.
> 
> Arnaud
> 
> 
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from Panalpina and 
> is confidential or privileged. 
> This information is intended only for the person (s) named above. If you are 
> not the intended recipient, any disclosure, copying, distribution or use or 
> any other action based on the contents of this
> information is strictly prohibited. 
> 
> If you receive this electronic transmission in error, please notify the 
> sender by e-mail, telephone or fax at the numbers listed above. Thank you.
> ****
> www.panalpina.com
> 
> 
> 
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
> Robert Talda
> Sent: Tuesday, September 18, 2018 10:48 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Netapp snapdiff issues having TSM client on a Linux machine, 
> making use of NFS mounts
> 
> Brion:
>  Here at Cornell, we have a TSM client on a Linux system backing up NFS 
> shares via NFS mounts and snapdiff differentials - and have been doing so for 
> years.
> 
>  First, the gory details:
> - For TSM
> TSM Client Version 7, Release 1, Level 6.0 (*red faced, thought I had 
> upgraded to SP 8.1.2.0 months ago*)
> Running on CentOS Linux release 7.5.1804 (kernel 3.10.0-862.2.3.el7.x86_64)
> Accessing TSM Server for Linux Version 7, Release 1, Level 7.200
> - For NetApp
> Filer Version Information: NetApp Release 9.3P6
> 
>  We aren’t seeing your issue, and I suspect I know why.  We are telling the 
> TSM client to create its own snapshots - and not use the snapshots being 
> created hourly (or daily or weekly even) by the team managing the Netapp - 
> said snapshots being equivalent to the ones you are trying to take advantage 
> of in your backups.
> 
>  That is, the relevant part of the daily backup schedule looks like:
> S

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[PAC Brion Arnaud]

Hi Robert,

Thanks a lot for appreciated feedback, unfortunately even when using TSM own 
snapshots the problem still exists :

root@xx:~ # dsmc i /Airwarder -snapdiff -diffsnapshot=create -snapdiffhttps 
-optfile=$OPTPATH >/tmp/snapshot_Airwarder.log

root@ xx:~ # more /tmp/snapshot_Airwarder.log
IBM Spectrum Protect
Command Line Backup-Archive Client Interface
  Client Version 8, Release 1, Level 4.0 
  Client date/time: 09/19/2018 17:17:41
(c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights Reserved. 

Node Name: NETAPP_CH1_NFS
Session established with server xx: Linux/ppc64le
  Server Version 8, Release 1, Level 4.000
  Server date/time: 09/19/2018 17:17:42  Last access: 09/18/2018 12:12:17


Incremental by snapshot difference of volume '/Airwarder'

Connected to NetApp Storage Virtual Machine
Management Filer Host/IP Address : xx
Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 13:07:06
 UTC 2018
Storage VM Name  : ch1ns100
Storage VM Host/IP Address   : xx
Storage VM Volume Style  : Flex
Login User   : tsm-backup-user
Transport: HTTPS

Performing a Full Incremental Backup of volume '/Airwarder'
Creating Base Snapshot.
Using Base Snapshot 'TSM_CH135BA2689781857_AIRWARDER' with timestamp 09/19/2018 
17:17:43
ANS4007E Error processing '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
acrimport_ref/ch': access to the object is denied
ANS4007E Error processing '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/
acrimport_test/ch': access to the object is denied
ANS1898I * Processed   500 files *
ANS4007E Error processing '/Airwarder/.snapshot/TSM_CH135BA2689781857_AIRWARDER/

I'm lost here ...

Cheers.

Arnaud


Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. 
This information is intended only for the person (s) named above. If you are 
not the intended recipient, any disclosure, copying, distribution or use or any 
other action based on the contents of this
 information is strictly prohibited. 

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.

www.panalpina.com



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Robert 
Talda
Sent: Tuesday, September 18, 2018 10:48 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Netapp snapdiff issues having TSM client on a Linux machine, 
making use of NFS mounts

Brion:
  Here at Cornell, we have a TSM client on a Linux system backing up NFS shares 
via NFS mounts and snapdiff differentials - and have been doing so for years.

  First, the gory details:
- For TSM
TSM Client Version 7, Release 1, Level 6.0 (*red faced, thought I had upgraded 
to SP 8.1.2.0 months ago*)
Running on CentOS Linux release 7.5.1804 (kernel 3.10.0-862.2.3.el7.x86_64)
Accessing TSM Server for Linux Version 7, Release 1, Level 7.200
- For NetApp
Filer Version Information: NetApp Release 9.3P6
 
  We aren’t seeing your issue, and I suspect I know why.  We are telling the 
TSM client to create its own snapshots - and not use the snapshots being 
created hourly (or daily or weekly even) by the team managing the Netapp - said 
snapshots being equivalent to the ones you are trying to take advantage of in 
your backups.

  That is, the relevant part of the daily backup schedule looks like:
 Schedule Name: DAILY.INCR.NFS.SNAPDIFF
   Description: Daily SnapDiff Incremental
Action: Incremental
 Subaction: 
   Options: -snapdiff -diffsnapshot=create -snapdiffhttps


  During a backup, then, the following type of information is written to the 
dsmsched.log:
09/13/2018 00:38:08 Incremental by snapshot difference of volume 
'/nas-backup/vol/cit-pea1-test'
09/13/2018 00:38:10 ANS2328I Using Snapshot Differential Change Log.
09/13/2018 00:38:10
Connected to NetApp Storage Virtual Machine
Management Filer Host/IP Address : ccc-cdot01.cit.cornell.edu
Filer Version Information: NetApp Release 9.3P6: Thu Jun 14 
20:21:25 UTC 2018
Storage VM Name  : cit-sfshared05
Storage VM Host/IP Address   : cit-sfshared05-backup.fil

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[PAC Brion Arnaud]

Del,

Thanks for your clarifications which are perfectly making sense. 
The way we operate our shop (users are always using the machine's node name as 
user ID) just made me forget that any admin might be granted authority on some 
other node, in our case the datamover ...

Cheers.
Arnaud


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Del 
Hoobler
Sent: Tuesday, September 18, 2018 10:16 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Netapp snapdiff issues having TSM client on a Linux machine, 
making use of NFS mounts

Hi Arnaud,

>> ... there's still the limitation that this GUI will be usable only once 
logged on the machine hosting the Spectrum Protect client


What do you mean by "that machine"? 

The Backup-Archive client is not in the data path for an NDMP backup or 
restore. The data path is between the NetApp filer and the server's tape 
library (or more generally speaking the storage pools).  The 
Backup-Archive client is reading metadata from the server to present 
information to the end user. 

This means that the user can log in to any machine (Windows/AIX/Solaris 
x86) that has a Backup-Archive client. When the user logs into the 
Spectrum Protect server (at launch of the Backup-Archive client) they are 
providing an admin id that is not tied to a specific machine.


Del





"ADSM: Dist Stor Manager"  wrote on 09/18/2018 
07:57:56 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 09/18/2018 07:58 AM
> Subject: Re: Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of  NFS mounts
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi Del,
> 
> Thanks for the information, which I got as well thru other channels 
> (IBM support), but there's still the limitation that this GUI will 
> be usable only once logged on the machine hosting the Spectrum Protect 
client.
> This means that I will be forced to grant some of my colleagues 
> (which were previously doing the restores on their own)  specific 
> logging rights on that machine. Was not the case before ...
> 
> Cheers.
> 
> Arnaud
> 
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
> Behalf Of Del Hoobler
> Sent: Tuesday, September 18, 2018 12:57 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of NFS mounts
> 
> Hi Arnaud,
> 
> Not related to SnapDiff ... but just some info on NDMP and B/A Client...
> 
> In Spectrum Protect BA client 8.1.6 that GAs this coming Friday (21 
> September) you will be able to use the native Java GUI to handle NDMP 
> backups/restores.
> 
> 
> Del
> 
> 
> 
> 
> 
> 
> "ADSM: Dist Stor Manager"  wrote on 09/18/2018 
> 06:20:52 AM:
> 
> > From: PAC Brion Arnaud 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 09/18/2018 06:21 AM
> > Subject: Netapp snapdiff issues having TSM client on a Linux 
> > machine, making use of  NFS mounts
> > Sent by: "ADSM: Dist Stor Manager" 
> > 
> > Hi All,
> > 
> > Following to IBM's decision to withdraw the web GUI in the latest 
> > versions of their clients (thus making NDMP hardly usable), as well 
> > as to the renewal of our NAS infrastructure (now using Netapp), I'm 
> > trying to implement netapp snapdiffs in our shop.
> > 
> > So far we succeeded in defining necessary user and roles on the 
> > Netapp, as well as to setup the TSM client in order to interact with
> > our filer, but I'm now facing an issue where some files cannot be 
> > backed up, apparently due to access rights.
> > 
> > Here an extract of our backup log :
> > 
> > IBM Spectrum Protect
> > Command Line Backup-Archive Client Interface
> >   Client Version 8, Release 1, Level 4.0
> >   Client date/time: 09/18/2018 10:26:28
> > (c) Copyright by IBM Corporation and other(s) 1990, 2017. All 
> RightsReserved.
> > 
> > Node Name: NETAPP_CH1_NFS
> > Session established with server XXX: Linux/ppc64le
> >   Server Version 8, Release 1, Level 4.000
> >   Server date/time: 09/18/2018 10:26:28  Last access: 09/18/2018 
> 10:00:37
> > 
> > 
> > Incremental by snapshot difference of volume '/Airwarder'
> > 
> > Connected to NetApp Storage Virtual Machine
> > Management Filer Host/IP Address : 
> > Filer Version Information: NetApp Release 9.3P3: Wed Apr
> > 04 13:07:06 UTC 2018
> > Storage VM Name  : xxx
>

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Robert Talda]

Brion:
  Here at Cornell, we have a TSM client on a Linux system backing up NFS shares 
via NFS mounts and snapdiff differentials - and have been doing so for years.

  First, the gory details:
- For TSM
TSM Client Version 7, Release 1, Level 6.0 (*red faced, thought I had upgraded 
to SP 8.1.2.0 months ago*)
Running on CentOS Linux release 7.5.1804 (kernel 3.10.0-862.2.3.el7.x86_64)
Accessing TSM Server for Linux Version 7, Release 1, Level 7.200
- For NetApp
Filer Version Information: NetApp Release 9.3P6
 
  We aren’t seeing your issue, and I suspect I know why.  We are telling the 
TSM client to create its own snapshots - and not use the snapshots being 
created hourly (or daily or weekly even) by the team managing the Netapp - said 
snapshots being equivalent to the ones you are trying to take advantage of in 
your backups.

  That is, the relevant part of the daily backup schedule looks like:
 Schedule Name: DAILY.INCR.NFS.SNAPDIFF
   Description: Daily SnapDiff Incremental
Action: Incremental
 Subaction: 
   Options: -snapdiff -diffsnapshot=create -snapdiffhttps


  During a backup, then, the following type of information is written to the 
dsmsched.log:
09/13/2018 00:38:08 Incremental by snapshot difference of volume 
'/nas-backup/vol/cit-pea1-test'
09/13/2018 00:38:10 ANS2328I Using Snapshot Differential Change Log.
09/13/2018 00:38:10
Connected to NetApp Storage Virtual Machine
Management Filer Host/IP Address : ccc-cdot01.cit.cornell.edu
Filer Version Information: NetApp Release 9.3P6: Thu Jun 14 
20:21:25 UTC 2018
Storage VM Name  : cit-sfshared05
Storage VM Host/IP Address   : cit-sfshared05-backup.files.cornell.edu
Storage VM Volume Style  : Flex
Login User   : tsmbackup
Transport: HTTPS

09/13/2018 00:38:10 Performing a Snapshot Differential Backup of volume 
'/nas-backup/vol/cit-pea1-test'
09/13/2018 00:38:10 Creating Diff Snapshot.
09/13/2018 00:38:10 Using Base Snapshot 
'TSM_SANT5B9759FC1AB6_CIT_PEA1_TEST_VOL' with timestamp 09/11/2018 06:00:28
09/13/2018 00:38:10 Using Diff Snapshot 
'TSM_SANT5B99E9B24F814_CIT_PEA1_TEST_VOL' with timestamp 09/13/2018 04:38:10

Then, on the next backup (48 hours later thanks to a large amount of data to 
backup on 9/13), the Diff Snapshot becomes the base:
09/15/2018 01:29:22 ANS2328I Using Snapshot Differential Change Log.
09/15/2018 01:29:22 
Connected to NetApp Storage Virtual Machine
Management Filer Host/IP Address : ccc-cdot01.cit.cornell.edu
Filer Version Information: NetApp Release 9.3P6: Thu Jun 14 
20:21:25 UTC 2018
Storage VM Name  : cit-sfshared05
Storage VM Host/IP Address   : cit-sfshared05-backup.files.cornell.edu
Storage VM Volume Style  : Flex
Login User   : tsmbackup
Transport: HTTPS

09/15/2018 01:29:22 Performing a Snapshot Differential Backup of volume 
'/nas-backup/vol/cit-pea1-test'
09/15/2018 01:29:22 Creating Diff Snapshot.
09/15/2018 01:29:22 Using Base Snapshot 
'TSM_SANT5B99E9B24F814_CIT_PEA1_TEST_VOL' with timestamp 09/13/2018 04:38:10
09/15/2018 01:29:22 Using Diff Snapshot 
'TSM_SANT5B9C98B1A15B2_CIT_PEA1_TEST_VOL' with timestamp 09/15/2018 05:29:21

I think that consistency is what is missing in your approach - I suspect that 
there are intervening snapshots with information about files that have been 
deleted or changed that are not part of what the TSM client is seeing.

I will add that our NetApp team is very happy with this approach.  The TSM 
client has been very efficient in its management of the snapshots, and in those 
rare cases when it doesn’t, the snapshots are easy to identify and clean up.

FWIW,
Bob


Robert Talda
EZ-Backup Systems Engineer
Cornell University
+1 607-255-8280
r...@cornell.edu


> On Sep 18, 2018, at 8:35 AM, PAC Brion Arnaud  
> wrote:
> 
> Hi Tommaso,
> 
> The backup command I'm using (dsmc i /Airwarder -snapdiff -snapdiffhttps  
> -diffsnapshotname="daily.*" -diffsnapshot=latest -useexistingbase 
> -optfile=dsm_netapp.opt)  includes the "useexistingbase" statement.
> Based on my understanding of the documentation, this means that the S.P. 
> client will not trigger the creation of a new snapshot on the Netapp device, 
> but instead, make use of a snapshot that had been created previously by some 
> internal mechanism of the Netapp. 
> Therefore, it seems very unlikely that any process still has a grip on any of 
> the files being part of this snapshot ...
> My humble opinion only of course, I'm waiting on others fellow members of 
> this list feedback !
> 
> In addition to this, I would like to avoid the use of any exclud

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Del Hoobler]

Hi Arnaud,

>> ... there's still the limitation that this GUI will be usable only once 
logged on the machine hosting the Spectrum Protect client


What do you mean by "that machine"? 

The Backup-Archive client is not in the data path for an NDMP backup or 
restore. The data path is between the NetApp filer and the server's tape 
library (or more generally speaking the storage pools).  The 
Backup-Archive client is reading metadata from the server to present 
information to the end user. 

This means that the user can log in to any machine (Windows/AIX/Solaris 
x86) that has a Backup-Archive client. When the user logs into the 
Spectrum Protect server (at launch of the Backup-Archive client) they are 
providing an admin id that is not tied to a specific machine.


Del





"ADSM: Dist Stor Manager"  wrote on 09/18/2018 
07:57:56 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 09/18/2018 07:58 AM
> Subject: Re: Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of  NFS mounts
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi Del,
> 
> Thanks for the information, which I got as well thru other channels 
> (IBM support), but there's still the limitation that this GUI will 
> be usable only once logged on the machine hosting the Spectrum Protect 
client.
> This means that I will be forced to grant some of my colleagues 
> (which were previously doing the restores on their own)  specific 
> logging rights on that machine. Was not the case before ...
> 
> Cheers.
> 
> Arnaud
> 
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
> Behalf Of Del Hoobler
> Sent: Tuesday, September 18, 2018 12:57 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of NFS mounts
> 
> Hi Arnaud,
> 
> Not related to SnapDiff ... but just some info on NDMP and B/A Client...
> 
> In Spectrum Protect BA client 8.1.6 that GAs this coming Friday (21 
> September) you will be able to use the native Java GUI to handle NDMP 
> backups/restores.
> 
> 
> Del
> 
> 
> 
> 
> 
> 
> "ADSM: Dist Stor Manager"  wrote on 09/18/2018 
> 06:20:52 AM:
> 
> > From: PAC Brion Arnaud 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 09/18/2018 06:21 AM
> > Subject: Netapp snapdiff issues having TSM client on a Linux 
> > machine, making use of  NFS mounts
> > Sent by: "ADSM: Dist Stor Manager" 
> > 
> > Hi All,
> > 
> > Following to IBM's decision to withdraw the web GUI in the latest 
> > versions of their clients (thus making NDMP hardly usable), as well 
> > as to the renewal of our NAS infrastructure (now using Netapp), I'm 
> > trying to implement netapp snapdiffs in our shop.
> > 
> > So far we succeeded in defining necessary user and roles on the 
> > Netapp, as well as to setup the TSM client in order to interact with
> > our filer, but I'm now facing an issue where some files cannot be 
> > backed up, apparently due to access rights.
> > 
> > Here an extract of our backup log :
> > 
> > IBM Spectrum Protect
> > Command Line Backup-Archive Client Interface
> >   Client Version 8, Release 1, Level 4.0
> >   Client date/time: 09/18/2018 10:26:28
> > (c) Copyright by IBM Corporation and other(s) 1990, 2017. All 
> RightsReserved.
> > 
> > Node Name: NETAPP_CH1_NFS
> > Session established with server XXX: Linux/ppc64le
> >   Server Version 8, Release 1, Level 4.000
> >   Server date/time: 09/18/2018 10:26:28  Last access: 09/18/2018 
> 10:00:37
> > 
> > 
> > Incremental by snapshot difference of volume '/Airwarder'
> > 
> > Connected to NetApp Storage Virtual Machine
> > Management Filer Host/IP Address : 
> > Filer Version Information: NetApp Release 9.3P3: Wed Apr
> > 04 13:07:06 UTC 2018
> > Storage VM Name  : xxx
> > Storage VM Host/IP Address   : XX
> > Storage VM Volume Style  : Flex
> > Login User   : tsm-backup-user
> > Transport: HTTPS
> > 
> > Performing a Full Incremental Backup of volume '/Airwarder'
> > Using Base Snapshot 'vserverdr.0.7b5a59c4-656c-11e8-
> > bb80-00a098d614c2.2018-09-18_102500' with timestamp 09/18/2018 
10:25:02
> > ANS4007E Error processing '/Airwarder/.snapshot/vserverdr.0.
> > 7b5a59c4-65

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[PAC Brion Arnaud]

Hi Tommaso,

The backup command I'm using (dsmc i /Airwarder -snapdiff -snapdiffhttps  
-diffsnapshotname="daily.*" -diffsnapshot=latest -useexistingbase 
-optfile=dsm_netapp.opt)  includes the "useexistingbase" statement.
Based on my understanding of the documentation, this means that the S.P. client 
will not trigger the creation of a new snapshot on the Netapp device, but 
instead, make use of a snapshot that had been created previously by some 
internal mechanism of the Netapp. 
Therefore, it seems very unlikely that any process still has a grip on any of 
the files being part of this snapshot ...
My humble opinion only of course, I'm waiting on others fellow members of this 
list feedback !

In addition to this, I would like to avoid the use of any exclude statement 
during the backup : I doubt that any of my customers would be very happy to 
hear that I'm not able to restore some of his precious files because the 
Spectrum Protect client had limitations ... Such issues never aroused with 
NDMP-based backups !

Cheers.

Arnaud 


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Tommaso Bollini
Sent: Tuesday, September 18, 2018 12:29 PM
To: ADSM-L@VM.MARIST.EDU
Subject: R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux 
machine, making use of NFS mounts

Hello Arnaud, maybe it would be: The NetApp snapshot indicates which files are 
modified on the LUN (for which their backup is triggered); but the 
Backup-Archive client cannot "take" them because they are locked by the 
application.
Perhaps you can solve by inserting the appropiate rules in incl.excl file to 
skip them.


-Messaggio originale-
Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di PAC 
Brion Arnaud
Inviato: martedì 18 settembre 2018 12:21
A: ADSM-L@VM.MARIST.EDU
Oggetto: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, 
making use of NFS mounts

Hi All,

Following to IBM's decision to withdraw the web GUI in the latest versions of 
their clients (thus making NDMP hardly usable), as well as to the renewal of 
our NAS infrastructure (now using Netapp), I'm trying to implement netapp 
snapdiffs in our shop.

So far we succeeded in defining necessary user and roles on the Netapp, as well 
as to setup the TSM client in order to interact with our filer, but I'm now 
facing an issue where some files cannot be backed up, apparently due to access 
rights.

Here an extract of our backup log :

IBM Spectrum Protect
Command Line Backup-Archive Client Interface
  Client Version 8, Release 1, Level 4.0
  Client date/time: 09/18/2018 10:26:28
(c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights Reserved.

Node Name: NETAPP_CH1_NFS
Session established with server XXX: Linux/ppc64le
  Server Version 8, Release 1, Level 4.000
  Server date/time: 09/18/2018 10:26:28  Last access: 09/18/2018 10:00:37


Incremental by snapshot difference of volume '/Airwarder'

Connected to NetApp Storage Virtual Machine
Management Filer Host/IP Address : 
Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
13:07:06 UTC 2018
Storage VM Name  : xxx
Storage VM Host/IP Address   : XX
Storage VM Volume Style  : Flex
Login User   : tsm-backup-user
Transport: HTTPS

Performing a Full Incremental Backup of volume '/Airwarder'
Using Base Snapshot 
'vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500' with 
timestamp 09/18/2018 10:25:02 ANS4007E Error processing 
'/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_ref/ch':
 access to the object is denied ANS4007E Error processing 
'/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_test/ch':
 access to the object is denied
ANS1898I * Processed   500 files *
Directory-->   4,096 /Airwarder/ [Sent]
Directory-->   4,096 /Airwarder/.etc [Sent]

(skipped some lines)

ANS1228E Sending of object '/Airwarder/acrimport_ref/config/log4j.conf' failed.
ANS4007E Error processing '/Airwarder/acrimport_ref/config/log4j.conf': access 
to the object is denied
Normal File-->   309 
/Airwarder/acrimport_ref/config/sapConnector.cfg  ** Unsuccessful **
ANS1228E Sending of object '/Airwarder/acrimport_ref/config/sapConnector.cfg' 
failed.
ANS4007E Error processing '/Airwarder/acrimport_ref/config/sapConnector.cfg': 
access to the object is denied
Normal File-->   330 
/Airwarder/acrimport_ref/config/sapConnector.cfg.bak  ** Unsuccessful **
ANS1228E Sending of object 
'/Airwarder/acrimport_ref/config/sapConnector.cfg.bak' failed.
ANS4007E Error processing 

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[PAC Brion Arnaud]

Hi Del,

Thanks for the information, which I got as well thru other channels (IBM 
support), but there's still the limitation that this GUI will be usable only 
once logged on the machine hosting the Spectrum Protect client.
This means that I will be forced to grant some of my colleagues (which were 
previously doing the restores on their own)  specific logging rights on that 
machine. Was not the case before ...

Cheers.

Arnaud

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Del 
Hoobler
Sent: Tuesday, September 18, 2018 12:57 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Netapp snapdiff issues having TSM client on a Linux machine, 
making use of NFS mounts

Hi Arnaud,

Not related to SnapDiff ... but just some info on NDMP and B/A Client...

In Spectrum Protect BA client 8.1.6 that GAs this coming Friday (21 
September) you will be able to use the native Java GUI to handle NDMP 
backups/restores.


Del






"ADSM: Dist Stor Manager"  wrote on 09/18/2018 
06:20:52 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 09/18/2018 06:21 AM
> Subject: Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of  NFS mounts
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi All,
> 
> Following to IBM's decision to withdraw the web GUI in the latest 
> versions of their clients (thus making NDMP hardly usable), as well 
> as to the renewal of our NAS infrastructure (now using Netapp), I'm 
> trying to implement netapp snapdiffs in our shop.
> 
> So far we succeeded in defining necessary user and roles on the 
> Netapp, as well as to setup the TSM client in order to interact with
> our filer, but I'm now facing an issue where some files cannot be 
> backed up, apparently due to access rights.
> 
> Here an extract of our backup log :
> 
> IBM Spectrum Protect
> Command Line Backup-Archive Client Interface
>   Client Version 8, Release 1, Level 4.0
>   Client date/time: 09/18/2018 10:26:28
> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All 
RightsReserved.
> 
> Node Name: NETAPP_CH1_NFS
> Session established with server XXX: Linux/ppc64le
>   Server Version 8, Release 1, Level 4.000
>   Server date/time: 09/18/2018 10:26:28  Last access: 09/18/2018 
10:00:37
> 
> 
> Incremental by snapshot difference of volume '/Airwarder'
> 
> Connected to NetApp Storage Virtual Machine
> Management Filer Host/IP Address : 
> Filer Version Information: NetApp Release 9.3P3: Wed Apr
> 04 13:07:06 UTC 2018
> Storage VM Name  : xxx
> Storage VM Host/IP Address   : XX
> Storage VM Volume Style  : Flex
> Login User   : tsm-backup-user
> Transport: HTTPS
> 
> Performing a Full Incremental Backup of volume '/Airwarder'
> Using Base Snapshot 'vserverdr.0.7b5a59c4-656c-11e8-
> bb80-00a098d614c2.2018-09-18_102500' with timestamp 09/18/2018 10:25:02
> ANS4007E Error processing '/Airwarder/.snapshot/vserverdr.0.
> 7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/
> acrimport_ref/ch': access to the object is denied
> ANS4007E Error processing '/Airwarder/.snapshot/vserverdr.0.
> 7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/
> acrimport_test/ch': access to the object is denied
> ANS1898I * Processed   500 files *
> Directory-->   4,096 /Airwarder/ [Sent]
> Directory-->   4,096 /Airwarder/.etc [Sent]
> 
> (skipped some lines)
> 
> ANS1228E Sending of object '/Airwarder/acrimport_ref/config/
> log4j.conf' failed.
> ANS4007E Error processing '/Airwarder/acrimport_ref/config/
> log4j.conf': access to the object is denied
> Normal File-->   309 /Airwarder/acrimport_ref/config/
> sapConnector.cfg  ** Unsuccessful **
> ANS1228E Sending of object '/Airwarder/acrimport_ref/config/
> sapConnector.cfg' failed.
> ANS4007E Error processing '/Airwarder/acrimport_ref/config/
> sapConnector.cfg': access to the object is denied
> Normal File-->   330 /Airwarder/acrimport_ref/config/
> sapConnector.cfg.bak  ** Unsuccessful **
> ANS1228E Sending of object '/Airwarder/acrimport_ref/config/
> sapConnector.cfg.bak' failed.
> ANS4007E Error processing '/Airwarder/acrimport_ref/config/
> sapConnector.cfg.bak': access to the object is denied
> 
> (skipped some lines)
> 
> ANS1802E Incremental backup of '/Airwarder' finished with 90 failure(s)
> 
> 
> Total number of objects inspected:5,881
>

Re: Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Del Hoobler]

Hi Arnaud,

Not related to SnapDiff ... but just some info on NDMP and B/A Client...

In Spectrum Protect BA client 8.1.6 that GAs this coming Friday (21 
September) you will be able to use the native Java GUI to handle NDMP 
backups/restores.


Del






"ADSM: Dist Stor Manager"  wrote on 09/18/2018 
06:20:52 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 09/18/2018 06:21 AM
> Subject: Netapp snapdiff issues having TSM client on a Linux 
> machine, making use of  NFS mounts
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi All,
> 
> Following to IBM's decision to withdraw the web GUI in the latest 
> versions of their clients (thus making NDMP hardly usable), as well 
> as to the renewal of our NAS infrastructure (now using Netapp), I'm 
> trying to implement netapp snapdiffs in our shop.
> 
> So far we succeeded in defining necessary user and roles on the 
> Netapp, as well as to setup the TSM client in order to interact with
> our filer, but I'm now facing an issue where some files cannot be 
> backed up, apparently due to access rights.
> 
> Here an extract of our backup log :
> 
> IBM Spectrum Protect
> Command Line Backup-Archive Client Interface
>   Client Version 8, Release 1, Level 4.0
>   Client date/time: 09/18/2018 10:26:28
> (c) Copyright by IBM Corporation and other(s) 1990, 2017. All 
RightsReserved.
> 
> Node Name: NETAPP_CH1_NFS
> Session established with server XXX: Linux/ppc64le
>   Server Version 8, Release 1, Level 4.000
>   Server date/time: 09/18/2018 10:26:28  Last access: 09/18/2018 
10:00:37
> 
> 
> Incremental by snapshot difference of volume '/Airwarder'
> 
> Connected to NetApp Storage Virtual Machine
> Management Filer Host/IP Address : 
> Filer Version Information: NetApp Release 9.3P3: Wed Apr
> 04 13:07:06 UTC 2018
> Storage VM Name  : xxx
> Storage VM Host/IP Address   : XX
> Storage VM Volume Style  : Flex
> Login User   : tsm-backup-user
> Transport: HTTPS
> 
> Performing a Full Incremental Backup of volume '/Airwarder'
> Using Base Snapshot 'vserverdr.0.7b5a59c4-656c-11e8-
> bb80-00a098d614c2.2018-09-18_102500' with timestamp 09/18/2018 10:25:02
> ANS4007E Error processing '/Airwarder/.snapshot/vserverdr.0.
> 7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/
> acrimport_ref/ch': access to the object is denied
> ANS4007E Error processing '/Airwarder/.snapshot/vserverdr.0.
> 7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/
> acrimport_test/ch': access to the object is denied
> ANS1898I * Processed   500 files *
> Directory-->   4,096 /Airwarder/ [Sent]
> Directory-->   4,096 /Airwarder/.etc [Sent]
> 
> (skipped some lines)
> 
> ANS1228E Sending of object '/Airwarder/acrimport_ref/config/
> log4j.conf' failed.
> ANS4007E Error processing '/Airwarder/acrimport_ref/config/
> log4j.conf': access to the object is denied
> Normal File-->   309 /Airwarder/acrimport_ref/config/
> sapConnector.cfg  ** Unsuccessful **
> ANS1228E Sending of object '/Airwarder/acrimport_ref/config/
> sapConnector.cfg' failed.
> ANS4007E Error processing '/Airwarder/acrimport_ref/config/
> sapConnector.cfg': access to the object is denied
> Normal File-->   330 /Airwarder/acrimport_ref/config/
> sapConnector.cfg.bak  ** Unsuccessful **
> ANS1228E Sending of object '/Airwarder/acrimport_ref/config/
> sapConnector.cfg.bak' failed.
> ANS4007E Error processing '/Airwarder/acrimport_ref/config/
> sapConnector.cfg.bak': access to the object is denied
> 
> (skipped some lines)
> 
> ANS1802E Incremental backup of '/Airwarder' finished with 90 failure(s)
> 
> 
> Total number of objects inspected:5,881
> Total number of objects backed up:5,791
> Total number of objects updated:  0
> Total number of objects rebound:  0
> Total number of objects deleted:  0
> Total number of objects expired:  0
> Total number of objects failed:  90
> Total number of objects encrypted:0
> Total number of objects grew: 0
> Total number of retries: 15
> Total number of bytes inspected:   1.27 GB
> Total number of bytes transferred: 1.37 GB
> Data transfer time:   18.61 sec
> Network data transfer rate:   77,367.92 KB/sec
> Aggregate data transfer rat

R: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[Tommaso Bollini]

Hello Arnaud, maybe it would be: The NetApp snapshot indicates which files are 
modified on the LUN (for which their backup is triggered); but the 
Backup-Archive client cannot "take" them because they are locked by the 
application.
Perhaps you can solve by inserting the appropiate rules in incl.excl file to 
skip them.


-Messaggio originale-
Da: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] Per conto di PAC 
Brion Arnaud
Inviato: martedì 18 settembre 2018 12:21
A: ADSM-L@VM.MARIST.EDU
Oggetto: [ADSM-L] Netapp snapdiff issues having TSM client on a Linux machine, 
making use of NFS mounts

Hi All,

Following to IBM's decision to withdraw the web GUI in the latest versions of 
their clients (thus making NDMP hardly usable), as well as to the renewal of 
our NAS infrastructure (now using Netapp), I'm trying to implement netapp 
snapdiffs in our shop.

So far we succeeded in defining necessary user and roles on the Netapp, as well 
as to setup the TSM client in order to interact with our filer, but I'm now 
facing an issue where some files cannot be backed up, apparently due to access 
rights.

Here an extract of our backup log :

IBM Spectrum Protect
Command Line Backup-Archive Client Interface
  Client Version 8, Release 1, Level 4.0
  Client date/time: 09/18/2018 10:26:28
(c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights Reserved.

Node Name: NETAPP_CH1_NFS
Session established with server XXX: Linux/ppc64le
  Server Version 8, Release 1, Level 4.000
  Server date/time: 09/18/2018 10:26:28  Last access: 09/18/2018 10:00:37


Incremental by snapshot difference of volume '/Airwarder'

Connected to NetApp Storage Virtual Machine
Management Filer Host/IP Address : 
Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
13:07:06 UTC 2018
Storage VM Name  : xxx
Storage VM Host/IP Address   : XX
Storage VM Volume Style  : Flex
Login User   : tsm-backup-user
Transport: HTTPS

Performing a Full Incremental Backup of volume '/Airwarder'
Using Base Snapshot 
'vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500' with 
timestamp 09/18/2018 10:25:02 ANS4007E Error processing 
'/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_ref/ch':
 access to the object is denied ANS4007E Error processing 
'/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_test/ch':
 access to the object is denied
ANS1898I * Processed   500 files *
Directory-->   4,096 /Airwarder/ [Sent]
Directory-->   4,096 /Airwarder/.etc [Sent]

(skipped some lines)

ANS1228E Sending of object '/Airwarder/acrimport_ref/config/log4j.conf' failed.
ANS4007E Error processing '/Airwarder/acrimport_ref/config/log4j.conf': access 
to the object is denied
Normal File-->   309 
/Airwarder/acrimport_ref/config/sapConnector.cfg  ** Unsuccessful **
ANS1228E Sending of object '/Airwarder/acrimport_ref/config/sapConnector.cfg' 
failed.
ANS4007E Error processing '/Airwarder/acrimport_ref/config/sapConnector.cfg': 
access to the object is denied
Normal File-->   330 
/Airwarder/acrimport_ref/config/sapConnector.cfg.bak  ** Unsuccessful **
ANS1228E Sending of object 
'/Airwarder/acrimport_ref/config/sapConnector.cfg.bak' failed.
ANS4007E Error processing 
'/Airwarder/acrimport_ref/config/sapConnector.cfg.bak': access to the object is 
denied

(skipped some lines)

ANS1802E Incremental backup of '/Airwarder' finished with 90 failure(s)


Total number of objects inspected:5,881
Total number of objects backed up:5,791
Total number of objects updated:  0
Total number of objects rebound:  0
Total number of objects deleted:  0
Total number of objects expired:  0
Total number of objects failed:  90
Total number of objects encrypted:0
Total number of objects grew: 0
Total number of retries: 15
Total number of bytes inspected:   1.27 GB
Total number of bytes transferred: 1.37 GB
Data transfer time:   18.61 sec
Network data transfer rate:   77,367.92 KB/sec
Aggregate data transfer rate: 36,807.03 KB/sec
Objects compressed by:0%
Total data reduction ratio:0.00%
Elapsed processing time:   00:00:39


Here, a closer look at some of the files making problems  :

root@xx:~ # ls -al /Airwarder/acrimport_ref/config/sapConnector.cfg.bak
-rw-r- 1 root root 330 May  6  2009 
/Airwarder/acrimport_ref/config/sapConnector.cfg.bak


root@xx:~ # ls -la 
/Airwarder/.snapshot/vserv

Netapp snapdiff issues having TSM client on a Linux machine, making use of NFS mounts

[PAC Brion Arnaud]

Hi All,

Following to IBM's decision to withdraw the web GUI in the latest versions of 
their clients (thus making NDMP hardly usable), as well as to the renewal of 
our NAS infrastructure (now using Netapp), I'm trying to implement netapp 
snapdiffs in our shop.

So far we succeeded in defining necessary user and roles on the Netapp, as well 
as to setup the TSM client in order to interact with our filer, but I'm now 
facing an issue where some files cannot be backed up, apparently due to access 
rights.

Here an extract of our backup log :

IBM Spectrum Protect
Command Line Backup-Archive Client Interface
  Client Version 8, Release 1, Level 4.0
  Client date/time: 09/18/2018 10:26:28
(c) Copyright by IBM Corporation and other(s) 1990, 2017. All Rights Reserved.

Node Name: NETAPP_CH1_NFS
Session established with server XXX: Linux/ppc64le
  Server Version 8, Release 1, Level 4.000
  Server date/time: 09/18/2018 10:26:28  Last access: 09/18/2018 10:00:37


Incremental by snapshot difference of volume '/Airwarder'

Connected to NetApp Storage Virtual Machine
Management Filer Host/IP Address : 
Filer Version Information: NetApp Release 9.3P3: Wed Apr 04 
13:07:06 UTC 2018
Storage VM Name  : xxx
Storage VM Host/IP Address   : XX
Storage VM Volume Style  : Flex
Login User   : tsm-backup-user
Transport: HTTPS

Performing a Full Incremental Backup of volume '/Airwarder'
Using Base Snapshot 
'vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500' with 
timestamp 09/18/2018 10:25:02
ANS4007E Error processing 
'/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_ref/ch':
 access to the object is denied
ANS4007E Error processing 
'/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_102500/acrimport_test/ch':
 access to the object is denied
ANS1898I * Processed   500 files *
Directory-->   4,096 /Airwarder/ [Sent]
Directory-->   4,096 /Airwarder/.etc [Sent]

(skipped some lines)

ANS1228E Sending of object '/Airwarder/acrimport_ref/config/log4j.conf' failed.
ANS4007E Error processing '/Airwarder/acrimport_ref/config/log4j.conf': access 
to the object is denied
Normal File-->   309 
/Airwarder/acrimport_ref/config/sapConnector.cfg  ** Unsuccessful **
ANS1228E Sending of object '/Airwarder/acrimport_ref/config/sapConnector.cfg' 
failed.
ANS4007E Error processing '/Airwarder/acrimport_ref/config/sapConnector.cfg': 
access to the object is denied
Normal File-->   330 
/Airwarder/acrimport_ref/config/sapConnector.cfg.bak  ** Unsuccessful **
ANS1228E Sending of object 
'/Airwarder/acrimport_ref/config/sapConnector.cfg.bak' failed.
ANS4007E Error processing 
'/Airwarder/acrimport_ref/config/sapConnector.cfg.bak': access to the object is 
denied

(skipped some lines)

ANS1802E Incremental backup of '/Airwarder' finished with 90 failure(s)


Total number of objects inspected:5,881
Total number of objects backed up:5,791
Total number of objects updated:  0
Total number of objects rebound:  0
Total number of objects deleted:  0
Total number of objects expired:  0
Total number of objects failed:  90
Total number of objects encrypted:0
Total number of objects grew: 0
Total number of retries: 15
Total number of bytes inspected:   1.27 GB
Total number of bytes transferred: 1.37 GB
Data transfer time:   18.61 sec
Network data transfer rate:   77,367.92 KB/sec
Aggregate data transfer rate: 36,807.03 KB/sec
Objects compressed by:0%
Total data reduction ratio:0.00%
Elapsed processing time:   00:00:39


Here, a closer look at some of the files making problems  :

root@xx:~ # ls -al /Airwarder/acrimport_ref/config/sapConnector.cfg.bak
-rw-r- 1 root root 330 May  6  2009 
/Airwarder/acrimport_ref/config/sapConnector.cfg.bak


root@xx:~ # ls -la 
/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_121000/acrimport_ref/ch
ls: cannot open directory 
/Airwarder/.snapshot/vserverdr.0.7b5a59c4-656c-11e8-bb80-00a098d614c2.2018-09-18_121000/acrimport_ref/ch:
 Permission denied

And who I am:

root@xx:~ # id
uid=0(root) gid=0(root) groups=0(root)

Any idea on how to solve such issues ?

Cheers.

Arnaud


Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. 

TSM v815 : Tape support enhancements

[Vikas Arora]

 Hi  Guys,

We recently upgraded one of tsm server to tsm v815 with interest to get
best of tape enhancement support announced in v815. But while running
normal operations like migration (disk to tape) or so, we doubt that it
still has poor performance. How could we be sure that these enhancements
are there in our tsm code v815 and are in effect?

Anyway to verify this?

Ref in youtube video link below at play time "20:23"

IBM Spectrum Protect 8.1.5 and Spectrum Protect Plus 10.1.1 - technical
overview - Presentation

https://www.youtube.com/watch?v=zz0qrh7vnOU

Many thanks!!



Grüße
Vikas Arora

*e Mail* : vikar...@gmail.com

Re: TSM Instance can't find its instance directory.

[Harris, Steven]

Hi Eric and list.

I found the problem.

The home directory of the user is /tsm/wtsmt01.  That is also the mount point 
of a file system

When mounted the permissions were fine 
drwxrwxrwx   11 wtsmt01  tsm   16384 25 May 16:55 wtsmt01

When unmounted however the permissions were not correct.
drwx--7 root system  256 10 Apr 14:37 wtsmt01

During start up db2 uses /bin/pwd which is symlinked to /usr/bin/pwd

I ran your db2idrop successfully but the db2icrt failed.  The log said 

1. Verify the access permissions of the home directory of the instance
   owner by using the command "su".

   Example:

   /usr/bin/su  -c /bin/pwd

2. If the permissions do not include "r" and "x", unmount the drive,
   change the permissions, and remount the drive.

   Example:

   unmount /
   chown  /
   chmod  755 /
   mount /

   Related information:
   File permission requirements for the instance and database
   directories

#1 failed, so I performed #2 and that fixed the issue. 

The symptom that I had seen before I started this debugging is that the 
instance user is getting a permissions error executing pwd

Many thanks

Steve

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
Eric van (ITOPT3) - KLM
Sent: Wednesday, 23 May 2018 10:27 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Instance can't find its instance directory.

Hi Steve,
Did you check the content of the .profile file of the instance user on the DR 
host? It should be identical to the one on the prime site.
If that's OK you could try and drop the instance and recatalog the database on 
the DR node:

# /opt/tivoli/tsm/db2/instance/db2idrop  # 
/opt/tivoli/tsm/db2/instance/db2icrt -a server -s ese -u  
 $ db2 catalog db tsmdb1 on /tsm/wtsmt01

Maybe that helps?
Kind regards,
Eric van Loon
Air France/KLM Storage Engineering

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Harris, Steven
Sent: woensdag 23 mei 2018 1:36
To: ADSM-L@VM.MARIST.EDU
Subject: TSM Instance can't find its instance directory.

Hi guys

I have a curly one here. TSM 7.1.1.300 on AIX  7.1

I'm preparing for a DR test.

The DR process is that the TSM instance directory, database, logs and disk/file 
storage pools reside on an IBM V840 flash device.  These are mirrored to the DR 
site using  remote copy.  The old way was to use the remote copy directly, the 
new way is to run a flashcopy backup of the remote-copied volumes and mount 
those. Old process or new I get the same error. It last worked November last 
year.

On my test instance,  when I start at prime site as the instance user, it finds 
the instance directory /tsm/wtsmt01.  It then opens the dsmserv.opt and 
initialization completes as usual.

When I start at the DR site, also as the instance user, with the same 
filesystems  mounted, and in the instance directory, it does not find the 
instance directory, but tries to use the root directory.  Obviously the 
dsmserv.opt file is not found and the process goes down hill from there.

Primary:
ANR0900I Processing options file /tsm/wtsmt01/dsmserv.opt.

DR:
ANR0905W Options file /dsmserv.opt not found.
ANR0010W Unable to open message catalog for language en_AU.8859-15. The default 
language message catalog will be used.
ANR7811I Using instance directory .

I have compared the two sets of environment variables and they are identical, 
apart from obvious things like PID and RANDOM.

According to the doc, the instance directory is supposed to be set from the DB2 
instance directory.  This is checked with db2 get dbm cfg | grep DFTDBPATH and 
is the same on both boxes.  I even explicitly set it again on the DR side to no 
effect.

TSM Support has been no help.

Can anyone point me in the right direction?  Even some idea of how to trace 
what's happening would be useful.

Thanks

Steve

Steven Harris
TSM Admin/Consultant
Canberra Australia

This message and any attachment is confidential and may be privileged or 
otherwise protected from disclosure. You should immediately delete the message 
if you are not the intended recipient. If you have received this email by 
mistake please delete it from your system; you should not copy the message or 
disclose its content to anyone. 

This electronic communication may contain general financial product advice but 
should not be relied upon or construed as a recommendation of any financial 
product. The information has been prepared without taking into account your 
objectives, financial situation or needs. You should consider the Product 
Disclosure Statement relating to the financial product and consult your 
financial adviser before making a decision about whether to acquire, hold or 
dispose of a financial product. 

For further details on the financial product please go to http://www.bt.com.au 

Past perfor

Re: TSM Instance can't find its instance directory.

[Harris, Steven]

Much appreciated,  Eric.

Yes I did look at the .profile and it is the same on both systems.
Dropping and recreating the instance is the way to go.  I can flashcopy the 
rootvg so its cheap to roll back if it all goes awry.
Will try tomorrow.

Thanks

Steve.

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Loon, 
Eric van (ITOPT3) - KLM
Sent: Wednesday, 23 May 2018 10:27 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] TSM Instance can't find its instance directory.

Hi Steve,
Did you check the content of the .profile file of the instance user on the DR 
host? It should be identical to the one on the prime site.
If that's OK you could try and drop the instance and recatalog the database on 
the DR node:

# /opt/tivoli/tsm/db2/instance/db2idrop 
# /opt/tivoli/tsm/db2/instance/db2icrt -a server -s ese -u  

$ db2 catalog db tsmdb1 on /tsm/wtsmt01

Maybe that helps?
Kind regards,
Eric van Loon
Air France/KLM Storage Engineering

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Harris, Steven
Sent: woensdag 23 mei 2018 1:36
To: ADSM-L@VM.MARIST.EDU
Subject: TSM Instance can't find its instance directory.

Hi guys

I have a curly one here. TSM 7.1.1.300 on AIX  7.1

I'm preparing for a DR test.

The DR process is that the TSM instance directory, database, logs and disk/file 
storage pools reside on an IBM V840 flash device.  These are mirrored to the DR 
site using  remote copy.  The old way was to use the remote copy directly, the 
new way is to run a flashcopy backup of the remote-copied volumes and mount 
those. Old process or new I get the same error. It last worked November last 
year.

On my test instance,  when I start at prime site as the instance user, it finds 
the instance directory /tsm/wtsmt01.  It then opens the dsmserv.opt and 
initialization completes as usual.

When I start at the DR site, also as the instance user, with the same 
filesystems  mounted, and in the instance directory, it does not find the 
instance directory, but tries to use the root directory.  Obviously the 
dsmserv.opt file is not found and the process goes down hill from there.

Primary:
ANR0900I Processing options file /tsm/wtsmt01/dsmserv.opt.

DR:
ANR0905W Options file /dsmserv.opt not found.
ANR0010W Unable to open message catalog for language en_AU.8859-15. The default 
language message catalog will be used.
ANR7811I Using instance directory .

I have compared the two sets of environment variables and they are identical, 
apart from obvious things like PID and RANDOM.

According to the doc, the instance directory is supposed to be set from the DB2 
instance directory.  This is checked with db2 get dbm cfg | grep DFTDBPATH and 
is the same on both boxes.  I even explicitly set it again on the DR side to no 
effect.

TSM Support has been no help.

Can anyone point me in the right direction?  Even some idea of how to trace 
what's happening would be useful.

Thanks

Steve

Steven Harris
TSM Admin/Consultant
Canberra Australia

This message and any attachment is confidential and may be privileged or 
otherwise protected from disclosure. You should immediately delete the message 
if you are not the intended recipient. If you have received this email by 
mistake please delete it from your system; you should not copy the message or 
disclose its content to anyone. 

This electronic communication may contain general financial product advice but 
should not be relied upon or construed as a recommendation of any financial 
product. The information has been prepared without taking into account your 
objectives, financial situation or needs. You should consider the Product 
Disclosure Statement relating to the financial product and consult your 
financial adviser before making a decision about whether to acquire, hold or 
dispose of a financial product. 

For further details on the financial product please go to http://www.bt.com.au 

Past performance is not a reliable indicator of future performance.

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM 

Re: TSM Instance can't find its instance directory.

[Loon, Eric van (ITOPT3) - KLM]

Hi Steve,
Did you check the content of the .profile file of the instance user on the DR 
host? It should be identical to the one on the prime site.
If that's OK you could try and drop the instance and recatalog the database on 
the DR node:

# /opt/tivoli/tsm/db2/instance/db2idrop 
# /opt/tivoli/tsm/db2/instance/db2icrt -a server -s ese -u  

$ db2 catalog db tsmdb1 on /tsm/wtsmt01

Maybe that helps?
Kind regards,
Eric van Loon
Air France/KLM Storage Engineering

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Harris, Steven
Sent: woensdag 23 mei 2018 1:36
To: ADSM-L@VM.MARIST.EDU
Subject: TSM Instance can't find its instance directory.

Hi guys

I have a curly one here. TSM 7.1.1.300 on AIX  7.1

I'm preparing for a DR test.

The DR process is that the TSM instance directory, database, logs and disk/file 
storage pools reside on an IBM V840 flash device.  These are mirrored to the DR 
site using  remote copy.  The old way was to use the remote copy directly, the 
new way is to run a flashcopy backup of the remote-copied volumes and mount 
those. Old process or new I get the same error. It last worked November last 
year.

On my test instance,  when I start at prime site as the instance user, it finds 
the instance directory /tsm/wtsmt01.  It then opens the dsmserv.opt and 
initialization completes as usual.

When I start at the DR site, also as the instance user, with the same 
filesystems  mounted, and in the instance directory, it does not find the 
instance directory, but tries to use the root directory.  Obviously the 
dsmserv.opt file is not found and the process goes down hill from there.

Primary:
ANR0900I Processing options file /tsm/wtsmt01/dsmserv.opt.

DR:
ANR0905W Options file /dsmserv.opt not found.
ANR0010W Unable to open message catalog for language en_AU.8859-15. The default 
language message catalog will be used.
ANR7811I Using instance directory .

I have compared the two sets of environment variables and they are identical, 
apart from obvious things like PID and RANDOM.

According to the doc, the instance directory is supposed to be set from the DB2 
instance directory.  This is checked with db2 get dbm cfg | grep DFTDBPATH and 
is the same on both boxes.  I even explicitly set it again on the DR side to no 
effect.

TSM Support has been no help.

Can anyone point me in the right direction?  Even some idea of how to trace 
what's happening would be useful.

Thanks

Steve

Steven Harris
TSM Admin/Consultant
Canberra Australia

This message and any attachment is confidential and may be privileged or 
otherwise protected from disclosure. You should immediately delete the message 
if you are not the intended recipient. If you have received this email by 
mistake please delete it from your system; you should not copy the message or 
disclose its content to anyone. 

This electronic communication may contain general financial product advice but 
should not be relied upon or construed as a recommendation of any financial 
product. The information has been prepared without taking into account your 
objectives, financial situation or needs. You should consider the Product 
Disclosure Statement relating to the financial product and consult your 
financial adviser before making a decision about whether to acquire, hold or 
dispose of a financial product. 

For further details on the financial product please go to http://www.bt.com.au 

Past performance is not a reliable indicator of future performance.

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286

Re: TSM Instance can't find its instance directory.

[Chavdar Cholev]

just shot in the dsrk
did check user thst run tsm and its permissioms

On Wednesday, May 23, 2018, Harris, Steven <
steven.har...@btfinancialgroup.com> wrote:

> Hi guys
>
> I have a curly one here. TSM 7.1.1.300 on AIX  7.1
>
> I'm preparing for a DR test.
>
> The DR process is that the TSM instance directory, database, logs and
> disk/file storage pools reside on an IBM V840 flash device.  These are
> mirrored to the DR site using  remote copy.  The old way was to use the
> remote copy directly, the new way is to run a flashcopy backup of the
> remote-copied volumes and mount those. Old process or new I get the same
> error. It last worked November last year.
>
> On my test instance,  when I start at prime site as the instance user, it
> finds the instance directory /tsm/wtsmt01.  It then opens the dsmserv.opt
> and initialization completes as usual.
>
> When I start at the DR site, also as the instance user, with the same
> filesystems  mounted, and in the instance directory, it does not find the
> instance directory, but tries to use the root directory.  Obviously the
> dsmserv.opt file is not found and the process goes down hill from there.
>
> Primary:
> ANR0900I Processing options file /tsm/wtsmt01/dsmserv.opt.
>
> DR:
> ANR0905W Options file /dsmserv.opt not found.
> ANR0010W Unable to open message catalog for language en_AU.8859-15. The
> default
> language message catalog will be used.
> ANR7811I Using instance directory .
>
> I have compared the two sets of environment variables and they are
> identical, apart from obvious things like PID and RANDOM.
>
> According to the doc, the instance directory is supposed to be set from
> the DB2 instance directory.  This is checked with db2 get dbm cfg | grep
> DFTDBPATH and is the same on both boxes.  I even explicitly set it again on
> the DR side to no effect.
>
> TSM Support has been no help.
>
> Can anyone point me in the right direction?  Even some idea of how to
> trace what's happening would be useful.
>
> Thanks
>
> Steve
>
> Steven Harris
> TSM Admin/Consultant
> Canberra Australia
>
> This message and any attachment is confidential and may be privileged or
> otherwise protected from disclosure. You should immediately delete the
> message if you are not the intended recipient. If you have received this
> email by mistake please delete it from your system; you should not copy the
> message or disclose its content to anyone.
>
> This electronic communication may contain general financial product advice
> but should not be relied upon or construed as a recommendation of any
> financial product. The information has been prepared without taking into
> account your objectives, financial situation or needs. You should consider
> the Product Disclosure Statement relating to the financial product and
> consult your financial adviser before making a decision about whether to
> acquire, hold or dispose of a financial product.
>
> For further details on the financial product please go to
> http://www.bt.com.au
>
> Past performance is not a reliable indicator of future performance.
>

TSM Instance can't find its instance directory.

[Harris, Steven]

Hi guys

I have a curly one here. TSM 7.1.1.300 on AIX  7.1

I'm preparing for a DR test.

The DR process is that the TSM instance directory, database, logs and disk/file 
storage pools reside on an IBM V840 flash device.  These are mirrored to the DR 
site using  remote copy.  The old way was to use the remote copy directly, the 
new way is to run a flashcopy backup of the remote-copied volumes and mount 
those. Old process or new I get the same error. It last worked November last 
year.

On my test instance,  when I start at prime site as the instance user, it finds 
the instance directory /tsm/wtsmt01.  It then opens the dsmserv.opt and 
initialization completes as usual.

When I start at the DR site, also as the instance user, with the same 
filesystems  mounted, and in the instance directory, it does not find the 
instance directory, but tries to use the root directory.  Obviously the 
dsmserv.opt file is not found and the process goes down hill from there.

Primary:
ANR0900I Processing options file /tsm/wtsmt01/dsmserv.opt.

DR:
ANR0905W Options file /dsmserv.opt not found.
ANR0010W Unable to open message catalog for language en_AU.8859-15. The default
language message catalog will be used.
ANR7811I Using instance directory .

I have compared the two sets of environment variables and they are identical, 
apart from obvious things like PID and RANDOM.

According to the doc, the instance directory is supposed to be set from the DB2 
instance directory.  This is checked with db2 get dbm cfg | grep DFTDBPATH and 
is the same on both boxes.  I even explicitly set it again on the DR side to no 
effect.

TSM Support has been no help.

Can anyone point me in the right direction?  Even some idea of how to trace 
what's happening would be useful.

Thanks

Steve

Steven Harris
TSM Admin/Consultant
Canberra Australia

This message and any attachment is confidential and may be privileged or 
otherwise protected from disclosure. You should immediately delete the message 
if you are not the intended recipient. If you have received this email by 
mistake please delete it from your system; you should not copy the message or 
disclose its content to anyone. 

This electronic communication may contain general financial product advice but 
should not be relied upon or construed as a recommendation of any financial 
product. The information has been prepared without taking into account your 
objectives, financial situation or needs. You should consider the Product 
Disclosure Statement relating to the financial product and consult your 
financial adviser before making a decision about whether to acquire, hold or 
dispose of a financial product. 

For further details on the financial product please go to http://www.bt.com.au 

Past performance is not a reliable indicator of future performance.