Re: HELP!!! EMERGENCY!!!
How are your disk pools? Full or still have room? What is your tape status? Do you have other communications problems? (Slow printing, slow terminals, slow FTP...) -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED] Sent: Friday, 04 April 2003 12:13 PM To: [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- Information from the mail header --- Sender: ADSM: Dist Stor Manager [EMAIL PROTECTED] Poster: Joni Moyer [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- - I am running TSM 4.1.5 on OS/390 2.10 I am getting calls from clients about slow backups, restores,etc. on TSM. I went out and looked and there are 100 sessions going for different clients, but they aren't ending. Some are in the IDLEW status, but a majority of them are in the run status. What should I look into? Thanks! Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: HELP!!! EMERGENCY!!!
Disk pools aren't full. Tape drives are available. There are plenty of tapes. The db is 88% full. Recovery log is 7% utilized. I did have the following message, but it said the server would continue: ANR0390W A server database deadlock situation has been encountered: lock request for transaction 0:6.517663675 will be denied to resolve the deadlock. ANRD DFUTIL(734): Lock acquisition (isLock) failed for bitfile root 0. ANR2183W AFMOVE3551: Transaction 0:6.517663675 was aborted. Any suggestions!!! I really need help! Should I contact support? If I do it'll take 3 hours to get a response... Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338 Kai Hintze [EMAIL PROTECTED] To: [EMAIL PROTECTED] TSONS.COM cc: Sent by: ADSM: Subject: Re: HELP!!! EMERGENCY!!! Dist Stor Manager [EMAIL PROTECTED] .EDU 04/04/2003 02:19 PM Please respond to ADSM: Dist Stor Manager How are your disk pools? Full or still have room? What is your tape status? Do you have other communications problems? (Slow printing, slow terminals, slow FTP...) -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED] Sent: Friday, 04 April 2003 12:13 PM To: [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- Information from the mail header --- Sender: ADSM: Dist Stor Manager [EMAIL PROTECTED] Poster: Joni Moyer [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- - I am running TSM 4.1.5 on OS/390 2.10 I am getting calls from clients about slow backups, restores,etc. on TSM. I went out and looked and there are 100 sessions going for different clients, but they aren't ending. Some are in the IDLEW status, but a majority of them are in the run status. What should I look into? Thanks! Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: HELP!!! EMERGENCY!!!
Joni, I had this happen on TSM 4.2.1.9 a long time ago, my only option was to bounce the TSM server. I did not have as many clients active only 10 or so. Sorry but that was my quickest fix at that time. No other way to free these sessions up that I know of. - Brian Brian L. Nick Systems Technician - Storage Solutions The Phoenix Companies Inc. 100 Bright Meadow Blvd Enfield CT. 06082-1900 E-MAIL: [EMAIL PROTECTED] PHONE: (860)403-2281 Joni Moyer [EMAIL PROTECTED]To: [EMAIL PROTECTED] HMARK.COM cc: Sent by: ADSM:Subject: Re: HELP!!! EMERGENCY!!! Dist Stor Manager [EMAIL PROTECTED] ST.EDU 04/04/03 02:22 PM Please respond to ADSM: Dist Stor Manager Disk pools aren't full. Tape drives are available. There are plenty of tapes. The db is 88% full. Recovery log is 7% utilized. I did have the following message, but it said the server would continue: ANR0390W A server database deadlock situation has been encountered: lock request for transaction 0:6.517663675 will be denied to resolve the deadlock. ANRD DFUTIL(734): Lock acquisition (isLock) failed for bitfile root 0. ANR2183W AFMOVE3551: Transaction 0:6.517663675 was aborted. Any suggestions!!! I really need help! Should I contact support? If I do it'll take 3 hours to get a response... Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338 Kai Hintze [EMAIL PROTECTED] To: [EMAIL PROTECTED] TSONS.COM cc: Sent by: ADSM: Subject: Re: HELP!!! EMERGENCY!!! Dist Stor Manager [EMAIL PROTECTED] .EDU 04/04/2003 02:19 PM Please respond to ADSM: Dist Stor Manager How are your disk pools? Full or still have room? What is your tape status? Do you have other communications problems? (Slow printing, slow terminals, slow FTP...) -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED] Sent: Friday, 04 April 2003 12:13 PM To: [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- Information from the mail header --- Sender: ADSM: Dist Stor Manager [EMAIL PROTECTED] Poster: Joni Moyer [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- - I am running TSM 4.1.5 on OS/390 2.10 I am getting calls from clients about slow backups, restores,etc. on TSM. I went out and looked and there are 100 sessions going for different clients, but they aren't ending. Some are in the IDLEW status, but a majority of them are in the run status. What should I look into? Thanks! Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: HELP!!! EMERGENCY!!!
I have seen a deadlock situation when trying to delete objects on the same FS. Are any of your sessions trying to backup the same fs names/files? Have you tried killing off the sessions? Is 100 client sessions at one time common? Are there any helpful messages in the individual client's dsmsched log? Have you have any network interruptions lately? Sometimes the big hammer approach is quickest (killing the server) but doesn't always point to the cause. Good luck, Mary Vraspir -Original Message- From: Salak Juraj [mailto:[EMAIL PROTECTED] Sent: Friday, April 04, 2003 1:22 PM To: [EMAIL PROTECTED] Subject: AW: HELP!!! EMERGENCY!!! quick ideas: Maybe your primary storage pool is full and the sessions are in MediaWait state? look at Q ACTL on server and dsmsched.log and dsmerror.log on clients Juraj -Ursprüngliche Nachricht- Von: Joni Moyer [mailto:[EMAIL PROTECTED] Gesendet: Freitag, 04. April 2003 21:13 An: [EMAIL PROTECTED] Betreff: HELP!!! EMERGENCY!!! I am running TSM 4.1.5 on OS/390 2.10 I am getting calls from clients about slow backups, restores,etc. on TSM. I went out and looked and there are 100 sessions going for different clients, but they aren't ending. Some are in the IDLEW status, but a majority of them are in the run status. What should I look into? Thanks! Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: HELP!!! EMERGENCY!!!
If you want support to look at the problem, take a svc dump. Then cycle your tsm server to clear the deadlock. David [EMAIL PROTECTED] 04/04/03 02:22PM Disk pools aren't full. Tape drives are available. There are plenty of tapes. The db is 88% full. Recovery log is 7% utilized. I did have the following message, but it said the server would continue: ANR0390W A server database deadlock situation has been encountered: lock request for transaction 0:6.517663675 will be denied to resolve the deadlock. ANRD DFUTIL(734): Lock acquisition (isLock) failed for bitfile root 0. ANR2183W AFMOVE3551: Transaction 0:6.517663675 was aborted. Any suggestions!!! I really need help! Should I contact support? If I do it'll take 3 hours to get a response... Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338 Kai Hintze [EMAIL PROTECTED] To: [EMAIL PROTECTED] TSONS.COM cc: Sent by: ADSM: Subject: Re: HELP!!! EMERGENCY!!! Dist Stor Manager [EMAIL PROTECTED] .EDU 04/04/2003 02:19 PM Please respond to ADSM: Dist Stor Manager How are your disk pools? Full or still have room? What is your tape status? Do you have other communications problems? (Slow printing, slow terminals, slow FTP...) -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED] Sent: Friday, 04 April 2003 12:13 PM To: [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- Information from the mail header --- Sender: ADSM: Dist Stor Manager [EMAIL PROTECTED] Poster: Joni Moyer [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- - I am running TSM 4.1.5 on OS/390 2.10 I am getting calls from clients about slow backups, restores,etc. on TSM. I went out and looked and there are 100 sessions going for different clients, but they aren't ending. Some are in the IDLEW status, but a majority of them are in the run status. What should I look into? Thanks! Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: HELP!!! EMERGENCY!!!
Joni, I had left you a v-mail, thought you would like a quick response. We also run OS/390 r2.10, and have several TSM servers. When we were still on 4.1 code we had this happen frequently. IBM was never able to find a cause or solution. However, when we upgraded to 4.2 and now 5.1 the problem went away. The only thing we could do when this happens, was as Brian said, bounce the server. Some of the symptoms you should see, processes will 'hang' also, they will show 'current file' as a size of 0 bytes, some sessions may also show this. The TSM server will continue to accept sessions and start new processes, it will not move any data however, and the sessions and processes will hang around until you stop/start the server. You will want to upgrade your code ASAP to avoid this. Sorry for no better answer, but this is the best we could ever come up with. Ryan Miller Principal Financial Group Tivoli Certified Consultant Tivoli Storage Manager v4.1 -Original Message- From: Brian L. Nick [mailto:[EMAIL PROTECTED] Sent: Friday, April 04, 2003 1:25 PM To: [EMAIL PROTECTED] Subject: Re: HELP!!! EMERGENCY!!! Joni, I had this happen on TSM 4.2.1.9 a long time ago, my only option was to bounce the TSM server. I did not have as many clients active only 10 or so. Sorry but that was my quickest fix at that time. No other way to free these sessions up that I know of. - Brian Brian L. Nick Systems Technician - Storage Solutions The Phoenix Companies Inc. 100 Bright Meadow Blvd Enfield CT. 06082-1900 E-MAIL: [EMAIL PROTECTED] PHONE: (860)403-2281 Joni Moyer [EMAIL PROTECTED]To: [EMAIL PROTECTED] HMARK.COM cc: Sent by: ADSM:Subject: Re: HELP!!! EMERGENCY!!! Dist Stor Manager [EMAIL PROTECTED] ST.EDU 04/04/03 02:22 PM Please respond to ADSM: Dist Stor Manager Disk pools aren't full. Tape drives are available. There are plenty of tapes. The db is 88% full. Recovery log is 7% utilized. I did have the following message, but it said the server would continue: ANR0390W A server database deadlock situation has been encountered: lock request for transaction 0:6.517663675 will be denied to resolve the deadlock. ANRD DFUTIL(734): Lock acquisition (isLock) failed for bitfile root 0. ANR2183W AFMOVE3551: Transaction 0:6.517663675 was aborted. Any suggestions!!! I really need help! Should I contact support? If I do it'll take 3 hours to get a response... Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338 Kai Hintze [EMAIL PROTECTED] To: [EMAIL PROTECTED] TSONS.COM cc: Sent by: ADSM: Subject: Re: HELP!!! EMERGENCY!!! Dist Stor Manager [EMAIL PROTECTED] .EDU 04/04/2003 02:19 PM Please respond to ADSM: Dist Stor Manager How are your disk pools? Full or still have room? What is your tape status? Do you have other communications problems? (Slow printing, slow terminals, slow FTP...) -Original Message- From: Joni Moyer [mailto:[EMAIL PROTECTED] Sent: Friday, 04 April 2003 12:13 PM To: [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- Information from the mail header --- Sender: ADSM: Dist Stor Manager [EMAIL PROTECTED] Poster: Joni Moyer [EMAIL PROTECTED] Subject: HELP!!! EMERGENCY!!! -- - I am running TSM 4.1.5 on OS/390 2.10 I am getting calls from clients about slow backups, restores,etc. on TSM. I went out and looked and there are 100 sessions going for different clients, but they aren't ending. Some are in the IDLEW status, but a majority of them are in the run status. What should I look into? Thanks! Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
Re: HELP!!! EMERGENCY!!!
Joni If you have any processes using server-to-server communications running concurrently with the backups, try cancelling the processes. This in itself will probably not free things up but if you then try cancel all the sessions created by the server-to-server communications, things may resume. You may have to do this on both source and destination servers. Worked for me in the past on 3.7 and 4.2. Regards Neil Schofield Visit our web site to find out about our award winning Cool Schools community campaign at http://www.yorkshirewater.com/yorkshirewater/schools.html The information in this e-mail is confidential and may also be legally privileged. The contents are intended for recipient only and are subject to the legal notice available at http://www.keldagroup.com/email.htm Yorkshire Water Services Limited Registered Office Western House Halifax Road Bradford BD6 2SZ Registered in England and Wales No 2366682
Re: HELP!!! EMERGENCY!!!
Joni, I have this happen with TSM 4.1.5 on AIX. You may or may not like the answer. Stop and restart the TSM server. This is the only way to clear up the sessions. Sias Get your own 800 number Voicemail, fax, email, and a lot more http://www.ureach.com/reg/tag On, Joni Moyer ([EMAIL PROTECTED]) wrote: I am running TSM 4.1.5 on OS/390 2.10 I am getting calls from clients about slow backups, restores,etc. on TSM. I went out and looked and there are 100 sessions going for different clients, but they aren't ending. Some are in the IDLEW status, but a majority of them are in the run status. What should I look into? Thanks! Joni Moyer Systems Programmer [EMAIL PROTECTED] (717)975-8338
