Hi Stefan, There is no noticable impact on the TSM performance when we use the tsmdiskperf tool on the database volumes. We recently discovered the issue seems to be related to the TDP for Oracle clients. When none are running the server response is OK, even during a reasonable load. But as soon as the TDP for Oracle clients start kicking in, the server becomes slow. We brought it to the attention of the developers, but no response yet... Thanks for your help!
Kind regards, Eric van Loon Air France/KLM Storage & Backup -----Original Message----- From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan Folkerts Sent: maandag 26 augustus 2019 10:39 To: ADSM-L@VM.MARIST.EDU Subject: Re: TSM server performance continuing Eric, What happens when you benchmark the DB volumes using the tool provided with the blueprints on an idle system, does the system also slow down with commands such as q stgpool or does it stay fast when the benchmark is running on all volumes? Also, what kind of a result does the benchmark give you and what do you use for your database storage? Regards, Stefan On Thu, Aug 22, 2019 at 3:36 PM PAC Brion Arnaud <arnaud.br...@panalpina.com> wrote: > Hi Eric, > > These seem to be default values when installing Red Hat Enterprise > Linux Server 7.3. > I checked several other machines, all are having the same value. > > And following page > https://stackoverflow.com/questions/55428812/how-the-values-of-kernel- > parameters-are-defined > seems to confirm my statement. > > Cheers. > > Arnaud > > > > ********************************************************************** > ********************************************************** > Backup and Recovery Systems Administrator Panalpina Management Ltd., > Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 > Basel/CH > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 > Direct: +41 (61) 226 19 78 > e-mail: arnaud.br...@panalpina.com > This electronic message transmission contains information from > Panalpina and is confidential or privileged. > This information is intended only for the person (s) named above. If > you are not the intended recipient, any disclosure, copying, > distribution or use or any other action based on the contents of this > information is strictly prohibited. > > If you receive this electronic transmission in error, please notify > the sender by e-mail, telephone or fax at the numbers listed above. Thank you. > > ********************************************************************** > ********************************************************** > www.panalpina.com > > > > -----Original Message----- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf > Of Loon, Eric van (ITOP NS) - KLM > Sent: Thursday, August 22, 2019 2:31 PM > To: ADSM-L@VM.MARIST.EDU > Subject: Re: TSM server performance continuing > > Hi Arnoud, > > I do not understand why your max seg size (kbytes) = 18014398509465599 > and max total shared memory (kbytes) = 18014397435740096 are so huge. > If you have 256 GB RAM, like I do, I would expect max seg size > (kbytes) = > 268435456 and max total shared memory (kbytes) = 268435456. > Thanks for your help! > > Kind regards, > Eric van Loon > Air France/KLM Storage & Backup > > > -----Original Message----- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf > Of PAC Brion Arnaud > Sent: donderdag 22 augustus 2019 09:02 > To: ADSM-L@VM.MARIST.EDU > Subject: Re: TSM server performance continuing > > Hi Eric, > > Running 4 TSM servers at 8.1.6.1 (PowerLinux servers with 256 GB RAM), > and making use of directory-container storage pools only. > No server performance or responsiveness issues here anymore, since we > changed our storage to make use of IBM Isilon. > > Here my values : > > vmstat 1 > procs -----------memory---------- ---swap-- -----io---- -system-- > ------cpu----- > r b swpd free buff cache si so bi bo in cs us sy id > wa st > 13 0 1420032 1152064 1429248 243848064 0 0 7500 3766 0 0 > 24 3 61 11 0 > 13 1 1420032 1046016 1428288 243943488 0 0 614256 80 22427 42342 > 50 4 39 7 0 > 12 0 1420032 1144640 1426688 243861952 0 0 542840 0 19109 33340 > 49 2 43 6 0 > 12 4 1420032 883904 1427136 244133824 0 0 577108 0 19777 34954 > 49 2 44 6 0 > 12 1 1420032 936576 1425344 244072768 0 0 613152 372 21110 37559 > 49 2 43 6 0 > 11 5 1420032 1028096 1425600 243977344 0 0 549024 0 24599 47296 > 48 2 42 8 0 > 12 0 1420032 1028032 1424832 243983424 0 0 566180 16 21693 39976 > 49 2 43 6 0 > > > ipcs -l > > ------ Messages Limits -------- > max queues system wide = 250880 > max size of message (bytes) = 65536 > default max size of queue (bytes) = 65536 > > ------ Shared Memory Limits -------- > max number of segments = 62720 > max seg size (kbytes) = 18014398509465599 max total shared memory > (kbytes) = 18014397435740096 min seg size (bytes) = 1 > > ------ Semaphore Limits -------- > max number of arrays = 62720 > max semaphores per array = 250 > max semaphores system wide = 256000 > max ops per semop call = 32 > semaphore max value = 32767 > > > Specific tuning done : disabling CPU virtualization (led to sever > hangs when active) > > ppc64_cpu --smt=off > > > Cheers. > > Arnaud > > > ********************************************************************** > ********************************************************** > Backup and Recovery Systems Administrator Panalpina Management Ltd., > Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002 > Basel/CH > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01 > Direct: +41 (61) 226 19 78 > e-mail: arnaud.br...@panalpina.com > This electronic message transmission contains information from > Panalpina and is confidential or privileged. > This information is intended only for the person (s) named above. If > you are not the intended recipient, any disclosure, copying, > distribution or use or any other action based on the contents of this > information is strictly prohibited. > > If you receive this electronic transmission in error, please notify > the sender by e-mail, telephone or fax at the numbers listed above. Thank you. > > ********************************************************************** > ********************************************************** > www.panalpina.com > > > -----Original Message----- > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf > Of Loon, Eric van (ITOP NS) - KLM > Sent: Wednesday, August 21, 2019 5:39 PM > To: ADSM-L@VM.MARIST.EDU > Subject: TSM server performance continuing > > Hi guys, > > A few weeks ago I already wrote about the severe performance issues we > have with our TSM 7.1 servers. In the 'old days' we used to back up > our clients to TSM 6.3 servers with Data Domains attached. Smaller > clients backed up through the LAN, large ones through the SAN. > Our newer servers use LAN-only with directory containers and the > performance of these servers really sucks. Setting up a session takes > sometimes almost one minute and a q stg also takes 30 to 50 seconds. I > noticed that performance is OK when there are no TDP for Oracle > sessions running, but as soon as they are started the performance > starts to drop drastically. > We are really lost on where to look for the cause. I sent numerous > logs and traces to IBM, but I guess they are out of ideas too since I > don't hear anything back from them lately. The only thing is that > supports notices delays in DB2, but they don't know why... > What I noticed on my TSM server is that as soon as there is a load on > the server, the blocked queue starts to rise. I would like to know if > that's something to focus on or not. > Can some of you please run the "vmstat 1" command on their Linux > server (preferably one with directory containers too) and let me know > if you too see values other than 0 in the B column? > Thank you very much for your help in advance! > > Kind regards, > Eric van Loon > Air France/KLM Storage & Backup > ******************************************************** > For information, services and offers, please visit our web site: > http://www.klm.com. This e-mail and any attachment may contain > confidential and privileged material intended for the addressee only. > If you are not the addressee, you are notified that no part of the > e-mail or any attachment may be disclosed, copied or distributed, and > that any other action related to this e-mail or attachment is strictly > prohibited, and may be unlawful. If you have received this e-mail by > error, please notify the sender immediately by return e-mail, and delete this > message. > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or > its employees shall not be liable for the incorrect or incomplete > transmission of this e-mail or any attachments, nor responsible for any delay > in receipt. > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal > Dutch > Airlines) is registered in Amstelveen, The Netherlands, with > registered number 33014286 > ******************************************************** > ******************************************************** > For information, services and offers, please visit our web site: > http://www.klm.com. This e-mail and any attachment may contain > confidential and privileged material intended for the addressee only. > If you are not the addressee, you are notified that no part of the > e-mail or any attachment may be disclosed, copied or distributed, and > that any other action related to this e-mail or attachment is strictly > prohibited, and may be unlawful. If you have received this e-mail by > error, please notify the sender immediately by return e-mail, and delete this > message. > > Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or > its employees shall not be liable for the incorrect or incomplete > transmission of this e-mail or any attachments, nor responsible for any delay > in receipt. > Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal > Dutch > Airlines) is registered in Amstelveen, The Netherlands, with > registered number 33014286 > ******************************************************** > ******************************************************** For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message. Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt. Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286 ********************************************************
