Re: Dsmcad listening port

[Efim]

HI,
CAD opens random port because the option WEBPORT has default value "0 0" and 
CAD randomly assign a free TCPport (the first parameter for CAD, the second for 
WEB client).
I think it’s impossible to prevent this.
As workaround you can set fixed port(s) and close it using firewall.
Example: WEBPORT 55000 0 or WEBPORT 55000 55001
Efim


> 18 дек. 2015 г., в 12:29, Henrik Ahlgren  написал(а):
> 
> Can someone explain why the Client Acceptor Daemon (dsmcad) opens
> a random port ("ANS3000I TCP/IP communications available on port X")
> when:
> 
> - SCHEDMODE is set to POLLING
> - MANAGEDSERVICES is set to SCHEDULE
> 
> How do you configure a TSM client to *never* listen to *any* port, but
> still using dsmcad to preserve memory etc.
> 
> To me this behaviour seems highly insecure and not clearly documented.

Dsmcad listening port

[Henrik Ahlgren]

Can someone explain why the Client Acceptor Daemon (dsmcad) opens
a random port ("ANS3000I TCP/IP communications available on port X")
when:

- SCHEDMODE is set to POLLING
- MANAGEDSERVICES is set to SCHEDULE

How do you configure a TSM client to *never* listen to *any* port, but
still using dsmcad to preserve memory etc.

To me this behaviour seems highly insecure and not clearly documented.

Re: Node replication information?

[Henrik Ahlgren]

On Fri, Dec 18, 2015 at 07:42:55AM -0800, Robert Clark wrote:
> I think I'll be running some basic network performance tests
> first. The throughput I'm seeing doesn't appear to match people's
> glowing descriptions of how big the pipes are.

If you have a decent network (10 Gbit), your problem might not be
network performance alone. Can your storage pools handle the
load? Are you simultanously running things like reclamations on the
same disks?

Tuning ReplBatchSize and ReplSizeThresh might also help, and I assume
you've already maxed out TCPWindowSize.

Re: Node replication information?

[Robert Clark]

Thank you for your gracious response.

To demonstrate just how long its been, I forgot to mention that I'm working 
with TSM 7.1.1, on some AIX (6.1 I believe) and RedHat on Intel (7.2?) servers.

I think I'll be running some basic network performance tests first. The 
throughput I'm seeing doesn't appear to match people's glowing descriptions of 
how big the pipes are.

Thanks,
[RC]


On Dec 18, 2015, at 6:34 AM, "Vandeventer, Harold [OITS]" 
 wrote:

> Welcome back Robert.  Lots of changes from the TSM 5 days.
> 
> I don't have any links to docs, but my experience has been...
> 
> I'm running Windows, 32 GB RAM, TSM 6.3.4.300.
> 
> I run 40 sessions on each Replication process.
> 
> The process is replicating a group that may have several members, 2 up to 15 
> or 20; semi-random on how I assigned.
> 
> I have a script that contains several REPLICATE NODE  commands.  
> Each command runs one group, with WAIT=YES, then another, with an exception 
> described next.
> 
> I have a group that has nodes with a very large numbers of files for those 
> nodes, thus the time required to determine what has to be migrated is long.  
> The amount of data to replicate isn't huge, just the time required to 
> inventory the status on a zillion files.
> 
> Therefore, that group is first in the script, with WAIT=NO, to immediately 
> start the next group in the script.  Subsequent replicate groups all have 
> WAIT=YES.
> 
> The only performance issue that I've seen as an issue is monitoring bandwidth 
> on the link between the source and target servers.  I've watched Network 
> stats in Windows Task Manager and depending on how many replication processes 
> are running, the impact on the bandwidth goes up.  I haven't seen any hits on 
> CPU or RAM.
> 
> 
> 
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
> Robert Clark
> Sent: Thursday, December 17, 2015 4:39 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: [ADSM-L] Node replication information?
> 
> After a long hiatus from working with TSM, I am back to working with it 
> again. (I left during the TSM 5 timeframe.)
> 
> 
> 
> Does anyone have pointers to docs useful in figuring out how node replication 
> determines how much to attempt to do in parallel, or general information on 
> how to troubleshoot its performance.
> 
> 
> 
> Thanks,
> 
> [RC]
> 
> [Confidentiality notice:]
> ***
> This e-mail message, including attachments, if any, is intended for the
> person or entity to which it is addressed and may contain confidential
> or privileged information.  Any unauthorized review, use, or disclosure
> is prohibited.  If you are not the intended recipient, please contact
> the sender and destroy the original message, including all copies,
> Thank you.
> ***

Re: Dsmcad listening port

[Henrik Ahlgren]

On Fri, Dec 18, 2015 at 12:40:46PM +0300, Efim wrote:

> CAD opens random port because the option WEBPORT has default value "0 0" and 
> CAD randomly assign a free TCPport (the first parameter for CAD, the second 
> for WEB client).
> I think it’s impossible to prevent this.
> As workaround you can set fixed port(s) and close it using firewall.
> Example: WEBPORT 55000 0 or WEBPORT 55000 55001

Am I the only one that finds this design totally unacceptable? If
you're not using the webclient functionality and are only using
schedmode polling, I don't see any reason why dsmcad (often running as
root, so the security implications are obvious) should listen to a
network port. Perhaps there is something I am not aware of?

One might think that setting tcpclientaddress to 127.0.0.1 (localhost)
would somewhat migitate this, but no - it does not have any effect if
you are not using schedmode prompted. Yes, of course it is always
possible to use host-based firewalls to close the ports, but it is a
workaround that really should not be necessary.

Re: Node replication information?

[Vandeventer, Harold [OITS]]

Welcome back Robert.  Lots of changes from the TSM 5 days.

I don't have any links to docs, but my experience has been...

I'm running Windows, 32 GB RAM, TSM 6.3.4.300.

I run 40 sessions on each Replication process.

The process is replicating a group that may have several members, 2 up to 15 or 
20; semi-random on how I assigned.

I have a script that contains several REPLICATE NODE  commands.  
Each command runs one group, with WAIT=YES, then another, with an exception 
described next.

I have a group that has nodes with a very large numbers of files for those 
nodes, thus the time required to determine what has to be migrated is long.  
The amount of data to replicate isn't huge, just the time required to inventory 
the status on a zillion files.

Therefore, that group is first in the script, with WAIT=NO, to immediately 
start the next group in the script.  Subsequent replicate groups all have 
WAIT=YES.

The only performance issue that I've seen as an issue is monitoring bandwidth 
on the link between the source and target servers.  I've watched Network stats 
in Windows Task Manager and depending on how many replication processes are 
running, the impact on the bandwidth goes up.  I haven't seen any hits on CPU 
or RAM.



-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Robert 
Clark
Sent: Thursday, December 17, 2015 4:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] Node replication information?

After a long hiatus from working with TSM, I am back to working with it again. 
(I left during the TSM 5 timeframe.)



Does anyone have pointers to docs useful in figuring out how node replication 
determines how much to attempt to do in parallel, or general information on how 
to troubleshoot its performance.



Thanks,

[RC]

[Confidentiality notice:]
***
This e-mail message, including attachments, if any, is intended for the
person or entity to which it is addressed and may contain confidential
or privileged information.  Any unauthorized review, use, or disclosure
is prohibited.  If you are not the intended recipient, please contact
the sender and destroy the original message, including all copies,
Thank you.
***

Re: Dsmcad listening port

[Mike De Gasperis]

You don't need to run the dsmcad technically speaking. You can just run a dsmc 
sched daemon or service. 

> On Dec 18, 2015, at 10:47 AM, Henrik Ahlgren  wrote:
> 
>> On Fri, Dec 18, 2015 at 12:40:46PM +0300, Efim wrote:
>> 
>> CAD opens random port because the option WEBPORT has default value "0 0" and 
>> CAD randomly assign a free TCPport (the first parameter for CAD, the second 
>> for WEB client).
>> I think it’s impossible to prevent this.
>> As workaround you can set fixed port(s) and close it using firewall.
>> Example: WEBPORT 55000 0 or WEBPORT 55000 55001
> 
> Am I the only one that finds this design totally unacceptable? If
> you're not using the webclient functionality and are only using
> schedmode polling, I don't see any reason why dsmcad (often running as
> root, so the security implications are obvious) should listen to a
> network port. Perhaps there is something I am not aware of?
> 
> One might think that setting tcpclientaddress to 127.0.0.1 (localhost)
> would somewhat migitate this, but no - it does not have any effect if
> you are not using schedmode prompted. Yes, of course it is always
> possible to use host-based firewalls to close the ports, but it is a
> workaround that really should not be necessary.

Re: Node replication information?

[Henrik Ahlgren]

On Fri, Dec 18, 2015 at 02:34:43PM +, Vandeventer, Harold [OITS] wrote:
> The process is replicating a group that may have several members, 2
> up to 15 or 20; semi-random on how I assigned.

One reason to do this is that (at least on 6.3) the nodes are locked against
any changes (update node etc.) during replication. If there are
hundreds of nodes and replication takes hours, this is very annoying.
But of course it would be nice if you could just replicate all nodes
in one go without maintaning node groups.

Re: Node replication information?

[Nixon, Charles D. (David)]

We stagger our replication during the morning, to avoid locking the node too 
long.  Some things that we have seen:

1.  If the node has a current backup session running and you attempt to 
replicate it, the backup session gets killed since replication takes priority.  
This has been a problem with our TDP nodes that backup during the day.

2.  We have a long running backup job (NODE LONG) that is taking 20 hours to 
complete.  We don't collocate and some data may be written to a tape/file from 
another node (NODE NORMAL) in the evening.  When issuing the replicate NODE 
NORMAL, it waits for the tape that is currently tied up by NODE LONG.  So we 
have 20-30 replication sessions tied up by one long backup job.

This could be fixed with some colocation changes but we are going to container 
pools very soon and this problem gets a work-around by using protect container 
command.



---
David Nixon
System Programmer II
Technology Services Group
Carilion Clinic
451 Kimball Ave.
Roanoke, VA 24015
Phone: 540-224-3903
cdni...@carilionclinic.org

Our mission: Improve the health of the communities we serve.




From: ADSM: Dist Stor Manager [ADSM-L@VM.MARIST.EDU] on behalf of Henrik 
Ahlgren [pa...@seestieto.com]
Sent: Friday, December 18, 2015 10:54 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Node replication information?

On Fri, Dec 18, 2015 at 07:42:55AM -0800, Robert Clark wrote:
> I think I'll be running some basic network performance tests
> first. The throughput I'm seeing doesn't appear to match people's
> glowing descriptions of how big the pipes are.

If you have a decent network (10 Gbit), your problem might not be
network performance alone. Can your storage pools handle the
load? Are you simultanously running things like reclamations on the
same disks?

Tuning ReplBatchSize and ReplSizeThresh might also help, and I assume
you've already maxed out TCPWindowSize.



Notice: The information and attachment(s) contained in this communication are 
intended for the addressee only, and may be confidential and/or legally 
privileged. If you have received this communication in error, please contact 
the sender immediately, and delete this communication from any computer or 
network system. Any interception, review, printing, copying, re-transmission, 
dissemination, or other use of, or taking of any action upon this information 
by persons or entities other than the intended recipient is strictly prohibited 
by law and may subject them to criminal or civil liability. Carilion Clinic 
shall not be liable for the improper and/or incomplete transmission of the 
information contained in this communication or for any delay in its receipt.

Re: Dsmcad listening port

[Henrik Ahlgren]

On Fri, Dec 18, 2015 at 11:00:46AM -0500, Mike De Gasperis wrote:
> You don't need to run the dsmcad technically speaking. You can just
> run a dsmc sched daemon or service.

Running the scheduler on-demand frees up the memory used by backup
jobs when it is no longer needed. Often the amount of memory used is
non-trivial. And long-running schedulers leak memory, at least with
some versions. Dsmcad solves this nicely, there is just this design
flaw, which I'd patch immediately if TSM wasn't proprietary
software. If the inbound port serves no purpose in polling-mode-only
configuration, there is no excuse in listening to it.

AFAIK, IBM recommends using CAS as best practice and Linux packages
install dsmcad service by default, so probably majority of users runs
it that way.

VE 7.1.4 install issues

[Schaub, Steve]

Anyone seen this?  I'm testing a fresh install of TSM for VE 7.1.4 on a newly 
built, never used Win2012R2 server.  running setup, using "typical" install.  
It looks like it is at the end of the datamover part of the install when I get 
a pop windows stating "Another instance of this setup is already running.  
Please wait for the other instance to finish and then try again."  When I click 
ok, I get "The wizard was interrupted before Data Protection for VMWare suite 
could be completely installed."  I have tried multiple reboots.  It looks like 
only the BA client was installed.  I don't see any other setup or install 
applications in task manager.

Thanks,

Steve Schaub
Systems Engineer II, Backup/Recovery
Blue Cross Blue Shield of Tennessee


--
Please see the following link for the BlueCross BlueShield of Tennessee E-mail 
disclaimer:  http://www.bcbst.com/email_disclaimer.shtm