If you're using your Isilon for NFS backend, I would make the following
recommendations:
* Use multiple mountpoints to the Isilon from your TSM server (e.g. Isilon
Client). I would use IP addresses or direct IP connection to the
underlying Isilon nodes to spread the load out against the entire Isilon
cluster (manually).
* Bypass SmartConnect, if that's a licensed thing for you. Don't worry
about failing over your mount points using smartconnect for this. In the
worst case emergencies, you're only impacting backups processing and can
easily reconfigure the mountpoint manually in case of a node outage. If
it's that important, then use a smartconnect policy that makes sense for
TSM deployments (probably round-robin, not one of those fancier
load-balancing options).
* Use many mount points and directories on the TSM server in the devclass.
I have 16 filesystems (aka mounts) per each TSM server devclass.
* Use subdirectories, as suggested in a earlier post, not whole new
exports, although that should work too, to separate the mount points so you
don't run into NFS client locking issues.
* Predefine your file volume sizes. I think directory container pools
might be a better option to use in the future, but that forces you into
dedup, IIRC.
Thanks!
SF
On Mon, May 14, 2018 at 10:25 AM, Skylar Thompson wrote:
> No, this is an NFS server setting, but I'm not sure that it's tunable on
> Isilon. On Linux and Solaris, it defaults to some very low value, which is
> fine for sequential I/O but really slows down random I/O. On Linux,
> RPCNFSDCOUNT can be tuned from the default of 8 to 512, which is fine as
> long as the NFS server is running on dedicated hardware.
>
> On Mon, May 14, 2018 at 10:06:31AM -0400, Zoltan Forray wrote:
> > We did some quick research and "NFS thread" controls don't apply in our
> > situation and can't be set. Or are you referring to the mountlimit value
> > for the devclass?
> >
> > On Mon, May 14, 2018 at 9:31 AM, Skylar Thompson wrote:
> >
> > > This sounds pretty good to me. If you can, I would boost your NFS
> thread
> > > count past the number of CPUs that you have, since a lot of NFS is just
> > > waiting for the disks to respond. You still need a thread for that,
> but it
> > > won't consume much CPU.
> > >
> > > On Mon, May 14, 2018 at 08:27:27AM -0400, Zoltan Forray wrote:
> > > > Very interesting. This supports my idea on how I want to layout the
> > > > new/replacement server. The old server is only 16-threads and
> certainly
> > > > could not handle dedup (we can't afford any appliances like DD)
> since it
> > > is
> > > > bucking under the current backups traffic. The new server has
> 72-threads
> > > as
> > > > well as 100TB internal disk. My idea is to use the fast internal
> 100TB
> > > > disk for inbound traffic and deduping and use the 200TB NFS/ISILON
> for
> > > > nextstoragepool (trying to get completely off 3592-tape storage for
> > > onsite
> > > > backups). Plus the DB will be on SSD.
> > > >
> > > > Any thoughts on this configuration?
> > > >
> > > > On Sun, May 13, 2018 at 8:38 PM, Harris, Steven <
> > > > steven.har...@btfinancialgroup.com> wrote:
> > > >
> > > > > Zoltan
> > > > >
> > > > > I have a similar issue TSM 7.1.1.300 AIX -> Data Domain. Have dual
> > > 10Gb
> > > > > links, but can only get ~4000 writes/sec and 120MB/sec throughput.
> AIX
> > > only
> > > > > supports NFS3, and as others have pointed out in this forum
> recently,
> > > the
> > > > > stack does not have a good reputation.
> > > > >
> > > > > I'm finding that the heavy NFS load has other knock on effects,
> e.g.
> > > > > TSMManager keeps reporting the instance offline when it's very
> busy as
> > > it
> > > > > gets a network error on some of its regular queries, but these
> work ok
> > > when
> > > > > load is light. Also getting a lot of Severed/reconnected sessions.
> > > > > CPU/IO/Paging are not a problem.
> > > > >
> > > > > Cheers
> > > > >
> > > > > Steve
> > > > >
> > > > > Steven Harris
> > > > > TSM Admin/Consultant
> > > > > Canberra Australia
> > > > >
> > > > > -Original Message-
> > > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
> Behalf
> > > Of
> > > > > Zoltan Forray
> > > > > Sent: Saturday, 12 May 2018 1:39 AM
> > > > > To: ADSM-L@VM.MARIST.EDU
> > > > > Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues
> > > > >
> > > > > Folks,
> > > > >
> > > > > ISP 7.1.7.300 on RHEL 6 10G connectivity
> > > > >
> > > > > We need some guidance on trying to figure out why ISP/TSM write
> > > perform to
> > > > > ISILON storage (via FILE DEVCLASS) is so horrible.
> > > > >
> > > > > We recently attached 200TB of ISILON storage to this server so we
> could
> > > > > empty the 36TB of onboard disk drives to move this server to new
> > > hardware.
> > > > >
> > > > > However, per my OS and SAN folks, we are only seeing 1Gbs level of
> data
> > > > > movement from the ISP server. Doing a regular file copy to this
> same
