Re: ISILON storage/FILE DEVCLASS performance issues

[Skylar Thompson]

No, this is an NFS server setting, but I'm not sure that it's tunable on
Isilon. On Linux and Solaris, it defaults to some very low value, which is
fine for sequential I/O but really slows down random I/O. On Linux,
RPCNFSDCOUNT can be tuned from the default of 8 to 512, which is fine as
long as the NFS server is running on dedicated hardware.

On Mon, May 14, 2018 at 10:06:31AM -0400, Zoltan Forray wrote:
> We did some quick research and "NFS thread" controls don't apply in our
> situation and can't be set. Or are you referring to the mountlimit value
> for the devclass?
>
> On Mon, May 14, 2018 at 9:31 AM, Skylar Thompson  wrote:
>
> > This sounds pretty good to me. If you can, I would boost your NFS thread
> > count past the number of CPUs that you have, since a lot of NFS is just
> > waiting for the disks to respond. You still need a thread for that, but it
> > won't consume much CPU.
> >
> > On Mon, May 14, 2018 at 08:27:27AM -0400, Zoltan Forray wrote:
> > > Very interesting.  This supports my idea on how I want to layout the
> > > new/replacement server.  The old server is only 16-threads and certainly
> > > could not handle dedup (we can't afford any appliances like DD) since it
> > is
> > > bucking under the current backups traffic. The new server has 72-threads
> > as
> > > well as 100TB internal disk.  My idea is to use the fast internal 100TB
> > > disk for inbound traffic and deduping and use the 200TB NFS/ISILON for
> > > nextstoragepool (trying to get completely off 3592-tape storage for
> > onsite
> > > backups). Plus the DB will be on SSD.
> > >
> > > Any thoughts on this configuration?
> > >
> > > On Sun, May 13, 2018 at 8:38 PM, Harris, Steven <
> > > steven.har...@btfinancialgroup.com> wrote:
> > >
> > > > Zoltan
> > > >
> > > > I have a similar issue TSM 7.1.1.300 AIX -> Data Domain.  Have dual
> > 10Gb
> > > > links, but can only get ~4000 writes/sec and 120MB/sec throughput. AIX
> > only
> > > > supports NFS3, and as others have pointed out in this forum recently,
> > the
> > > > stack does not have a good reputation.
> > > >
> > > > I'm finding that the heavy NFS load has other knock on effects, e.g.
> > > > TSMManager keeps reporting the instance offline when it's very busy as
> > it
> > > > gets a network error on some of its regular queries, but these work ok
> > when
> > > > load is light.  Also getting a lot of Severed/reconnected sessions.
> > > > CPU/IO/Paging are not a problem.
> > > >
> > > > Cheers
> > > >
> > > > Steve
> > > >
> > > > Steven Harris
> > > > TSM Admin/Consultant
> > > > Canberra Australia
> > > >
> > > > -Original Message-
> > > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
> > Of
> > > > Zoltan Forray
> > > > Sent: Saturday, 12 May 2018 1:39 AM
> > > > To: ADSM-L@VM.MARIST.EDU
> > > > Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues
> > > >
> > > > Folks,
> > > >
> > > > ISP 7.1.7.300 on RHEL 6   10G connectivity
> > > >
> > > > We need some guidance on trying to figure out why ISP/TSM write
> > perform to
> > > > ISILON storage (via FILE DEVCLASS) is so horrible.
> > > >
> > > > We recently attached 200TB of ISILON storage to this server so we could
> > > > empty the 36TB of onboard disk drives to move this server to new
> > hardware.
> > > >
> > > > However, per my OS and SAN folks, we are only seeing 1Gbs level of data
> > > > movement from the ISP server.  Doing a regular file copy to this same
> > > > storage peaks at 10Gbs speeds.
> > > >
> > > > So what, if anything, are we doing wrong when it comes to configuring
> > the
> > > > storage for ISP to use?  Are there some secret
> > controls/settings/options to
> > > > tell it to use the storage at max-speeds?
> > > >
> > > > We tried changing the Est/Max capacity thinking larger files would
> > reduce
> > > > the overhead of allocating new pieces constantly.  Changed the Mount
> > Limit
> > > > to a bigger number.  Nothing has helped.
> > > >
> > > > Only thing uses the storage right now is migrations from the original
> > disk
> > > > stgpool.
> > > >
> > > >
> > > >
> > > > --
> > > > *Zoltan Forray*
> > > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon
> > > > Monitor Administrator VMware Administrator Virginia Commonwealth
> > University
> > > > UCC/Office of Technology Services www.ucc.vcu.edu zfor...@vcu.edu -
> > > > 804-828-4807 Don't be a phishing victim - VCU and other reputable
> > > > organizations will never use email to request that you reply with your
> > > > password, social security number or confidential personal information.
> > For
> > > > more details visit http://phishing.vcu.edu/
> > > >
> > > >
> > > > This message and any attachment is confidential and may be privileged
> > or
> > > > otherwise protected from disclosure. You should immediately delete the
> > > > message if you are not the intended recipient. If you have received
> > this
> > > > email by mistake please delete it from 

Re: ISILON storage/FILE DEVCLASS performance issues

[Zoltan Forray]

We did some quick research and "NFS thread" controls don't apply in our
situation and can't be set. Or are you referring to the mountlimit value
for the devclass?

On Mon, May 14, 2018 at 9:31 AM, Skylar Thompson  wrote:

> This sounds pretty good to me. If you can, I would boost your NFS thread
> count past the number of CPUs that you have, since a lot of NFS is just
> waiting for the disks to respond. You still need a thread for that, but it
> won't consume much CPU.
>
> On Mon, May 14, 2018 at 08:27:27AM -0400, Zoltan Forray wrote:
> > Very interesting.  This supports my idea on how I want to layout the
> > new/replacement server.  The old server is only 16-threads and certainly
> > could not handle dedup (we can't afford any appliances like DD) since it
> is
> > bucking under the current backups traffic. The new server has 72-threads
> as
> > well as 100TB internal disk.  My idea is to use the fast internal 100TB
> > disk for inbound traffic and deduping and use the 200TB NFS/ISILON for
> > nextstoragepool (trying to get completely off 3592-tape storage for
> onsite
> > backups). Plus the DB will be on SSD.
> >
> > Any thoughts on this configuration?
> >
> > On Sun, May 13, 2018 at 8:38 PM, Harris, Steven <
> > steven.har...@btfinancialgroup.com> wrote:
> >
> > > Zoltan
> > >
> > > I have a similar issue TSM 7.1.1.300 AIX -> Data Domain.  Have dual
> 10Gb
> > > links, but can only get ~4000 writes/sec and 120MB/sec throughput. AIX
> only
> > > supports NFS3, and as others have pointed out in this forum recently,
> the
> > > stack does not have a good reputation.
> > >
> > > I'm finding that the heavy NFS load has other knock on effects, e.g.
> > > TSMManager keeps reporting the instance offline when it's very busy as
> it
> > > gets a network error on some of its regular queries, but these work ok
> when
> > > load is light.  Also getting a lot of Severed/reconnected sessions.
> > > CPU/IO/Paging are not a problem.
> > >
> > > Cheers
> > >
> > > Steve
> > >
> > > Steven Harris
> > > TSM Admin/Consultant
> > > Canberra Australia
> > >
> > > -Original Message-
> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
> Of
> > > Zoltan Forray
> > > Sent: Saturday, 12 May 2018 1:39 AM
> > > To: ADSM-L@VM.MARIST.EDU
> > > Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues
> > >
> > > Folks,
> > >
> > > ISP 7.1.7.300 on RHEL 6   10G connectivity
> > >
> > > We need some guidance on trying to figure out why ISP/TSM write
> perform to
> > > ISILON storage (via FILE DEVCLASS) is so horrible.
> > >
> > > We recently attached 200TB of ISILON storage to this server so we could
> > > empty the 36TB of onboard disk drives to move this server to new
> hardware.
> > >
> > > However, per my OS and SAN folks, we are only seeing 1Gbs level of data
> > > movement from the ISP server.  Doing a regular file copy to this same
> > > storage peaks at 10Gbs speeds.
> > >
> > > So what, if anything, are we doing wrong when it comes to configuring
> the
> > > storage for ISP to use?  Are there some secret
> controls/settings/options to
> > > tell it to use the storage at max-speeds?
> > >
> > > We tried changing the Est/Max capacity thinking larger files would
> reduce
> > > the overhead of allocating new pieces constantly.  Changed the Mount
> Limit
> > > to a bigger number.  Nothing has helped.
> > >
> > > Only thing uses the storage right now is migrations from the original
> disk
> > > stgpool.
> > >
> > >
> > >
> > > --
> > > *Zoltan Forray*
> > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon
> > > Monitor Administrator VMware Administrator Virginia Commonwealth
> University
> > > UCC/Office of Technology Services www.ucc.vcu.edu zfor...@vcu.edu -
> > > 804-828-4807 Don't be a phishing victim - VCU and other reputable
> > > organizations will never use email to request that you reply with your
> > > password, social security number or confidential personal information.
> For
> > > more details visit http://phishing.vcu.edu/
> > >
> > >
> > > This message and any attachment is confidential and may be privileged
> or
> > > otherwise protected from disclosure. You should immediately delete the
> > > message if you are not the intended recipient. If you have received
> this
> > > email by mistake please delete it from your system; you should not
> copy the
> > > message or disclose its content to anyone.
> > >
> > > This electronic communication may contain general financial product
> advice
> > > but should not be relied upon or construed as a recommendation of any
> > > financial product. The information has been prepared without taking
> into
> > > account your objectives, financial situation or needs. You should
> consider
> > > the Product Disclosure Statement relating to the financial product and
> > > consult your financial adviser before making a decision about whether
> to
> > > acquire, hold or dispose of a financial product.
> > >
> > > 

Re: ISILON storage/FILE DEVCLASS performance issues

[Steven Harris]

Thanks for the ideas. Much appreciated.  I also looked at some AIX stats
today and they have pointed me at a few hypervisor tuning options.

Cheers

Steve.

On Mon, 14 May 2018, 23:32 Skylar Thompson,  wrote:

> This sounds pretty good to me. If you can, I would boost your NFS thread
> count past the number of CPUs that you have, since a lot of NFS is just
> waiting for the disks to respond. You still need a thread for that, but it
> won't consume much CPU.
>
> On Mon, May 14, 2018 at 08:27:27AM -0400, Zoltan Forray wrote:
> > Very interesting.  This supports my idea on how I want to layout the
> > new/replacement server.  The old server is only 16-threads and certainly
> > could not handle dedup (we can't afford any appliances like DD) since it
> is
> > bucking under the current backups traffic. The new server has 72-threads
> as
> > well as 100TB internal disk.  My idea is to use the fast internal 100TB
> > disk for inbound traffic and deduping and use the 200TB NFS/ISILON for
> > nextstoragepool (trying to get completely off 3592-tape storage for
> onsite
> > backups). Plus the DB will be on SSD.
> >
> > Any thoughts on this configuration?
> >
> > On Sun, May 13, 2018 at 8:38 PM, Harris, Steven <
> > steven.har...@btfinancialgroup.com> wrote:
> >
> > > Zoltan
> > >
> > > I have a similar issue TSM 7.1.1.300 AIX -> Data Domain.  Have dual
> 10Gb
> > > links, but can only get ~4000 writes/sec and 120MB/sec throughput. AIX
> only
> > > supports NFS3, and as others have pointed out in this forum recently,
> the
> > > stack does not have a good reputation.
> > >
> > > I'm finding that the heavy NFS load has other knock on effects, e.g.
> > > TSMManager keeps reporting the instance offline when it's very busy as
> it
> > > gets a network error on some of its regular queries, but these work ok
> when
> > > load is light.  Also getting a lot of Severed/reconnected sessions.
> > > CPU/IO/Paging are not a problem.
> > >
> > > Cheers
> > >
> > > Steve
> > >
> > > Steven Harris
> > > TSM Admin/Consultant
> > > Canberra Australia
> > >
> > > -Original Message-
> > > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
> Of
> > > Zoltan Forray
> > > Sent: Saturday, 12 May 2018 1:39 AM
> > > To: ADSM-L@VM.MARIST.EDU
> > > Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues
> > >
> > > Folks,
> > >
> > > ISP 7.1.7.300 on RHEL 6   10G connectivity
> > >
> > > We need some guidance on trying to figure out why ISP/TSM write
> perform to
> > > ISILON storage (via FILE DEVCLASS) is so horrible.
> > >
> > > We recently attached 200TB of ISILON storage to this server so we could
> > > empty the 36TB of onboard disk drives to move this server to new
> hardware.
> > >
> > > However, per my OS and SAN folks, we are only seeing 1Gbs level of data
> > > movement from the ISP server.  Doing a regular file copy to this same
> > > storage peaks at 10Gbs speeds.
> > >
> > > So what, if anything, are we doing wrong when it comes to configuring
> the
> > > storage for ISP to use?  Are there some secret
> controls/settings/options to
> > > tell it to use the storage at max-speeds?
> > >
> > > We tried changing the Est/Max capacity thinking larger files would
> reduce
> > > the overhead of allocating new pieces constantly.  Changed the Mount
> Limit
> > > to a bigger number.  Nothing has helped.
> > >
> > > Only thing uses the storage right now is migrations from the original
> disk
> > > stgpool.
> > >
> > >
> > >
> > > --
> > > *Zoltan Forray*
> > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon
> > > Monitor Administrator VMware Administrator Virginia Commonwealth
> University
> > > UCC/Office of Technology Services www.ucc.vcu.edu zfor...@vcu.edu -
> > > 804-828-4807 Don't be a phishing victim - VCU and other reputable
> > > organizations will never use email to request that you reply with your
> > > password, social security number or confidential personal information.
> For
> > > more details visit http://phishing.vcu.edu/
> > >
> > >
> > > This message and any attachment is confidential and may be privileged
> or
> > > otherwise protected from disclosure. You should immediately delete the
> > > message if you are not the intended recipient. If you have received
> this
> > > email by mistake please delete it from your system; you should not
> copy the
> > > message or disclose its content to anyone.
> > >
> > > This electronic communication may contain general financial product
> advice
> > > but should not be relied upon or construed as a recommendation of any
> > > financial product. The information has been prepared without taking
> into
> > > account your objectives, financial situation or needs. You should
> consider
> > > the Product Disclosure Statement relating to the financial product and
> > > consult your financial adviser before making a decision about whether
> to
> > > acquire, hold or dispose of a financial product.
> > >
> > > For further 

Re: ISILON storage/FILE DEVCLASS performance issues

[Skylar Thompson]

This sounds pretty good to me. If you can, I would boost your NFS thread
count past the number of CPUs that you have, since a lot of NFS is just
waiting for the disks to respond. You still need a thread for that, but it
won't consume much CPU.

On Mon, May 14, 2018 at 08:27:27AM -0400, Zoltan Forray wrote:
> Very interesting.  This supports my idea on how I want to layout the
> new/replacement server.  The old server is only 16-threads and certainly
> could not handle dedup (we can't afford any appliances like DD) since it is
> bucking under the current backups traffic. The new server has 72-threads as
> well as 100TB internal disk.  My idea is to use the fast internal 100TB
> disk for inbound traffic and deduping and use the 200TB NFS/ISILON for
> nextstoragepool (trying to get completely off 3592-tape storage for onsite
> backups). Plus the DB will be on SSD.
>
> Any thoughts on this configuration?
>
> On Sun, May 13, 2018 at 8:38 PM, Harris, Steven <
> steven.har...@btfinancialgroup.com> wrote:
>
> > Zoltan
> >
> > I have a similar issue TSM 7.1.1.300 AIX -> Data Domain.  Have dual 10Gb
> > links, but can only get ~4000 writes/sec and 120MB/sec throughput. AIX only
> > supports NFS3, and as others have pointed out in this forum recently, the
> > stack does not have a good reputation.
> >
> > I'm finding that the heavy NFS load has other knock on effects, e.g.
> > TSMManager keeps reporting the instance offline when it's very busy as it
> > gets a network error on some of its regular queries, but these work ok when
> > load is light.  Also getting a lot of Severed/reconnected sessions.
> > CPU/IO/Paging are not a problem.
> >
> > Cheers
> >
> > Steve
> >
> > Steven Harris
> > TSM Admin/Consultant
> > Canberra Australia
> >
> > -Original Message-
> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
> > Zoltan Forray
> > Sent: Saturday, 12 May 2018 1:39 AM
> > To: ADSM-L@VM.MARIST.EDU
> > Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues
> >
> > Folks,
> >
> > ISP 7.1.7.300 on RHEL 6   10G connectivity
> >
> > We need some guidance on trying to figure out why ISP/TSM write perform to
> > ISILON storage (via FILE DEVCLASS) is so horrible.
> >
> > We recently attached 200TB of ISILON storage to this server so we could
> > empty the 36TB of onboard disk drives to move this server to new hardware.
> >
> > However, per my OS and SAN folks, we are only seeing 1Gbs level of data
> > movement from the ISP server.  Doing a regular file copy to this same
> > storage peaks at 10Gbs speeds.
> >
> > So what, if anything, are we doing wrong when it comes to configuring the
> > storage for ISP to use?  Are there some secret controls/settings/options to
> > tell it to use the storage at max-speeds?
> >
> > We tried changing the Est/Max capacity thinking larger files would reduce
> > the overhead of allocating new pieces constantly.  Changed the Mount Limit
> > to a bigger number.  Nothing has helped.
> >
> > Only thing uses the storage right now is migrations from the original disk
> > stgpool.
> >
> >
> >
> > --
> > *Zoltan Forray*
> > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon
> > Monitor Administrator VMware Administrator Virginia Commonwealth University
> > UCC/Office of Technology Services www.ucc.vcu.edu zfor...@vcu.edu -
> > 804-828-4807 Don't be a phishing victim - VCU and other reputable
> > organizations will never use email to request that you reply with your
> > password, social security number or confidential personal information. For
> > more details visit http://phishing.vcu.edu/
> >
> >
> > This message and any attachment is confidential and may be privileged or
> > otherwise protected from disclosure. You should immediately delete the
> > message if you are not the intended recipient. If you have received this
> > email by mistake please delete it from your system; you should not copy the
> > message or disclose its content to anyone.
> >
> > This electronic communication may contain general financial product advice
> > but should not be relied upon or construed as a recommendation of any
> > financial product. The information has been prepared without taking into
> > account your objectives, financial situation or needs. You should consider
> > the Product Disclosure Statement relating to the financial product and
> > consult your financial adviser before making a decision about whether to
> > acquire, hold or dispose of a financial product.
> >
> > For further details on the financial product please go to
> > http://www.bt.com.au
> >
> > Past performance is not a reliable indicator of future performance.
> >
>
>
>
> --
> *Zoltan Forray*
> Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator
> Xymon Monitor Administrator
> VMware Administrator
> Virginia Commonwealth University
> UCC/Office of Technology Services
> www.ucc.vcu.edu
> zfor...@vcu.edu - 804-828-4807
> Don't be a phishing victim - VCU and other reputable 

Re: ISILON storage/FILE DEVCLASS performance issues

[Skylar Thompson]

Do you see consistent NFS throughput, or is it bursty? We've never used
Isilon as storage for TSM, but we have had problems generally with too-low
NFS timeouts causing NFS to back off for too long. You can also see this
problem manifest itself with NFS timeout messages in the kernel log.

On Fri, May 11, 2018 at 11:39:08AM -0400, Zoltan Forray wrote:
> Folks,
>
> ISP 7.1.7.300 on RHEL 6   10G connectivity
>
> We need some guidance on trying to figure out why ISP/TSM write perform to
> ISILON storage (via FILE DEVCLASS) is so horrible.
>
> We recently attached 200TB of ISILON storage to this server so we could
> empty the 36TB of onboard disk drives to move this server to new hardware.
>
> However, per my OS and SAN folks, we are only seeing 1Gbs level of data
> movement from the ISP server.  Doing a regular file copy to this same
> storage peaks at 10Gbs speeds.
>
> So what, if anything, are we doing wrong when it comes to configuring the
> storage for ISP to use?  Are there some secret controls/settings/options to
> tell it to use the storage at max-speeds?
>
> We tried changing the Est/Max capacity thinking larger files would reduce
> the overhead of allocating new pieces constantly.  Changed the Mount Limit
> to a bigger number.  Nothing has helped.
>
> Only thing uses the storage right now is migrations from the original disk
> stgpool.
>
>
>
> --
> *Zoltan Forray*
> Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator
> Xymon Monitor Administrator
> VMware Administrator
> Virginia Commonwealth University
> UCC/Office of Technology Services
> www.ucc.vcu.edu
> zfor...@vcu.edu - 804-828-4807
> Don't be a phishing victim - VCU and other reputable organizations will
> never use email to request that you reply with your password, social
> security number or confidential personal information. For more details
> visit http://phishing.vcu.edu/

--
-- Skylar Thompson (skyl...@u.washington.edu)
-- Genome Sciences Department, System Administrator
-- Foege Building S046, (206)-685-7354
-- University of Washington School of Medicine

Re: ISILON storage/FILE DEVCLASS performance issues

[Zoltan Forray]

Very interesting.  This supports my idea on how I want to layout the
new/replacement server.  The old server is only 16-threads and certainly
could not handle dedup (we can't afford any appliances like DD) since it is
bucking under the current backups traffic. The new server has 72-threads as
well as 100TB internal disk.  My idea is to use the fast internal 100TB
disk for inbound traffic and deduping and use the 200TB NFS/ISILON for
nextstoragepool (trying to get completely off 3592-tape storage for onsite
backups). Plus the DB will be on SSD.

Any thoughts on this configuration?

On Sun, May 13, 2018 at 8:38 PM, Harris, Steven <
steven.har...@btfinancialgroup.com> wrote:

> Zoltan
>
> I have a similar issue TSM 7.1.1.300 AIX -> Data Domain.  Have dual 10Gb
> links, but can only get ~4000 writes/sec and 120MB/sec throughput. AIX only
> supports NFS3, and as others have pointed out in this forum recently, the
> stack does not have a good reputation.
>
> I'm finding that the heavy NFS load has other knock on effects, e.g.
> TSMManager keeps reporting the instance offline when it's very busy as it
> gets a network error on some of its regular queries, but these work ok when
> load is light.  Also getting a lot of Severed/reconnected sessions.
> CPU/IO/Paging are not a problem.
>
> Cheers
>
> Steve
>
> Steven Harris
> TSM Admin/Consultant
> Canberra Australia
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
> Zoltan Forray
> Sent: Saturday, 12 May 2018 1:39 AM
> To: ADSM-L@VM.MARIST.EDU
> Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues
>
> Folks,
>
> ISP 7.1.7.300 on RHEL 6   10G connectivity
>
> We need some guidance on trying to figure out why ISP/TSM write perform to
> ISILON storage (via FILE DEVCLASS) is so horrible.
>
> We recently attached 200TB of ISILON storage to this server so we could
> empty the 36TB of onboard disk drives to move this server to new hardware.
>
> However, per my OS and SAN folks, we are only seeing 1Gbs level of data
> movement from the ISP server.  Doing a regular file copy to this same
> storage peaks at 10Gbs speeds.
>
> So what, if anything, are we doing wrong when it comes to configuring the
> storage for ISP to use?  Are there some secret controls/settings/options to
> tell it to use the storage at max-speeds?
>
> We tried changing the Est/Max capacity thinking larger files would reduce
> the overhead of allocating new pieces constantly.  Changed the Mount Limit
> to a bigger number.  Nothing has helped.
>
> Only thing uses the storage right now is migrations from the original disk
> stgpool.
>
>
>
> --
> *Zoltan Forray*
> Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon
> Monitor Administrator VMware Administrator Virginia Commonwealth University
> UCC/Office of Technology Services www.ucc.vcu.edu zfor...@vcu.edu -
> 804-828-4807 Don't be a phishing victim - VCU and other reputable
> organizations will never use email to request that you reply with your
> password, social security number or confidential personal information. For
> more details visit http://phishing.vcu.edu/
>
>
> This message and any attachment is confidential and may be privileged or
> otherwise protected from disclosure. You should immediately delete the
> message if you are not the intended recipient. If you have received this
> email by mistake please delete it from your system; you should not copy the
> message or disclose its content to anyone.
>
> This electronic communication may contain general financial product advice
> but should not be relied upon or construed as a recommendation of any
> financial product. The information has been prepared without taking into
> account your objectives, financial situation or needs. You should consider
> the Product Disclosure Statement relating to the financial product and
> consult your financial adviser before making a decision about whether to
> acquire, hold or dispose of a financial product.
>
> For further details on the financial product please go to
> http://www.bt.com.au
>
> Past performance is not a reliable indicator of future performance.
>



--
*Zoltan Forray*
Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator
Xymon Monitor Administrator
VMware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
www.ucc.vcu.edu
zfor...@vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://phishing.vcu.edu/

Re: [EXTERNAL] Re: ISILON storage/FILE DEVCLASS performance issues

[Rhodes, Richard L.]

AIX does supported NFSv4.  I'd love to get the time to try it out, but 
DataDomains only support NVSv4 with the newest release and we are several 
levels back.
https://www.redbooks.ibm.com/redbooks/pdfs/sg246657.pdf

When our admins found that multiple/concurrent mount points helped for 
Oracle/RMAN backups, I converted out DataDomain mounts from single mounts to 
multiple mounts.  I then changed the file pool devclass to use multiple 
directories (one per new mount).  I pulled the original directory out of the 
file pool definition, marked all it's volumes R/O, and movedata all vols.  TSM 
put the vols across the multiple mount points.  Lets just say this took quite 
some time!  The scary things was convincing myself (via testing) that I could 
remove a directory from the devclass and still access those volumes.


original single mount:  
  dd-10g:/data/col1/tsm1  1274748715008 323228344320   75%   103091 1% 
/DD/tsm1
file pool was at /DD/tsm1/backup-pri-dd

changed to these mounts
  dd-10g:/data/col1/tsm1  1274748715008 323228344320   75%   103091 1% 
/DD/tsm1
  dd-10g:/data/col1/tsm1/backup-pri-dd-1  1274748715008 323228344320   75%  
 103091 1% /DD/backup-pri-dd-1
  dd-10g:/data/col1/tsm1/backup-pri-dd-2  1274748715008 323228344320   75%  
 103091 1% /DD/backup-pri-dd-2
  dd-10g:/data/col1/tsm1/backup-pri-dd-3  1274748715008 323228344320   75%  
 103091 1% /DD/backup-pri-dd-3
  dd-10g:/data/col1/tsm1/backup-pri-dd-4  1274748715008 323228344320   75%  
 103091 1% /DD/backup-pri-dd-4
 
  Note, this did not require any additional exports from the DataDomain, just 
creating some directories in it.

This is the devclass for the Datadomain mount

 Device Class Name: BACKUP-PRI-DD
Storage Pool Count: 1
   Device Type: FILE
 Directory: 
/DD/backup-pri-dd-1,/DD/backup-pri-dd-2,/DD/backup-pri-dd-3,/DD/backup-pri-dd-4

Note - prior to this change the devclass only hadDirectory: 
/DD/tsm1/backup-pri-dd





-Original Message-
From: ADSM: Dist Stor Manager  On Behalf Of Harris, Steven
Sent: Sunday, May 13, 2018 8:39 PM
To: ADSM-L@VM.MARIST.EDU
Subject: [EXTERNAL] Re: ISILON storage/FILE DEVCLASS performance issues

Zoltan

I have a similar issue TSM 7.1.1.300 AIX -> Data Domain.  Have dual 10Gb links, 
but can only get ~4000 writes/sec and 120MB/sec throughput. AIX only supports 
NFS3, and as others have pointed out in this forum recently, the stack does not 
have a good reputation.

I'm finding that the heavy NFS load has other knock on effects, e.g. TSMManager 
keeps reporting the instance offline when it's very busy as it gets a network 
error on some of its regular queries, but these work ok when load is light.  
Also getting a lot of Severed/reconnected sessions.  CPU/IO/Paging are not a 
problem.

Cheers

Steve

Steven Harris
TSM Admin/Consultant
Canberra Australia

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Zoltan 
Forray
Sent: Saturday, 12 May 2018 1:39 AM
To: ADSM-L@VM.MARIST.EDU
Subject: [ADSM-L] ISILON storage/FILE DEVCLASS performance issues

Folks,

ISP 7.1.7.300 on RHEL 6   10G connectivity

We need some guidance on trying to figure out why ISP/TSM write perform to 
ISILON storage (via FILE DEVCLASS) is so horrible.

We recently attached 200TB of ISILON storage to this server so we could empty 
the 36TB of onboard disk drives to move this server to new hardware.

However, per my OS and SAN folks, we are only seeing 1Gbs level of data 
movement from the ISP server.  Doing a regular file copy to this same storage 
peaks at 10Gbs speeds.

So what, if anything, are we doing wrong when it comes to configuring the 
storage for ISP to use?  Are there some secret controls/settings/options to 
tell it to use the storage at max-speeds?

We tried changing the Est/Max capacity thinking larger files would reduce the 
overhead of allocating new pieces constantly.  Changed the Mount Limit to a 
bigger number.  Nothing has helped.

Only thing uses the storage right now is migrations from the original disk 
stgpool.



--
*Zoltan Forray*
Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon Monitor 
Administrator VMware Administrator Virginia Commonwealth University UCC/Office 
of Technology Services www.ucc.vcu.edu zfor...@vcu.edu - 804-828-4807 Don't be 
a phishing victim - VCU and other reputable organizations will never use email 
to request that you reply with your password, social security number or 
confidential personal information. For more details visit 
http://phishing.vcu.edu/


This message and any attachment is confidential and may be privileged or 
otherwise protected from disclosure. You should immediately delete the message 
if you are not the intended recipient. If you have received this email by 
mistake please delete it from your system; you should not copy the message