Re: Real world deduplication rates with TSM 7.1 and container pools

[Stefan Folkerts]

We see around 50-65% deduplication savings on the fileclass storagepools,
most common seems to be around 55%.
It requires what I call "deep reclaims" with very low values that need a
lot of time.
We are seeing 60-70% on containerpools but on average it is more like 65%
but that is based on a much smaller install base.
Both in heterogeneous environments.


On Fri, Mar 18, 2016 at 5:06 PM, Ken Bury  wrote:

> I have two 7.1.4 servers, one with devclass file with dedupe, and the other
> is using containers. The two servers are in a node replication pair so the
> data on each server is exactly the same. The workload is almost exclusively
> vmware backups with datamover dedupe and compression. The data reduction
> for both pools is 89%. I like what I am getting from container pools and
> replication.
> On Fri, Mar 18, 2016 at 11:35 Ryder, Michael S 
> wrote:
>
> > Hi Arnaud
> >
> > If IBM made that commitment in black and white, then you should hold
> their
> > feet to the fire.  But I am willing to bet this was a salesman promising
> > "similar performance."
> >
> > There is no technology I know where any deduplication factor can be
> > guaranteed.  Perhaps "UP to 4" for certain kinds of data...  And overall
> > reduction of storage is what you should be comparing, not simply the
> > deduplication percentage.
> >
> > Here, try reading at least the introduction of this document, " Effective
> > Planning and Use of TSM V6 and V7 Deduplication"
> >
> >
> >
> http://www.ibm.com/developerworks/community/wikis/form/anonymous/api/wiki/f731037e-c0cf-436e-88b5-862b9a6597c3/page/82e361b4-8e96-42cf-b559-0b77df9aed2c/attachment/5cf980b3-807f-464b-a1c0-b896b0cec7e6/media/TSM%20Dedup%20Best%20Practices%20-%20v2.1.pdf
> >
> > We haven't adopted the directory-container pools yet due to their lacking
> > of support for important features like migration and copy pools, but I
> have
> > no doubt that IBM will be delivering those abilities soon; otherwise,
> there
> > are very limited use-cases for directory-containers.
> >
> > Best regards,
> >
> > Mike
> > RMD IT Client Services
> >
> > On Fri, Mar 18, 2016 at 10:41 AM, PAC Brion Arnaud <
> > arnaud.br...@panalpina.com> wrote:
> >
> > > Hi All,
> > >
> > > We are currently testing TSM 7.1 deduplication feature, in conjunction
> > > with container based storage pools.
> > > So far, my test TSM instances, installed with such a setup are
> reporting
> > > dedup percentage of 45 %, means dedup factor around 1.81, using a
> sample
> > of
> > > clients which are representative of our production environment.
> > > This is unfortunately pretty far from what was promised by IBM (dedup
> > > factor of 4) ...
> > >
> > > I'm wondering if anybody making use of container based storage pools
> and
> > > deduplication would be sharing his deduplication factor, so that I
> could
> > > have a better appreciation of real world figures.
> > > If you would be so kind to share your information (possibly with the
> kind
> > > of backed-up data  i.e. VM, DB, NAS, Exchange, and retention values
> ...)
> > I
> > > would be very grateful !
> > >
> > > Thanks in advance for appreciated feedback.
> > >
> > > Cheers.
> > >
> > > Arnaud
> > >
> > >
> > >
> >
> **
> > > Backup and Recovery Systems Administrator
> > > Panalpina Management Ltd., Basle, Switzerland,
> > > CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> > > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > > Direct: +41 (61) 226 19 78
> > > e-mail: arnaud.br...@panalpina.com
> > > This electronic message transmission contains information from
> Panalpina
> > > and is confidential or privileged. This information is intended only
> for
> > > the person (s) named above. If you are not the intended recipient, any
> > > disclosure, copying, distribution or use or any other action based on
> the
> > > contents of this information is strictly prohibited.
> > >
> > > If you receive this electronic transmission in error, please notify the
> > > sender by e-mail, telephone or fax at the numbers listed above. Thank
> > you.
> > >
> > >
> >
> **
> > >
> > >
> >
> --
> Ken
>

Re: Real world deduplication rates with TSM 7.1 and container pools

[Ryder, Michael S]

Hi Arnaud

If IBM made that commitment in black and white, then you should hold their
feet to the fire.  But I am willing to bet this was a salesman promising
"similar performance."

There is no technology I know where any deduplication factor can be
guaranteed.  Perhaps "UP to 4" for certain kinds of data...  And overall
reduction of storage is what you should be comparing, not simply the
deduplication percentage.

Here, try reading at least the introduction of this document, " Effective
Planning and Use of TSM V6 and V7 Deduplication"

http://www.ibm.com/developerworks/community/wikis/form/anonymous/api/wiki/f731037e-c0cf-436e-88b5-862b9a6597c3/page/82e361b4-8e96-42cf-b559-0b77df9aed2c/attachment/5cf980b3-807f-464b-a1c0-b896b0cec7e6/media/TSM%20Dedup%20Best%20Practices%20-%20v2.1.pdf

We haven't adopted the directory-container pools yet due to their lacking
of support for important features like migration and copy pools, but I have
no doubt that IBM will be delivering those abilities soon; otherwise, there
are very limited use-cases for directory-containers.

Best regards,

Mike
RMD IT Client Services

On Fri, Mar 18, 2016 at 10:41 AM, PAC Brion Arnaud <
arnaud.br...@panalpina.com> wrote:

> Hi All,
>
> We are currently testing TSM 7.1 deduplication feature, in conjunction
> with container based storage pools.
> So far, my test TSM instances, installed with such a setup are reporting
> dedup percentage of 45 %, means dedup factor around 1.81, using a sample of
> clients which are representative of our production environment.
> This is unfortunately pretty far from what was promised by IBM (dedup
> factor of 4) ...
>
> I'm wondering if anybody making use of container based storage pools and
> deduplication would be sharing his deduplication factor, so that I could
> have a better appreciation of real world figures.
> If you would be so kind to share your information (possibly with the kind
> of backed-up data  i.e. VM, DB, NAS, Exchange, and retention values ...) I
> would be very grateful !
>
> Thanks in advance for appreciated feedback.
>
> Cheers.
>
> Arnaud
>
>
> **
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from Panalpina
> and is confidential or privileged. This information is intended only for
> the person (s) named above. If you are not the intended recipient, any
> disclosure, copying, distribution or use or any other action based on the
> contents of this information is strictly prohibited.
>
> If you receive this electronic transmission in error, please notify the
> sender by e-mail, telephone or fax at the numbers listed above. Thank you.
>
> **
>
>

Re: Real world deduplication rates with TSM 7.1 and container pools

[Ken Bury]

I have two 7.1.4 servers, one with devclass file with dedupe, and the other
is using containers. The two servers are in a node replication pair so the
data on each server is exactly the same. The workload is almost exclusively
vmware backups with datamover dedupe and compression. The data reduction
for both pools is 89%. I like what I am getting from container pools and
replication.
On Fri, Mar 18, 2016 at 11:35 Ryder, Michael S 
wrote:

> Hi Arnaud
>
> If IBM made that commitment in black and white, then you should hold their
> feet to the fire.  But I am willing to bet this was a salesman promising
> "similar performance."
>
> There is no technology I know where any deduplication factor can be
> guaranteed.  Perhaps "UP to 4" for certain kinds of data...  And overall
> reduction of storage is what you should be comparing, not simply the
> deduplication percentage.
>
> Here, try reading at least the introduction of this document, " Effective
> Planning and Use of TSM V6 and V7 Deduplication"
>
>
> http://www.ibm.com/developerworks/community/wikis/form/anonymous/api/wiki/f731037e-c0cf-436e-88b5-862b9a6597c3/page/82e361b4-8e96-42cf-b559-0b77df9aed2c/attachment/5cf980b3-807f-464b-a1c0-b896b0cec7e6/media/TSM%20Dedup%20Best%20Practices%20-%20v2.1.pdf
>
> We haven't adopted the directory-container pools yet due to their lacking
> of support for important features like migration and copy pools, but I have
> no doubt that IBM will be delivering those abilities soon; otherwise, there
> are very limited use-cases for directory-containers.
>
> Best regards,
>
> Mike
> RMD IT Client Services
>
> On Fri, Mar 18, 2016 at 10:41 AM, PAC Brion Arnaud <
> arnaud.br...@panalpina.com> wrote:
>
> > Hi All,
> >
> > We are currently testing TSM 7.1 deduplication feature, in conjunction
> > with container based storage pools.
> > So far, my test TSM instances, installed with such a setup are reporting
> > dedup percentage of 45 %, means dedup factor around 1.81, using a sample
> of
> > clients which are representative of our production environment.
> > This is unfortunately pretty far from what was promised by IBM (dedup
> > factor of 4) ...
> >
> > I'm wondering if anybody making use of container based storage pools and
> > deduplication would be sharing his deduplication factor, so that I could
> > have a better appreciation of real world figures.
> > If you would be so kind to share your information (possibly with the kind
> > of backed-up data  i.e. VM, DB, NAS, Exchange, and retention values ...)
> I
> > would be very grateful !
> >
> > Thanks in advance for appreciated feedback.
> >
> > Cheers.
> >
> > Arnaud
> >
> >
> >
> **
> > Backup and Recovery Systems Administrator
> > Panalpina Management Ltd., Basle, Switzerland,
> > CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > Direct: +41 (61) 226 19 78
> > e-mail: arnaud.br...@panalpina.com
> > This electronic message transmission contains information from Panalpina
> > and is confidential or privileged. This information is intended only for
> > the person (s) named above. If you are not the intended recipient, any
> > disclosure, copying, distribution or use or any other action based on the
> > contents of this information is strictly prohibited.
> >
> > If you receive this electronic transmission in error, please notify the
> > sender by e-mail, telephone or fax at the numbers listed above. Thank
> you.
> >
> >
> **
> >
> >
>
--
Ken

Re: Real world deduplication rates with TSM 7.1 and container pools

[PAC Brion Arnaud]

Michael, Ken and Stephan,

Thanks a lot for valuable input !

Based on your feedback, I believe that IBM is effectively overselling it's 
product  : best value announced is around 65 %, which means dedup factor around 
2.5 ...

Reason why I asked is that we are currently making use of Data Domain VTL's in 
our shop, which at present time have a dedup factor of 7.7, but are aging and 
should soon be retired.
I was wondering if their replacement with a combination of cheap disk storage 
and TSM deduped container would be a good idea ... 
So far I still need to be convinced : disk (IBM, Hitachi ...) is way cheaper 
than a VTL, but TSM dedup rates are seeming to be less than expected : this 
will probably force us to buy more disks, thus making such a solution less 
attractive.

Cheers.

Arnaud

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Stefan 
Folkerts
Sent: Friday, March 18, 2016 5:32 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Real world deduplication rates with TSM 7.1 and container pools

We see around 50-65% deduplication savings on the fileclass storagepools,
most common seems to be around 55%.
It requires what I call "deep reclaims" with very low values that need a
lot of time.
We are seeing 60-70% on containerpools but on average it is more like 65%
but that is based on a much smaller install base.
Both in heterogeneous environments.


On Fri, Mar 18, 2016 at 5:06 PM, Ken Bury  wrote:

> I have two 7.1.4 servers, one with devclass file with dedupe, and the other
> is using containers. The two servers are in a node replication pair so the
> data on each server is exactly the same. The workload is almost exclusively
> vmware backups with datamover dedupe and compression. The data reduction
> for both pools is 89%. I like what I am getting from container pools and
> replication.
> On Fri, Mar 18, 2016 at 11:35 Ryder, Michael S 
> wrote:
>
> > Hi Arnaud
> >
> > If IBM made that commitment in black and white, then you should hold
> their
> > feet to the fire.  But I am willing to bet this was a salesman promising
> > "similar performance."
> >
> > There is no technology I know where any deduplication factor can be
> > guaranteed.  Perhaps "UP to 4" for certain kinds of data...  And overall
> > reduction of storage is what you should be comparing, not simply the
> > deduplication percentage.
> >
> > Here, try reading at least the introduction of this document, " Effective
> > Planning and Use of TSM V6 and V7 Deduplication"
> >
> >
> >
> http://www.ibm.com/developerworks/community/wikis/form/anonymous/api/wiki/f731037e-c0cf-436e-88b5-862b9a6597c3/page/82e361b4-8e96-42cf-b559-0b77df9aed2c/attachment/5cf980b3-807f-464b-a1c0-b896b0cec7e6/media/TSM%20Dedup%20Best%20Practices%20-%20v2.1.pdf
> >
> > We haven't adopted the directory-container pools yet due to their lacking
> > of support for important features like migration and copy pools, but I
> have
> > no doubt that IBM will be delivering those abilities soon; otherwise,
> there
> > are very limited use-cases for directory-containers.
> >
> > Best regards,
> >
> > Mike
> > RMD IT Client Services
> >
> > On Fri, Mar 18, 2016 at 10:41 AM, PAC Brion Arnaud <
> > arnaud.br...@panalpina.com> wrote:
> >
> > > Hi All,
> > >
> > > We are currently testing TSM 7.1 deduplication feature, in conjunction
> > > with container based storage pools.
> > > So far, my test TSM instances, installed with such a setup are
> reporting
> > > dedup percentage of 45 %, means dedup factor around 1.81, using a
> sample
> > of
> > > clients which are representative of our production environment.
> > > This is unfortunately pretty far from what was promised by IBM (dedup
> > > factor of 4) ...
> > >
> > > I'm wondering if anybody making use of container based storage pools
> and
> > > deduplication would be sharing his deduplication factor, so that I
> could
> > > have a better appreciation of real world figures.
> > > If you would be so kind to share your information (possibly with the
> kind
> > > of backed-up data  i.e. VM, DB, NAS, Exchange, and retention values
> ...)
> > I
> > > would be very grateful !
> > >
> > > Thanks in advance for appreciated feedback.
> > >
> > > Cheers.
> > >
> > > Arnaud
> > >
> > >
> > >
> >
> **
> > > Backup and Recovery Systems A

Re: Real world deduplication rates with TSM 7.1 and container pools

[Hans Christian Riksheim]

We are also getting around 60% in dedup savings. When I compare similar
data going to tape I find that simple compression saves us about 40%.

Hans Chr.

On Mon, Mar 21, 2016 at 8:54 AM, PAC Brion Arnaud <
arnaud.br...@panalpina.com> wrote:

> Michael, Ken and Stephan,
>
> Thanks a lot for valuable input !
>
> Based on your feedback, I believe that IBM is effectively overselling it's
> product  : best value announced is around 65 %, which means dedup factor
> around 2.5 ...
>
> Reason why I asked is that we are currently making use of Data Domain
> VTL's in our shop, which at present time have a dedup factor of 7.7, but
> are aging and should soon be retired.
> I was wondering if their replacement with a combination of cheap disk
> storage and TSM deduped container would be a good idea ...
> So far I still need to be convinced : disk (IBM, Hitachi ...) is way
> cheaper than a VTL, but TSM dedup rates are seeming to be less than
> expected : this will probably force us to buy more disks, thus making such
> a solution less attractive.
>
> Cheers.
>
> Arnaud
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of
> Stefan Folkerts
> Sent: Friday, March 18, 2016 5:32 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Real world deduplication rates with TSM 7.1 and container
> pools
>
> We see around 50-65% deduplication savings on the fileclass storagepools,
> most common seems to be around 55%.
> It requires what I call "deep reclaims" with very low values that need a
> lot of time.
> We are seeing 60-70% on containerpools but on average it is more like 65%
> but that is based on a much smaller install base.
> Both in heterogeneous environments.
>
>
> On Fri, Mar 18, 2016 at 5:06 PM, Ken Bury  wrote:
>
> > I have two 7.1.4 servers, one with devclass file with dedupe, and the
> other
> > is using containers. The two servers are in a node replication pair so
> the
> > data on each server is exactly the same. The workload is almost
> exclusively
> > vmware backups with datamover dedupe and compression. The data reduction
> > for both pools is 89%. I like what I am getting from container pools and
> > replication.
> > On Fri, Mar 18, 2016 at 11:35 Ryder, Michael S <
> michael_s.ry...@roche.com>
> > wrote:
> >
> > > Hi Arnaud
> > >
> > > If IBM made that commitment in black and white, then you should hold
> > their
> > > feet to the fire.  But I am willing to bet this was a salesman
> promising
> > > "similar performance."
> > >
> > > There is no technology I know where any deduplication factor can be
> > > guaranteed.  Perhaps "UP to 4" for certain kinds of data...  And
> overall
> > > reduction of storage is what you should be comparing, not simply the
> > > deduplication percentage.
> > >
> > > Here, try reading at least the introduction of this document, "
> Effective
> > > Planning and Use of TSM V6 and V7 Deduplication"
> > >
> > >
> > >
> >
> http://www.ibm.com/developerworks/community/wikis/form/anonymous/api/wiki/f731037e-c0cf-436e-88b5-862b9a6597c3/page/82e361b4-8e96-42cf-b559-0b77df9aed2c/attachment/5cf980b3-807f-464b-a1c0-b896b0cec7e6/media/TSM%20Dedup%20Best%20Practices%20-%20v2.1.pdf
> > >
> > > We haven't adopted the directory-container pools yet due to their
> lacking
> > > of support for important features like migration and copy pools, but I
> > have
> > > no doubt that IBM will be delivering those abilities soon; otherwise,
> > there
> > > are very limited use-cases for directory-containers.
> > >
> > > Best regards,
> > >
> > > Mike
> > > RMD IT Client Services
> > >
> > > On Fri, Mar 18, 2016 at 10:41 AM, PAC Brion Arnaud <
> > > arnaud.br...@panalpina.com> wrote:
> > >
> > > > Hi All,
> > > >
> > > > We are currently testing TSM 7.1 deduplication feature, in
> conjunction
> > > > with container based storage pools.
> > > > So far, my test TSM instances, installed with such a setup are
> > reporting
> > > > dedup percentage of 45 %, means dedup factor around 1.81, using a
> > sample
> > > of
> > > > clients which are representative of our production environment.
> > > > This is unfortunately pretty far from what was promised by IBM (dedup
> > > > factor of 4) ...
> > > >
> > > > I'm wondering if anybody making use of container bas

Re: Real world deduplication rates with TSM 7.1 and container pools

[Bas van Kampen]

We have around 900 VM's in 2 container pools and we get 35% dedup
percentage.

Regards,

Bas van Kampen

On 18-3-2016 15:41, PAC Brion Arnaud wrote:

Hi All,

We are currently testing TSM 7.1 deduplication feature, in conjunction with 
container based storage pools.
So far, my test TSM instances, installed with such a setup are reporting dedup 
percentage of 45 %, means dedup factor around 1.81, using a sample of clients 
which are representative of our production environment.
This is unfortunately pretty far from what was promised by IBM (dedup factor of 
4) ...

I'm wondering if anybody making use of container based storage pools and 
deduplication would be sharing his deduplication factor, so that I could have a 
better appreciation of real world figures.
If you would be so kind to share your information (possibly with the kind of 
backed-up data  i.e. VM, DB, NAS, Exchange, and retention values ...) I would 
be very grateful !

Thanks in advance for appreciated feedback.

Cheers.

Arnaud

**
Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. This information is intended only for the person 
(s) named above. If you are not the intended recipient, any disclosure, 
copying, distribution or use or any other action based on the contents of this 
information is strictly prohibited.

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.
**

Re: Real world deduplication rates with TSM 7.1 and container pools

[PAC Brion Arnaud]

Hi Bas,

Thanks for feedback. 

On our side average dedup is 38% for VM's data , with daily incremental and 30 
days retention ...

Cheers.

Arnaud

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Bas 
van Kampen
Sent: Monday, March 21, 2016 12:15 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Real world deduplication rates with TSM 7.1 and container pools

We have around 900 VM's in 2 container pools and we get 35% dedup
percentage.

Regards,

Bas van Kampen

On 18-3-2016 15:41, PAC Brion Arnaud wrote:
> Hi All,
>
> We are currently testing TSM 7.1 deduplication feature, in conjunction with 
> container based storage pools.
> So far, my test TSM instances, installed with such a setup are reporting 
> dedup percentage of 45 %, means dedup factor around 1.81, using a sample of 
> clients which are representative of our production environment.
> This is unfortunately pretty far from what was promised by IBM (dedup factor 
> of 4) ...
>
> I'm wondering if anybody making use of container based storage pools and 
> deduplication would be sharing his deduplication factor, so that I could have 
> a better appreciation of real world figures.
> If you would be so kind to share your information (possibly with the kind of 
> backed-up data  i.e. VM, DB, NAS, Exchange, and retention values ...) I would 
> be very grateful !
>
> Thanks in advance for appreciated feedback.
>
> Cheers.
>
> Arnaud
>
> **
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
> This electronic message transmission contains information from Panalpina and 
> is confidential or privileged. This information is intended only for the 
> person (s) named above. If you are not the intended recipient, any 
> disclosure, copying, distribution or use or any other action based on the 
> contents of this information is strictly prohibited.
>
> If you receive this electronic transmission in error, please notify the 
> sender by e-mail, telephone or fax at the numbers listed above. Thank you.
> **
>

Re: Real world deduplication rates with TSM 7.1 and container pools

[Del Hoobler]

I think most of you know Spectrum Protect just added in-line compression 
to the container and cloud deduplicated pools in version 7.1.5:

 
http://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.5/srv.common/r_techchg_srv_compress_715.html

Adding incremental forever, the new in-line deduplication - client or 
server-based (7.1.3), new in-line compression (7.1.5) I think you will 
find that Protect continues to drive overall data reduction. This is being 
done in the software, so you can choose what disk you want to use.

I encourage folks to try out the the new deduplication along with the new 
compression to see how it helps with the overall data reduction. 


Thank you,

Del





"ADSM: Dist Stor Manager"  wrote on 03/18/2016 
10:41:06 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/18/2016 10:41 AM
> Subject: Real world deduplication rates with TSM 7.1 and container pools
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi All,
> 
> We are currently testing TSM 7.1 deduplication feature, in 
> conjunction with container based storage pools.
> So far, my test TSM instances, installed with such a setup are 
> reporting dedup percentage of 45 %, means dedup factor around 1.81, 
> using a sample of clients which are representative of our production
> environment.
> This is unfortunately pretty far from what was promised by IBM 
> (dedup factor of 4) ...
> 
> I'm wondering if anybody making use of container based storage pools
> and deduplication would be sharing his deduplication factor, so that
> I could have a better appreciation of real world figures.
> If you would be so kind to share your information (possibly with the
> kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention 
> values ...) I would be very grateful !
> 
> Thanks in advance for appreciated feedback.
> 
> Cheers.
> 
> Arnaud
> 
> 
**
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from 
> Panalpina and is confidential or privileged. This information is 
> intended only for the person (s) named above. If you are not the 
> intended recipient, any disclosure, copying, distribution or use or 
> any other action based on the contents of this information is 
> strictly prohibited.
> 
> If you receive this electronic transmission in error, please notify 
> the sender by e-mail, telephone or fax at the numbers listed above. 
Thank you.
> 
**
> 

Re: Real world deduplication rates with TSM 7.1 and container pools

[PAC Brion Arnaud]

Hi Del,

Thanks for notifying that new feature : I will test it immediately !
I'm however sorry to inform you that contrarily to your assumption, and despite 
the fact that I subscribed to "IBM my notifications"  as well as to " Technical 
Support Newsletter", both related to Spectrum protect, that information did not 
make its way to me ... 
I assume that lots of TSM admins, all of them being pretty occupied by daily 
business activities, and not mandatorily having enough time to seek information 
are in the same case !
I incidentally learnt as well thru your link that " Repair damaged data on a 
replication target" feature had been added in TSM 7.1.5, which is definitively 
a good thing, but I find it a shame that such information is not pushed to 
customers thru the means of newsletters, and that we have to dig in the depths 
of IBM's web site to get such  - important - information.
I fully understand that TSM / Spectrum protect has been subject to numerous 
bugs fixes and improvements in the latest months ( I never lived such an 
upgrade waltz during 15 years of TSM administration), but to my eyes there's 
still lot of space for communication improvement by IBM ! 
Of course, only MHO ...

Cheers.

Arnaud

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Del 
Hoobler
Sent: Monday, March 21, 2016 10:53 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Real world deduplication rates with TSM 7.1 and container pools

I think most of you know Spectrum Protect just added in-line compression 
to the container and cloud deduplicated pools in version 7.1.5:

 
http://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.5/srv.common/r_techchg_srv_compress_715.html

Adding incremental forever, the new in-line deduplication - client or 
server-based (7.1.3), new in-line compression (7.1.5) I think you will 
find that Protect continues to drive overall data reduction. This is being 
done in the software, so you can choose what disk you want to use.

I encourage folks to try out the the new deduplication along with the new 
compression to see how it helps with the overall data reduction. 


Thank you,

Del





"ADSM: Dist Stor Manager"  wrote on 03/18/2016 
10:41:06 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/18/2016 10:41 AM
> Subject: Real world deduplication rates with TSM 7.1 and container pools
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi All,
> 
> We are currently testing TSM 7.1 deduplication feature, in 
> conjunction with container based storage pools.
> So far, my test TSM instances, installed with such a setup are 
> reporting dedup percentage of 45 %, means dedup factor around 1.81, 
> using a sample of clients which are representative of our production
> environment.
> This is unfortunately pretty far from what was promised by IBM 
> (dedup factor of 4) ...
> 
> I'm wondering if anybody making use of container based storage pools
> and deduplication would be sharing his deduplication factor, so that
> I could have a better appreciation of real world figures.
> If you would be so kind to share your information (possibly with the
> kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention 
> values ...) I would be very grateful !
> 
> Thanks in advance for appreciated feedback.
> 
> Cheers.
> 
> Arnaud
> 
> 
**
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
> This electronic message transmission contains information from 
> Panalpina and is confidential or privileged. This information is 
> intended only for the person (s) named above. If you are not the 
> intended recipient, any disclosure, copying, distribution or use or 
> any other action based on the contents of this information is 
> strictly prohibited.
> 
> If you receive this electronic transmission in error, please notify 
> the sender by e-mail, telephone or fax at the numbers listed above. 
Thank you.
> 
**
> 

Re: Real world deduplication rates with TSM 7.1 and container pools

[Del Hoobler]

Hi Arnaud,

I was having a conversation about this other day. With today's social 
media blitz, it can become an overload of information and different people 
like to learn about things in different ways. Some people told me they 
like something in their InBox. Others said their InBox was too full 
already. Others told me Twitter. Others LinkedIn. Some said ADSM-L. You 
indicate below that you would like to see information about new releases 
in the "IBM my notifications" or the "Technical Support Newsletter". One 
option is post it everywhere and it. This is all good feedback.

Thank you,

Del




"ADSM: Dist Stor Manager"  wrote on 03/22/2016 
05:36:46 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/22/2016 05:37 AM
> Subject: Re: Real world deduplication rates with TSM 7.1 and container 
pools
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi Del,
> 
> Thanks for notifying that new feature : I will test it immediately !
> I'm however sorry to inform you that contrarily to your assumption, 
> and despite the fact that I subscribed to "IBM my notifications"  as
> well as to " Technical Support Newsletter", both related to Spectrum
> protect, that information did not make its way to me ... 
> I assume that lots of TSM admins, all of them being pretty occupied 
> by daily business activities, and not mandatorily having enough time
> to seek information are in the same case !
> I incidentally learnt as well thru your link that " Repair damaged 
> data on a replication target" feature had been added in TSM 7.1.5, 
> which is definitively a good thing, but I find it a shame that such 
> information is not pushed to customers thru the means of 
> newsletters, and that we have to dig in the depths of IBM's web site
> to get such  - important - information.
> I fully understand that TSM / Spectrum protect has been subject to 
> numerous bugs fixes and improvements in the latest months ( I never 
> lived such an upgrade waltz during 15 years of TSM administration), 
> but to my eyes there's still lot of space for communication 
> improvement by IBM ! 
> Of course, only MHO ...
> 
> Cheers.
> 
> Arnaud
> 
> -----Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
> Behalf Of Del Hoobler
> Sent: Monday, March 21, 2016 10:53 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Real world deduplication rates with TSM 7.1 and container 
pools
> 
> I think most of you know Spectrum Protect just added in-line compression 

> to the container and cloud deduplicated pools in version 7.1.5:
> 
> 
> http://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.5/srv.common/
> r_techchg_srv_compress_715.html
> 
> Adding incremental forever, the new in-line deduplication - client or 
> server-based (7.1.3), new in-line compression (7.1.5) I think you will 
> find that Protect continues to drive overall data reduction. This is 
being 
> done in the software, so you can choose what disk you want to use.
> 
> I encourage folks to try out the the new deduplication along with the 
new 
> compression to see how it helps with the overall data reduction. 
> 
> 
> Thank you,
> 
> Del
> 
> 
> 
> 
> 
> "ADSM: Dist Stor Manager"  wrote on 03/18/2016 
> 10:41:06 AM:
> 
> > From: PAC Brion Arnaud 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 03/18/2016 10:41 AM
> > Subject: Real world deduplication rates with TSM 7.1 and container 
pools
> > Sent by: "ADSM: Dist Stor Manager" 
> > 
> > Hi All,
> > 
> > We are currently testing TSM 7.1 deduplication feature, in 
> > conjunction with container based storage pools.
> > So far, my test TSM instances, installed with such a setup are 
> > reporting dedup percentage of 45 %, means dedup factor around 1.81, 
> > using a sample of clients which are representative of our production
> > environment.
> > This is unfortunately pretty far from what was promised by IBM 
> > (dedup factor of 4) ...
> > 
> > I'm wondering if anybody making use of container based storage pools
> > and deduplication would be sharing his deduplication factor, so that
> > I could have a better appreciation of real world figures.
> > If you would be so kind to share your information (possibly with the
> > kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention 
> > values ...) I would be very grateful !
> > 
> > Thanks in advance for appreciated feedback.
> > 
> > Cheers.
> > 
> > Arnaud
> > 
> 

Re: Real world deduplication rates with TSM 7.1 and container pools

[David Ehresman]

Del,

After upgrading to 7.1.5 is there a way to get pre-existing container data 
compressed?

David

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Del 
Hoobler
Sent: Monday, March 21, 2016 5:53 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Real world deduplication rates with TSM 7.1 and container 
pools

I think most of you know Spectrum Protect just added in-line compression 
to the container and cloud deduplicated pools in version 7.1.5:

 
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ibm.com_support_knowledgecenter_SSGSG7-5F7.1.5_srv.common_r-5Ftechchg-5Fsrv-5Fcompress-5F715.html&d=AwIFAg&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=OMHhA4Nmsv9Buo24T83zOFo8w4mvoMyvC_nagZn36T0&s=C4shp8VD_EyXjruT5Lu2v68tMVpfpzHcluAuRJfaIK0&e=
 

Adding incremental forever, the new in-line deduplication - client or 
server-based (7.1.3), new in-line compression (7.1.5) I think you will 
find that Protect continues to drive overall data reduction. This is being 
done in the software, so you can choose what disk you want to use.

I encourage folks to try out the the new deduplication along with the new 
compression to see how it helps with the overall data reduction. 


Thank you,

Del





"ADSM: Dist Stor Manager"  wrote on 03/18/2016 
10:41:06 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/18/2016 10:41 AM
> Subject: Real world deduplication rates with TSM 7.1 and container pools
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi All,
> 
> We are currently testing TSM 7.1 deduplication feature, in 
> conjunction with container based storage pools.
> So far, my test TSM instances, installed with such a setup are 
> reporting dedup percentage of 45 %, means dedup factor around 1.81, 
> using a sample of clients which are representative of our production
> environment.
> This is unfortunately pretty far from what was promised by IBM 
> (dedup factor of 4) ...
> 
> I'm wondering if anybody making use of container based storage pools
> and deduplication would be sharing his deduplication factor, so that
> I could have a better appreciation of real world figures.
> If you would be so kind to share your information (possibly with the
> kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention 
> values ...) I would be very grateful !
> 
> Thanks in advance for appreciated feedback.
> 
> Cheers.
> 
> Arnaud
> 
> 
**
> Backup and Recovery Systems Administrator
> Panalpina Management Ltd., Basle, Switzerland,
> CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> Direct: +41 (61) 226 19 78
> e-mail: arnaud.br...@panalpina.com
> This electronic message transmission contains information from 
> Panalpina and is confidential or privileged. This information is 
> intended only for the person (s) named above. If you are not the 
> intended recipient, any disclosure, copying, distribution or use or 
> any other action based on the contents of this information is 
> strictly prohibited.
> 
> If you receive this electronic transmission in error, please notify 
> the sender by e-mail, telephone or fax at the numbers listed above. 
Thank you.
> 
**
> 

Re: Real world deduplication rates with TSM 7.1 and container pools

[Del Hoobler]

Hi David,

No. Only newly stored data will be compressed.


Del




"ADSM: Dist Stor Manager"  wrote on 03/22/2016 
09:41:27 AM:

> From: David Ehresman 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/22/2016 09:42 AM
> Subject: Re: Real world deduplication rates with TSM 7.1 and container 
pools
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Del,
> 
> After upgrading to 7.1.5 is there a way to get pre-existing 
> container data compressed?
> 
> David
> 
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
> Behalf Of Del Hoobler
> Sent: Monday, March 21, 2016 5:53 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] Real world deduplication rates with TSM 7.1 
> and container pools
> 
> I think most of you know Spectrum Protect just added in-line compression 

> to the container and cloud deduplicated pools in version 7.1.5:
> 
> 
> https://urldefense.proofpoint.com/v2/url?
> u=http-3A__www.ibm.com_support_knowledgecenter_SSGSG7-5F7.1.
> 
5_srv.common_r-5Ftechchg-5Fsrv-5Fcompress-5F715.html&d=AwIFAg&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=OMHhA4Nmsv9Buo24T83zOFo8w4mvoMyvC_nagZn36T0&s=C4shp8VD_EyXjruT5Lu2v68tMVpfpzHcluAuRJfaIK0&e=
> 
> Adding incremental forever, the new in-line deduplication - client or 
> server-based (7.1.3), new in-line compression (7.1.5) I think you will 
> find that Protect continues to drive overall data reduction. This is 
being 
> done in the software, so you can choose what disk you want to use.
> 
> I encourage folks to try out the the new deduplication along with the 
new 
> compression to see how it helps with the overall data reduction. 
> 
> 
> Thank you,
> 
> Del
> 
> 
> 
> 
> 
> "ADSM: Dist Stor Manager"  wrote on 03/18/2016 
> 10:41:06 AM:
> 
> > From: PAC Brion Arnaud 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 03/18/2016 10:41 AM
> > Subject: Real world deduplication rates with TSM 7.1 and container 
pools
> > Sent by: "ADSM: Dist Stor Manager" 
> > 
> > Hi All,
> > 
> > We are currently testing TSM 7.1 deduplication feature, in 
> > conjunction with container based storage pools.
> > So far, my test TSM instances, installed with such a setup are 
> > reporting dedup percentage of 45 %, means dedup factor around 1.81, 
> > using a sample of clients which are representative of our production
> > environment.
> > This is unfortunately pretty far from what was promised by IBM 
> > (dedup factor of 4) ...
> > 
> > I'm wondering if anybody making use of container based storage pools
> > and deduplication would be sharing his deduplication factor, so that
> > I could have a better appreciation of real world figures.
> > If you would be so kind to share your information (possibly with the
> > kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention 
> > values ...) I would be very grateful !
> > 
> > Thanks in advance for appreciated feedback.
> > 
> > Cheers.
> > 
> > Arnaud
> > 
> > 
> 
**
> > Backup and Recovery Systems Administrator
> > Panalpina Management Ltd., Basle, Switzerland,
> > CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > Direct: +41 (61) 226 19 78
> > e-mail: arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
> > This electronic message transmission contains information from 
> > Panalpina and is confidential or privileged. This information is 
> > intended only for the person (s) named above. If you are not the 
> > intended recipient, any disclosure, copying, distribution or use or 
> > any other action based on the contents of this information is 
> > strictly prohibited.
> > 
> > If you receive this electronic transmission in error, please notify 
> > the sender by e-mail, telephone or fax at the numbers listed above. 
> Thank you.
> > 
> 
**
> > 
> 

Re: Real world deduplication rates with TSM 7.1 and container pools

[Alexander Heindl]

Hi,

but already client side compressed data isn't affected, isn't it?
what if data is replicated. Is that data then compressed?

Regards,
Alex




Von:Del Hoobler 
An: ADSM-L@VM.MARIST.EDU,
Datum:  22.03.2016 15:14
Betreff:Re: [ADSM-L] Real world deduplication rates with TSM 7.1
and container pools
Gesendet von:   "ADSM: Dist Stor Manager" 



Hi David,

No. Only newly stored data will be compressed.


Del




"ADSM: Dist Stor Manager"  wrote on 03/22/2016
09:41:27 AM:

> From: David Ehresman 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/22/2016 09:42 AM
> Subject: Re: Real world deduplication rates with TSM 7.1 and container
pools
> Sent by: "ADSM: Dist Stor Manager" 
>
> Del,
>
> After upgrading to 7.1.5 is there a way to get pre-existing
> container data compressed?
>
> David
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
> Behalf Of Del Hoobler
> Sent: Monday, March 21, 2016 5:53 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] Real world deduplication rates with TSM 7.1
> and container pools
>
> I think most of you know Spectrum Protect just added in-line compression


> to the container and cloud deduplicated pools in version 7.1.5:
>
>
> https://urldefense.proofpoint.com/v2/url?
> u=http-3A__www.ibm.com_support_knowledgecenter_SSGSG7-5F7.1.
>
5_srv.common_r-5Ftechchg-5Fsrv-5Fcompress-5F715.html&d=AwIFAg&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=OMHhA4Nmsv9Buo24T83zOFo8w4mvoMyvC_nagZn36T0&s=C4shp8VD_EyXjruT5Lu2v68tMVpfpzHcluAuRJfaIK0&e=
>
> Adding incremental forever, the new in-line deduplication - client or
> server-based (7.1.3), new in-line compression (7.1.5) I think you will
> find that Protect continues to drive overall data reduction. This is
being
> done in the software, so you can choose what disk you want to use.
>
> I encourage folks to try out the the new deduplication along with the
new
> compression to see how it helps with the overall data reduction.
>
>
> Thank you,
>
> Del
>
> 
>
>
>
> "ADSM: Dist Stor Manager"  wrote on 03/18/2016
> 10:41:06 AM:
>
> > From: PAC Brion Arnaud 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 03/18/2016 10:41 AM
> > Subject: Real world deduplication rates with TSM 7.1 and container
pools
> > Sent by: "ADSM: Dist Stor Manager" 
> >
> > Hi All,
> >
> > We are currently testing TSM 7.1 deduplication feature, in
> > conjunction with container based storage pools.
> > So far, my test TSM instances, installed with such a setup are
> > reporting dedup percentage of 45 %, means dedup factor around 1.81,
> > using a sample of clients which are representative of our production
> > environment.
> > This is unfortunately pretty far from what was promised by IBM
> > (dedup factor of 4) ...
> >
> > I'm wondering if anybody making use of container based storage pools
> > and deduplication would be sharing his deduplication factor, so that
> > I could have a better appreciation of real world figures.
> > If you would be so kind to share your information (possibly with the
> > kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention
> > values ...) I would be very grateful !
> >
> > Thanks in advance for appreciated feedback.
> >
> > Cheers.
> >
> > Arnaud
> >
> >
>
**
> > Backup and Recovery Systems Administrator
> > Panalpina Management Ltd., Basle, Switzerland,
> > CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > Direct: +41 (61) 226 19 78
> > e-mail: arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
> > This electronic message transmission contains information from
> > Panalpina and is confidential or privileged. This information is
> > intended only for the person (s) named above. If you are not the
> > intended recipient, any disclosure, copying, distribution or use or
> > any other action based on the contents of this information is
> > strictly prohibited.
> >
> > If you receive this electronic transmission in error, please notify
> > the sender by e-mail, telephone or fax at the numbers listed above.
> Thank you.
> >
>
**
> >
>

Re: Real world deduplication rates with TSM 7.1 and container pools

[PAC Brion Arnaud]

@Del,

Additional  question : if replicating compressed data, will the data be 
transmitted compressed or uncompressed ?

Cheers.

Arnaud


-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of 
Alexander Heindl
Sent: Tuesday, March 22, 2016 3:19 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Real world deduplication rates with TSM 7.1 and container pools

Hi,

but already client side compressed data isn't affected, isn't it?
what if data is replicated. Is that data then compressed?

Regards,
Alex




Von:Del Hoobler 
An: ADSM-L@VM.MARIST.EDU,
Datum:  22.03.2016 15:14
Betreff:Re: [ADSM-L] Real world deduplication rates with TSM 7.1
and container pools
Gesendet von:   "ADSM: Dist Stor Manager" 



Hi David,

No. Only newly stored data will be compressed.


Del




"ADSM: Dist Stor Manager"  wrote on 03/22/2016
09:41:27 AM:

> From: David Ehresman 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/22/2016 09:42 AM
> Subject: Re: Real world deduplication rates with TSM 7.1 and container
pools
> Sent by: "ADSM: Dist Stor Manager" 
>
> Del,
>
> After upgrading to 7.1.5 is there a way to get pre-existing
> container data compressed?
>
> David
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
> Behalf Of Del Hoobler
> Sent: Monday, March 21, 2016 5:53 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] Real world deduplication rates with TSM 7.1
> and container pools
>
> I think most of you know Spectrum Protect just added in-line compression


> to the container and cloud deduplicated pools in version 7.1.5:
>
>
> https://urldefense.proofpoint.com/v2/url?
> u=http-3A__www.ibm.com_support_knowledgecenter_SSGSG7-5F7.1.
>
5_srv.common_r-5Ftechchg-5Fsrv-5Fcompress-5F715.html&d=AwIFAg&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=OMHhA4Nmsv9Buo24T83zOFo8w4mvoMyvC_nagZn36T0&s=C4shp8VD_EyXjruT5Lu2v68tMVpfpzHcluAuRJfaIK0&e=
>
> Adding incremental forever, the new in-line deduplication - client or
> server-based (7.1.3), new in-line compression (7.1.5) I think you will
> find that Protect continues to drive overall data reduction. This is
being
> done in the software, so you can choose what disk you want to use.
>
> I encourage folks to try out the the new deduplication along with the
new
> compression to see how it helps with the overall data reduction.
>
>
> Thank you,
>
> Del
>
> 
>
>
>
> "ADSM: Dist Stor Manager"  wrote on 03/18/2016
> 10:41:06 AM:
>
> > From: PAC Brion Arnaud 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 03/18/2016 10:41 AM
> > Subject: Real world deduplication rates with TSM 7.1 and container
pools
> > Sent by: "ADSM: Dist Stor Manager" 
> >
> > Hi All,
> >
> > We are currently testing TSM 7.1 deduplication feature, in
> > conjunction with container based storage pools.
> > So far, my test TSM instances, installed with such a setup are
> > reporting dedup percentage of 45 %, means dedup factor around 1.81,
> > using a sample of clients which are representative of our production
> > environment.
> > This is unfortunately pretty far from what was promised by IBM
> > (dedup factor of 4) ...
> >
> > I'm wondering if anybody making use of container based storage pools
> > and deduplication would be sharing his deduplication factor, so that
> > I could have a better appreciation of real world figures.
> > If you would be so kind to share your information (possibly with the
> > kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention
> > values ...) I would be very grateful !
> >
> > Thanks in advance for appreciated feedback.
> >
> > Cheers.
> >
> > Arnaud
> >
> >
>
**
> > Backup and Recovery Systems Administrator
> > Panalpina Management Ltd., Basle, Switzerland,
> > CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
> > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > Direct: +41 (61) 226 19 78
> > e-mail: arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
> > This electronic message transmission contains information from
> > Panalpina and is confidential or privileged. This information is
> > intended only for the person (s) named above. If you are not the
> > intended recipient, any disclosure, copying, distribution or use or
> > any other action based on the contents of this information is
> > strictly prohibited.
> >
> > If you receive this electronic transmission in error, please notify
> > the sender by e-mail, telephone or fax at the numbers listed above.
> Thank you.
> >
>
**
> >
>

Re: Real world deduplication rates with TSM 7.1 and container pools

[Del Hoobler]

Hello,

Correct, current client-side compression is not affected.

Spectrum Protect is targeting client-side compression integrated with 
dedup and this new compression later this year.

As for replication, if the target storage pool is a container pool with 
compression enabled, the data extents are sent over compressed.

Thank you,

Del



"ADSM: Dist Stor Manager"  wrote on 03/22/2016 
10:18:53 AM:

> From: Alexander Heindl 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/22/2016 10:19 AM
> Subject: Re: Real world deduplication rates with TSM 7.1 and container 
pools
> Sent by: "ADSM: Dist Stor Manager" 
> 
> Hi,
> 
> but already client side compressed data isn't affected, isn't it?
> what if data is replicated. Is that data then compressed?
> 
> Regards,
> Alex
> 
> 
> 
> 
> Von:Del Hoobler 
> An: ADSM-L@VM.MARIST.EDU,
> Datum:  22.03.2016 15:14
> Betreff:Re: [ADSM-L] Real world deduplication rates with TSM 7.1
> and container pools
> Gesendet von:   "ADSM: Dist Stor Manager" 
> 
> 
> 
> Hi David,
> 
> No. Only newly stored data will be compressed.
> 
> 
> Del
> 
> 
> 
> 
> "ADSM: Dist Stor Manager"  wrote on 03/22/2016
> 09:41:27 AM:
> 
> > From: David Ehresman 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 03/22/2016 09:42 AM
> > Subject: Re: Real world deduplication rates with TSM 7.1 and container
> pools
> > Sent by: "ADSM: Dist Stor Manager" 
> >
> > Del,
> >
> > After upgrading to 7.1.5 is there a way to get pre-existing
> > container data compressed?
> >
> > David
> >
> > -Original Message-
> > From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On
> > Behalf Of Del Hoobler
> > Sent: Monday, March 21, 2016 5:53 PM
> > To: ADSM-L@VM.MARIST.EDU
> > Subject: Re: [ADSM-L] Real world deduplication rates with TSM 7.1
> > and container pools
> >
> > I think most of you know Spectrum Protect just added in-line 
compression
> 
> 
> > to the container and cloud deduplicated pools in version 7.1.5:
> >
> >
> > https://urldefense.proofpoint.com/v2/url?
> > u=http-3A__www.ibm.com_support_knowledgecenter_SSGSG7-5F7.1.
> >
> 
5_srv.common_r-5Ftechchg-5Fsrv-5Fcompress-5F715.html&d=AwIFAg&c=SgMrq23dbjbGX6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L8&m=OMHhA4Nmsv9Buo24T83zOFo8w4mvoMyvC_nagZn36T0&s=C4shp8VD_EyXjruT5Lu2v68tMVpfpzHcluAuRJfaIK0&e=
> >
> > Adding incremental forever, the new in-line deduplication - client or
> > server-based (7.1.3), new in-line compression (7.1.5) I think you will
> > find that Protect continues to drive overall data reduction. This is
> being
> > done in the software, so you can choose what disk you want to use.
> >
> > I encourage folks to try out the the new deduplication along with the
> new
> > compression to see how it helps with the overall data reduction.
> >
> >
> > Thank you,
> >
> > Del
> >
> > 
> >
> >
> >
> > "ADSM: Dist Stor Manager"  wrote on 03/18/2016
> > 10:41:06 AM:
> >
> > > From: PAC Brion Arnaud 
> > > To: ADSM-L@VM.MARIST.EDU
> > > Date: 03/18/2016 10:41 AM
> > > Subject: Real world deduplication rates with TSM 7.1 and container
> pools
> > > Sent by: "ADSM: Dist Stor Manager" 
> > >
> > > Hi All,
> > >
> > > We are currently testing TSM 7.1 deduplication feature, in
> > > conjunction with container based storage pools.
> > > So far, my test TSM instances, installed with such a setup are
> > > reporting dedup percentage of 45 %, means dedup factor around 1.81,
> > > using a sample of clients which are representative of our production
> > > environment.
> > > This is unfortunately pretty far from what was promised by IBM
> > > (dedup factor of 4) ...
> > >
> > > I'm wondering if anybody making use of container based storage pools
> > > and deduplication would be sharing his deduplication factor, so that
> > > I could have a better appreciation of real world figures.
> > > If you would be so kind to share your information (possibly with the
> > > kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention
> > > values ...) I would be very grateful !
> > >
> > > Thanks in advance for appreciated feedback.

Re: Real world deduplication rates with TSM 7.1 and container pools

[J. Pohlmann]

If you are using node replication, you could re-replicate the data and get
it compressed (both ways). It may be a bit much work, feasible perhaps for
selected nodes.

Joerg Pohlmann

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Del
Hoobler
Sent: March 22, 2016 07:11
To: ADSM-L@VM.MARIST.EDU
Subject: Re: [ADSM-L] Real world deduplication rates with TSM 7.1 and
container pools

Hi David,

No. Only newly stored data will be compressed.


Del




"ADSM: Dist Stor Manager"  wrote on 03/22/2016
09:41:27 AM:

> From: David Ehresman 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/22/2016 09:42 AM
> Subject: Re: Real world deduplication rates with TSM 7.1 and container
pools
> Sent by: "ADSM: Dist Stor Manager" 
>
> Del,
>
> After upgrading to 7.1.5 is there a way to get pre-existing container
> data compressed?
>
> David
>
> -Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf
> Of Del Hoobler
> Sent: Monday, March 21, 2016 5:53 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: [ADSM-L] Real world deduplication rates with TSM 7.1 and
> container pools
>
> I think most of you know Spectrum Protect just added in-line
> compression

> to the container and cloud deduplicated pools in version 7.1.5:
>
>
> https://urldefense.proofpoint.com/v2/url?
> u=http-3A__www.ibm.com_support_knowledgecenter_SSGSG7-5F7.1.
>
5_srv.common_r-5Ftechchg-5Fsrv-5Fcompress-5F715.html&d=AwIFAg&c=SgMrq23dbjbG
X6e0ZsSHgEZX6A4IAf1SO3AJ2bNrHlk&r=dOGCMY197NTNH1k_wcsrWS3_fxedKW4rpKJ8cHCD2L
8&m=OMHhA4Nmsv9Buo24T83zOFo8w4mvoMyvC_nagZn36T0&s=C4shp8VD_EyXjruT5Lu2v68tMV
pfpzHcluAuRJfaIK0&e=
>
> Adding incremental forever, the new in-line deduplication - client or
> server-based (7.1.3), new in-line compression (7.1.5) I think you will
> find that Protect continues to drive overall data reduction. This is
being
> done in the software, so you can choose what disk you want to use.
>
> I encourage folks to try out the the new deduplication along with the
new
> compression to see how it helps with the overall data reduction.
>
>
> Thank you,
>
> Del
>
> 
>
>
>
> "ADSM: Dist Stor Manager"  wrote on 03/18/2016
> 10:41:06 AM:
>
> > From: PAC Brion Arnaud 
> > To: ADSM-L@VM.MARIST.EDU
> > Date: 03/18/2016 10:41 AM
> > Subject: Real world deduplication rates with TSM 7.1 and container
pools
> > Sent by: "ADSM: Dist Stor Manager" 
> >
> > Hi All,
> >
> > We are currently testing TSM 7.1 deduplication feature, in
> > conjunction with container based storage pools.
> > So far, my test TSM instances, installed with such a setup are
> > reporting dedup percentage of 45 %, means dedup factor around 1.81,
> > using a sample of clients which are representative of our production
> > environment.
> > This is unfortunately pretty far from what was promised by IBM
> > (dedup factor of 4) ...
> >
> > I'm wondering if anybody making use of container based storage pools
> > and deduplication would be sharing his deduplication factor, so that
> > I could have a better appreciation of real world figures.
> > If you would be so kind to share your information (possibly with the
> > kind of backed-up data  i.e. VM, DB, NAS, Exchange, and retention
> > values ...) I would be very grateful !
> >
> > Thanks in advance for appreciated feedback.
> >
> > Cheers.
> >
> > Arnaud
> >
> >
>

**
> > Backup and Recovery Systems Administrator Panalpina Management Ltd.,
> > Basle, Switzerland, CIT Department Viadukstrasse 42, P.O. Box 4002
> > Basel/CH
> > Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
> > Direct: +41 (61) 226 19 78
> > e-mail:
> > arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
> > This electronic message transmission contains information from
> > Panalpina and is confidential or privileged. This information is
> > intended only for the person (s) named above. If you are not the
> > intended recipient, any disclosure, copying, distribution or use or
> > any other action based on the contents of this information is
> > strictly prohibited.
> >
> > If you receive this electronic transmission in error, please notify
> > the sender by e-mail, telephone or fax at the numbers listed above.
> Thank you.
> >
>

**
> >
>

Re: Real world deduplication rates with TSM 7.1 and container pools

[Del Hoobler]

Posting again... seems like the first one was rejected...



Hello,

Correct, current client-side compression is not affected.

Spectrum Protect is targeting client-side compression integrated with 
dedup and this new compression later this year.

As for replication, if the target storage pool is a container pool with 
compression enabled, the data extents are sent over compressed.

Thank you,

Del



"ADSM: Dist Stor Manager"  wrote on 03/22/2016 
10:47:24 AM:

> From: PAC Brion Arnaud 
> To: ADSM-L@VM.MARIST.EDU
> Date: 03/22/2016 10:48 AM
> Subject: Re: Real world deduplication rates with TSM 7.1 and container 
pools
> Sent by: "ADSM: Dist Stor Manager" 
> 
> @Del,
> 
> Additional  question : if replicating compressed data, will the data
> be transmitted compressed or uncompressed ?
> 
> Cheers.
> 
> Arnaud


-Original Message-
> From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On 
> Behalf Of Alexander Heindl
> Sent: Tuesday, March 22, 2016 3:19 PM
> To: ADSM-L@VM.MARIST.EDU
> Subject: Re: Real world deduplication rates with TSM 7.1 and container 
pools
> 
> Hi,
> 
> but already client side compressed data isn't affected, isn't it?
> what if data is replicated. Is that data then compressed?
> 
> Regards,
> Alex

Re: Real world deduplication rates with TSM 7.1 and container pools

[Erwann SIMON]

Hi all,

TSM deduplication is effective combined with compression. Without compression, 
I'm not sure that it worths what it costs (1:2,5 ratio or 65% is what I 
generally see with mixed data and "standard" retention).

You all should whatch this Tricia Jiang's video on YouTube (from 7:11) : 
https://www.youtube.com/watch?v=ISWRrkY5RS8

-- 
Best regards / Cordialement / مع تحياتي
Erwann SIMON

- Mail original -
De: "PAC Brion Arnaud" 
À: ADSM-L@VM.MARIST.EDU
Envoyé: Vendredi 18 Mars 2016 15:41:06
Objet: [ADSM-L] Real world deduplication rates with TSM 7.1 and container pools

Hi All,

We are currently testing TSM 7.1 deduplication feature, in conjunction with 
container based storage pools.
So far, my test TSM instances, installed with such a setup are reporting dedup 
percentage of 45 %, means dedup factor around 1.81, using a sample of clients 
which are representative of our production environment.
This is unfortunately pretty far from what was promised by IBM (dedup factor of 
4) ...

I'm wondering if anybody making use of container based storage pools and 
deduplication would be sharing his deduplication factor, so that I could have a 
better appreciation of real world figures.
If you would be so kind to share your information (possibly with the kind of 
backed-up data  i.e. VM, DB, NAS, Exchange, and retention values ...) I would 
be very grateful !

Thanks in advance for appreciated feedback.

Cheers.

Arnaud

**
Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. This information is intended only for the person 
(s) named above. If you are not the intended recipient, any disclosure, 
copying, distribution or use or any other action based on the contents of this 
information is strictly prohibited.

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.
**

Re: Real world deduplication rates with TSM 7.1 and container pools

[PAC Brion Arnaud]

Erwann,

Thanks for your input : I had a look at the video which clarified few points, 
but not all of them ...
One question : if using deduped and compressed pool, shall the "compression" 
option on the node definition be left to default "client" value, or must I 
update it to "no" ?
I'm asking because I updated my container pools yesterday to use compression, 
and this morning compression rate on this storage pool is still zero :-(

Cheers.

Arnaud

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Erwann 
SIMON
Sent: Tuesday, March 22, 2016 11:18 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Real world deduplication rates with TSM 7.1 and container pools

Hi all,

TSM deduplication is effective combined with compression. Without compression, 
I'm not sure that it worths what it costs (1:2,5 ratio or 65% is what I 
generally see with mixed data and "standard" retention).

You all should whatch this Tricia Jiang's video on YouTube (from 7:11) : 
https://www.youtube.com/watch?v=ISWRrkY5RS8

-- 
Best regards / Cordialement / مع تحياتي
Erwann SIMON

- Mail original -
De: "PAC Brion Arnaud" 
À: ADSM-L@VM.MARIST.EDU
Envoyé: Vendredi 18 Mars 2016 15:41:06
Objet: [ADSM-L] Real world deduplication rates with TSM 7.1 and container pools

Hi All,

We are currently testing TSM 7.1 deduplication feature, in conjunction with 
container based storage pools.
So far, my test TSM instances, installed with such a setup are reporting dedup 
percentage of 45 %, means dedup factor around 1.81, using a sample of clients 
which are representative of our production environment.
This is unfortunately pretty far from what was promised by IBM (dedup factor of 
4) ...

I'm wondering if anybody making use of container based storage pools and 
deduplication would be sharing his deduplication factor, so that I could have a 
better appreciation of real world figures.
If you would be so kind to share your information (possibly with the kind of 
backed-up data  i.e. VM, DB, NAS, Exchange, and retention values ...) I would 
be very grateful !

Thanks in advance for appreciated feedback.

Cheers.

Arnaud

**
Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. This information is intended only for the person 
(s) named above. If you are not the intended recipient, any disclosure, 
copying, distribution or use or any other action based on the contents of this 
information is strictly prohibited.

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.
**

Re: Real world deduplication rates with TSM 7.1 and container pools

[Erwann SIMON]

Arnaud,

If an object is already compressed (client side compression activated), it will 
be ignored/excluded by server side compression.

-- 
Best regards / Cordialement / مع تحياتي
Erwann SIMON

- Mail original -
De: "PAC Brion Arnaud" 
À: ADSM-L@VM.MARIST.EDU
Envoyé: Mercredi 23 Mars 2016 10:27:58
Objet: Re: [ADSM-L] Real world deduplication rates with TSM 7.1 and container 
pools

Erwann,

Thanks for your input : I had a look at the video which clarified few points, 
but not all of them ...
One question : if using deduped and compressed pool, shall the "compression" 
option on the node definition be left to default "client" value, or must I 
update it to "no" ?
I'm asking because I updated my container pools yesterday to use compression, 
and this morning compression rate on this storage pool is still zero :-(

Cheers.

Arnaud

-Original Message-
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Erwann 
SIMON
Sent: Tuesday, March 22, 2016 11:18 PM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Real world deduplication rates with TSM 7.1 and container pools

Hi all,

TSM deduplication is effective combined with compression. Without compression, 
I'm not sure that it worths what it costs (1:2,5 ratio or 65% is what I 
generally see with mixed data and "standard" retention).

You all should whatch this Tricia Jiang's video on YouTube (from 7:11) : 
https://www.youtube.com/watch?v=ISWRrkY5RS8

-- 
Best regards / Cordialement / مع تحياتي
Erwann SIMON

- Mail original -
De: "PAC Brion Arnaud" 
À: ADSM-L@VM.MARIST.EDU
Envoyé: Vendredi 18 Mars 2016 15:41:06
Objet: [ADSM-L] Real world deduplication rates with TSM 7.1 and container pools

Hi All,

We are currently testing TSM 7.1 deduplication feature, in conjunction with 
container based storage pools.
So far, my test TSM instances, installed with such a setup are reporting dedup 
percentage of 45 %, means dedup factor around 1.81, using a sample of clients 
which are representative of our production environment.
This is unfortunately pretty far from what was promised by IBM (dedup factor of 
4) ...

I'm wondering if anybody making use of container based storage pools and 
deduplication would be sharing his deduplication factor, so that I could have a 
better appreciation of real world figures.
If you would be so kind to share your information (possibly with the kind of 
backed-up data  i.e. VM, DB, NAS, Exchange, and retention values ...) I would 
be very grateful !

Thanks in advance for appreciated feedback.

Cheers.

Arnaud

**
Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.br...@panalpina.com<mailto:arnaud.br...@panalpina.com>
This electronic message transmission contains information from Panalpina and is 
confidential or privileged. This information is intended only for the person 
(s) named above. If you are not the intended recipient, any disclosure, 
copying, distribution or use or any other action based on the contents of this 
information is strictly prohibited.

If you receive this electronic transmission in error, please notify the sender 
by e-mail, telephone or fax at the numbers listed above. Thank you.
**