Re: ZFS dedup and temporary freeze

[George Kontostanos]

Very useful thank you!

DDT-sha256-zap-duplicate: 177 entries, size 9493 on disk, 12010 in core
DDT-sha256-zap-unique: 20016 entries, size 1232 on disk, 1524 in core

DDT histogram (aggregated over all DDTs):

bucket  allocated   referenced
__   __   __
refcnt   blocks   LSIZE   PSIZE   DSIZE   blocks   LSIZE   PSIZE   DSIZE
--   --   -   -   -   --   -   -   -
 119.5K   2.44G   2.44G   2.44G19.5K   2.44G   2.44G   2.44G
 2  175   21.9M   21.9M   21.9M  407   50.9M   50.9M   50.9M
 41128K128K128K6768K768K768K
   2561128K128K128K  297   37.1M   37.1M   37.1M
 Total19.7K   2.46G   2.46G   2.46G20.2K   2.53G   2.53G   2.53G

dedup = 1.03, compress = 1.00, copies = 1.00, dedup * compress / copies =
1.03



On Wed, Mar 30, 2011 at 7:11 PM, Sean M. Collins  wrote:

> Can you attach the output from zdb -DD zhome ?
>
> --
> Sean Collins
> Core IT Pro, LLC
> www.coreitpro.com
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>



-- 
George Kontostanos
aisecure.net 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ZFS dedup and temporary freeze

[Sean M. Collins]

Can you attach the output from zdb -DD zhome ?

-- 
Sean Collins
Core IT Pro, LLC
www.coreitpro.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ZFS dedup and temporary freeze

[George Kontostanos]

As far as I am aware ezjail has the option to use compression and disable
access time updates.

ezjail_zfs_properties="-o compression=lzjb -o atime=off"

In my case the dedup option was turned on only in the basejail causing the
system to temporarily freeze. Once the dedup option was removed everything
went back to normal.
What I am trying to figure out here is why did this occur.

Thanks

On Wed, Mar 30, 2011 at 10:53 AM, Alexander Leidinger <
alexan...@leidinger.net> wrote:

> Quoting George Kontostanos  (from Tue, 29 Mar 2011
> 17:57:49 +0300):
>
>  I thought that jails would be a good candidate for dedup. So far I am
>> using
>> it without any issues in my /usr/src and /virtualBox directories.
>>
>
> No idea about your problem, but ezjail is already doing some dedup. The
> basejail is the dedup-place, and the jails do a null-mount of it. Inside the
> jails are only some links to the basejail. So if you do not install the same
> programs (the same packages, so make sure there is no different compile
> timestamp (some programs have this) inside the newly generated binaries) or
> data inside the jail-specific area, you will not have a benefit from this.
>
> Bye,
> Alexander.
>
> --
> Real programmers never work 9 to 5.  If any real
> programmers are around at 9 a.m., it's because they
> were up all night.
>
> http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7
> http://www.FreeBSD.org   netchild @ FreeBSD.org  : PGP ID = 72077137
>



-- 
George Kontostanos
aisecure.net 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ZFS dedup and temporary freeze

[Alexander Leidinger]

Quoting George Kontostanos  (from Tue, 29 Mar  
2011 17:57:49 +0300):



I thought that jails would be a good candidate for dedup. So far I am using
it without any issues in my /usr/src and /virtualBox directories.


No idea about your problem, but ezjail is already doing some dedup.  
The basejail is the dedup-place, and the jails do a null-mount of it.  
Inside the jails are only some links to the basejail. So if you do not  
install the same programs (the same packages, so make sure there is no  
different compile timestamp (some programs have this) inside the newly  
generated binaries) or data inside the jail-specific area, you will  
not have a benefit from this.


Bye,
Alexander.

--
Real programmers never work 9 to 5.  If any real
programmers are around at 9 a.m., it's because they
were up all night.

http://www.Leidinger.netAlexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org   netchild @ FreeBSD.org  : PGP ID = 72077137
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ZFS dedup and temporary freeze

[George Kontostanos]

The system has 4Gs of RAM. I don''t use any devices for L2ARC and my tuning
is like this:

vm.kmem_size="3072M"
vfs.zfs.arc_max="2048M"

I had my eyes on the memory during the whole procedure and things looked
"normal".

Thanks

On Tue, Mar 29, 2011 at 10:53 PM, Sean M. Collins wrote:

> How much memory do you have in the system? Any L2ARC devices? Depending
> on the amount of memory in your box, it could be that ddt is spilling
> over ARC?
>
>
> --
> Sean Collins
> Core IT Pro, LLC
> www.coreitpro.com
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>



-- 
George Kontostanos
aisecure.net 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ZFS dedup and temporary freeze

[Sean M. Collins]

How much memory do you have in the system? Any L2ARC devices? Depending
on the amount of memory in your box, it could be that ddt is spilling
over ARC?


-- 
Sean Collins
Core IT Pro, LLC
www.coreitpro.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

ZFS dedup and temporary freeze

[George Kontostanos]

Hi List,

I am using dedup=on for my ezjail /jails directory. I noticed a strange
behavior during the installation of the ports tree in the basejail. The
system appears to freeze for a while and the cpu appears to be idle during
that time. The same behavior is shown when building ports inside the jail.

Here is some information about the pool:

   capacity operationsbandwidth
poolalloc   free   read  write   read  write
--  -  -  -  -  -  -
zhome   15.0G  59.5G  4 14  61.2K   468K
zhome   15.0G  59.5G 38 66   107K  68.9K
zhome   15.0G  59.5G 43 71   119K  45.0K
zhome   15.0G  59.5G 37 67   103K  45.0K
zhome   15.0G  59.5G 39103   102K  4.71M
zhome   15.0G  59.5G 42 72   114K  45.0K
zhome   15.0G  59.5G 36 59  97.9K  37.0K
zhome   15.0G  59.5G 42 69   114K  48.0K
zhome   15.0G  59.5G 39 72   105K  43.5K
zhome   15.0G  59.5G 33 58  93.4K  42.0K
zhome   15.0G  59.5G 44 77   117K  53.4K

NAMESIZE  ALLOC   FREECAP  DEDUP  HEALTH  ALTROOT
zhome  74.5G  15.8G  58.7G21%  1.06x  ONLINE  -

I thought that jails would be a good candidate for dedup. So far I am using
it without any issues in my /usr/src and /virtualBox directories.

Best Regards

-- 
George Kontostanos
aisecure.net 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"