Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
It's supposed to be
7111576: arc shrinks in the absence of memory pressure
currently in status "accepted" and an RPE escalation pending.
-Original Message-
From: zfs-discuss-boun...@opensolaris.org
[mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Tomas Forsman
Sent: Donnerstag, 5. Januar 2012 10:35
To: zfs-discuss@opensolaris.org
Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
On 04 January, 2012 - Steve Gonczi sent me these 2,5K bytes:
> The interesting bit is what happens inside arc_reclaim_needed(), that
> is, how it arrives at the conclusion that there is memory pressure.
>
> Maybe we could trace arg0, which gives the location where we have left
> the function. This would finger which return path
> arc_reclaim_needed() took.
It's new code, basically comparing the inuse/total/free from kstat -n
zfs_file_data, which seems buggered.
> Steve
>
>
> - Original Message -
>
>
>
> Well it looks like the only place this get's changed is in the
> arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see what
> is going on and investigate what is happening to the return code of the
> arc_reclaim_needed is.
>
>
>
> http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/comm
> on/fs/zfs/arc.c#2089
>
>
> maybe
>
>
> dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }'
>
>
> Dave
>
>
>
>
>
/Tomas
--
Tomas Forsman, st...@acc.umu.se, http://www.acc.umu.se/~stric/
|- Student at Computing Science, University of Umeå
`- Sysadmin at {cs,acc}.umu.se
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Thanks. The guys from Oracle are currently looking at some new code that was
introduced in arc_reclaim_thread() between b151a and b175.
Peter Radig, Ahornstrasse 34, 85774 Unterföhring, Germany
tel: +49 89 99536751 - fax: +49 89 99536754 - mobile: +49 171 2652977
email: pe...@radig.de<mailto:pe...@radig.de>
From: David Blasingame [mailto:dbla...@yahoo.com]
Sent: Mittwoch, 4. Januar 2012 17:35
To: Peter Radig; st...@acc.umu.se
Cc: zfs-discuss@opensolaris.org
Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Well it looks like the only place this get's changed is in the
arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see what
is going on and investigate what is happening to the return code of the
arc_reclaim_needed is.
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#2089
maybe
dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }'
Dave
Original Message
Subject:
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Date:
Tue, 3 Jan 2012 23:50:04 +0000
From:
Peter Radig <mailto:pe...@radig.de>
To:
Tomas Forsman <mailto:st...@acc.umu.se>
CC:
zfs-discuss@opensolaris.org<mailto:zfs-discuss@opensolaris.org>
<mailto:zfs-discuss@opensolaris.org>
Tomas,
Yup, same here.
free0
mem_inuse 16162750464
mem_total 17171480576
Page SummaryPagesMB %Tot
Kernel 842305 3290 20%
ZFS File Data2930110%
Anon44038 1721%
Exec and libs3731140%
Page cache 8580330%
Free (cachelist) 5504210%
Free (freelist) 3284924 12831 78%
Total 4192012 16375
Physical 4192011 16375
I will create an SR with Oracle.
Thanks,
Peter
-Original Message-
From: Tomas Forsman [mailto:st...@acc.umu.se]
Sent: Mittwoch, 4. Januar 2012 00:39
To: Peter Radig
Cc: zfs-discuss@opensolaris.org<mailto:zfs-discuss@opensolaris.org>
Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
On 03 January, 2012 - Peter Radig sent me these 3,5K bytes:
> Hello.
>
> I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple of
> weeks ago).
>
> Without no obvious reason (at least for me), after an uptime of 1 to 2 days
> (observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it
> back to 0. ARC is being shrunk to less than 1 GB -- needless to say that
> performance is terrible. There is not much load on this system.
>
> Memory seems to be not an issue (see below).
>
> I looked at the old Nevada code base of onnv_147 and can't find a reason for
> this happening.
>
> How can I find out what's causing this?
New code that seems to be counting wrong.. I was planning on filing a bug, but
am currently struggling to convince oracle that we bought support..
Try this:
kstat -n zfs_file_data
In my case, I get:
free15322
mem_inuse 24324866048
mem_total 25753026560
.. where ::memstat says:
Kernel2638984 10308 42%
ZFS File Data 39260 1531%
Anon 873549 3412 14%
Exec and libs5199200%
Page cache 20019780%
Free (cachelist) 6608250%
Free (freelist) 2703509 10560 43%
On another reboot, it refused to go over 130MB on a 24GB system..
> BTW: I was not seeing this on SolEx 11/10.
Dito.
> Thanks,
> Peter
>
>
>
> *** ::memstat ***
> Page SummaryPagesMB %Tot
>
> Kernel 860254 3360 21%
> ZFS File Data3047110%
> Anon38246 1491%
> Exec and libs3765140%
> Page cache 8517330%
> Free (cachelist) 5866220%
> Free (freelist) 3272317 12782 78%
> Total 4192012 16375
> Physical 4192011 16375
>
> mem_inuse
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Tomas,
Yup, same here.
free0
mem_inuse 16162750464
mem_total 17171480576
Page SummaryPagesMB %Tot
Kernel 842305 3290 20%
ZFS File Data2930110%
Anon44038 1721%
Exec and libs3731140%
Page cache 8580330%
Free (cachelist) 5504210%
Free (freelist) 3284924 12831 78%
Total 4192012 16375
Physical 4192011 16375
I will create an SR with Oracle.
Thanks,
Peter
-Original Message-
From: Tomas Forsman [mailto:st...@acc.umu.se]
Sent: Mittwoch, 4. Januar 2012 00:39
To: Peter Radig
Cc: zfs-discuss@opensolaris.org
Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
On 03 January, 2012 - Peter Radig sent me these 3,5K bytes:
> Hello.
>
> I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple of
> weeks ago).
>
> Without no obvious reason (at least for me), after an uptime of 1 to 2 days
> (observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it
> back to 0. ARC is being shrunk to less than 1 GB -- needless to say that
> performance is terrible. There is not much load on this system.
>
> Memory seems to be not an issue (see below).
>
> I looked at the old Nevada code base of onnv_147 and can't find a reason for
> this happening.
>
> How can I find out what's causing this?
New code that seems to be counting wrong.. I was planning on filing a bug, but
am currently struggling to convince oracle that we bought support..
Try this:
kstat -n zfs_file_data
In my case, I get:
free15322
mem_inuse 24324866048
mem_total 25753026560
.. where ::memstat says:
Kernel2638984 10308 42%
ZFS File Data 39260 1531%
Anon 873549 3412 14%
Exec and libs5199200%
Page cache 20019780%
Free (cachelist) 6608250%
Free (freelist) 2703509 10560 43%
On another reboot, it refused to go over 130MB on a 24GB system..
> BTW: I was not seeing this on SolEx 11/10.
Dito.
> Thanks,
> Peter
>
>
>
> *** ::memstat ***
> Page SummaryPagesMB %Tot
>
> Kernel 860254 3360 21%
> ZFS File Data3047110%
> Anon38246 1491%
> Exec and libs3765140%
> Page cache 8517330%
> Free (cachelist) 5866220%
> Free (freelist) 3272317 12782 78%
> Total 4192012 16375
> Physical 4192011 16375
>
> mem_inuse 4145901568
> mem_total 1077466365952
>
> *** ::arc ***
> hits = 186279921
> misses= 14366462
> demand_data_hits = 4648464
> demand_data_misses= 8605873
> demand_metadata_hits = 171803126
> demand_metadata_misses= 3805675
> prefetch_data_hits=772678
> prefetch_data_misses = 1464457
> prefetch_metadata_hits= 9055653
> prefetch_metadata_misses =490457
> mru_hits = 12295087
> mru_ghost_hits= 0
> mfu_hits = 175281066
> mfu_ghost_hits= 0
> deleted = 14462192
> mutex_miss=30
> hash_elements = 3752768
> hash_elements_max = 3752770
> hash_collisions = 11409790
> hash_chains = 8256
> hash_chain_max=20
> p =48 MB
> c = 781 MB
> c_min =64 MB
> c_max = 15351 MB
> size = 788 MB
> buf_size = 185 MB
> data_size = 289 MB
> other_size= 313 MB
> l2_hits = 0
> l2_misses = 14366462
> l2_feeds
[zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Hello.
I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple of
weeks ago).
Without no obvious reason (at least for me), after an uptime of 1 to 2 days
(observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it
back to 0. ARC is being shrunk to less than 1 GB -- needless to say that
performance is terrible. There is not much load on this system.
Memory seems to be not an issue (see below).
I looked at the old Nevada code base of onnv_147 and can't find a reason for
this happening.
How can I find out what's causing this?
BTW: I was not seeing this on SolEx 11/10.
Thanks,
Peter
*** ::memstat ***
Page SummaryPagesMB %Tot
Kernel 860254 3360 21%
ZFS File Data3047110%
Anon38246 1491%
Exec and libs3765140%
Page cache 8517330%
Free (cachelist) 5866220%
Free (freelist) 3272317 12782 78%
Total 4192012 16375
Physical 4192011 16375
mem_inuse 4145901568
mem_total 1077466365952
*** ::arc ***
hits = 186279921
misses= 14366462
demand_data_hits = 4648464
demand_data_misses= 8605873
demand_metadata_hits = 171803126
demand_metadata_misses= 3805675
prefetch_data_hits=772678
prefetch_data_misses = 1464457
prefetch_metadata_hits= 9055653
prefetch_metadata_misses =490457
mru_hits = 12295087
mru_ghost_hits= 0
mfu_hits = 175281066
mfu_ghost_hits= 0
deleted = 14462192
mutex_miss=30
hash_elements = 3752768
hash_elements_max = 3752770
hash_collisions = 11409790
hash_chains = 8256
hash_chain_max=20
p =48 MB
c = 781 MB
c_min =64 MB
c_max = 15351 MB
size = 788 MB
buf_size = 185 MB
data_size = 289 MB
other_size= 313 MB
l2_hits = 0
l2_misses = 14366462
l2_feeds = 0
l2_rw_clash = 0
l2_read_bytes = 0 MB
l2_write_bytes= 0 MB
l2_writes_sent= 0
l2_writes_done= 0
l2_writes_error = 0
l2_writes_hdr_miss= 0
l2_evict_lock_retry = 0
l2_evict_reading = 0
l2_abort_lowmem = 490
l2_cksum_bad = 0
l2_io_error = 0
l2_hdr_size = 0 MB
memory_throttle_count = 0
meta_used = 499 MB
meta_max = 1154 MB
meta_limit= 0 MB
arc_no_grow = 1
arc_tempreserve = 0 MB
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Snapshot size as reported by the USED property
I create snapshots on my datasets quite frequently. My understanding of the USED property of a snapshot is that it indicates the amount of data that was written to the dataset after the snapshot was taken. But now I'm seeing a snapshot with USED == 0 where there was definitely write activity after it was taken. Is my understanding wrong or do I see a thing that is not supposed to happen? I am on NCP3. Thanks, Peter -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Sharing Issues
As I explained earlier, this is not possible with CIFS. This is the RFE entry: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6582165 And the explanation is here: http://mail.opensolaris.org/pipermail/cifs-discuss/2009-March/001397.html Peter -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Tau Sent: 22. February 2010 22:40 To: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Sharing Issues It will not let me set sharesmb=none. Also I dont see how ACL's would do what im after here... I want to create a nested dataset inside another one so that I can create snapshots, and export that dataset when needed. Though I do not want the nested dataset to have its own share... (if that makes sence) Scenario; tank1/websites/website1 /website2 /website3 Now when i create website1-3 datasets inside of the websites dataset it shares them as well. So from my windows/nix boxes i see; websites websites_website1 websites_website2 websites_website3 now i JUST want to see "websites" then be able to access the website1-3 datasets from navigating through the websites share. The way it is sharing right now im going to end up with 100's of shares were i only really need to have 6 or 8, and the rest accessible through the root datasets. I hope this is a better explanation. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Sharing Issues
It doesn't work with CIFS. There is an open RFE on that for quite some time now. Peter On 22.02.2010, at 08:09, "Tau" wrote: > I am having a bit of an issue I have an opensolaris box setup as a > fileserver. Running through CIFS to provide shares to some windows > machines. > > Now lets call my zpool /tank1, when i create a zfs filesystem > called /test it gets shared as /test and i can see it as "test" on > my windows machines... Now when i create a child system inside the > test system (lets call this /tank1/test/child) the child system gets > shared as well on its own as test_child as seen on the windows system. > > I want to be able to create nested filesystems, and not have the > nested systems shared through cifs i want to access them > through the root system, and only have the root systems shared to > the windows machines... > > I have been trolling through the manuals, and forums, but cant seem > to find the ansear. > > I'm sure im missing something simple, can someone shed some light > onto this issue? > -- > This message posted from opensolaris.org > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] File created with CIFS is not immediately deletable on local file system
Box running osol_133 with smb/server enabled. I create a file on a Windows box that has a remote ZFS fs mounted. I go to the Solaris box and try to remove the file and get "permission denied" for up 30 sec. Than it works. A "sync" immediately before the rm seems to speed things up and rm is successful immediately afterwards, too. The permissions of the file don't change in between and look as follow: --+ 1 petersysprog0 Feb 21 11:37 NEWFILE3.txt user:peter:rwxpdDaARWcCos:---:allow group:2147483648:rwxpdDaARWcCos:---:allow Is this a bug or a feature? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Impact of an enterprise class SSD on ZIL performance
I was interested in the impact the type of an SSD has on the performance of the ZIL. So I did some benchmarking and just want to share the results. My test case is simply untarring the latest ON source (528 MB, 53k files) on an Linux system that has a ZFS file system mounted via NFS over gigabit ethernet. I got the following results: - locally on the Solaris box: 30 sec - remotely with no dedicated ZIL device: 36 min 37 sec (factor 73 compared to local) - remotely with ZIL disabled: 1 min 54 sec (factor 3.8 compared to local) - remotely with a OCZ VERTEX SATA II 120 GB as ZIL device: 14 min 40 sec (factor 29.3 compared to local) - remotely with an Intel X25-E 32 GB as ZIL device: 3 min 11 sec (factor 6.4 compared to local) So it really makes a difference what type of SSD you use for your ZIL device. I was expecting a good performance from the X25-E, but was really suprised that it is that good (only 1.7 times slower than it takes with ZIL completely disabled). So I will use the X25-E as ZIL device on my box and will not consider disabling ZIL at all to improve NFS performance. -- Peter -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
