[zfs-code] mze_insert() panic (avl_find() succeeded inside avl_add())

2009-09-09 Thread Matthew Ahrens
Pawel Jakub Dawidek wrote: > On Wed, Sep 09, 2009 at 10:07:08AM -0700, Matthew Ahrens wrote: >> Unfortunately, I don't think that the send stream would help us diagnose it >> further. What we really need is a way to reproduce how to get into this >> situation in the fi

[zfs-code] mze_insert() panic (avl_find() succeeded inside avl_add())

2009-09-09 Thread Matthew Ahrens
Unfortunately, I don't think that the send stream would help us diagnose it further. What we really need is a way to reproduce how to get into this situation in the first place. --matt Pawel Jakub Dawidek wrote: > Hi. > > There is a bug (6709336), which was closed with 'Not Reproducible' reas

[zfs-code] Expanding a raidz pool?

2009-08-28 Thread Matthew Ahrens
Ty Newton wrote: > Hi, > I've read a few articles about the lack of 'simple' raidz pool expansion > capability in ZFS. I am interested in having a go at developing this > functionality. Is anyone working on this at the moment? FYI, Adam Leventhal and I worked out how to do this a while back.

[zfs-code] Can ZFS super-size (scale) me ?

2009-06-22 Thread Matthew Ahrens
gary mazz wrote: > Hi, > > I'd like to use ZFS in a highly scalable data application. > > My requirements are very loose at this point. but I know I need object > locking and I'd like to use the DMU interface. I would like the DMU > transactions to be distributed across machines ? The preferr

[zfs-code] arc.c: variable types observation

2009-06-02 Thread Matthew Ahrens
Marc Moreau wrote: > * This comment is noted from reading > * > http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c > * which may or may not be the current version. > > I was reading through the code and I noticed that arc_no_grow and arc_dead > are declared as

[zfs-code] Superfluous inits in arc.c (and perhaps elsewhere)

2009-06-01 Thread Matthew Ahrens
Jeremy Archer wrote: > Also, someone may think that those init's are necessary, and chase a > non-existent bug when one of them is missing. :-) We try to keep all the mutex_init() & friends there for the ZFS ports to other operating systems. Some of them need it. --matt

[zfs-code] PSARC/2009/204 ZFS user/group quotas & space accounting

2009-04-21 Thread Matthew Ahrens
Mark Maybee wrote: > Pawel Jakub Dawidek wrote: >> On Sat, Apr 18, 2009 at 06:05:56PM -0700, Matthew.Ahrens at sun.com wrote: >>> Author: Matthew Ahrens >>> Repository: /hg/onnv/onnv-gate >>> Latest revision: f41cf682d0d3e3cf5c4ec17669b903ae621ef882 >

[zfs-code] Have an idea about ZFS improvements - step to the user. Is it actual?

2009-03-30 Thread Matthew Ahrens
Eugene Turkulevich wrote: > Hi, I have an idea about improvements of ZFS from the one side and > improvements for of the GUI interface from the other side. Let me describe my > idea. > >>From the one side we have an ZFS on-th-fly snapshots, and starting from >>OpenSolaris 2008.11 we have TimeSl

[zfs-code] How does zfs COW deal with '..' in brother directory?

2008-11-12 Thread Matthew Ahrens
Eddie Edwards wrote: > In principle, I think you're right, in that an objset could hold both > versions of a file. It would contain physical pointers to each copy. > They would start out the same and diverge when one or other copy was > modified. > > For instance, zfs could support a fast "cp" wh

[zfs-code] How does zfs COW deal with '..' in brother directory?

2008-11-11 Thread Matthew Ahrens
Chen Zheng wrote: > Hi matt, > > I have some problems about understanding zfs COW implemention. Suppose > b and c are both children dir of a, if c changes, there will be new > versions of both a and c, namely c' and a'. > > a a' > b c c' > Actually, there will not be a new version of A unl

[zfs-code] acitive dataset & object directory [C1]

2008-11-04 Thread Matthew Ahrens
Cyril ROUHASSIA wrote: > > Dear all, > I have some problems with zfs implementation issues: > > * I do not really understand what is a DSL directory (I though is > was kind of structure listing all datasets + properties..) and > what is the unique "Active Dataset" within a DSL

[zfs-code] panic dirtying dbuf obj=3d199 lvl=1 blkid=1 but not tx_held

2008-09-30 Thread Matthew Ahrens
yes, please do, and include a crash dump. --matt Nils Goroll wrote: > Has happened again after I replaced the faulty cable, seems to be > reproducible > with my zfs send/receive script. > > Should I open a bug? > > panic[cpu1]/thread=ff0167f4b200: > dirtying dbuf obj=3d199 lvl=1 blkid=1 b

[zfs-code] Review for 6729208 Optimize macros in sys/byteorder.h (due Sept. 3)

2008-09-02 Thread Matthew Ahrens
How do we get 4% faster when running on the same endianness as the fs was created on? (created on x86, run on amd64) Where are the bulk of the calls to BSWAP_* happening in that case? --matt Dan Anderson wrote: > Here's some performance results running " find . -exec ls -l" on separate ZFS >

[zfs-code] Design for EAs in dnode.

2008-02-08 Thread Matthew Ahrens
Andreas Dilger wrote: > On Feb 07, 2008 17:14 +, Ricardo Correia wrote: >> On Qui, 2008-02-07 at 09:44 -0700, Mark Shellenbaum wrote: >>> Section 2.2.4 >>> >>> Not sure what you mean by storing all xattrs in a single object? Are you >>> saying all of the Lustre attributes would be stored in a

[zfs-code] Design for EAs in dnode.

2008-02-08 Thread Matthew Ahrens
Ricardo M. Correia wrote: > On Qui, 2008-02-07 at 22:51 -0700, Neil Perrin wrote: >> I believe when a prototype using 1K dnodes was tested it showed an >> unacceptable (30%?) hit on some benchmarks. So if can possibly >> avoid increasing the dnode size (by default) then we should do so. > > Hmm, i

[zfs-code] Design for EAs in dnode.

2008-02-07 Thread Matthew Ahrens
Ricardo M. Correia wrote: > > On Qui, 2008-02-07 at 12:48 -0800, Matthew Ahrens wrote: >> First, note that the problem is quite constrained: >> >> There are a fixed number of attribute names (for any given software version) >> since these are interpreted sy

[zfs-code] Design for EAs in dnode.

2008-02-07 Thread Matthew Ahrens
Mark Shellenbaum wrote: > Girish Shilamkar wrote: >> Hi, >> With reference to my previous mail ([zfs-code] Nvpair for storing EAs >> in dnode) which had expressed our intent to use libnvpair for EAs in >> dnode. I am now sending a high level design document for the same. >> >> Any suggestions/

[zfs-code] A question about zfs code - dbuf related

2008-02-05 Thread Matthew Ahrens
jay xiong wrote: > I have two questions for zfs code: > > In dbuf_hold_imple(): > In this piece of code, it will copy the data to the extra buffer if the > current one is syncing into disks. My question is that why not deferring this > data copy to dbuf_dirty, where we check the db_data_pending

[zfs-code] New ZAP object based on Micro-ZAP

2007-12-14 Thread Matthew Ahrens
Girish Shilamkar wrote: > Hi, > While using DMU for lustre we need to store some extra information in > directory entries which is required for lustre operations. The size of > typical entries to be stored in directory entries are 16 bytes, hence we > can't use micro zap. And we would like to

[zfs-code] R/W lock portability issue

2007-11-26 Thread Matthew Ahrens
Neil Perrin wrote: > Matthew Ahrens wrote: >> Neil Perrin wrote: >>> >>> Ricardo M. Correia wrote: >>>> Ricardo M. Correia wrote: >>>>> boolean_t need_lock = !RW_LOCK_HELD(&dp->dp_config_rwlock); >>>>> >>>>>

[zfs-code] R/W lock portability issue

2007-11-26 Thread Matthew Ahrens
Neil Perrin wrote: > > Ricardo M. Correia wrote: >> Ricardo M. Correia wrote: >>> boolean_t need_lock = !RW_LOCK_HELD(&dp->dp_config_rwlock); >>> >>> if (need_lock) >>> rw_enter(&dp->dp_config_rwlock, RW_READER); >>> >> Maybe I posted to soon. >> Am I right that this specific code (in dsl_

[zfs-code] Start of the MOS

2007-11-13 Thread Matthew Ahrens
Mark Furner wrote: > Hi > > Am not sure who to address this query to, there is no list for ZFS forensics > (yet). I'm looking at a DD dump of a VTOC partition containing a ZFS test > file system, follow the Uberblock DVAs to 3 separate locations (ditto blocks, > as expected) but don't find an o

[zfs-code] System call to create a clone of a file on a ZFS filesystem?

2007-10-12 Thread Matthew Ahrens
Pawel Jakub Dawidek wrote: > On Fri, Oct 12, 2007 at 11:10:31AM +0100, Darren J Moffat wrote: >> Pawel Jakub Dawidek wrote: >>> I understand it's not trivial, but beeing able to reference the same >>> block from different datasets would be a really nice feature to have. >>> The functionality discus

[zfs-code] System call to create a clone of a file on a ZFS filesystem?

2007-10-11 Thread Matthew Ahrens
Pawel Jakub Dawidek wrote: > On Thu, Oct 11, 2007 at 10:47:44AM +0100, Robert Milkowski wrote: >> Hello Matthew, >> >> Thursday, October 11, 2007, 9:10:13 AM, you wrote: >> >> MA> Robert Milkowski wrote: I haven't looked into details but in theory one should be able to copy/move a file wi

[zfs-code] System call to create a clone of a file on a ZFS filesystem?

2007-10-11 Thread Matthew Ahrens
Robert Milkowski wrote: > I haven't looked into details but in theory one should be > able to copy/move a file within the same datapool between datasets > without having to actually copy data blocks... or maybe there's some > detail which actually makes it hard to implement... Once a block is refe

[zfs-code] Increasing dnode size

2007-09-20 Thread Matthew Ahrens
Darren J Moffat wrote: > Andreas Dilger wrote: > >> I agree, but I suspect large dnodes could also be of use to ZFS at >> some point, either for fast EAs and/or small files, so we wanted to >> get some buy-in from the ZFS developers on an approach that would >> be suitable for ZFS also. In partic

[zfs-code] Which ZFS DMU objects should be encrypted ?

2007-09-20 Thread Matthew Ahrens
eric kustarz wrote: > On Sep 12, 2007, at 4:31 AM, Darren J Moffat wrote: > >> The DMU layer will determine which objects in be encrypted when being >> placed in a dataset that has the encryption property turned on. >> >> http://opensolaris.org/os/project/zfs-crypto/phase1/dmu_ot/ >> >> Which of t

[zfs-code] Extending RAIDZ.

2007-09-19 Thread Matthew Ahrens
Pawel Jakub Dawidek wrote: > On Tue, Aug 07, 2007 at 11:28:31PM +0100, James Blackburn wrote: >> Well I read this email having just written a mammoth one in the other >> thread, my thoughts: >> >> The main difficulty in this, as far as I see it, is you're >> intentionally moving data on a checksumm

[zfs-code] Adding New Zpool Property

2007-09-19 Thread Matthew Ahrens
Duff, You should resync with Lin Ling's putback for: PSARC 2007/342 "Enhanced ZFS Pool Properties" from this morning. It substantially changes the way the pool properties are implemented, and should make implementing a new pool property much easier. (Also note, I fixed particular nastyness th

[zfs-code] Which ZFS DMU objects should be encrypted ?

2007-09-19 Thread Matthew Ahrens
Darren J Moffat wrote: > The DMU layer will determine which objects in be encrypted when being > placed in a dataset that has the encryption property turned on. > > http://opensolaris.org/os/project/zfs-crypto/phase1/dmu_ot/ > > Which of these marked as FALSE can be and should be *TRUE* ? > Ac

[zfs-code] DMU as general purpose transaction engine?

2007-09-05 Thread Matthew Ahrens
Atul Vidwansa wrote: > ZFS Experts, > > Is it possible to use DMU as general purpose transaction engine? More > specifically, in following order: > > 1. Create transaction: > tx = dmu_tx_create(os); > error = dmu_tx_assign(tx, TXG_WAIT) > > 2. Decide what to modify(say create new object): > dmu_

[zfs-code] statvfs change

2007-09-05 Thread Matthew Ahrens
Don Cragun wrote: > Chris, > If ZFS is ever going to pass the test suites so we can claim > that ZFS meets POSIX and UNIX filesystem requirements, you'll find that > the conformance test suites will be trying things similar to what I've > been describing. The test suites certainly aren't per

[zfs-code] ZFS vs SPA version

2007-08-20 Thread Matthew Ahrens
Darren J Moffat wrote: > George Wilson wrote: >> New code should either update the SPA_VERSION whenever you change the >> on-disk format and/or the new ZPL_VERSION whenever you change the Posix >> layer. > > That is what I thought but as you say it is currently a little confusing. > > Given tha

[zfs-code] Read-only (forensic) mounts of ZFS

2007-07-27 Thread Matthew Ahrens
Mark Furner wrote: > Thanks for the clarification, Darren, and sorry for cross-posting. > > OK, physical device -> pool -> file-system(s) > > Some questions: > > 1) zpool import allows options similar to zfs. Can I set the same or similar > read-only (RO) options for the whole pool > (noexec,n

[zfs-code] zbookmark_t to dataset & pool name

2007-07-04 Thread Matthew Ahrens
Darren J Moffat wrote: > While doing a zio we have the information about what pool and dataset we > are operating on encoded in a zbookmark_t. > > I found I can go from a object to a dataset name using > dsl_dsobj_to_dsname() but that needs a pool name. > > The reason I need the dataset name is

[zfs-code] properties not visible via zfs(1)

2007-07-03 Thread Matthew Ahrens
George Wilson wrote: > Darren J Moffat wrote: >> Is it possible to have dataset properties that are managed using the >> dsl_prop_set() / dsl_prop_get() interfaces that aren't made available >> via zfs(1), in fact I probably don't want them in userland at all. >> > You can set the pd_visible f

[zfs-code] Space allocation failure

2007-06-27 Thread Matthew Ahrens
Manoj Joseph wrote: > Hi, > > Replying to myself again. :) > > I see this problem only if I attempt to use a zpool that already exists. > If I create one (using files instead of devices, don't know if it > matters) like ztest does, it works like a charm. You should probably be posting on zfs-d

[zfs-code] Dataset Key storage

2007-05-08 Thread Matthew Ahrens
Darren J Moffat wrote: > Matthew Ahrens wrote: >> Darren J Moffat wrote: >>> My initial thought was using *dsl_dataset_phys_t* however that >>> doesn't have sufficient space so it may be better to store a >>> "pointer" there to some external

[zfs-code] Dataset Key storage

2007-05-04 Thread Matthew Ahrens
Darren J Moffat wrote: > My initial thought was using > *dsl_dataset_phys_t* however that doesn't have sufficient space so it > may be better to store a "pointer" there to some external object. Yep, I would recommend that you store each key in an object in the MOS, and keep that object's number

[zfs-code] ZFS_DIRENT_*().

2007-05-01 Thread Matthew Ahrens
Pawel Jakub Dawidek wrote: > On Tue, May 01, 2007 at 05:48:15PM -0700, Matthew Ahrens wrote: >> Pawel Jakub Dawidek wrote: >>> so I started to use it. The thing is that it broke some functionality. >>> For example 'zpool status -v' doesn't show file n

[zfs-code] ZFS_DIRENT_*().

2007-05-01 Thread Matthew Ahrens
Pawel Jakub Dawidek wrote: > so I started to use it. The thing is that it broke some functionality. > For example 'zpool status -v' doesn't show file names anymore. > > I tracked it down to the zap_value_search() function and changing: > > if (za->za_first_integer == value) { > into: >

[zfs-code] What is the DTL for?

2007-04-06 Thread Matthew Ahrens
Jeremy Teo wrote: > Hello, > > I'm curious as to what the Dirty Time Log structures are for : namely > > 1) vdev_dtl_map > 2) vdev_dtl_scrub The DTL keeps track of times (txg ranges) when a device was offline. For example, in a mirrored pair, one side of the mirror may go offline for some time

[zfs-code] dmu_tx_wait and dmu_tx_try_assign

2007-01-30 Thread Matthew Ahrens
Jason Banham wrote: > Hello, > > I've been looking at the source code to the transaction engine / DMU > and was wondering what the purpose of dn->dn_assigned_txg was > for? An object can be modified in many transactions (dmu_tx_t's) simultaneously, but they must all be for the same transaction

[zfs-code] How does zfs mount at boot? How to let the system not to mount zfs?

2006-12-14 Thread Matthew Ahrens
No?l Dellofano wrote: >> How to I tell the system not to mount certain zfs at boot? > you can set the 'canmount' property to 'off' for whatever filesystem you > do not want mounted: Or if you don't want anything mounted, 'zfs set mountpoint=none '. (the mountpoint property is inherited.) --mat

[zfs-code] Re: ZFS Available Space

2006-10-09 Thread Matthew Ahrens
Adhari C Mahendra wrote: > Sorry being offline for moment. > >> --matt >> >> ps. do you have any snapshots? Please send the >> output of 'du -hs >> ', 'zfs list', 'zfs get -r all '. > No I do not use any snapshot. Sorry, I have ripped out the zfs so I am unable > to give you the output of those

[zfs-code] Re: ASSERT failed dn->dn_nlevels > level (0x0 > 0x0) dbuf.c, line: 1523

2006-09-29 Thread Matthew Ahrens
Darren J Moffat wrote: > Thanks again for trying. Now what do I do ;-( The bug doesn't exist in nevada, right? So try removing your code until you find what's broken. --matt

[zfs-code] Re: ASSERT failed dn->dn_nlevels > level (0x0 > 0x0) dbuf.c, line: 1523

2006-09-28 Thread Matthew Ahrens
Darren J Moffat wrote: > eric kustarz wrote: >> Darren J Moffat wrote: >>> eric kustarz wrote: >>> Not sure if you've already posted this or not, but do you have a webrev? >>> >>> Yes it was in the original message: >>> >>> http://cr.grommit.com/~darrenm/zfs-crypto/ >>> >> >> I didn't se

[zfs-code] ZFS Available Space

2006-09-21 Thread Matthew Ahrens
Adhari C Mahendra wrote: > [Sol 10 6/6 x64] > > I am very newbie in ZFS. > > I have created 30GB storage pool and become a root_pool. If I run "du -hs" > from root directory , it reports only used 5.4G. But when I run "df -h", it > reports used 26G. Why it is happens? How to reclaim back to 5.4

[zfs-code] Code review request: new features and misc fixes

2006-08-30 Thread Matthew Ahrens
Eric Schrock wrote: > http://cr.grommit.com/~eschrock/zfs-prop/ Shouldn't dsl_prop_get* all take into account the inheritability of properties? It looks like non-inheritable properties will be inherited unless you are using dsl_prop_get_all(). It seems like the check for inheritability needs

[zfs-code] Looking for confirmation.

2006-08-29 Thread Matthew Ahrens
Pawel Jakub Dawidek wrote: > Hi. > > I've almost all file system functions working. > > I started to run some heavy file system regression tests. They work. fsx > wasn't able to break my port, but the test you can find here: > > http://people.freebsd.org/~kan/fsstress.tar.gz > > broke it.