Re: Putney's First Rule

2012-02-06 Thread David Nicol
On Mon, Feb 6, 2012 at 6:14 AM, David Nicol wrote: > > > Look! It's Putney's First Rule! > > On Thu, Oct 13, 2011 at 10:51 AM, Jeff Putney > wrote: >> >> >> Anyone can make up an axiom and call it the first rule of something. > > > --

waiting for deferred cleanup operations

2012-06-26 Thread David Nicol
t source would there be any problem with tracking it for mainstream inclusion? David Nicol -- Run it up the flagpole and see who salutes it -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo i

Re: waiting for deferred cleanup operations

2012-06-26 Thread David Nicol
I see that the #undef spells the macro wrong, so I'll fix that of course too. On Tue, Jun 26, 2012 at 8:55 PM, David Nicol wrote: > The whole picture is in the introduction to the last submitted version > of the patch, > http://permalink.gmane.org/gmane.comp.file-systems.bt

Re: waiting for deferred cleanup operations

2012-06-27 Thread David Nicol
silly gmail rich-text... On Wed, Jun 27, 2012 at 1:42 PM, David Nicol wrote: > > > > On Wed, Jun 27, 2012 at 9:31 AM, David Sterba wrote: >> >> On Tue, Jun 26, 2012 at 08:55:20PM -0500, David Nicol wrote: >> > [let's discuss the] architecture > > &

Re: IOCTL #21 part two: btrfs progs patch, including iso 8601 timeout support

2010-10-07 Thread David Nicol
On Thu, Oct 7, 2010 at 1:10 AM, Goffredo Baroncelli wrote: > On Wednesday, 06 October, 2010, David Nicol wrote: >> the ISO 8601 duration support is very loose, but I believe it is >> accurate for valid > In the man page and in the help the syntax is reported as: > >  

Re: IOCTL #21 part two: btrfs progs patch, including iso 8601 timeout support

2010-10-07 Thread David Nicol
anyway I think I got the logic wrong: the function as given yesterday would misparse PT1M1M -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: IOCTL #21 part two: btrfs progs patch, including iso 8601 timeout support

2010-10-07 Thread David Nicol
On Thu, Oct 7, 2010 at 9:25 AM, David Nicol wrote: > anyway I think I got the logic wrong: the function as given yesterday > would misparse > >   PT1M1M no it wouldn't! because of the post-increment on the how-to-parse 'M' state variable, the BIGM label can only get ju

Re: IOCTL #21 part two: btrfs progs patch, including iso 8601 timeout support

2010-10-07 Thread David Nicol
On Thu, Oct 7, 2010 at 1:10 AM, Goffredo Baroncelli wrote: > Please the next time put your patch inline or it is more difficult to > highlight a suggestion. * drop support for years and months, except as identified usage errors * lower-case 'm' now minutes * escalate syntax warnings to fatal ex

Re: Can you please define "snapshot" and "subvolume"?

2010-10-07 Thread David Nicol
On Thu, Oct 7, 2010 at 6:39 AM, Francis Galiegue wrote: > While I fully understand (and use) the purpose of snapshots, I don't > quite fathom the use case for subvolumes, apart from btrfs-convert... > Why has btrfs grown such a feature in the first place? Can someone > give me a use case for them?

Re: Can you please define "snapshot" and "subvolume"?

2010-10-07 Thread David Nicol
On Thu, Oct 7, 2010 at 8:18 AM, Mike Hommey wrote: > BTW, it would be very useful to be able to turn existing directories > into subvolumes. does a (link,unlink) move work across subvolumes? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majo

revisiting "Hard link across subvolumes"

2010-10-09 Thread David Nicol
could i-node numbers be made unique on a whole device, or somehow adjusted to avoid collisions, instead of simply disallowing this useful operation? On Thu, Oct 7, 2010 at 4:13 PM, Goffredo Baroncelli wrote: > The link across subvolumes is not allowable.  In the beginning it was > possible, but

Re: revisiting "Hard link across subvolumes"

2010-10-09 Thread David Nicol
when i have some time to work on this I will figure out a way to pull inodes from a larger pool or include the subvolume id at creation time in them or something so that the collisions go away. Surely it must be possible to prefix or suffix the inode number with a few bits that are per-subvolume un

Re: On-Disk Format

2010-10-11 Thread David Nicol
how are inode numbers determined? "dynamic inode numbering" is included amongst BTRFS's features. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

PATCH: IOCTL#21 progs, including -dq OPTIONS FOR btrfs fi sy

2010-10-13 Thread David Nicol
diff --git a/Makefile b/Makefile index 525676e..7442e14 100644 --- a/Makefile +++ b/Makefile @@ -37,12 +37,13 @@ all: version $(progs) manpages  version:        bash version.sh -btrfs: $(objects) btrfs.o btrfs_cmds.o -       gcc $(CFLAGS) -o btrfs btrfs.o btrfs_cmds.o \ +btrfs: $(objects) btrfs.o

Re: PATCH: IOCTL#21 progs, including -dq OPTIONS FOR btrfs fi sy

2010-10-13 Thread David Nicol
Drat, I left in a file descriptor leak. Is anyone trying this stuff? On Wed, Oct 13, 2010 at 9:37 AM, David Nicol wrote: > >        res = ioctl(fd, BTRFS_IOC_SYNC); > -       close(fd); >        if( res < 0 ){ -- To unsubscribe from this list: send the line "unsubscribe linu

Re: the idea for improving the performance of b-tree search

2010-10-13 Thread David Nicol
On Wed, Oct 13, 2010 at 6:24 AM, Chris Mason wrote: > 3) Do delayed inode deletion.  See above, but change the word insert > with delete ;) And the reserved-for-the-future-but-not-used-yet flags field in the ioctl#21 control structure gets a use: bits to indicate for the completion of which of t

Re: the idea for improving the performance of b-tree search

2010-10-13 Thread David Nicol
>> > 3) Do delayed inode deletion.  See above, but change the word insert >> > with delete ;) >> >> And the reserved-for-the-future-but-not-used-yet flags field in the >> ioctl#21 control structure gets a use: bits to indicate for the >> completion of which of the potential delayed things the calle

Re: btrfs filesystem df not working

2010-10-14 Thread David Nicol
On Thu, Oct 14, 2010 at 6:45 AM, Leonidas Spyropoulos wrote: > My question though is, if I use experimental kernels can I then load > an "old" kernel and still use the btrfs filesystem? > Or the newer kernels write anything specials on ionodes which the old > ones cannot read? My understanding i

PATCH zeroing ioctl21 flags progs-side

2010-10-14 Thread David Nicol
This is once again the whole patch starting from 075587c96c2f39e227847d13ca0ef305b13cd7d3 (Chris Mason, April 06 2010) The difference between this one and yesterday's is: 1: the file descriptor leak is corrected 2: the ioctl21 flags field is explicitly zeroed, for forwards compatibility. The inte

PATCH added flags to cleaner-wait structure kernel-side

2010-10-14 Thread David Nicol
This is once again based on 2ebc3464781ad24474abcbd2274e6254689853b5 (Dan Rosenberg July 19 2010) The delta between this and the previous ioctl#21 kernel patch I posted is that this one defines the flags field in the arguments structure, and has a comment about the intended semantics of it, and te

newbie question about struct btrfs_dir_item *btrfs_match_dir_item_name

2010-10-18 Thread David Nicol
reading, in dir-item.c, the code in struct btrfs_dir_item *btrfs_match_dir_item_name (...) I am a little surprised to see an O(n) iterative name comparison check instead of something that would efficiently support directories with lots of items in them. Is this function a fall-back if a O(1) table

IOCTL #21 could also wait for btrfs_run_delayed_iputs

2010-10-18 Thread David Nicol
That's the other thing the cleaner_kthread does; conceivably some new delayed iputs could get queued during snapshot deletions. Thoughts? -- "In one instance, a rai being transported by canoe was accidentally dropped and sunk to the sea floor. Although it was never seen again, everyone agreed th

Re: newbie question about inode numbers

2010-10-20 Thread David Nicol
Wow, vger categorically refuses HTML e-mail! On Tue, Oct 19, 2010 at 4:41 PM, David Nicol wrote: > > the inode number is an "unsigned long" which on a 32 bit system may be merely > 32 bits, yet btrfs allows a full 64 bits of files in a file system (which is > impossib

Re: [PATCH] Btrfs: add support for mixed data+metadata block groups V3

2010-10-21 Thread David Nicol
Does this mixing constitute a forbidden change of on-disk format, and if not how not? On Wed, Oct 20, 2010 at 11:21 PM, Mitch Harder wrote: > Other than that, it seemed to perform well.  I did not encounter any > stability issues as long as I was using a patched kernel. -- To unsubscribe from th

Re: Mark btrfsctl deprecated

2010-10-25 Thread David Nicol
I am certainly not in a position to answer for Chris Mason, but I am happy to share my response to the question, coming from a perspective of being somewhat obsessive about not breaking back-compat. Let's not. As I am certainly within the "lot of people" in question, having just done exactly that,

Re: [PATCH v2] Btrfs: allow subvol deletion by unprivileged user with -o user_subvol_rm_allowed

2010-10-28 Thread David Nicol
do the resulting semantics match (or resemble) the semantics provided by other facilities that allow non-superusers to mount and unmount file systems? If all "user mountable file systems" are provided either by the FUSE layer (but that is concerned with providing the FS, not mounting it) or by set

Re: [PATCH v2] btrfs-progs: btrfs: implement async option for snapshot command

2010-11-01 Thread David Nicol
this would be yet another thing that ioctl#21 could have a flag to wait for. On Mon, Nov 1, 2010 at 11:34 AM, Sage Weil wrote: > Add an 'async' option to the snapshot creating command, that will let you > avoid waiting for a new snapshot to commit to disk. > > Signed-off-by: Sage Weil -- To unsu

hard links across snapshots/subvolumes are actually a bad idea.

2010-11-24 Thread David Nicol
I've been thinking about this for a while, from a perspective of how to make it work by allocating i-node numbers from a global pool, but yesterday I realized that offering the feature would be a bad idea because it violates the semantics of file systems. I will be happy to expand on that point if

Re: hard links across snapshots/subvolumes are actually a bad idea.

2010-11-25 Thread David Nicol
>> "COW hardlinks" are ref-links (as far as I'm concerned).  I said >> partially implemented, because that's exactly what a snapshot is.  I'm >> just not certain whether bcp works across subvolumes or not.  An >> actual hardlink (i.e., all writes appear in all hardlinks) across any >> file-system-l

VFS support for fast copy on deduplicating FSes

2010-11-25 Thread David Nicol
unresearched question/suggestion: Is there general support for a "fast copy" ioctl in the VFS layer, which would be hooked by file systems that support COW or other forms of deduplication and can provide copy semantics by manipulating metadata only? -- "It is merely a matter of persistence." --

PATCH: btrfs ioctl for waiting for kernel cleaner thread task completions kernel code

2010-12-12 Thread David Nicol
the attached patch is against ubuntu maverick latest git, and I believe it is final. It is forward-compatible, as there is space in it to define 29 more deferred things to wait for, if needed, as well as a flag bit reserved for strict versioning. Calling it with a flags field of 0xFFFA will wai

revised ioctl#21 btrfsprogs patch

2010-12-15 Thread David Nicol
the below is against 70c6c10134b502fa69955746554031939b85fb0c which is the head of the "next" branch. 1: absolute-sized fields in the interface object 2: now getting error from errno instead of the return value of ioctl(2) -- Forwarded message -- From: project user Date: Wed, Dec

Re: What do the arguments of btrfs filesystem defragment do?

2010-12-16 Thread David Nicol
On Wed, Dec 15, 2010 at 1:20 PM, Chris Mason wrote: >> >> Is there a decent way to have btrfs compress already existing files >> (that were written before compression was enabled) without hurting any >> of the internal structures such as snapshots? > > I'm afraid not yet.  There is code for this

Re: revised ioctl#21 btrfsprogs patch

2010-12-24 Thread David Nicol
Attached is another patch, that includes the new files, against a progs clone taken earlier today. It's attached, not in-line, because that seems better, as there are lines that wrap in-line. I have 2.6.35-24-generic_2.6.35-24.42_i386 Maverick kernel image and header debs available with the ioctl#

ioctl #21 kernel patch that (1) actually works, to wait for dead_roots cleaning and (2) provides framework for adding facility for user-space waiting for anything else deferred

2011-01-16 Thread David Nicol
Here is the latest ioctl #21 kernel patch; it (1) actually works, unlike the last one, which only waited until the to-do list was emptied at the beginning of the cleaning function, which wasn't actually helpful and (2) provides framework for adding facility for user-space waiting for anything else

Re: [PATCH v3 0/5]add new ioctls to do metadata readahead in btrfs

2011-01-19 Thread David Nicol
On Wed, Jan 19, 2011 at 2:34 PM, Andrew Morton wrote: > It would be much more valuable if those 3.5 seconds were available to > devices which really really care about bootup times, but very few of > those devices use rotating disks nowadays, I expect? And don't rotating disk modules read and buf

thinking about deferred things

2011-02-28 Thread David Nicol
What if there was a new structure, with two members, a list_head and an atomic_t, that could replace the current list_head structures in the file system info structure with a minimum of patch size, to facilitate monitoring the completion of deferred actions? That way, the new additional atomic_t el

Re: how to know when all space from a snapshot delete is freed?

2011-03-02 Thread David Nicol
the patch has been working in production at the sponsor's site for some weeks. It has not been merged, or even built by anyone else, to my knowledge. The working kernel patch is at https://patchwork.kernel.org/patch/482721/ The open question, that everyone has more important things to do than tal

Re: how to know when all space from a snapshot delete is freed?

2011-03-02 Thread David Nicol
On Wed, Mar 2, 2011 at 7:12 AM, Brian J. Murrell wrote: > Given that the return from the snapshot delete command is immediate and > the space is freed asynchronously, how can I determine absolutely that > the snapshot has been entirely removed and the space freeing operation > is complete? > > Ch

Re: [PATCH V2] btrfs: implement delayed inode items operation

2011-03-04 Thread David Nicol
On Thu, Feb 17, 2011 at 7:09 PM, Chris Mason > > This work is really cool, thanks for doing it.  I'm starting a run on > this tonight and if all goes well I'll review in detail and try to queue > it along with the per-subvolume storage bits for .39. > > Thanks! > > -chris This is the kind of thi

completely revised ioctl for notification of when all space from a snapshot delete is freed and similar

2011-04-14 Thread David Nicol
thing is to try compiling and replace all references to the replaced list heads with the new syntax. Any takers? David Nicol -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://

Re: btrfs defra

2011-06-16 Thread David Nicol
>   Defrag works on individual files, and tries to find a contiguous > sequence of bytes to write the file's data to. In the process, the > current implementation will break any CoW duplication -- either within > single files (copies with cp --reflink=always) or files copied via > snapshotting. W

Re: BTRFS should increase the hard-link in the same directory limit

2011-08-23 Thread David Nicol
so there are these hundreds of message files, and when one is read, a new link to The Markfile appears with a similar name as the read file? Is that right? if the point is to save inodes by making a directory entry that's a hardlink to something already existing, why not link to the message file?

Re: Swap file on btrfs fails ("swapfile has holes")

2011-08-23 Thread David Nicol
it seems to me that the swap file interface doesn't make sense; to "eliminate the middleman" and provide a direct swap tree type would be the way to go, so a btrfs can hold swapped pages. Doing this would AIUI require: 1) direct interface to btrfs trees, not via file-system interface 2) modifying s

wishlist item: sqlite

2009-10-07 Thread David Nicol
my wishlist item is for btrfs to expose to user-space enough functionality that keeping sqlite databases directly in the file system instead of through its own abstraction in a file maintained in the fs will be a matter of replacing sqlite's btree.c file, and referring to an sqlite database using

an idea for increasing responsiveness during write bursts

2009-10-26 Thread David Nicol
I don't know if btrfs does this already; the idea is, defer reference count decrements until the spindle is available, instead of keeping up with them as writes occur. Risk: a system crash could occur between change and decrementing, causing space that should get recycled to get stuck Engineering

little fixme in btrfsctl.c [PATCH]

2009-10-26 Thread David Nicol
diff --git a/btrfsctl.c b/btrfsctl.c index 66c4e89..3ab0c26 100644 --- a/btrfsctl.c +++ b/btrfsctl.c @@ -144,7 +144,7 @@ int main(int ac, char **av) len = strlen(name); if (len == 0 || len >= BTRFS_VOL_NAME_MAX) { fprin

announcement of deduplication in Sun ZFS

2009-11-02 Thread David Nicol
Is it inappropriate to post a relevant news item to this list? Please correct me off-list if so. http://blogs.sun.com/bonwick/en_US/entry/zfs_dedup -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: Way to quickly "revert back" to a snapshot?

2009-11-04 Thread David Nicol
On Fri, Oct 30, 2009 at 12:52 AM, TARUISI Hiroaki wrote: > I'm trying to make this snapshots/subvolumes listing feature, > I wonder how the interface should be. > > Is there any other idea? It's layers above where you're working, but how about providing a virtual directory off the mounting root,

Re: [RFC] proposal for a btrfs filesystem layout

2009-11-20 Thread David Nicol
On Fri, Nov 20, 2009 at 12:50 PM, Goffredo Baroncelli wrote: > Any comments ? > BR > G.Baroncelli since COW semantics require touching directory entries all the way up to the root of the subvolume, for transaction-intensive applications it would make sense to provide something that works like "

Re: [RFC] proposal for a btrfs filesystem layout

2009-11-20 Thread David Nicol
On Fri, Nov 20, 2009 at 1:50 PM, Chris Mason wrote: > COW semantics require touching btree nodes all the way up to the root of > the btree, but this is different from the directory.  Directories are > stored in the btree, but you won't have to touch more than 8 or so btree > levels regardless of h

survey of command line interfaces for working with snapshots

2009-11-25 Thread David Nicol
On Tue, Nov 24, 2009 at 12:27 PM, Goffredo Baroncelli > Yes, I can remount the sub-volume with the mount command anywhere and with an > arbitrary name. In fact the thing that seems strange to me is that when I > create a snapshot, immediately it is mounted: it is not a real problem, it is > on

Re: [wiki] btrfs logo

2009-11-28 Thread David Nicol
> It, of course, may be argued whether a filesystem needs a logo or not > (i think it does), but i made one: > http://lomm.es/pub/misc/btrfs_logo.svg Wah, it doesn't look like a stick of butter being sliced into hard drive pats. Boo hoo. -- To unsubscribe from this list: send the line "unsubscribe

Re: BUG: Link from sub volume, then remove the subvolume -> wrong link

2009-12-15 Thread David Nicol
how about supporting the operation, as a feature, and allowing a file to be in two different subvolumes at the same time? Would that make the link semantics too confusing? -- Is it the time when there isn't time to discuss but there is time to act yet? -- To unsubscribe from this list: send the l

Re: remote mirroring in the works?

2010-09-06 Thread David Nicol
On Mon, Aug 30, 2010 at 1:14 PM, K. Richard Pixley wrote: > In terms of fault tolerance, I'd call this a tolerance of about a half a > fault since the system cannot return to it's initial configuration without > breaking continuity of service. > > And there really isn't any way to extend this. It'

reserving an IOCTL number; other details

2010-09-12 Thread David Nicol
(the next one from btrfs-unstable) or receive advice on what ioctl number to use instead. Also, if you would like to test this function on a trade-secret basis (until the sponsorship comes through) please contact me off-list. David Nicol -- l'égalité des droits pour les ambidextres -- To u

Re: reserving an IOCTL number; other details

2010-09-12 Thread David Nicol
this I can release, the user-facing API. diff --git a/man/btrfsctl.8.in b/man/btrfsctl.8.in index c2d4488..94403a4 100644 --- a/man/btrfsctl.8.in +++ b/man/btrfsctl.8.in @@ -35,6 +35,10 @@ Scans all devices present in the system for btrfs filesystem. .TP \fB\-c\fR Forces a filesystem sync. +.T

Re: reserving an IOCTL number; other details

2010-09-13 Thread David Nicol
https://btrfs.wiki.kernel.org/index.php/Category:Documentation does not mention any btrfs command. the btrfsctl command has the -c sync tool, and the tools for creating and deleting snapshots and subvolumes, so it seems that unless this btrfs program does all of that I have amended the right tool t

Re: reserving an IOCTL number; other details

2010-09-13 Thread David Nicol
I see btrfs utility appears here: http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs-unstable.git;a=commit;h=6d2cf042471cc728b5399b2beae54603739bc66a I shall be patching it, too, with the new ioctl which I made this thread to reserve. -- To unsubscribe from this list: send the line "uns

Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]

2010-09-15 Thread David Nicol
Does the manpatch below describe an acceptable API for a hook to a facility for waiting for the completion of deleted snapshot cleanup? diff --git a/man/btrfs.8.in b/man/btrfs.8.in index 26ef982..290d202 100644 --- a/man/btrfs.8.in +++ b/man/btrfs.8.in @@ -19,6 +19,8 @@ btrfs \- control a btrfs fi

Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]

2010-09-15 Thread David Nicol
btrfs s ... is imprecise, as the operation isn't on a subvolume, but on the fi where the s used to be but isn't any more. With your approval I'll go with "filesystem garbage-collection" which can be abbrevd to btrfs fi g /butter 3 || echo there should be some more space now \ but it wou

Re: Updating the wiki pages adding btrfs command [Was Re: reserving an IOCTL number; other details]

2010-09-15 Thread David Nicol
On Wed, Sep 15, 2010 at 1:41 PM, Josh Berry wrote: > Personally, I like "gc" or "clean".  "wait-subvolume-deletion", > "garbage-collect" or similar is too verbose for my taste. > > -- Josh the question IMO is how much implementation details to saddle the user with. Anything with the word "garbage

Re: Btrfs, NFS (v3) and ESTALE

2010-09-23 Thread David Nicol
I wonder how difficult it would be to use GFS storage allocation, so the right way to cluster btrfs would be at the block device level instead of file system level. Like the way btrfs can coexist in ext4 partition, but coexisting with GFS. On Thu, Sep 23, 2010 at 6:02 AM, David Flynn wrote: > D

patch intoducing IOctl #21 allowing for waiting for completion of deleted subvol recovery

2010-10-06 Thread David Nicol
Written against the btrfs-unstable git tree, the attached patch installs cleanly against the latest kernel (as of two weeks ago.) The patches for btrfsctl and btrfs programs will be coming soon; I'm adding support for ISO8061 duration strings instead of (or in addition to) milliseconds and thinkin

IOCTL #21 part two: btrfs progs patch, including iso 8601 timeout support

2010-10-06 Thread David Nicol
the ISO 8601 duration support is very loose, but I believe it is accurate for valid input. Without any non-numeric designators, the timeout is interpreted as seconds, so btrfs fi reclaim 10.3321 /my_btrfs_mount || echo timed out will wait 10332 ms before echoing, if the pending subvolume delet