Re: Diff using send-receive code / RFC
On Sun, Feb 10, 2013 at 10:21:31PM +0530, nafisa mandliwala wrote:
> Hello,
> We're a team of 4 final year computer science students and are
> working on generating a diff between file system snapshots using the
> send receive code.
This was just implemented in snapper. Unfortunately I was in a
hurry so the code doesn't look so good. Instead of improving the
code in snapper (C++) I thought about implementing it in C so
that it can be included in btrfsprogs (and libbtrfs).
Here is an example:
# btrfs send -a -p /testsuite/.snapshots/1/snapshot
/testsuite/.snapshots/2/snapshot | cat
At subvol /testsuite/.snapshots/2/snapshot
Comparing testsuite/.snapshots/1/snapshot and testsuite/.snapshots/2/snapshot.
+ /foo
+ /foo/bar
+ /foo/bar/world
Attached you can find my current state. It's completely
unfinished and only works from some test-cases. To get it
compiled you either need some patches posted here earlier
(e.g. NO_FILE_DATA) or must make minor modifications.
But I would like to get feedback about this feature on the list.
Kind Regards,
Arvin
--
Arvin Schnell,
Senior Software Engineer, Research & Development
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB
16746 (AG Nürnberg)
Maxfeldstraße 5
90409 Nürnberg
Germany
/*
* Copyright (c) 2013 Arvin Schnell
*
* All Rights Reserved.
*
* This program is free software; you can redistribute it and/or modify it
* under the terms of version 2 of the GNU General Public License as published
* by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
* You should have received a copy of the GNU General Public License along
* with this program; if not, write to the Free Software Foundation, Inc., 59
* Temple Place - Suite 330, Boston, MA 021110-1307, USA.
*/
#define _GNU_SOURCE
#include
#include
#include
#include
#include
#include "btrfs/send.h"
#include "btrfs/send-stream.h"
#include "btrfs/rbtree.h"
#include "btrfs/send-analyser.h"
struct tree_root {
struct rb_root rb_root;
};
struct tree_node {
struct rb_node rb_node;
char *name;
unsigned int status;
struct tree_root children;
};
static void tree_root_init(struct tree_root *root)
{
root->rb_root = RB_ROOT;
}
static struct tree_node *tree_node_init(const char *name)
{
struct tree_node *node = malloc(sizeof(struct tree_node));
node->name = strdup(name);
node->status = 0;
tree_root_init(&node->children);
return node;
}
static struct tree_node *tree_find(struct tree_root *root, const char *name)
{
struct rb_node *rb_node = root->rb_root.rb_node;
while (rb_node)
{
struct tree_node *data = rb_entry(rb_node, struct tree_node, rb_node);
int result = strcmp(name, data->name);
if (result < 0)
rb_node = rb_node->rb_left;
else if (result > 0)
rb_node = rb_node->rb_right;
else
return data;
}
return NULL;
}
static int tree_insert(struct tree_root *root, struct tree_node *node)
{
struct rb_node **new = &(root->rb_root.rb_node);
struct rb_node *parent = NULL;
while (*new)
{
struct tree_node *this = rb_entry(*new, struct tree_node, rb_node);
parent = *new;
int result = strcmp(node->name, this->name);
if (result < 0)
new = &(*new)->rb_left;
else if (result > 0)
new = &(*new)->rb_right;
else
return 0;
}
rb_link_node(&node->rb_node, parent, new);
rb_insert_color(&node->rb_node, &root->rb_root);
return 1;
}
/* find node or create new one, return 1 if already existing */
static int tree_find_or_insert(struct tree_root *root, const char *name,
struct tree_node **node)
{
struct rb_node **new = &(root->rb_root.rb_node);
struct rb_node *parent = NULL;
while (*new)
{
struct tree_node *this = rb_entry(*new, struct tree_node, rb_node);
parent = *new;
int result = strcmp(name, this->name);
if (result < 0)
new = &(*new)->rb_left;
else if (result > 0)
new = &(*new)->rb_right;
else
{
*node = this;
return 1;
}
}
*node = tree_node_init(name);
rb_link_node(&(*node)->rb_node, parent, new);
rb_insert_color(&(*node)->rb_node, &root->rb_root);
return 0;
}
static void tree_remove(struct tree_root *root, struct tree_node *node)
{
rb_erase(&node->rb_node, &root->rb_root);
free(node->name);
free(node);
}
static void tree_cut(struct tree_root *root, struct tree_node *node)
{
rb_erase(&node->rb_node, &root->rb_root);
}
static struct tree_node *tree_node_first(struct tree_root *root)
{
struct rb_node *rb_node = rb_first(&root->rb_root);
if (!rb_node)
return NULL;
return rb_entry(rb_node, struct tree_node, rb_node);
}
static struct tree_node *tree_node_next(struct tree_node *node)
{
struct rb_node *rb_node = rb_next(&node->rb_node);
if (!rb_node)
return NULL;
return rb_entry(rb_node, struct tree_node, rb_no
Re: Deleted subvolume reappears and other cleaner issues
Thanks for your comments, Josef. Another thing that confuses me is that there are some cases, in which btrfs_drop_snapshot() has a failure, but still returns 0, like for example, if btrfs_del_root() fails. (For cases when btrfs_drop_snapshot() returns non-zero there is a BUG_ON). So in this case for me __btrfs_abort_transaction() sees that trans->blocks_used==0, so it doesn't call __btrfs_std_error, which would further force the filesystem to become RO. So after that btrfs_drop_snapshot successfully completes, and, basically, nobody will retry the subvol deletion. In addition, in this case, after couple of seconds the machine completely freezes for me. I have not yet succeeded to determine why. Thanks, Alex. On Wed, Feb 6, 2013 at 5:14 PM, Josef Bacik wrote: > On Thu, Jan 31, 2013 at 06:03:06AM -0700, Alex Lyakas wrote: >> Hi, >> I want to check if any of the below issues are worth/should be fixed: >> >> # btrfs_ioctl_snap_destroy() does not commit a transaction. As a >> result, user can ask to delete a subvol, he receives "ok" back. Even >> if user does "btrfs sub list", >> he will not see the deleted subvol (even though the transaction was >> not committed yet). But if a crash happens, ORPHAN_ITEM will not >> re-appear after crash. >> So after crash, the subvolume still exists perfectly fine (happened >> couple of times here). > > Same thing happens to normal unlinks, I don't see a reason to have different > rules for subvols. > >> >> # btrfs_drop_snapshot() does not commit a transaction after >> btrfs_del_orphan_item(). So if the subvol deletion completed in one go >> (did not have to detach and re-attach to transaction, thus committing >> the ORPHAN_ITEM and drop_progress/level), then after crash ORPHAN_ITEM >> will not be in the tree, and subvolume still exists. >> > > Again same thing happens with normal files. > >> # btrfs_drop_snapshot() checks btrfs_should_end_transaction(), and >> then does btrfs_end_transaction_throttle() and >> btrfs_start_transaction(). However, it looks like it can rejoin the >> same transaction if transaction was not not blocked yet. Minor issue, >> perhaps? > > No if we didn't block then its ok and we wait longer, we only throttle to give > the transaction stuff a chance to commit, so if the join logic decides its ok > to > go on then we're good. > >> >> # umount may get delayed because of pending-for-deletion subvolumes: >> btrfs_commit_super() locks the cleaner_mutex, so it will wait for the >> cleaner to complete. >> On the other hand, cleaner will not give up until it completes >> processing all its splice. If currently cleaner is not running, then >> btrfs_commit_super() >> calls btrfs_clean_old_snapshots() directly. So does it make sense: >> - btrfs_commit_super() will not call btrfs_clean_old_snapshots() >> - close_ctree() calls kthread_stop(cleaner_kthread) early, and cleaner >> thread periodically checks if it needs to exit > > I don't quite follow this, but sure? Thanks, > > Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs and LVM snapshots (Re: kernel BUG at fs/btrfs/extent-tree.c:1772)
On 02/10/2013 01:32 AM, Piotr Pawłow wrote: > Hello, >> Yeah you can't mount images, we clear out the chunk tree so >> nothing works. Let me know if you run into any problems in the >> future. Thanks, > > That's surprising, I haven't seen it mentioned anywhere. > > With any other filesystem I could use an LVM snapshot to save the > original state, but with a multi-device btrfs it would be very > risky. Origin and snapshot volumes could easily get mixed up by > kernel and tools. > > I tried that in qemu, and I've seen btrfs happily mount 1 origin and > 1 snapshot device as a single FS. And then I've seen it mount both > origin devices, even though one of them had old content. Naturally > it complained a lot about bad checksums. I can confirm that, even with a single-device btrfs filesystem. However I am curious why you want to use the lvm snapshot capability instead of the btrfs one. > > Is there any way to avoid such mix-ups? To somehow mark devices so > that btrfs would know these devices belong together? Btrfs assume that every device has an "unique" uuid. However when a device is snapshotted it uuid is copied too, so it is not unique any more. This is the reason of the btrfs confusing. > Regards -- To unsubscribe from this list: send the line "unsubscribe > linux-btrfs" in the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 ss -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
