Re: How to debug very very slow file delete? (btrfs on md-raid5)
Le 25 mars 2014 à 12:13, Martin a écrit: On 25/03/14 01:49, Marc MERLIN wrote: It took 18H to rm it in 3 tries: And is not *the 512kByte raid chunk* going to give you horrendous write amplification?! For example, rm updates a few bytes in one 4kByte metadata block and the system has to then do a read-modify-write on 512kBytes... My question would be naive, but would it be possible to have a syscall or something to do a fast rm -rf or du ? I think ceph might have the later actually. -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: atime and filesystems with snapshots (especially Btrfs)
Le 25 May 2012 ? 11:42, Josef Bacik a écrit: Just mount with -o noatime, there's no chance of turning something like that on by default since it will break some applications (notably mutt). Thanks, I've just updated the wiki: https://btrfs.wiki.kernel.org/index.php/Mount_options#Performance -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Balance RAID10 with odd device count
Le 21 February 2012 ? 07:54, Hugo Mills a écrit: Some time ago, I proposed the following scheme: nCmSpP where n is the number of copies (suffixed by C), m is the number of stripes for that data (suffixed by S), and p is the number of parity blocks (suffixed by P). Values of zero are omitted. So btrfs's RAID-1 would be 2C, RAID-0 would be 1CnS, RAID-5 would be 1CnS1P, and RAID-6 would be 1CnS2P. DUP would need a special indicator to show that it wasn't redundant in the face of a whole-disk failure: 2CN Seems clear. However, is the S really relevant ? It would be simpler without it, wouldn't it ? -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs subvolume snapshot syntax too smart
Le 04 avril 2011 à 12:57, Freddie Cash a écrit: Isn't this a situation where supporting a trailing / would help? For example, with the / at the end, means put the snapshot into the folder. Thus btrfs subvolume snapshot /ssd/sub1 /ssd/5/ would create a sub1 snapshot inside the 5/ folder. Running it a second time would error out since /ssd/5/sub1/ already exists. And if the 5/ folder doesn't exist, it would error out. And without the / at the end, means name the snapshot. Thus btrfs subvolume snapshot /ssd/sub1 /ssd/5 would create a snapshot named /ssd/5. Running the command again would error out due to the snapshot already existing. And if the 5/ folder doesn't exist, it's created. And it errors out if the 5/ folder already exists. Or, something along those lines. Similar to how other apps work with/without a trailing /. I am not an expert, but I think it is the opposite with rsync. Cheers, -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
mini bug report
Hi, I use btrfs on my laptop, 2.6.37-rc2 from kernel.org, under dm-crypt, as /. I use space_cache and compression (not forced). Today, my computer froze. At reboot, the kernel could not mount. The dmesg output, which I haven't saved was speaking of a null dereference. After that I rebooted on 2.6.34, which was not very happy: mount errors (space cache ?). Now I am on 2.6.37-rc2 again, which seems to work. So I guess it might come from space_cache. I would note on a paper if this comes back. However if anybody has any clue on what specific part of the dmesg should be reported, it would be very helpfull ! Cheers, -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
possible bug with space_cache 2.6.37-rc2
Hi, today I accidentally shut my laptop down (batteries were empty). It is a / btrfs partition, on ubuntu 10.10, kernel 2.6.37-rc2 from kernel.org. It is mounted with: rw,relatime,compress,space_cache Init could not be found on reboot (repeated twice). I rebooted to 2.6.36, which made the partition mounted read-only, and then on 2.6.37-rc2, and everything seems to work pretty well. Please tell me if you need anything about this. -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Update to Project_ideas wiki page
Le 17 novembre 2010 à 16:12, Bart Noordervliet a écrit: Can I suggest we combine this new RAID level management with a modernisation of the terminology for storage redundancy, as has been discussed previously in the Raid1 with 3 drives thread of March this year? I.e. abandon the burdened raid* terminology in favour of something that makes more sense for a filesystem. I would agree with that. -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 0/2] Control filesystem balances (kernel side)
Le 30 octobre 2010 à 19:44, Goffredo Baroncelli a écrit: I like the idea that these info should be put under sysfs. Something like /sys/btrfs/filesystem-uuid/ balance - info on balancing devices - list of device (a directory of links or a file which contains the list of devices) subvolumes/ - info on subvolume(s) label - label of the filesystem other btrfs filesystem related knoba Well, mdstat stats are under /proc/mdstat. Is sysfs the ideal place ? Just asking. -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: some issues with lots of snapshots
Le 26 octobre 2010 à 15:15, Pat Regan a écrit: I turned off the 5-minute snapshots and I'm now just keeping 4 weekly, 7 daily, and 24 hourly snapshots alive. I have just rebooted and I am going with /15 minutes interval. -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
some issues with lots of snapshots
I can't reproduce this right know, but it seems that creating lots of snapshots, one per minute (keeping only a dozens of them) may block the system. Every btrfs snap processes stay idle and the load skyrockets to 900. I use an old version: 2.6.34.5 kernel, and the system is still very responsive (web and mysql server). I would apreciate if somebody could tell me about the current kernel (I had stability issues with 2.6.35 series). # btrfs filesys show Label: 'btrfs' uuid: e13357f4-c40c-4940-87bd-4f5142d32c49 Total devices 2 FS bytes used 8.67GB devid1 size 1.81TB used 12.03GB path /dev/sda2 devid2 size 1.81TB used 12.01GB path /dev/sdb2 # btrfs filesys df /var Data, RAID1: total=11.00GB, used=8.24GB Metadata, RAID1: total=1.00GB, used=436.21MB System, RAID1: total=8.00MB, used=4.00KB Data: total=8.00MB, used=0.00 Metadata: total=8.00MB, used=0.00 System: total=4.00MB, used=0.00 Regards, PS: extract from dmesg if somebody is interested. http://pastebin.com/8zE4GKXu -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 2.6.36-rc1 btrfs still unstable
Le 25 août 2010 à 18:57, Johannes Hirte a écrit: the other big question is: Is btrfs with 2.6.36 really rockstable and ready to use in productive environments? I don't think so. There is at least one checksum bug and ENOSPC problems are also still present. I am planning to put 2 dedicated web hosting servers in production (backuped every day), with 2.6.34.5 vanilla kernel. I also use btrfs on a 2.6.34 kernel on the backup server (rsync) for some time. -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: birgers RFE #2
Le 03 août 2010 à 14:44, birger a écrit: I have a neat snapshot script for my laptop that I wouldn't mind sharing. Works mostly like the NetApp snapshots. Set up a cron job with a prefix for the snapshot name and the number of snapshots to keep. It's very easy to set up for schemes like 24 hourly snapshots, 7 daily and 4 weekly. I have roughly the same thing in python. I can post it if someone is interested. Cheers, -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: birgers RFE #2
Le 03 août 2010 à 15:42, Benjamin Griese a écrit:
Hello guys,
I would be interested in that script, maybe in both of yours,
It is not supposedly bulletproof. Use at your own risks.
backup1:/usr/local/bin# crontab -l
# m h dom mon dow command
# hourly backup
0 * * * * /usr/local/bin/snapshots.py -h
# daily backup
10 6 * * * /usr/local/bin/snapshots.py -d
# weekly backup
#20 6 * * 0 /usr/local/bin/snapshots.py -w
# monthly backup
#30 6 1 * * /usr/local/bin/snapshots.py -m
# yearly backup
#40 6 1 1 * /usr/local/bin/snapshots.py -y
#! /usr/bin/env python
Daily snapshots for all our backups
import os
import os.path
import glob
import sys
import subprocess
import time
import re
def usage():
Print usage and exit
print usage: %s [-y|-m|-d|-h] % sys.argv[0]
sys.exit(2)
def chdir_force(path):
Chdir even if we have to make the path
try:
os.makedirs(path)
except OSError:
pass
os.chdir(path)
class Tuning:
tuning = {
'-y' : [ 'yearly', 1 ], # we keep 1 snapshots
'-m' : [ 'monthly', 1 ], # we keep 1 snapshots
'-w' : [ 'weekly', 1 ], # we keep 1 weekly snapshots
'-d' : [ 'daily', 4 ],
'-h' : [ 'hourly', 2 ],
}
def __init__(self):
try:
option = sys.argv[1]
(self.mode, self.keep) = Tuning.tuning[option]
except (IndexError, KeyError):
usage()
def get_backup_name():
Return the backup dir name.
It depends on the current location and time.
name = time.strftime(backup-%y-%m-%d-%H:%M:%S)
if os.path.exists(name):
print name already exists
sys.exit(2)
return name
def main():
tuning = Tuning()
backup_path = os.path.join(/mnt/btrfs/history, tuning.mode)
chdir_force(backup_path)
# Make the new snapshot
cmd = /usr/local/bin/btrfs subvolume snapshot.split()
backup_name = get_backup_name()
cmd.extend( [/backup/, backup_name] )
ret = subprocess.call(cmd)
if ret != 0:
print return from btrfs subvol snap: + str(ret)
# Filter the number of snapshots
files = sorted( (file for file in glob.glob( backup-*) if not
file.endswith(.old)) )
if len(files) tuning.keep:
for file in files[:-tuning.keep]:
cmd = /usr/local/bin/btrfs subvolume delete .split()
cmd.append( file )
ret = subprocess.call(cmd)
if ret != 0:
print return from btrfs subvol del: + ret
if __name__ == '__main__':
main()
--
Xavier Nicollet
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Number of hard links limit
Le 02 août 2010 à 14:40, Sami Liedes a écrit: [BTRFS supports only 256 hard-links per directory ...] but if it indeed needs a disk format change, I think this should be considered before the format is set in stone. I won't personally lose my sleep if this is not fixed - I can use other filesystems for backuppc and other similar systems, Wouldn't it be even better to actually patch BackupPC to handle btrfs snapshots and COW (bcp) ? -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs: unlinked X orphans messages
Hi, I am using btrfs for remote backups (via rsync), with daily and weekly snapshots. I see these messages in kern.log: Jul 18 07:09:43 backup1 kernel: [3437126.458374] btrfs: unlinked 9 orphans Jul 18 12:01:01 backup1 kernel: [3454604.905856] btrfs: unlinked 1 orphans Jul 18 13:01:51 backup1 kernel: [3458254.990199] btrfs: unlinked 1 orphans Jul 19 04:01:41 backup1 kernel: [3512244.236347] btrfs: unlinked 1 orphans Is this something I have to be afraid of ? Linux debian lenny, pure btrfs partition with no raid, vanilla kernel: 2.6.34. Thanks, -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] Move all btrfs command to only one command
Le 22 janvier 2010 à 01:11, Michael Niederle a écrit: btrfs clone|-c source [dest/]name Clone the subvolume source with the name name in the dest directory. Maybe you should add the writable snapshot in the description, since everyone is using the word snapshot on the mailing list. When I hear clone, I understand dump. btrfs delete|-d subvolume Delete the subvolume subvolume. btrfs rem|-R dev path Remove a device to a filesystem Just a small suggestion: btrfs remove instead of btrfs rem Why not btrfs rm, which reminds git rm ? Actually, I would do something like: btrfs vol add btrfs vol rm btrfs dev add btrfs dev rm It would bet simpler, wouldn't it ? -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
compile btrfs for 2.6.27-rc9
Hello everyone, I am trying to compile btrfs.ko (unstable) for 2.6.27-rc9. I copied btrfs part of Kbuild, and fs/btrfs to my 2.6.27-rc9 tree. Then make modules modules_install However, I can't insmod btrfs.ko. Dmesg ouput: [ 8973.920435] btrfs: no symbol version for struct_module [ 8994.375255] btrfs: no symbol version for struct_module [ 9049.174891] btrfs: no symbol version for struct_module [ 9125.963989] btrfs: no symbol version for struct_module [ 9178.504381] btrfs: no version for struct_module found: kernel tainted. [ 9178.513086] btrfs: Unknown symbol security_inode_mkdir Is there a better way to do it ? I am running ubuntu. Cheers, -- Xavier Nicollet -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
