Re: scrub "correcting" tons of errors ?
On that note, is btrfs doing automatic background scrubs of its own or do I have to use crontab to schedule scrubs? Thanks! On Fri, Mar 29, 2013 at 1:58 PM, Josef Bacik wrote: > On Fri, Mar 29, 2013 at 03:50:15AM -0600, Swāmi Petaramesh wrote: >> Hi there, >> >> I've started "btrfs scrub start /" on one of my machines (Kernel >> 3.8.0-15 Ubuntu AMD64), which typically "behaves well" so I wasn't >> suspected any disk issue. >> >> After having ran for only 165 seconds, "scrub status" shows it pretends >> having found and corrected 22926 CSUM errors ??!?!?!?!!??? >> >> This is a rather new HDD, in perfect shape (SMART all OK, never >> reallocated a single sector, less than 200 hours total runtime...) >> >> WTF ?!? >> >> I've cancelled scrub for now, until I get further understanding of what >> can be happening... >> > > So this is probably because of the extent tree corruption you had, it's just > cleaning things up and you should be fine once it finishes. Thanks, > > Josef > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Space usage metering
Hi, I'm somewhat confused with who is right on what when it comes to showing disk usage and space free. I have a file system that I'm using for testing with a full linux installed as well as X and a desktop. I've created a good amount of snapshots and I've also done a lot of changes to the system throuhgout the process. I've run a balance before taking these meassurements: -> df -h reports 59G total space, 18G used, 7G free. -> btrfs fi df / reports the following: Data, RAID0: total=16.00GB, used=15.58GB System, RAID1: total=32.00MB, used=4.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=6.00GB, used=974.36MB So from what I garther the df reports used space somewhat correctly. Is it true that my total disk usage should only be around 16.6G now? Does that include all space required to store the snapshots as well? I'd be pretty amazed at that number because I've changed A LOT between snapshots. Why is df reporting only 7G free? Wouldn't it be possible to make it report the sum of total minus used instead which would be at least a lot closer to the truth? Thanks -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Space usage metering
Sorry I forgot to post that, -> btrfs fi show Label: none uuid: 2feccf06-5af8-4d8a-ad8d-a090cf4ef69a Total devices 2 FS bytes used 16.54GB devid1 size 40.90GB used 14.04GB path /dev/sda7 devid2 size 17.75GB used 14.03GB path /dev/sda6 The array is more of a for-fun and testing thing than a real world example. I've simply combined two differently sized partitions on the same disk together. So how much space is actually used now? On Fri, Mar 29, 2013 at 3:12 PM, Josef Bacik wrote: > On Fri, Mar 29, 2013 at 07:58:40AM -0600, Harald Glatt wrote: >> Hi, >> >> I'm somewhat confused with who is right on what when it comes to >> showing disk usage and space free. >> >> I have a file system that I'm using for testing with a full linux >> installed as well as X and a desktop. I've created a good amount of >> snapshots and I've also done a lot of changes to the system throuhgout >> the process. >> >> I've run a balance before taking these meassurements: >> >> -> df -h reports 59G total space, 18G used, 7G free. >> -> btrfs fi df / reports the following: >> Data, RAID0: total=16.00GB, used=15.58GB >> System, RAID1: total=32.00MB, used=4.00KB >> System: total=4.00MB, used=0.00 >> Metadata, RAID1: total=6.00GB, used=974.36MB >> >> So from what I garther the df reports used space somewhat correctly. >> Is it true that my total disk usage should only be around 16.6G now? >> Does that include all space required to store the snapshots as well? >> I'd be pretty amazed at that number because I've changed A LOT between >> snapshots. >> >> Why is df reporting only 7G free? Wouldn't it be possible to make it >> report the sum of total minus used instead which would be at least a >> lot closer to the truth? >> > > So you need to include btrfs fi show to get at why df -h reports what it > reports. Basically df does this > > total = total bytes > used = total data used * raid multiplier + total system used * raid > multiplier + > metadata used * raid multiplier > avail (or free) = (data total - data used) + (total bytes - allocated) > > where allocated is the sum of all the data chunks * their raid multipliers. > How > many disks do you have in this array? Take whatever that number is and > multiply > it by the numbers for the RAID1 chunks and thats how much real space is used. > Thanks, > > Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Space usage metering
Ah, now it makes sense!! Thanks! On Fri, Mar 29, 2013 at 3:27 PM, Hugo Mills wrote: > On Fri, Mar 29, 2013 at 03:18:03PM +0100, Harald Glatt wrote: >> Sorry I forgot to post that, >> >> -> btrfs fi show >> Label: none uuid: 2feccf06-5af8-4d8a-ad8d-a090cf4ef69a >> Total devices 2 FS bytes used 16.54GB >> devid1 size 40.90GB used 14.04GB path /dev/sda7 >> devid2 size 17.75GB used 14.03GB path /dev/sda6 >> >> The array is more of a for-fun and testing thing than a real world >> example. I've simply combined two differently sized partitions on the >> same disk together. >> >> So how much space is actually used now? >> >> On Fri, Mar 29, 2013 at 3:12 PM, Josef Bacik wrote: >> > On Fri, Mar 29, 2013 at 07:58:40AM -0600, Harald Glatt wrote: >> >> I've run a balance before taking these meassurements: >> >> >> >> -> df -h reports 59G total space, 18G used, 7G free. >> >> -> btrfs fi df / reports the following: >> >> Data, RAID0: total=16.00GB, used=15.58GB >> >> System, RAID1: total=32.00MB, used=4.00KB >> >> System: total=4.00MB, used=0.00 >> >> Metadata, RAID1: total=6.00GB, used=974.36MB > >OK, so you have 14.04+14.03 = 28.07 GiB allocated for immediate use > on the disk. > >Of that 28.07 GiB, > > - 16.00 GiB is assigned for RAID-0, and 15.58 GiB of that is actually > occupied by useful data. > > - 6.00*2 = 12.00 GiB is assigned for use by RAID-1 metadata, giving > 6.00 GiB of usable metadata. 974.36 MiB of that contains actual > metadata. > > - 68.00 MiB (2*32.00 + 4.00) is the remainder, assigned to the system > chunks (actually the chunk tree), of which 4 KiB is actually useful > data. > > Now, since you have RAID-0 data, the FS requires at least two > devices to stripe across. So the best it can manage is to add a > further 3.75 GiB of data chunks on each device (because /dev/sda6 has > 3.75 GiB not allocated). 3.75 GiB on two deivces for RAID-0 is 2*3.75 > = 7.5 GiB reported free by df. > >HTH, >Hugo. > > -- > === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === > PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk > --- Gentlemen! You can't fight here! This is the War Room! --- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] Online dedup for Btrfs
On Mon, Apr 1, 2013 at 2:50 PM, Josef Bacik wrote: > Hello, > > I was bored this weekend so I hacked up online dedup for Btrfs. It's working > quite well so I think it can be more widely tested. There are two ways to use > it > > 1) Compatible mode - this is a bit slower but will handle being used by older > kernels. We use the csum tree to find duplicate blocks. Since it is > relatively > easy to have crc32c collisions this also involves reading the block from disk > and doing a memcmp with the block we want to write to verify it has the same > data. This is way slow but hey, no incompat flag! > > 2) Incompatible mode - so this is the way you probably want to use it if you > don't care about being able to go back to older kernels. You select your > hashing function (at the momement I only support sha1 but there is room in the > format to have different functions). This creates a btree indexed by the hash > and the bytenr. Then we lookup the hash and just link the extent in if it > matches the hash. You can use -o paranoid-dedup if you are paranoid about > hash > collisions and this will force it to do the memcmp() dance to make sure that > the > extent we are deduping really matches the extent. > > So performance wise obviously the compat mode sucks. It's about 50% slower on > disk and about 20% slower on my Fusion card. We get pretty good space > savings, > about 10% in my horrible test (just copy a git tree onto the fs), but IMHO not > worth the performance hit. > > The incompat mode is a bit better, only 15% drop on disk and about 10% on my > fusion card. Closer to the crc numbers if we have -o paranoid-dedup. The > space > savings is better since it uses the original extent sizes, we get about 15% > space savings. Please feel free to pull and try it, you can get it here > > git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git dedup > > Thanks! > > Josef > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Hey Josef, that's really cool! Can this be used together with lzo compression for example? How high (roughly) is the impact of something like force-compress=lzo compared to the 15% hit from this dedup? Thanks! Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] Online dedup for Btrfs
Oh man :D It was so elaborate that I really believed it :P On Mon, Apr 1, 2013 at 5:38 PM, Josef Bacik wrote: > On Mon, Apr 01, 2013 at 08:50:34AM -0400, Josef Bacik wrote: >> Hello, >> >> I was bored this weekend so I hacked up online dedup for Btrfs. It's working >> quite well so I think it can be more widely tested. There are two ways to >> use >> it >> >> 1) Compatible mode - this is a bit slower but will handle being used by older >> kernels. We use the csum tree to find duplicate blocks. Since it is >> relatively >> easy to have crc32c collisions this also involves reading the block from disk >> and doing a memcmp with the block we want to write to verify it has the same >> data. This is way slow but hey, no incompat flag! >> >> 2) Incompatible mode - so this is the way you probably want to use it if you >> don't care about being able to go back to older kernels. You select your >> hashing function (at the momement I only support sha1 but there is room in >> the >> format to have different functions). This creates a btree indexed by the >> hash >> and the bytenr. Then we lookup the hash and just link the extent in if it >> matches the hash. You can use -o paranoid-dedup if you are paranoid about >> hash >> collisions and this will force it to do the memcmp() dance to make sure that >> the >> extent we are deduping really matches the extent. >> >> So performance wise obviously the compat mode sucks. It's about 50% slower >> on >> disk and about 20% slower on my Fusion card. We get pretty good space >> savings, >> about 10% in my horrible test (just copy a git tree onto the fs), but IMHO >> not >> worth the performance hit. >> >> The incompat mode is a bit better, only 15% drop on disk and about 10% on my >> fusion card. Closer to the crc numbers if we have -o paranoid-dedup. The >> space >> savings is better since it uses the original extent sizes, we get about 15% >> space savings. Please feel free to pull and try it, you can get it here >> >> git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git dedup >> >> Thanks! >> > > It's been pointed out to me that this is probably too serious, so just FYI > it's > April 1st where I am. Thanks, > > Josef > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Moving...
On Tue, Apr 2, 2013 at 7:14 PM, Swâmi Petaramesh wrote: > Le 02/04/2013 19:04, Roman Mamedov a écrit : >> but at this point I trust my data to BTRFS more, than I would trust ZFS. > > My experience with ZFS on Linux is still somewhat limited, the only > thing that I can say is that I've used it for about 2 years and a half > on my (quite loaded) home server without ever encoutering a single issue > (besides some tricky Ubuntu version upgrades, only due to the fact that > ZFS is "not in the standard kernel" by opposition to BTRFS). > > On the BTRFS side, I've used it on 5 machine for about a year on a daily > basis ; I've completely lost a filesystem, beyond repair, twice ; I > found myself with a machine that wouldn't boot anymore and would > necessitate serious and tricky FS maintenance, 3 times, I had to > completely reinstall the FS because of extremely degraded performance, 4 > times. > > Add to this that the FS management tools and properly working FS > features are an order of magnitude better in ZFS than in BTRFS... > > That's enough for me... > > I've already converted 2 machines from BTRFS to ZFS since yesterday. > > My netbook (now ZFS) boots to GDM in less than 30 seconds. My son's same > machine (except for the FS) takes 75 MORE seconds. > > You get it. > > Kind regards. > > -- > Swâmi Petaramesh http://petaramesh.org PGP 9076E32E > Ne cherchez pas : Je ne suis pas sur Facebook. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html I don't find it too suprising that you have more problems with a filesystem that is not final than with one that has seen several years of post-final improvements. I would be shocked if ZFS were in worse shape than btrfs right now. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Backup Options
Hi list, what are currently the backup options for btrfs? Is it possible to somehow duplicate the entire filesystem including all snapshots without using the real space for each snapshot onto a remote server? (Maybe given the remote server uses btrfs too?) Are these features planned? Right now the only solution I know of is dd if=eachDevice, netcat and xz or something combined. But that is not exactly a great solution. The other thing is rsyncing a single snapshot, but if I rsync several they occupy the full space on the remote machine each which is also less than optimal... Any other suggestions? Thanks! Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Backup Options
On Tue, Apr 2, 2013 at 7:37 PM, Harald Glatt wrote: > Hi list, > > what are currently the backup options for btrfs? > > Is it possible to somehow duplicate the entire filesystem including > all snapshots without using the real space for each snapshot onto a > remote server? (Maybe given the remote server uses btrfs too?) Are > these features planned? > > Right now the only solution I know of is dd if=eachDevice, netcat and > xz or something combined. But that is not exactly a great solution. > > The other thing is rsyncing a single snapshot, but if I rsync several > they occupy the full space on the remote machine each which is also > less than optimal... > > Any other suggestions? > > Thanks! > Harald I was informed (via private mail) of the existance of btrfs send / receive. I'm looking into that now to see how many of the things I'd like to see are (or will be) covered by it! Thanks Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Backup Options
On Wed, Apr 3, 2013 at 1:29 AM, Harald Glatt wrote: > On Tue, Apr 2, 2013 at 7:37 PM, Harald Glatt wrote: >> Hi list, >> >> what are currently the backup options for btrfs? >> >> Is it possible to somehow duplicate the entire filesystem including >> all snapshots without using the real space for each snapshot onto a >> remote server? (Maybe given the remote server uses btrfs too?) Are >> these features planned? >> >> Right now the only solution I know of is dd if=eachDevice, netcat and >> xz or something combined. But that is not exactly a great solution. >> >> The other thing is rsyncing a single snapshot, but if I rsync several >> they occupy the full space on the remote machine each which is also >> less than optimal... >> >> Any other suggestions? >> >> Thanks! >> Harald > > I was informed (via private mail) of the existance of btrfs send / > receive. I'm looking into that now to see how many of the things I'd > like to see are (or will be) covered by it! > > Thanks > Harald Hi all, I've attempted to try out the send/receive functionality but I can't get it to work. I've taken a random snapshot of my laptops drive and wanted to replicate it into a brand new btrfs volume in a VM. On the laptop (source): # btrfs send /snapshot | nc 10.10.10.108 On the VM (receiver): # nc -l -p | btrfs receive /mnt/restore I'm getting the following messages on the receiving end, and an immediate cancellation: At subvol snapshot receiving subvol snapshot uuid=f6c01679-a373-d74e-b0ab-2074330239bc, stransid=3991 chown - uid=0, gid=0 chmod - mode=0755 utimes ERROR: utimes failed. Bad file descriptor I'm running on kernel 3.8.5 on the sending side and kernel 3.8.0 on the receiving side. btrfs-progs are from the end of January in both cases. I saw the code hasn't been maintained since August last year. Is it broken or am I making some mistake?? Thanks Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Backup Options
On Wed, Apr 3, 2013 at 4:23 AM, Harald Glatt wrote: > On Wed, Apr 3, 2013 at 1:29 AM, Harald Glatt wrote: >> On Tue, Apr 2, 2013 at 7:37 PM, Harald Glatt wrote: >>> Hi list, >>> >>> what are currently the backup options for btrfs? >>> >>> Is it possible to somehow duplicate the entire filesystem including >>> all snapshots without using the real space for each snapshot onto a >>> remote server? (Maybe given the remote server uses btrfs too?) Are >>> these features planned? >>> >>> Right now the only solution I know of is dd if=eachDevice, netcat and >>> xz or something combined. But that is not exactly a great solution. >>> >>> The other thing is rsyncing a single snapshot, but if I rsync several >>> they occupy the full space on the remote machine each which is also >>> less than optimal... >>> >>> Any other suggestions? >>> >>> Thanks! >>> Harald >> >> I was informed (via private mail) of the existance of btrfs send / >> receive. I'm looking into that now to see how many of the things I'd >> like to see are (or will be) covered by it! >> >> Thanks >> Harald > > Hi all, > > I've attempted to try out the send/receive functionality but I can't > get it to work. > > I've taken a random snapshot of my laptops drive and wanted to > replicate it into a brand new btrfs volume in a VM. > > On the laptop (source): > # btrfs send /snapshot | nc 10.10.10.108 > > On the VM (receiver): > # nc -l -p | btrfs receive /mnt/restore > > I'm getting the following messages on the receiving end, and an > immediate cancellation: > > At subvol snapshot > receiving subvol snapshot uuid=f6c01679-a373-d74e-b0ab-2074330239bc, > stransid=3991 > chown - uid=0, gid=0 > chmod - mode=0755 > utimes > ERROR: utimes failed. Bad file descriptor > > I'm running on kernel 3.8.5 on the sending side and kernel 3.8.0 on > the receiving side. btrfs-progs are from the end of January in both > cases. > > I saw the code hasn't been maintained since August last year. Is it > broken or am I making some mistake?? > > Thanks > Harald Once I had sent the previous mail I noticed that had accidentially lied about the command on the receiver side: # nc -l -p | btrfs receive /mnt/restore However what I actually did was: # cd /mnt/restore # nc -l -p | btrfs receive . After noticing this difference I had to try it again as described in my mail and - oh wonder - it works now!! Giving 'btrfs receive' a dot as a parameter seems to fail in this case. Is this expected behavior or a bug? Anyway I fixed my own problem and I'll continue trying the send / receive out. Sorry for the trouble :) Thanks! Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [BUG] btrfs.fsck failing to fix corrupted block
On Sun, Apr 7, 2013 at 3:56 AM, Shripad Ratnaparkhi wrote: > [Replying to my own email] > > Found a patch which seems to be discusses the fix in a patch provided here: > https://patchwork.kernel.org/patch/1946561/ > > Still now sure whether that is already there in v0.20-rc1 or > applicable to fix my issue. > > thanks, > ~shripadr > > PS: BCC'ing the patch provider. > > On Sun, Apr 7, 2013 at 7:16 AM, Shripad Ratnaparkhi > wrote: >> Hi there, >> I am newbie and recently started using btrfs. Now facing a weird problem. >> >> FWIW, I am on archlinux, kenel v3.8.0, having Btrfs v0.20-rc1. >> After an abnormal reboot, getting these errors while boot: >> >> systemd.fsck[289]: checking extents >> systemd.fsck[289]: checking fs roots >> systemd.fsck[289]: checking root refs >> systemd.fsck[289]: found 23728128 bytes used err is 0 >> systemd.fsck[289]: total csum bytes: 23092 >> systemd.fsck[289]: total tree bytes: 81920 >> systemd.fsck[289]: total fs tree bytes: 24576 >> systemd.fsck[289]: btree space waste bytes: 38038 >> systemd.fsck[289]: file data blocks allocated: 23646208 >> systemd.fsck[289]: referenced 23646208 >> systemd.fsck[289]: Btrfs v0.20-rc1 >> [ OK ] Started File System Check on /dev/sda1. >>Mounting /boot... >> systemd.fsck[289]: checking extents >> [ OK ] Mounted /boot. >> [ OK ] Reached target Sound Card. >> systemd.fsck[289]: checking fs roots >> systemd.fsck[289]: checking root refs >> systemd.fsck[289]: extent buffer leak: start 206893056 len 4096 >> systemd.fsck[289]: *** Error in `fsck.btrfs`: corrupted double-linked >> list: 0x02081d80 *** >> >> and its not proceeding and stuck here. >> I have the rescue disk handy, but not sure how can I fix this issue. >> Any help please. >> >> thanks, >> >> -- >> ~shripadr > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html The version that you can get from btrfs doesn't say a whole lot sadly. You can build the newest tools from git and try with the btrfsck there. Now this might not be entirely correct, but as far as I know the btrfsck tool isn't ready to be used in the traditional fsck capacity (automatically at bootup, etc). They don't symlink the tool on arch and per default arch builds it's initrd without a fsck for btrfs available. If you manually symlinked the tool to get rid of that error message during initrd creation I'd suggestion removing that symlink again and rebuilding your initrd without a btrfsck in place. As far as I know the best and most reliable way at the moment to confirm or repair errors is by btrfs scrub. Once you removed the startup fsck it might boot normally. You can then start a scrub with 'btrfs scrub start /' and see its status and progress via 'btrfs scrub status /'. If the scrub reports no errors, or finds some but repairs them, that is (as far as I know) a more reliable message than whatever btrfsck thinks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs can't complete balance
Hi all, I've managed to ruin my btrfs filesystem, but I don't know how. I basically have it split up into two subvolumes, one for the root and one for home both of which are creating hourly snapshots. I've 'btrfs send' some of those to an external machine every now and then. (At least once a day) At some point 'btrfs send' stopped working but gave no better error message than invalid argument. I did a balance on the drive which resulted in a kernel panic after a while of working. It then repeated the balance automatically on the next bootup until it finally said 'balance failed' when I checked with btrfs fi balance status. I'm running a funky setup here where two partitions on the same drive are both added into the same btrfs volume resulting in this setup: Label: none uuid: 4e74fa05-4bf2-40da-946b-8cd53ce92242 Total devices 2 FS bytes used 54.60GB devid1 size 60.52GB used 52.04GB path /dev/sda7 devid2 size 16.32GB used 8.03GB path /dev/sda5 Data: total=54.00GB, used=53.51GB System, RAID1: total=32.00MB, used=12.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=3.00GB, used=1.11GB The state that I'm in is that everything is working, I haven't found any data corruption, scrubs work and confirm that there are no errors but I can no longer use 'btrfs send' to create differential images and I can also no longer complete a balance. I tried running balance start -dconvert=single to get rid of my second partition but that also failed (without a panic though) I have backed up the newest state of the filesystem via rsync and am about to wipe and restore from backup with a new fs. Is anyone interested in a btrfs-image to debug this state? If so please tell me what command to run since I have never done this before... Thanks Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs can't complete balance
The dmesg from the kernel panic can be seen here: http://pastebin.com/3QZqrqAX I've tried to create a btrfs-image but I'm getting a crash of the utility after only 50 MB... Check tree block failed, want=320304799744, have=0 Check tree block failed, want=320305049600, have=0 Check tree block failed, want=320298184704, have=0 Check tree block failed, want=320298184704, have=0 Check tree block failed, want=320305221632, have=0 Check tree block failed, want=320308678656, have=0 Check tree block failed, want=320308678656, have=0 Check tree block failed, want=320297295872, have=0 Check tree block failed, want=320297295872, have=0 Check tree block failed, want=320297295872, have=721320548710005777 Check tree block failed, want=320297295872, have=721320548710005777 Check tree block failed, want=320297295872, have=721320548710005777 read block failed check_tree_block btrfs-image: btrfs-image.c:518: create_metadump: Assertion `!(ret < 0)' failed. [1]32513 abort (core dumped) On Mon, Apr 8, 2013 at 9:48 AM, Harald Glatt wrote: > Hi all, > > I've managed to ruin my btrfs filesystem, but I don't know how. I > basically have it split up into two subvolumes, one for the root and > one for home both of which are creating hourly snapshots. > > I've 'btrfs send' some of those to an external machine every now and > then. (At least once a day) > > At some point 'btrfs send' stopped working but gave no better error > message than invalid argument. > > I did a balance on the drive which resulted in a kernel panic after a > while of working. It then repeated the balance automatically on the > next bootup until it finally said 'balance failed' when I checked with > btrfs fi balance status. > > I'm running a funky setup here where two partitions on the same drive > are both added into the same btrfs volume resulting in this setup: > > Label: none uuid: 4e74fa05-4bf2-40da-946b-8cd53ce92242 > Total devices 2 FS bytes used 54.60GB > devid1 size 60.52GB used 52.04GB path /dev/sda7 > devid2 size 16.32GB used 8.03GB path /dev/sda5 > > Data: total=54.00GB, used=53.51GB > System, RAID1: total=32.00MB, used=12.00KB > System: total=4.00MB, used=0.00 > Metadata, RAID1: total=3.00GB, used=1.11GB > > The state that I'm in is that everything is working, I haven't found > any data corruption, scrubs work and confirm that there are no errors > but I can no longer use 'btrfs send' to create differential images and > I can also no longer complete a balance. > > I tried running balance start -dconvert=single to get rid of my second > partition but that also failed (without a panic though) > > I have backed up the newest state of the filesystem via rsync and am > about to wipe and restore from backup with a new fs. > > Is anyone interested in a btrfs-image to debug this state? If so > please tell me what command to run since I have never done this > before... > > Thanks > Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs can't complete balance
On Mon, Apr 8, 2013 at 10:45 AM, Harald Glatt wrote: > The dmesg from the kernel panic can be seen here: > http://pastebin.com/3QZqrqAX > > I've tried to create a btrfs-image but I'm getting a crash of the > utility after only 50 MB... > > Check tree block failed, want=320304799744, have=0 > Check tree block failed, want=320305049600, have=0 > Check tree block failed, want=320298184704, have=0 > Check tree block failed, want=320298184704, have=0 > Check tree block failed, want=320305221632, have=0 > Check tree block failed, want=320308678656, have=0 > Check tree block failed, want=320308678656, have=0 > Check tree block failed, want=320297295872, have=0 > Check tree block failed, want=320297295872, have=0 > Check tree block failed, want=320297295872, have=721320548710005777 > Check tree block failed, want=320297295872, have=721320548710005777 > Check tree block failed, want=320297295872, have=721320548710005777 > read block failed check_tree_block > btrfs-image: btrfs-image.c:518: create_metadump: Assertion `!(ret < 0)' > failed. > [1]32513 abort (core dumped) > > > On Mon, Apr 8, 2013 at 9:48 AM, Harald Glatt wrote: >> Hi all, >> >> I've managed to ruin my btrfs filesystem, but I don't know how. I >> basically have it split up into two subvolumes, one for the root and >> one for home both of which are creating hourly snapshots. >> >> I've 'btrfs send' some of those to an external machine every now and >> then. (At least once a day) >> >> At some point 'btrfs send' stopped working but gave no better error >> message than invalid argument. >> >> I did a balance on the drive which resulted in a kernel panic after a >> while of working. It then repeated the balance automatically on the >> next bootup until it finally said 'balance failed' when I checked with >> btrfs fi balance status. >> >> I'm running a funky setup here where two partitions on the same drive >> are both added into the same btrfs volume resulting in this setup: >> >> Label: none uuid: 4e74fa05-4bf2-40da-946b-8cd53ce92242 >> Total devices 2 FS bytes used 54.60GB >> devid1 size 60.52GB used 52.04GB path /dev/sda7 >> devid2 size 16.32GB used 8.03GB path /dev/sda5 >> >> Data: total=54.00GB, used=53.51GB >> System, RAID1: total=32.00MB, used=12.00KB >> System: total=4.00MB, used=0.00 >> Metadata, RAID1: total=3.00GB, used=1.11GB >> >> The state that I'm in is that everything is working, I haven't found >> any data corruption, scrubs work and confirm that there are no errors >> but I can no longer use 'btrfs send' to create differential images and >> I can also no longer complete a balance. >> >> I tried running balance start -dconvert=single to get rid of my second >> partition but that also failed (without a panic though) >> >> I have backed up the newest state of the filesystem via rsync and am >> about to wipe and restore from backup with a new fs. >> >> Is anyone interested in a btrfs-image to debug this state? If so >> please tell me what command to run since I have never done this >> before... >> >> Thanks >> Harald I didn't realize it had to be unmounted for the image. I've uploaded a successful image with -c9. I'll give out the URL on demand. However there are news: I managed after creating that image to successfully balance down to single and remove the second drive. I readded it again and now everything is back to normal and working... It is possible that after that kernel panic it indeed finished the balance correctly and I misread the error message saying something along the lines of 'error while getting status: balance not in progress' for something that was telling me the balance failed... Ultimately that reduces this issue down to, something caused 'btrfs send' to fail, a balance then kernel paniced and afterwards everything was back to normal. I don't think the image that I have will be of any help because that imagine is of course from after that kernel panic and I suppose completeling that balance then fixed the issues - even if it paniced along the way. Thanks Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs can't complete balance
You're right, I'm seeing this 'object already exists' message in my dmesg log. Nice catch!! I'm posting it again in case others missed it too, this was the kernel panic dmesg that happened during the balance that I did in order to fix the 'btrfs send' not working anymore problem: http://pastebin.com/3QZqrqAX On Mon, Apr 8, 2013 at 1:14 PM, Roman Mamedov wrote: > On Mon, 8 Apr 2013 09:48:55 +0200 > Harald Glatt wrote: > >> At some point 'btrfs send' stopped working but gave no better error >> message than invalid argument. > > Usually in cases like this there are more details in 'dmesg'. > >> I did a balance on the drive which resulted in a kernel panic after a >> while of working. > > Maybe you faced the same problem that I had recently with "Object already > exists" . Could've been useful if you saved the actual kernel panic message. > > -- > With respect, > Roman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs can't complete balance
That's not gonna be a problem, I'm always running on the most recent final kernel - in the case of my panic it was 3.8.6. Thanks! On Mon, Apr 8, 2013 at 1:22 PM, Roman Mamedov wrote: > On Mon, 8 Apr 2013 13:19:13 +0200 > Harald Glatt wrote: > >> You're right, I'm seeing this 'object already exists' message in my >> dmesg log. Nice catch!! > > See this thread: > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg23514.html > > This patch fixes the problem: > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg23534.html > > But it is / will be only in the very latest kernels (including 'stable' > updates), because it was posted only like a week ago. > > -- > With respect, > Roman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS 3.8.7 Kernel Crash Report
On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley wrote: > Thanks, David. > > What causes this corruption and how can I fix it? > > I'm very worried about running btrfs.fsck as last time it made a slight > corruption like this worse and the whole volume had to be trashed. > > After fsck the "available space" on df ended up being negative so nothing > could be written to the volume. > > Mark > > On 22/04/2013 14:42, "David Sterba" wrote: > >>On Mon, Apr 22, 2013 at 01:19:41PM +, Mark Ridley wrote: >>> If I then use rsync --inplace to update the images, I get a btrfs stack >>>trace >>> and btrfs hangs: >>> >>> This happens every night. >> >>> [] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >> >>The check fails because it finds keys in reverted order. Given the >>conditions under which it happens I think it's an on-disk corruption and >>fsck should be able to at least detect it. >> >>david > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html This happened without --repair? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS 3.8.7 Kernel Crash Report
Yeah, --repair is not recommended as of now. Maybe posting the btrfsck output just from checking would be helpful in this case. On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley wrote: > No. > > Without --repair pointed out some problems, but whats the point of knowing > about the problems if they can't be fixed so I ran with --repair and it > broke the volume. > > On 22/04/2013 15:02, "Harald Glatt" wrote: > >>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>wrote: >>> Thanks, David. >>> >>> What causes this corruption and how can I fix it? >>> >>> I'm very worried about running btrfs.fsck as last time it made a slight >>> corruption like this worse and the whole volume had to be trashed. >>> >>> After fsck the "available space" on df ended up being negative so >>>nothing >>> could be written to the volume. >>> >>> Mark >>> >>> On 22/04/2013 14:42, "David Sterba" wrote: >>> >>>>On Mon, Apr 22, 2013 at 01:19:41PM +, Mark Ridley wrote: >>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>stack >>>>>trace >>>>> and btrfs hangs: >>>>> >>>>> This happens every night. >>>> >>>>> [] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>> >>>>The check fails because it finds keys in reverted order. Given the >>>>conditions under which it happens I think it's an on-disk corruption and >>>>fsck should be able to at least detect it. >>>> >>>>david >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" >>>in >>> the body of a message to majord...@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >>This happened without --repair? > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS 3.8.7 Kernel Crash Report
Only data errors (from CRC checks), maybe also some structure errors - I'm not sure. A remount should fix all errors. If it doesn't I think it's considered a bug - ultimately the goal is that no --repair should ever be required... Scrub is generally safe though, unlike btrfsck --repair. On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley wrote: > What does btrfs scrub do? > > Is that meant to detect and fix problems? > > Thanks, > > Mark > > On 22/04/2013 15:13, "Harald Glatt" wrote: > >>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>output just from checking would be helpful in this case. >> >>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley >>wrote: >>> No. >>> >>> Without --repair pointed out some problems, but whats the point of >>>knowing >>> about the problems if they can't be fixed so I ran with --repair and it >>> broke the volume. >>> >>> On 22/04/2013 15:02, "Harald Glatt" wrote: >>> >>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>wrote: >>>>> Thanks, David. >>>>> >>>>> What causes this corruption and how can I fix it? >>>>> >>>>> I'm very worried about running btrfs.fsck as last time it made a >>>>>slight >>>>> corruption like this worse and the whole volume had to be trashed. >>>>> >>>>> After fsck the "available space" on df ended up being negative so >>>>>nothing >>>>> could be written to the volume. >>>>> >>>>> Mark >>>>> >>>>> On 22/04/2013 14:42, "David Sterba" wrote: >>>>> >>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +, Mark Ridley wrote: >>>>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>>>stack >>>>>>>trace >>>>>>> and btrfs hangs: >>>>>>> >>>>>>> This happens every night. >>>>>> >>>>>>> [] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>> >>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>conditions under which it happens I think it's an on-disk corruption >>>>>>and >>>>>>fsck should be able to at least detect it. >>>>>> >>>>>>david >>>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" >>>>>in >>>>> the body of a message to majord...@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>>>This happened without --repair? >>> > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS 3.8.7 Kernel Crash Report
I heard it's coming out next week. On Mon, Apr 22, 2013 at 4:40 PM, Mark Ridley wrote: > Any idea when the 3.9 kernel will be out. > > I see there are lots of btrfs fixes going into it. > > I'm currently running FC18 on 3.8.7, although 10 minutes ago it got > updated to 3.8.8. > > On 22/04/2013 15:28, "Harald Glatt" wrote: > >>I think it won't... That would just be the goal eventually. If scrub >>sees no errors all the data should be in tact and your best bet to get >>things working perfectly again is to create a new filesystem and >>transfer the data into it. >> >>I'm not a dev and I can't say if a btrfs-image for debugging purposes >>would be helpful in this case or not, maybe this kind of corrution has >>been fixed a long time ago and wouldn't happen again in up-to-date >>versions of btrfs anyway. >> >>On Mon, Apr 22, 2013 at 4:24 PM, Mark Ridley >>wrote: >>> Thanks for getting back to me. >>> >>> Do you think remount fixes the keys reverted problem? >>> >>> On 22/04/2013 15:18, "Harald Glatt" wrote: >>> >>>>Only data errors (from CRC checks), maybe also some structure errors - >>>>I'm not sure. A remount should fix all errors. If it doesn't I think >>>>it's considered a bug - ultimately the goal is that no --repair should >>>>ever be required... Scrub is generally safe though, unlike btrfsck >>>>--repair. >>>> >>>>On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley >>>>wrote: >>>>> What does btrfs scrub do? >>>>> >>>>> Is that meant to detect and fix problems? >>>>> >>>>> Thanks, >>>>> >>>>> Mark >>>>> >>>>> On 22/04/2013 15:13, "Harald Glatt" wrote: >>>>> >>>>>>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>>>>>output just from checking would be helpful in this case. >>>>>> >>>>>>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley >>>>>> >>>>>>wrote: >>>>>>> No. >>>>>>> >>>>>>> Without --repair pointed out some problems, but whats the point of >>>>>>>knowing >>>>>>> about the problems if they can't be fixed so I ran with --repair and >>>>>>>it >>>>>>> broke the volume. >>>>>>> >>>>>>> On 22/04/2013 15:02, "Harald Glatt" wrote: >>>>>>> >>>>>>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>>>>> >>>>>>>>wrote: >>>>>>>>> Thanks, David. >>>>>>>>> >>>>>>>>> What causes this corruption and how can I fix it? >>>>>>>>> >>>>>>>>> I'm very worried about running btrfs.fsck as last time it made a >>>>>>>>>slight >>>>>>>>> corruption like this worse and the whole volume had to be trashed. >>>>>>>>> >>>>>>>>> After fsck the "available space" on df ended up being negative so >>>>>>>>>nothing >>>>>>>>> could be written to the volume. >>>>>>>>> >>>>>>>>> Mark >>>>>>>>> >>>>>>>>> On 22/04/2013 14:42, "David Sterba" wrote: >>>>>>>>> >>>>>>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +, Mark Ridley wrote: >>>>>>>>>>> If I then use rsync --inplace to update the images, I get a >>>>>>>>>>>btrfs >>>>>>>>>>>stack >>>>>>>>>>>trace >>>>>>>>>>> and btrfs hangs: >>>>>>>>>>> >>>>>>>>>>> This happens every night. >>>>>>>>>> >>>>>>>>>>> [] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>>>>>> >>>>>>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>>>>>conditions under which it happens I think it's an on-disk >>>>>>>>>>corruption >>>>>>>>>>and >>>>>>>>>>fsck should be able to at least detect it. >>>>>>>>>> >>>>>>>>>>david >>>>>>>>> >>>>>>>>> -- >>>>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>>>>linux-btrfs" >>>>>>>>>in >>>>>>>>> the body of a message to majord...@vger.kernel.org >>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>>> >>>>>>>>This happened without --repair? >>>>>>> >>>>> >>> > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: BTRFS 3.8.7 Kernel Crash Report
I think it won't... That would just be the goal eventually. If scrub sees no errors all the data should be in tact and your best bet to get things working perfectly again is to create a new filesystem and transfer the data into it. I'm not a dev and I can't say if a btrfs-image for debugging purposes would be helpful in this case or not, maybe this kind of corrution has been fixed a long time ago and wouldn't happen again in up-to-date versions of btrfs anyway. On Mon, Apr 22, 2013 at 4:24 PM, Mark Ridley wrote: > Thanks for getting back to me. > > Do you think remount fixes the keys reverted problem? > > On 22/04/2013 15:18, "Harald Glatt" wrote: > >>Only data errors (from CRC checks), maybe also some structure errors - >>I'm not sure. A remount should fix all errors. If it doesn't I think >>it's considered a bug - ultimately the goal is that no --repair should >>ever be required... Scrub is generally safe though, unlike btrfsck >>--repair. >> >>On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley >>wrote: >>> What does btrfs scrub do? >>> >>> Is that meant to detect and fix problems? >>> >>> Thanks, >>> >>> Mark >>> >>> On 22/04/2013 15:13, "Harald Glatt" wrote: >>> >>>>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>>>output just from checking would be helpful in this case. >>>> >>>>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley >>>>wrote: >>>>> No. >>>>> >>>>> Without --repair pointed out some problems, but whats the point of >>>>>knowing >>>>> about the problems if they can't be fixed so I ran with --repair and >>>>>it >>>>> broke the volume. >>>>> >>>>> On 22/04/2013 15:02, "Harald Glatt" wrote: >>>>> >>>>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>>> >>>>>>wrote: >>>>>>> Thanks, David. >>>>>>> >>>>>>> What causes this corruption and how can I fix it? >>>>>>> >>>>>>> I'm very worried about running btrfs.fsck as last time it made a >>>>>>>slight >>>>>>> corruption like this worse and the whole volume had to be trashed. >>>>>>> >>>>>>> After fsck the "available space" on df ended up being negative so >>>>>>>nothing >>>>>>> could be written to the volume. >>>>>>> >>>>>>> Mark >>>>>>> >>>>>>> On 22/04/2013 14:42, "David Sterba" wrote: >>>>>>> >>>>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +, Mark Ridley wrote: >>>>>>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>>>>>stack >>>>>>>>>trace >>>>>>>>> and btrfs hangs: >>>>>>>>> >>>>>>>>> This happens every night. >>>>>>>> >>>>>>>>> [] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>>>> >>>>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>>>conditions under which it happens I think it's an on-disk corruption >>>>>>>>and >>>>>>>>fsck should be able to at least detect it. >>>>>>>> >>>>>>>>david >>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>>linux-btrfs" >>>>>>>in >>>>>>> the body of a message to majord...@vger.kernel.org >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>>>This happened without --repair? >>>>> >>> > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [BUG] "btrfs csum failed" on v3.9.0-rc7
On Wed, Apr 24, 2013 at 1:24 PM, Tom Gundersen wrote: > I'm having lots of problems with wrong checksums on the most recent > kernels. Note that this is not a regression as far as I know, just > more pronounced now than before (the increase in severity might be due > to changes in my setup). > > I see that this was discussed on the ML a few months back, but it was > not clear to me if the problem is still open or if a solution should > have landed upstream. > > > > This is what I'm seeing: > > Pretty much on every reboot some (but not all) of the files written to > or created before the reboot are broken. If the offending files are > deleted / overwritten the problem goes away (at least until next > reboot when other files are affected). A random selection of "dmesg | > grep btrfs" is attached below. > > As I can easily reproduce, please let me know how I can help debugging > further. For instance, how can I tell btrfs to ignore the checksum > error and give me the file it has anyway (to see if the file is > garbled, or just the checksum is wrong)? > > My btrfs volume is made up of two partitions, and is split into three > subvolumes. When mounting the rootfs I see this in dmesg: > > Apr 24 01:31:47 toms-air kernel: device fsid > 0d7a2474-3523-413e-8611-1f489b8a9891 devid 1 transid 141284 /dev/sda4 > Apr 24 01:31:47 toms-air kernel: device fsid > 0d7a2474-3523-413e-8611-1f489b8a9891 devid 2 transid 141284 /dev/sda2 > Apr 24 01:31:47 toms-air kernel: device fsid > 0d7a2474-3523-413e-8611-1f489b8a9891 devid 2 transid 141284 /dev/sda2 > Apr 24 01:31:47 toms-air kernel: device fsid > 0d7a2474-3523-413e-8611-1f489b8a9891 devid 1 transid 141284 /dev/sda4 > Apr 24 01:31:47 toms-air kernel: device fsid > 0d7a2474-3523-413e-8611-1f489b8a9891 devid 2 transid 141284 /dev/sda2 > Apr 24 01:31:47 toms-air kernel: btrfs: use ssd allocation scheme > Apr 24 01:31:47 toms-air kernel: btrfs: use lzo compression > Apr 24 01:31:47 toms-air kernel: btrfs: disk space caching is enabled > Apr 24 01:31:47 toms-air kernel: btrfs: bdev /dev/sda2 errs: wr 0, rd > 0, flush 0, corrupt 2056270, gen 6 > Apr 24 01:31:47 toms-air kernel: btrfs: bdev /dev/sda4 errs: wr 0, rd > 0, flush 0, corrupt 2049061, gen 6 > Apr 24 01:31:47 toms-air kernel: device fsid > 0d7a2474-3523-413e-8611-1f489b8a9891 devid 2 transid 141284 /dev/sda2 > > And the output of findmnt is: > > TARGET SOURCEFSTYPE OPTIONS > /home UUID=0d7a2474-3523-413e-8611-1f489b8a9891 btrfs > subvol=home,ssd,compress=lzo,x-systemd.automount,nofail > /var UUID=0d7a2474-3523-413e-8611-1f489b8a9891 btrfs > subvol=var,ssd,compress=lzo > /usr UUID=0d7a2474-3523-413e-8611-1f489b8a9891 btrfs > subvol=usr,ssd,compress=lzo > > > > Errors reported in dmesg: > > [10520.530437] btrfs csum failed ino 1988603 off 1277952 csum > 2566472073 private 2887162790 > [10520.535299] btrfs csum failed ino 1988542 off 172032 csum > 1032373158 private 2555710917 > [10520.535489] btrfs csum failed ino 1988542 off 172032 csum > 1032373158 private 2555710917 > [10520.536448] btrfs csum failed ino 1988542 off 307200 csum > 2566472073 private 4189934277 > [10521.404738] btrfs csum failed ino 1988603 off 1277952 csum > 2566472073 private 2887162790 > [10521.406514] btrfs csum failed ino 1988542 off 192512 csum > 2359321615 private 259683409 > [10521.407797] btrfs csum failed ino 1988542 off 372736 csum > 2566472073 private 1399566794 > [10521.620012] btrfs csum failed ino 1988603 off 1277952 csum > 2566472073 private 2887162790 > [10521.621371] btrfs csum failed ino 1988542 off 192512 csum > 2359321615 private 259683409 > [10521.622048] btrfs csum failed ino 1988542 off 372736 csum > 2566472073 private 1399566794 > [10546.115794] btrfs_readpage_end_io_hook: 26 callbacks suppressed > [10546.115806] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.116811] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.117847] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.118527] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.118910] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.119436] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.119856] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.120292] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.120683] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10546.121086] btrfs csum failed ino 1988548 off 28672 csum 2066685480 > private 49363816 > [10553.246253] btrfs_readpage_end_io_hook: 2 callbacks suppressed > [10553.246269] btrfs csum failed ino 114348 off 45056 csum 1787155441 > private 2298707641 > [10553.246541] btrfs csum failed ino 114348 off 45056 csum 1787155441 > private 2298707641 > [10554.761105] btrfs csum failed ino 198
Re: Btrfs performance problem; metadata size to blame?
On Sun, Apr 28, 2013 at 9:04 PM, John . wrote: > Hi guys, > > My Btrfs fs has a performance problem which I hope you can help me > solve. I have a dataset of around 3.15 TiB, that has lived on a ZFS > volume for almost two years (ZRAID1, 4 2TiB disks). In order to move > to Btrfs I bought myself a 4TiB disk with the idea of buying a new one > next week and balance it to a RAID1 of 2 4TiB disks. > > I created a single disk Btrfs volume with the default mkfs options (no > data duplication, metadata duplication on). Next I transferred my > dataset to this disk (no problems there). Today when I tried to create > a directory and I noticed the Btrfs volume was awefully slow; it took > a few seconds to create the directory and a few to delete it (which > should be milliseconds as you know). In fact each and every operation > on the volume grinded the fs down to a halt. > > FS information: > > # btrfs fi df /storage > Data: total=3.29TB, used=3.15TB > System, DUP: total=8.00MB, used=360.00KB > System: total=4.00MB, used=0.00 > Metadata, DUP: total=4.00GB, used=3.88GB > Metadata: total=8.00MB, used=0.00 > > # btrfs fi show > Label: 'storage' uuid: 3fa262cd-baa9-46dc-92a8-318c87166186 > Total devices 1 FS bytes used 3.16TB > devid1 size 3.64TB used 3.30TB path /dev/sdb > > I suspect my performance blow has everything to do with the abysmally > low amount of space for metadata that is left, but since I am not a > Btrfs guru I don't now whether this is truly the case and/or how to > solve it. btrfs fi balance start -dusage=5 did not help. > > Yours, > > John > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Try to defragment the root of the volume (e.g the mountpoint). While it's mounted: btrfs fi defrag /path/to/mnt Then try performance again -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs performance problem; metadata size to blame?
On Sun, Apr 28, 2013 at 9:18 PM, John . wrote: > I use Ubuntu with kernel 3.8.0-19-generic. I also tested with the > latest live disk of Arch Linux; write performance was the same (bad). > > My mount options: rw,compress=lzo. > Iotop does not show any strange disk activity. > > 2013/4/28 Harald Glatt : >> On Sun, Apr 28, 2013 at 9:10 PM, John . wrote: >>> Hi Harald, >>> >>> I did perform a defrag of the volume a few hours ago. This did not >>> make a difference. :( >>> >>> Yours, >>> >>> John >>> >>> 2013/4/28 Harald Glatt : >>>> On Sun, Apr 28, 2013 at 9:04 PM, John . wrote: >>>>> Hi guys, >>>>> >>>>> My Btrfs fs has a performance problem which I hope you can help me >>>>> solve. I have a dataset of around 3.15 TiB, that has lived on a ZFS >>>>> volume for almost two years (ZRAID1, 4 2TiB disks). In order to move >>>>> to Btrfs I bought myself a 4TiB disk with the idea of buying a new one >>>>> next week and balance it to a RAID1 of 2 4TiB disks. >>>>> >>>>> I created a single disk Btrfs volume with the default mkfs options (no >>>>> data duplication, metadata duplication on). Next I transferred my >>>>> dataset to this disk (no problems there). Today when I tried to create >>>>> a directory and I noticed the Btrfs volume was awefully slow; it took >>>>> a few seconds to create the directory and a few to delete it (which >>>>> should be milliseconds as you know). In fact each and every operation >>>>> on the volume grinded the fs down to a halt. >>>>> >>>>> FS information: >>>>> >>>>> # btrfs fi df /storage >>>>> Data: total=3.29TB, used=3.15TB >>>>> System, DUP: total=8.00MB, used=360.00KB >>>>> System: total=4.00MB, used=0.00 >>>>> Metadata, DUP: total=4.00GB, used=3.88GB >>>>> Metadata: total=8.00MB, used=0.00 >>>>> >>>>> # btrfs fi show >>>>> Label: 'storage' uuid: 3fa262cd-baa9-46dc-92a8-318c87166186 >>>>> Total devices 1 FS bytes used 3.16TB >>>>> devid1 size 3.64TB used 3.30TB path /dev/sdb >>>>> >>>>> I suspect my performance blow has everything to do with the abysmally >>>>> low amount of space for metadata that is left, but since I am not a >>>>> Btrfs guru I don't now whether this is truly the case and/or how to >>>>> solve it. btrfs fi balance start -dusage=5 did not help. >>>>> >>>>> Yours, >>>>> >>>>> John >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>>>> the body of a message to majord...@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>>> Try to defragment the root of the volume (e.g the mountpoint). While >>>> it's mounted: >>>> btrfs fi defrag /path/to/mnt >>>> >>>> Then try performance again >> >> What kernel version do you use? What are your mount options? Try to >> run iotop and see if there is any unusual activity... Try mount -o inode_cache,space_cache,autodefrag - first mount with the new options might take a while, also there might be disk activity for a while after mounting with this... -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs performance problem; metadata size to blame?
On Sun, Apr 28, 2013 at 9:53 PM, John . wrote: > I just started up my usenet reader (which generate a lot of small > files) and transferred two large files (7,5GiB) at the same time. > Performance seems all right again! :D Thanks! > > Could you explain to me why each of the options could have a positive > effect on performance? The wiki explains what the option imply, but > not how they could help boost performance. > > root@host:/storage# time mkdir test > > real 0m0.001s > user 0m0.000s > sys 0m0.000s > root@test:/storage# time rm -Rf test > > real 0m0.001s > user 0m0.000s > sys 0m0.000s > > Because performance was good again I was able to spam the volume with > data and the metadata size also grew. No problems in that department > either. ;-) > > 2013/4/28 Harald Glatt : >> On Sun, Apr 28, 2013 at 9:18 PM, John . wrote: >>> I use Ubuntu with kernel 3.8.0-19-generic. I also tested with the >>> latest live disk of Arch Linux; write performance was the same (bad). >>> >>> My mount options: rw,compress=lzo. >>> Iotop does not show any strange disk activity. >>> >>> 2013/4/28 Harald Glatt : >>>> On Sun, Apr 28, 2013 at 9:10 PM, John . wrote: >>>>> Hi Harald, >>>>> >>>>> I did perform a defrag of the volume a few hours ago. This did not >>>>> make a difference. :( >>>>> >>>>> Yours, >>>>> >>>>> John >>>>> >>>>> 2013/4/28 Harald Glatt : >>>>>> On Sun, Apr 28, 2013 at 9:04 PM, John . wrote: >>>>>>> Hi guys, >>>>>>> >>>>>>> My Btrfs fs has a performance problem which I hope you can help me >>>>>>> solve. I have a dataset of around 3.15 TiB, that has lived on a ZFS >>>>>>> volume for almost two years (ZRAID1, 4 2TiB disks). In order to move >>>>>>> to Btrfs I bought myself a 4TiB disk with the idea of buying a new one >>>>>>> next week and balance it to a RAID1 of 2 4TiB disks. >>>>>>> >>>>>>> I created a single disk Btrfs volume with the default mkfs options (no >>>>>>> data duplication, metadata duplication on). Next I transferred my >>>>>>> dataset to this disk (no problems there). Today when I tried to create >>>>>>> a directory and I noticed the Btrfs volume was awefully slow; it took >>>>>>> a few seconds to create the directory and a few to delete it (which >>>>>>> should be milliseconds as you know). In fact each and every operation >>>>>>> on the volume grinded the fs down to a halt. >>>>>>> >>>>>>> FS information: >>>>>>> >>>>>>> # btrfs fi df /storage >>>>>>> Data: total=3.29TB, used=3.15TB >>>>>>> System, DUP: total=8.00MB, used=360.00KB >>>>>>> System: total=4.00MB, used=0.00 >>>>>>> Metadata, DUP: total=4.00GB, used=3.88GB >>>>>>> Metadata: total=8.00MB, used=0.00 >>>>>>> >>>>>>> # btrfs fi show >>>>>>> Label: 'storage' uuid: 3fa262cd-baa9-46dc-92a8-318c87166186 >>>>>>> Total devices 1 FS bytes used 3.16TB >>>>>>> devid1 size 3.64TB used 3.30TB path /dev/sdb >>>>>>> >>>>>>> I suspect my performance blow has everything to do with the abysmally >>>>>>> low amount of space for metadata that is left, but since I am not a >>>>>>> Btrfs guru I don't now whether this is truly the case and/or how to >>>>>>> solve it. btrfs fi balance start -dusage=5 did not help. >>>>>>> >>>>>>> Yours, >>>>>>> >>>>>>> John >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" >>>>>>> in >>>>>>> the body of a message to majord...@vger.kernel.org >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>>> Try to defragment the root of the volume (e.g the mountpoint). While >>>>>> it's mounted: >>>>>> btrfs fi defrag /path/to/mnt >>>>>> >>>>>> Then try performance again >>>> >>>> What kernel version do you use? What are your mount options? Try to >>>> run iotop and see if there is any unusual activity... >> >> Try mount -o inode_cache,space_cache,autodefrag - first mount with the >> new options might take a while, also there might be disk activity for >> a while after mounting with this... Great, glad it helped! I'm not dev so I can only give vague and possibly wrong answers here: - autodefrag: would actually negatively impact immediate performance but will make a difference compared to no defrag over time - inode_cache: is apparently caching the latest inode number(?) that has been used so that whena new one has to be given it is immediately available instead of searching for it again - space_cache: caches the amount of space free, otherwise each space free 'question' to the volume would require it to recaculate it If you want better answers you should wait until a dev answers or corrects me :) Or come onto #btrfs on irc.freenode.net and ask there. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: grub/grub2 boot into btrfs raid root and with no initrd
On Fri, May 3, 2013 at 10:42 PM, Martin wrote:
> I've made a few attempts to boot into a root filesystem created using:
>
> mkfs.btrfs -d raid1 -m raid1 -L btrfs_root_3 /dev/sda3 /dev/sdb3
>
> Both grub and grub2 pick up a kernel image fine from an ext4 /boot on
> /dev/sda1 for exaample, but then fail to find or assemble the btrfs root.
>
> Setting up an initrd and grub operates fine for the btrfs raid.
>
>
> What is the special magic to do this without the need for an initrd?
>
> Is the comment/patch below from last year languishing unknown? Or is
> there some problem with that kernel approach?
>
>
> Thanks,
> Martin
>
>
> See:
>
> http://forums.gentoo.org/viewtopic-t-923554-start-0.html
>
>
> Below is my patch, which is working fine for me with 3.8.2.
> Code:
>
> $ cat /etc/portage/patches/sys-kernel/gentoo-sources/earlydevtmpfs.patch
> --- init/do_mounts.c.orig 2013-03-24 20:49:53.446971127 +0100
> +++ init/do_mounts.c 2013-03-24 20:51:46.408237541 +0100
> @@ -529,6 +529,7 @@
> create_dev("/dev/root", ROOT_DEV);
> if (saved_root_name[0]) {
>create_dev(saved_root_name, ROOT_DEV);
> + devtmpfs_mount("dev");
>mount_block_root(saved_root_name, root_mountflags);
> } else {
>create_dev("/dev/root", ROOT_DEV);
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
The initrd has to run btrfs-scan so that btrfs can find the other
devices that have btrfs on them. Alternatively you can give all
involved devices in the fstab and kernel command line with
device=/dev/name
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: hard freezes with 3.9.0 during io-intensive loads
I have this problem too, and I cannot reproduce it properly... When is
that patch in btrfs-next going to be in the mainline kernel?
On Mon, May 6, 2013 at 10:55 AM, Jan Schmidt wrote:
> On Sun, May 05, 2013 at 18:10 (+0200), Kai Krakow wrote:
>> Hello list,
>>
>> Kai Krakow schrieb:
>>
>>> I've upgraded to 3.9.0 mainly for the snapshot-aware defragging patches.
>>> I'm running bedup[1] on a regular basis and it is now the third time that
>>> I got back to my PC just to find it hard-frozen and I needed to use the
>>> reset button.
>>>
>>> It looks like this happens only while running bedup on my two btrfs
>>> filesystems but I'm not sure if it happens for any of the filesystems or
>>> only one. This is my setup:
>>>
>>> # cat /etc/fstab (shortened)
>>> UUID=d2bb232a-2e8f-4951-8bcc-97e237f1b536 / btrfs
>>> compress=lzo,subvol=root64 0 1 # /dev/sd{a,b,c}3
>>> LABEL=usb-backup /mnt/private/usb-backup btrfs noauto,compress-
>>> force=zlib,subvolid=0,autodefrag,comment=systemd.automount 0 0 # external
>>> usb3 disk
>>>
>>> # btrfs filesystem show
>>> Label: 'usb-backup' uuid: 7038c8fa-4293-49e9-b493-a9c46e5663ca
>>> Total devices 1 FS bytes used 1.13TB
>>> devid1 size 1.82TB used 1.75TB path /dev/sdd1
>>>
>>> Label: 'system' uuid: d2bb232a-2e8f-4951-8bcc-97e237f1b536
>>> Total devices 3 FS bytes used 914.43GB
>>> devid3 size 927.26GB used 426.03GB path /dev/sdc3
>>> devid2 size 927.26GB used 426.03GB path /dev/sdb3
>>> devid1 size 927.26GB used 427.07GB path /dev/sda3
>>>
>>> Btrfs v0.20-rc1
>>>
>>> Since the system hard-freezes I have no messages from dmesg. But I suspect
>>> it to be related to the defragmentation option in bedup (I've switched to
>>> bedub with --defrag since 3.9.0, and autodefrag for the backup drive).
>>> Just in case, I'm going to try without this option now and see if it won't
>>> freeze.
>>>
>>> I was able to take a "physical" screenshot with a real camera of a kernel
>>> backtrace one time when the freeze happened. I wonder if it is useful to
>>> you and where to send it. I just don't want to upload jpegs right here to
>>> the list without asking first.
>>>
>>> The big plus is: Altough I had to hard-reset the frozen system several
>>> times now, btrfs survived the procedure without any impact (just boot
>>> times increases noticeably, probably due to log-replays or something). So
>>> thumbs up for the developers on that point.
>>
>> Thanks to the great cwillu netcat service here's my backtrace:
>
> That one should be fixed in btrfs-next. If you can reliably reproduce the bug
> I'd be glad to get a confirmation - you can probably even save putting it on
> bugzilla then ;-)
>
> -Jan
>
>> 4,1072,17508258745,-;[ cut here ]
>> 2,1073,17508258772,-;kernel BUG at fs/btrfs/ctree.c:1144!
>> 4,1074,17508258791,-;invalid opcode: [#1] SMP
>> 4,1075,17508258811,-;Modules linked in: bnep bluetooth af_packet vmci(O)
>> vmmon(O) vmblock(O) vmnet(O) vsock reiserfs snd_usb_audio snd_usbmidi_lib
>> snd_rawmidi snd_seq_device gspca_sonixj gpio_ich gspca_main videodev
>> coretemp hwmon kvm_intel kvm crc32_pclmul crc32c_intel 8250 serial_core
>> lpc_ich microcode mfd_core i2c_i801 pcspkr evdev usb_storage zram(C) unix
>> 4,1076,17508258966,-;CPU 0
>> 4,1077,17508258977,-;Pid: 7212, comm: btrfs-endio-wri Tainted: G C O
>> 3.9.0-gentoo #2 To Be Filled By O.E.M. To Be Filled By O.E.M./Z68 Pro3
>> 4,1078,17508259023,-;RIP: 0010:[] []
>> __tree_mod_log_rewind+0x4c/0x121
>> 4,1079,17508259064,-;RSP: 0018:8801966718e8 EFLAGS: 00010293
>> 4,1080,17508259085,-;RAX: 0003 RBX: 8801ee8d33b0 RCX:
>> 880196671888
>> 4,1081,17508259112,-;RDX: 0a4596a4 RSI: 0eee RDI:
>> 8804087be700
>> 4,1082,17508259138,-;RBP: 0071 R08: 1000 R09:
>> 880196671898
>> 4,1083,17508259165,-;R10: R11: R12:
>> 880406c2e000
>> 4,1084,17508259191,-;R13: 8a11 R14: 8803b5aa1200 R15:
>> 0001
>> 4,1085,17508259218,-;FS: () GS:88041f20()
>> knlGS:
>> 4,1086,17508259248,-;CS: 0010 DS: ES: CR0: 80050033
>> 4,1087,17508259270,-;CR2: 026f0390 CR3: 01a0b000 CR4:
>> 000407f0
>> 4,1088,17508259297,-;DR0: DR1: DR2:
>>
>> 4,1089,17508259323,-;DR3: DR6: 0ff0 DR7:
>> 0400
>> 4,1090,17508259350,-;Process btrfs-endio-wri (pid: 7212, threadinfo
>> 88019667, task 8801b82e5400)
>> 4,1091,17508259383,-;Stack:
>> 4,1092,17508259391,-; 8801ee8d38f0 880021b6f360 88013a5b2000
>> 8a11
>> 4,1093,17508259423,-; 8802d0a14000 81167606 0246
>> 8801ee8d33b0
>> 4,1094,17508259455,-; 880406c2e000 8801966719bf 880021b6f360
>>
>> 4,1095,17508259488,-;Call Trace:
>> 4,
Re: How many subvols/snapshots are possible? (limits?)
I can only speak from experience, a snapshot can take up to a minute to create later on, so minutely snapshots are out of the question then... I have snapshots every hour and I have no problems with that at all :) On Thu, May 9, 2013 at 12:10 PM, Martin wrote: > Dear Devs, > > This is more a use case question of what is a good idea to do... > > > Can btrfs support snapshots of the filesystem at very regular intervals, > say minute by minute or even second by second? > > Or are there limits that will be hit with metadata overheads or > links/reference limits or CPU overheads if 'too many' snapshots/subvols > are made? > > If snapshots were to be taken once a minute and retained, what breaks first? > > > What are 'reasonable' (maximum) numbers for frequency and number of held > versions? > > > Thanks, > Martin > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Remove a materially failed device from a Btrfs "single-raid" using partitions
On Mon, May 13, 2013 at 4:15 PM, Vincent wrote:
> Hello,
>
> I am on Ubuntu Server 13.04 with Linux 3.8.
>
> I've created a "single-raid" using /dev/sd{a,b,c,d}{1,3}. One of my hard
> drives has failed, I mean it's materially dead.
>
> :~$ sudo btrfs filesystem show
> Label: none uuid: 40886f51-8c9b-4be1-8721-83bf5653d2a0
> Total devices 5 FS bytes used 226.90GB
> devid4 size 37.27GB used 31.01GB path /dev/sdd1
> devid3 size 37.27GB used 31.01GB path /dev/sdc1
> devid2 size 37.31GB used 31.00GB path /dev/sdb1
> devid1 size 139.73GB used 132.02GB path /dev/sda3
> *** Some devices missing
>
>
> Many tutorials I found about it never mention the simple deletion of a
> non-remountable disk in case of a "single-raid" (where the datas doesn't
> matter, I've used the only "d=single" option, insinuating "m=mirrored").
>
> I've read this page http://www.howtoforge.com/a-beginners-guide-to-btrfs
> until "8 Adding/Deleting Hard Drives To/From A btrfs File System" section.
> But this page want to make me mount the drive, but it's dead.
>
> When my Btrfs partition is not mounted and when I do:
> :~$ sudo btrfs device delete missing
> btrfs device delete: too few arguments
> or
> :~$ sudo btrfs device delete missing /media/single-raid/
> Nothing happen.
>
> If I try to mount the failed device, and remove /dev/sde1 from the
> mountpoint, my console doesn't respond anymore.
>
> I've also read the official documentation
> https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Removing_devices
> using degraded mode: mount -o degraded /dev/sda3 /media/single-raid/
>
> The fstab line is however: /dev/sda3 /media/single-raid/
> btrfs device=/dev/sda3,device=/dev/sdb1,device=/dev/sdc1,device=/dev/sdd1 0
> 2
>
> Then perform :~$ sudo btrfs filesystem show
> Label: none uuid: 40886f51-8c9b-4be1-8721-83bf5653d2a0
> Total devices 5 FS bytes used 226.30GB
> devid4 size 37.27GB used 31.01GB path /dev/sdd1
> devid3 size 37.27GB used 31.01GB path /dev/sdc1
> devid2 size 37.31GB used 31.00GB path /dev/sdb1
> devid1 size 139.73GB used 132.02GB path /dev/sda3
> Some devices missing*
>
> I don't understand why in degraded mode I can't remove the failed device.
> Could you help me please?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
If you have used d=single, it means that the data that was on the
drive that failed is now gone. I think btrfs refuses to remove devices
if it means data loss, but I could be wrong here..
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Remove a materially failed device from a Btrfs "single-raid" using partitions
On Mon, May 13, 2013 at 4:33 PM, Vincent wrote:
> Le 13/05/2013 16:29, Harald Glatt a écrit :
>
>> On Mon, May 13, 2013 at 4:15 PM, Vincent wrote:
>>>
>>> Hello,
>>>
>>> I am on Ubuntu Server 13.04 with Linux 3.8.
>>>
>>> I've created a "single-raid" using /dev/sd{a,b,c,d}{1,3}. One of my hard
>>> drives has failed, I mean it's materially dead.
>>>
>>> :~$ sudo btrfs filesystem show
>>> Label: none uuid: 40886f51-8c9b-4be1-8721-83bf5653d2a0
>>> Total devices 5 FS bytes used 226.90GB
>>> devid4 size 37.27GB used 31.01GB path /dev/sdd1
>>> devid3 size 37.27GB used 31.01GB path /dev/sdc1
>>> devid2 size 37.31GB used 31.00GB path /dev/sdb1
>>> devid1 size 139.73GB used 132.02GB path /dev/sda3
>>> *** Some devices missing
>>>
>>>
>>> Many tutorials I found about it never mention the simple deletion of a
>>> non-remountable disk in case of a "single-raid" (where the datas doesn't
>>> matter, I've used the only "d=single" option, insinuating "m=mirrored").
>>>
>>> I've read this page http://www.howtoforge.com/a-beginners-guide-to-btrfs
>>> until "8 Adding/Deleting Hard Drives To/From A btrfs File System"
>>> section.
>>> But this page want to make me mount the drive, but it's dead.
>>>
>>> When my Btrfs partition is not mounted and when I do:
>>> :~$ sudo btrfs device delete missing
>>> btrfs device delete: too few arguments
>>> or
>>> :~$ sudo btrfs device delete missing /media/single-raid/
>>> Nothing happen.
>>>
>>> If I try to mount the failed device, and remove /dev/sde1 from the
>>> mountpoint, my console doesn't respond anymore.
>>>
>>> I've also read the official documentation
>>>
>>> https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices#Removing_devices
>>> using degraded mode: mount -o degraded /dev/sda3 /media/single-raid/
>>>
>>> The fstab line is however: /dev/sda3 /media/single-raid/
>>> btrfs device=/dev/sda3,device=/dev/sdb1,device=/dev/sdc1,device=/dev/sdd1
>>> 0
>>> 2
>>>
>>> Then perform :~$ sudo btrfs filesystem show
>>> Label: none uuid: 40886f51-8c9b-4be1-8721-83bf5653d2a0
>>> Total devices 5 FS bytes used 226.30GB
>>> devid4 size 37.27GB used 31.01GB path /dev/sdd1
>>> devid3 size 37.27GB used 31.01GB path /dev/sdc1
>>> devid2 size 37.31GB used 31.00GB path /dev/sdb1
>>> devid1 size 139.73GB used 132.02GB path /dev/sda3
>>> Some devices missing*
>>>
>>> I don't understand why in degraded mode I can't remove the failed device.
>>> Could you help me please?
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majord...@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> If you have used d=single, it means that the data that was on the
>> drive that failed is now gone. I think btrfs refuses to remove devices
>> if it means data loss, but I could be wrong here..
>
>
> I've no problem with data loss, I just want to have a kind of sharing area,
> but data on other drive are always here, why couldn't I retrieve them alive?
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
I think it's a safety mechanism, you'll have to wait until a dev
responds... Otherwise you can always rsync the data into a new
filesystem and replace the old one if you can't wait.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/free-space-cache.c:1567!, Kernel 3.9.1
On Wed, May 15, 2013 at 4:19 PM, Philipp Dreimann wrote: > Hello, > > my btrfs filesystem was not mountable anymore after a loss of power: > > kernel BUG at fs/btrfs/free-space-cache.c:1567! > invalid opcode: [#1] SMP > Modules linked in: btrfs libcrc32c xor zlib_deflate raid6_pq i915(+) > i2c_algo_bit drm_kms_helper drm i2c_core video uinput > CPU 3 > Pid: 147, comm: mount Not tainted 3.9.1-301.fc19.x86_64 #1 LENOVO > 4290W4H/4290W4H > RIP: 0010:[] [] > remove_from_bitmap+0x1f4/0x210 [btrfs] > RSP: 0018:88020fddd698 EFLAGS: 00010287 > RAX: 001454248000 RBX: 880211399180 RCX: 001454253000 > RDX: 00013000 RSI: 51c0 RDI: 2e40 > RBP: 88020fddd6e8 R08: 88020fecf5c8 R09: 00145940 > R10: 8802124ce240 R11: 880211399198 R12: 880210025240 > R13: 88020fddd708 R14: 88020fddd700 R15: 8802113991a8 > FS: 7f019f213880() GS:88021e2c() knlGS: > CS: 0010 DS: ES: CR0: 80050033 > CR2: 7f4cdd4223f0 CR3: 00020ff03000 CR4: 000407e0 > DR0: DR1: DR2: > DR3: DR6: 0ff0 DR7: 0400 > Process mount (pid: 147, threadinfo 88020fddc000, task 8802100b4650) > Stack: > 88020fddd6e8 00145940 880210025264 001454253000 > 00013000 880210025240 880210025264 > 880210025248 88020ff4ec00 88020fddd738 a01c714b > Call Trace: > [] btrfs_remove_free_space+0x5b/0x290 [btrfs] > [] ? wake_up_bit+0x30/0x30 > [] btrfs_alloc_logged_file_extent+0x1af/0x1d0 [btrfs] > [] ? btrfs_free_path+0x26/0x30 [btrfs] > [] replay_one_extent+0x607/0x690 [btrfs] > [] replay_one_buffer+0x2cb/0x390 [btrfs] > [] ? alloc_extent_buffer+0x94/0x3f0 [btrfs] > [] walk_down_log_tree+0x224/0x410 [btrfs] > [] walk_log_tree+0xbf/0x1f0 [btrfs] > [] btrfs_recover_log_trees+0x1f1/0x360 [btrfs] > [] ? replay_one_extent+0x690/0x690 [btrfs] > [] open_ctree+0x18eb/0x1f90 [btrfs] > [] ? disk_name+0x54/0xb0 > [] btrfs_mount+0x5ce/0x6c0 [btrfs] > [] ? selinux_sb_copy_data+0x14d/0x220 > [] mount_fs+0x39/0x1b0 > [] vfs_kern_mount+0x5f/0xf0 > [] btrfs_mount+0x17c/0x6c0 [btrfs] > [] ? selinux_sb_copy_data+0x14d/0x220 > [] mount_fs+0x39/0x1b0 > [] vfs_kern_mount+0x5f/0xf0 > [] do_mount+0x23e/0xa20 > [] ? copy_mount_options+0x36/0x170 > [] sys_mount+0x83/0xc0 > [] system_call_fastpath+0x16/0x1b > Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 40 00 31 c0 48 83 7b 20 > 00 75 e4 48 89 de 4c 89 e7 89 45 c0 e8 b1 ed ff ff 8b 45 c0 eb d1 <0f> > 0b e8 69 a4 02 00 0f 1f 44 00 00 e8 59 a4 02 00 66 66 2e 0f > RIP [] remove_from_bitmap+0x1f4/0x210 [btrfs] > RSP > > Mounting the fs with "-o recovery,ro", and using btrfsck did not work. > btrfsck segfaulted, but I can not provide the error message. > btrfs-zero-log allowed me to mount the fs again. > > BR, > Philipp Dreimann > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Looks to me like you should have been able to mount with -o clear_cache. Did you try that? Does it work again now? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: subvol copying
On Wed, May 15, 2013 at 6:43 PM, Chris Murphy wrote: > > On May 15, 2013, at 1:40 AM, Gabriel de Perthuis wrote: > >> >> You can move subvolumes at any time, as if they were regular directories. > > In the example case, the subvolumes are read-only. So is it possible to make > a read-only subvolume (snapshot) read-writable? And is it possible to make a > read-writeable snapshot of a read-only subvolume? > > > Chris Murphy > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html You make a ro snapshot rw by creating a snapshot of it that is rw. So yes to both questions, by doing the same thing in both cases. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: subvol copying
On Wed, May 15, 2013 at 7:28 PM, Chris Murphy wrote: > > On May 15, 2013, at 10:44 AM, Harald Glatt wrote: > >> >> You make a ro snapshot rw by creating a snapshot of it that is rw. So >> yes to both questions, by doing the same thing in both cases. > > In other words, a normal snapshot (without -r) of a read-only snapshot will > create a rw snapshot? Yes -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Recommended settings for SSD
Data that already exists will only be compressed on re-write. You can
do it with btrfs fi defrag and a script that traverses the fs to call
defrag on every file. Another good way is the find command that has
been outlined here:
https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#Defragmenting_a_directory_doesn.27t_work
On Sun, May 26, 2013 at 12:29 AM, Leonidas Spyropoulos
wrote:
> On Sat, May 25, 2013 at 1:13 PM, Martin Steigerwald
> wrote:
>> Am Samstag, 25. Mai 2013, 03:58:12 schrieb Duncan:
>> [...]
>> And can be verified by:
>>
>> martin@merkaba:~> grep ssd /proc/mounts
>> /dev/mapper/merkaba-debian / btrfs rw,noatime,compress=lzo,ssd,space_cache 0 >> 0
>> /dev/mapper/merkaba-debian /mnt/debian-zeit btrfs
>> rw,noatime,compress=lzo,ssd,space_cache 0 0
>> /dev/mapper/merkaba-home /home btrfs rw,noatime,compress=lzo,ssd,space_cache >> 0
>> 0
>> /dev/mapper/merkaba-home /mnt/home-zeit btrfs
>> rw,noatime,compress=lzo,ssd,space_cache 0 0
>> martin@merkaba:~> grep ssd /etc/fstab
>> martin@merkaba:~#1>
>> [...]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> I see you are using compression. I don't have compression at the
> moment and I would like to use it. What will happen to the data that
> are already on the partitions? Will it be compressed when I use them?
> Do I have to re-write them? Would it be compressed with btrfs defrag
> command?
>
> Thanks for the information
>
> --
> Caution: breathing may be hazardous to your health.
>
> #include
> int main(){printf("%s","\x4c\x65\x6f\x6e\x69\x64\x61\x73");}
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Recommended settings for SSD
I don't know a better way to check than doing df -h before and
after... If you use space_cache you have to clear_cache though to make
the numbers be current for sure each time before looking at df.
Old kernel can be a problem. You can use the Arch CDs to do it, they
usually come with the newest kernels.
https://www.archlinux.org/download/
If you need to install anything a quick guide to the package managing:
# Update Repos and Upgrade system:
pacman -Suy
# Install a specific package:
pacman -S packagename
# Search for a package
pacman -Ss search term
On Sun, May 26, 2013 at 12:00 PM, Leonidas Spyropoulos
wrote:
> On Sat, May 25, 2013 at 11:33 PM, Harald Glatt wrote:
>> Data that already exists will only be compressed on re-write. You can
>> do it with btrfs fi defrag and a script that traverses the fs to call
>> defrag on every file. Another good way is the find command that has
>> been outlined here:
>>
>> https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#Defragmenting_a_directory_doesn.27t_work
>>
>
> I tried to my home partition the 'find' command and worked without
> errors, not sure if it did compress (how can I check?). I tried also
> on the root partition and every file that was in use (obviously) it
> didn't defrag it. I am guessing I have to mount the partition from a
> LiveCD but since the LiceCD kernel is usually old (in terms of btrfs)
> do you reckon there will be any problems?
>
> Thanks
> --
> Caution: breathing may be hazardous to your health.
>
> #include
> int main(){printf("%s","\x4c\x65\x6f\x6e\x69\x64\x61\x73");}
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
On Sat, Mar 9, 2013 at 9:31 PM, Hugo Mills wrote: >Some time ago, and occasionally since, we've discussed altering the > "RAID-n" terminology to change it to an "nCmSpP" format, where n is the > number of copies, m is the number of (data) devices in a stripe per copy, > and p is the number of parity devices in a stripe. > >The current kernel implementation uses as many devices as it can in the > striped modes (RAID-0, -10, -5, -6), and in this implementation, that is > written as "mS" (with a literal "m"). The mS and pP sections are omitted > if the value is 1S or 0P. > >The magic look-up table for old-style / new-style is: > > single 1C (or omitted, in btrfs fi df output) > RAID-0 1CmS > RAID-1 2C > DUP 2CD > RAID-10 2CmS > RAID-5 1CmS1P > RAID-6 1CmS2P > >The following patch set modifies userspace tools to accept C/S/P formats > in input (mkfs and the restriper). The older formats are also accepted. It > also prints the newer formats by default in btrfs fi df, with an option to > show the older format for the traditionalists. > >I'm not sure whether we should omit the 1C in btrfs fi df output, or > make it explicit even in the "single" case. > >Hugo. > > Hugo Mills (5): > Use nCmSpP format for mkfs > Move parse_profile to utils.c > Convert balance filter parser to use common nCmSpP replication-level > parser > Change output of btrfs fi df to report new (or old) RAID names > Add man page description for nCmSpP replication levels > > cmds-balance.c | 23 +++-- > cmds-filesystem.c | 135 > +++ > man/btrfs.8.in |9 > man/mkfs.btrfs.8.in | 24 - > mkfs.c | 35 - > utils.c | 94 +++ > utils.h |1 + > 7 files changed, 258 insertions(+), 63 deletions(-) > > -- > 1.7.10.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html I vote make it explicit to give people a chance to understand the new way it works even with just one device. The idea for this new naming scheme is great, I like it!! Will btrfs allow for things like 2C10SP2 too? Basically two RAID6 arrays with 12 drives each (10 for data, 2 for parity), if I understand it right with the entire array being mirrored to another one. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
On Sun, Mar 10, 2013 at 6:41 AM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 09/03/13 17:44, Hugo Mills wrote: >> You've got at least three independent parameters to the system in order >> to make that choice, though, and it's a fairly fuzzy decision problem. >> You've got: >> >> - Device redundancy - Storage overhead - Performance > > Overhead and performance aren't separate goals. More accurately the goal > is best performance given the devices available and constrained by redundancy. > > If I have 1GB of unique data and 10GB of underlying space available then > feel free to make 9 additional copies of each piece of data if that helps > performance. As I increase the unique data the overhead available will > decrease, but I doubt anyone has a goal of micromanaging overhead usage. > Why can't the filesystem just figure it out and do the best job available > given minimal constraints? > >> I definitely want to report the results in nCmSpP form, which tells you >> what it's actually done. The internal implementation, while not >> expressing the full gamut of possibilities, maps directly from the >> internal configuration to that form, and so it should at least be an >> allowable input for configuration (e.g. mkfs.btrfs and the restriper). > > Agreed on that for the micromanagers :-) > >> If you'd like to suggest a usable set of configuration axes [say, >> (redundancy, overhead) ], and a set of rules for converting those >> requirements to the internal representation, then there's no reason we >> can't add them as well in a later set of patches. > > The only constraints that matter are surviving N device failures, and data > not lost if at least N devices are still present. Under the hood the best > way of meeting those can be heuristically determined, and I'd expect > things like overhead to dynamically adjust as storage fills up or empties. > > Roger > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.11 (GNU/Linux) > > iEYEARECAAYFAlE8HR0ACgkQmOOfHg372QSyngCgpE9PTyBl3MsJ1kCYODtQWno/ > 85cAn0dcqE8ZWhOpFbZnQISmpe/KYceN > =LTf8 > -END PGP SIGNATURE- > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Very good points, I was also gonna write something by the lines of 'all that matters is achieving the minimum amount of redundancy, as requested by the user, at the maximum possible performance'. After reading your post now, Roger, I'm much more clear on what I actually wanted to say, which is pretty much the same thing: In paradise really all I would have to tell btrfs is how many drives I'm willing to give away so that they will be used exclusively for redundancy. Everything else btrfs should figure out by itself. Not just because it's simpler for the user, but also because btrfs actually is in a position to KNOW better. As Roger said, as long as the given minimum redundancy quota is filled, btrfs could make choices that favor either even more redundancy or more performance based on my usage of the filesystem by meassuring things like throughput, ioops or space used. I could imagine that a snail filesystem that barely fills and doesn't do whole lot could easily work itself up on building huge redunancy, while a filesystem that requires high performance would do excessive striping to achieve maximum performance, while only keeping the minimum requested redundancy intact. It sounds quite futuristic to me, but it is definitely something that we have to achieve hopefully rather sooner than later :) I'm looking forward to it! -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
> Very good points, > > I was also gonna write something by the lines of 'all that matters is > achieving the minimum amount of redundancy, as requested by the user, > at the maximum possible performance'. > > After reading your post now, Roger, I'm much more clear on what I > actually wanted to say, which is pretty much the same thing: > > In paradise really all I would have to tell btrfs is how many drives > I'm willing to give away so that they will be used exclusively for > redundancy. Everything else btrfs should figure out by itself. Not > just because it's simpler for the user, but also because btrfs > actually is in a position to KNOW better. > > As Roger said, as long as the given minimum redundancy quota is > filled, btrfs could make choices that favor either even more > redundancy or more performance based on my usage of the filesystem by > meassuring things like throughput, ioops or space used. I could > imagine that a snail filesystem that barely fills and doesn't do whole > lot could easily work itself up on building huge redunancy, while a > filesystem that requires high performance would do excessive striping > to achieve maximum performance, while only keeping the minimum > requested redundancy intact. > > It sounds quite futuristic to me, but it is definitely something that > we have to achieve hopefully rather sooner than later :) > > I'm looking forward to it! I have to add something to my own message: Even the notion of thinking in 'how many devices do I want to give away for redundancy' is outdated... What it really comes down to is how much space am I willing to sacrifice so that my reliablilty is increasing. Rather than addressing that at a per-drive level with a futuristic fs like btrfs I think setting a percent value of total space would be best. Here are my n hard drives, I want to give up thirty percent of maximum space so that basically I can lose that amount of space across any device combination and be safe. Do this for me and do it at maximum possible performance with the device count, and type, and sizes that I've given you. And if I'm not using the filesystem much, if I have tons of space free, feel free to build even more redundancy while idle. This is pretty much what it would be like in a perfect world, in my opinion :) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
On Sun, Mar 10, 2013 at 10:36 PM, Hugo Mills wrote: > >Oh, sorry. It's "reduced redundancy", aka DUP -- i.e. you get that > number of copies, but not guarantee that the copies all live on > different devices. I'm not devoted to showing it this way. Other > suggestions for making this distinction are welcomed. :) > > >Hugo. > > -- > === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === > PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk >--- There's many a slip 'twixt wicket-keeper and gully. --- I've noticed through my own tests that on a single device I can corrupt around 5% of the data completely before btrfs fails. Up to that point both filesystem as well as data integrity stays at 100%. However the default layout for one disk seems to be having the data once, the metadata DUP and the system DUP too. Having these 5% isn't mentioned anywhere... Is this a value that could maybe be manipulated and could it be introduced into a naming scheme like this? Also where do the 5% redundancy come from? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
I created a btrfs volume on a 4GB drive using the entire drive (VirtualBox VM). Of this drive btrfs immediately used 400 MB. I then filled it up with random data, left around 300 MB free and made a md5sum of said data. Then I umounted the volume and wrote random data into it the drive with dd at 1GB, 2GB and 3GB offsets. I increased the amount of data written each time between my tests. Btrfs kept the entire filesystem in tact (I did have to use btrfs-zero-log though) and the checksum stayed correct the entire time. I wrote 120 MB of corrupt data into the drive and that worked, on my next test writing 150 MB of corrupt data resulted in btrfs not being mountable anymore. 150 MB was around 5% of the 3.3 GB which was the size of my test file how I got to the 5%. The output of btrfs when I couldn't mount anymore said something about seeing a corruption larger than 145 MB which it cannot recover from (heavily paraphrased) so I knew that my 150 MB test was very close to the limit. On Sun, Mar 10, 2013 at 11:59 PM, Goffredo Baroncelli wrote: > On 03/10/2013 10:45 PM, Harald Glatt wrote: > >> I've noticed through my own tests that on a single device I can >> corrupt around 5% of the data completely before btrfs fails. Up to >> that point both filesystem as well as data integrity stays at 100%. >> However the default layout for one disk seems to be having the data >> once, the metadata DUP and the system DUP too. > > How make you the corruption ? Does btrfs return wrong data ? How is > calculated the 5% ? > > >> Having these 5% isn't >> mentioned anywhere... Is this a value that could maybe be manipulated >> and could it be introduced into a naming scheme like this? Also where >> do the 5% redundancy come from? > > On a single device, the metadata are DUPlicated but the data have only 1 > copy. > > This means that if you corrupt the 1 copy of the metadata, btrfs > survives using the other copy. Instead if you corrupt the data btrfs > return an error. > > >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > > -- > gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it> > Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/5] [RFC] RAID-level terminology change
On Sun, Mar 10, 2013 at 11:24 PM, Hugo Mills wrote: >> On 03/09/2013 09:31 PM, Hugo Mills wrote: >> >Some time ago, and occasionally since, we've discussed altering the >> > "RAID-n" terminology to change it to an "nCmSpP" format, where n is the >> > number of copies, m is the number of (data) devices in a stripe per copy, >> > and p is the number of parity devices in a stripe. >> > > >Hugo. > > -- > === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === > PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk >--- Great oxymorons of the world, no. 6: Mature Student --- It's important that the userland tools will output the things in both a 'human readable' format as well as the short forms. I would say when giving btrfs a parameter, it should only accept the short forms. But in order to allow users to figure out what they actually mean when doing something like 'btrfs fi show' it should tell you sometihng by the lines of '1 copy, 3 stripes, 1 parity (1C3S1P)' instead of just the short form. I would also make the case that leaving out 'defaults' from the output is bad for the learning curve as well. Even when it's just 1C I wouldn't remove that from the output, but the input parameter when you work with it should know what you mean when you leave 1C out, of course, and not require it. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Unable to boot btrfs filesystem, and btrfsck aborts
On Mon, Mar 11, 2013 at 11:44 PM, Matthew Booth wrote: > My laptop crashed hard earlier today. It reset immediately to a black > screen followed by the BIOS. I have no idea why. > > However, it now fails to boot. I took a picture of the kernel panic > that results from trying to mount the root filesystem: > https://plus.google.com/107763699965053810188/posts/QZZt7GYzBZi > > To make things worse, btrfsck aborts with a double free, without > fixing it. I took a picture of that, too: > https://plus.google.com/107763699965053810188/posts/gKYqGgFhWyT > > As the kernel panic mentions btrfs_remove_free_space, I also tried > mounting with clear_cache. Unfortunately it didn't dislodge anything. > > This is on a fully updated Fedora 18 system. I would really like to > get this data back. If anybody could offer a suggestion I'd be very > grateful. > > Thanks, > > Matt > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html If you can make a complete image backup of the drive before trying any things to bring it back. Try mounting with -o nospace_cache, also try -o ro and -o recovery as well as -o recovery,ro. If you can bringt it back in ro mode you can at least copy your data out of it if all else fails... I'm not a dev, just a random guy having an interest in btrfs, so if you don't have a backup and aren't able to create a dd copy of it right now you might wanna wait for a reply of someone who actually knows the code... Good luck -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Unable to boot btrfs filesystem, and btrfsck aborts
If you are going to use btrfs-zero-log please create a btrfs-image first that you can then upload to a bug report so that this can be fixed. # btrfs-image -c 9 -t 8 /dev/yourbtrfs /tmp/fs_image On Mon, Mar 11, 2013 at 11:53 PM, Jan Steffens wrote: > On Mon, Mar 11, 2013 at 11:49 PM, Harald Glatt wrote: >> On Mon, Mar 11, 2013 at 11:44 PM, Matthew Booth >> wrote: >>> My laptop crashed hard earlier today. It reset immediately to a black >>> screen followed by the BIOS. I have no idea why. >>> >>> However, it now fails to boot. I took a picture of the kernel panic >>> that results from trying to mount the root filesystem: >>> https://plus.google.com/107763699965053810188/posts/QZZt7GYzBZi >>> >>> To make things worse, btrfsck aborts with a double free, without >>> fixing it. I took a picture of that, too: >>> https://plus.google.com/107763699965053810188/posts/gKYqGgFhWyT >>> >>> As the kernel panic mentions btrfs_remove_free_space, I also tried >>> mounting with clear_cache. Unfortunately it didn't dislodge anything. >>> >>> This is on a fully updated Fedora 18 system. I would really like to >>> get this data back. If anybody could offer a suggestion I'd be very >>> grateful. >> If you can make a complete image backup of the drive before trying any >> things to bring it back. >> Try mounting with -o nospace_cache, also try -o ro and -o recovery as >> well as -o recovery,ro. > > I think the bug happens during log recovery, so btrfs-zero-log might > get it mountable again, with the caveat of losing the most recently > fsynced changes. > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Debian 3.7.1 BTRFS crash
If you care about the data, create a backup if you haven't already done so. Then you can try btrfsck, maybe you are in luck! On Wed, Mar 13, 2013 at 2:38 AM, Russell Coker wrote: > I have a workstation running the Debian packaged 3.7.1 kernel from 24th > December last year. After some period of uptime (maybe months) it crashed and > mounted the root filesystem read-only. Now when I boot it the root filesystem > gets mounted read-only. > > I have attached the dmesg output from the last boot. > > The system has an Intel 120G SSD and apart from 4G of swap and 400M of /boot > it's all a single encrypted BTRFS filesystem. > > Any suggestions on what I should do next? > > -- > My Main Blog http://etbe.coker.com.au/ > My Documents Bloghttp://doc.coker.com.au/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Any method to 'umount' a subvolume?
That's because the test shouldn't be in your root in the first place. The common way of thinking now is to create a btrfs volume with a structure for holding subvolumes inside of which your system root is a member. You then mount the system root via -o subvol= and will only see it and nothing else. If you want access to the "control" structure you mount that without -o subvol. If you want to mount another subvolume (even in place of your root) you mount that with -o subvol to whereever you want. Examples: real btrfs root: /root/default /root/snap1 /root/snap2 /home/default /home/snap1 The system root mounts /root/default on boot via -o subvol=root/default as well as the /home/default subvolume for /home. Then if you want to make a snapshot you mount btrfs to something like /mnt/ctrl and do 'btrfs sub snap / /mnt/ctrl/root/snap3' and umount /mnt/ctrl again. Rolling back a snapshot is as easy as mounting /mnt/ctrl and doing 'mv /mnt/ctrl/root/default /mnt/ctrl/root/default-old && mv /mnt/ctrl/root/snap3 /mnt/ctrl/root/default' followed by a reboot. No changes are necessary to your boot system and on the next boot you are rolled back and can delete the default-old subvol if you desire. On Thu, Mar 14, 2013 at 5:24 AM, Queen Adam wrote: > Hi all, > > When using "btrfs sub create test" commands to create a subvolume, it > will create test directory in the current directory. > > But the subvol directory can only be delete by "btrfs sub delete" > command, which seems also delete all the data in the subvolume. > > Any method about the 'umount'-like method to 'umount' the subvolume > and delete the subvolume directory without deleting all the data? > The method now seems less flexible (can mount -o subvol=test to other > path but the test/ can't be deleted) > > Thanks. > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Unable to mount btrfs after crash
Hi, I have set up a btrfs within a 200 GB file that I mount via -o loop. It worked fine so far but today it crashed when there was alot of concurrent writing/reading going on. I'm testing it with a torrent client, so the data is really irrelevant - but it's a good amount of stress. When it crashed dmesg got logged onto my /var/log which filled up the entire harddrive within a few minutes... Since I wasn't able to reboot properly I deleted the relevant log files and am not able to give a dmesg output of the crash :( I adjusted my logging now so I should be able to provide more info next time. When I try to mount the filesystem now mount hangs forever though and dmesg is flooding this message several times per second: parent transid verify failed on 164143104 wanted 22325 found 22209 I don't really care about the data but I would like to restore one config file from the filesystem. Is there any way to get it mounted? Or to extract single files? If you need any more info feel free to ask, I'll try to help as much as possible. Thanks Harald -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
